Line structured light based calibration method, system, device and storage medium

Abstract

The application provides a kind of calibration method, system, equipment and storage medium based on line structured light, the method comprises: in the camera field of view that has placed calibration board, respectively, collect the first image with line structured light projection and the second image without line structured light projection;Camera calibration is carried out using the second image, the internal parameter of the camera and the attitude transformation matrix between each calibration board are determined;The three-dimensional coordinates of the concentric circle center feature points are determined by the second image, and the three-dimensional coordinate data of the laser center point in the first image is reconstructed using the internal parameter and the attitude transformation matrix between each calibration board;The line structured light plane of the first image is fitted in combination with the three-dimensional coordinate data, and the calibration equation of line structured light plane is determined.The application calibrates camera and line structured light using multiple small calibration boards, and improves the accuracy of calibration in large field of view environment.

Description

Technical Field

[0001] This application belongs to the technical field of three-dimensional vision measurement, and relates to a calibration method, particularly a calibration method, system, device and storage medium based on line structured light. Background Technology

[0002] 3D measurement involves measuring an object from all angles to determine its three-dimensional coordinates. Its measurement principles can be categorized into four aspects: distance measurement, angular displacement, scanning, and orientation. Instruments developed based on these principles include photogrammetric (structured light) 3D scanners, laser 3D scanners, and coordinate measuring machines (CMMs).

[0003] With the continuous development and increasing maturity of 3D measurement technology, 3D scanning equipment has gradually become commercialized. The significant advantage of 3D laser scanners lies in their ability to quickly scan objects without the need for reflective prisms, directly obtaining high-precision point cloud data. This allows for efficient 3D modeling and virtual reconstruction of the real world. Therefore, it has become a current research hotspot and has wide applications in fields such as digital preservation of cultural relics, civil engineering, industrial surveying, natural disaster investigation, digital city terrain visualization, and urban and rural planning.

[0004] For example, 3D measurement has been widely used in aerospace, biomedicine, mechanical engineering and other fields. Among them, line structured light vision technology is an active 3D measurement technology that has attracted much attention due to its advantages such as non-contact, high speed, high precision, high robustness and low cost.

[0005] In industrial production, the flatness of steel plates is a crucial factor affecting product quality. Unevenness in steel plates can cause problems during processing or welding, leading to dimensional deviations or shape defects in the finished product, thus impacting its performance and reliability. Therefore, accurate measurement of steel plate flatness is a critical step in industrial production. However, in practical industrial environments where multi-line structured light is used to measure the surface flatness of steel plates, using large calibration plates for calibrating cameras and line structured light systems presents challenges, including difficulties in moving and fabricating such large calibration plates. Summary of the Invention

[0006] This application provides a calibration method, system, device, and storage medium based on line structured light, which solves the problems of difficulty in moving the calibration board and difficulty in processing large calibration boards when calibrating cameras and line structured light using large calibration boards.

[0007] In a first aspect, this application provides a calibration method based on line structured light, the method comprising: acquiring a first image projected by line structured light and a second image projected by wireless structured light in the field of view of a camera with calibration plates placed thereon; performing camera calibration using the second image to determine the intrinsic parameters of the camera and the attitude transformation matrix between each calibration plate; determining the three-dimensional coordinates of the center feature point of concentric circles using the second image, and reconstructing the three-dimensional coordinate data of the laser center point in the first image using the intrinsic parameters and the attitude transformation matrix between each calibration plate; and fitting the line structured light plane of the first image with the three-dimensional coordinate data to determine the calibration equation of the line structured light plane.

[0008] In one implementation of the first aspect, the step of acquiring a first image of wired structured light projection and a second image of wireless structured light projection respectively in the field of view of a camera with a calibration plate already placed includes: randomly changing the position of the camera a preset number of times; and acquiring the first image of wired structured light projection and the second image of wireless structured light projection at each position.

[0009] In one implementation of the first aspect, the step of using the second image to perform camera calibration and determine the intrinsic parameters of the camera and the attitude transformation matrix between each calibration plate includes: determining a linear solution of the camera intrinsic parameters; using the linear solution as an initial assumption, performing nonlinear optimization, and determining the attitude transformation matrix between each calibration plate.

[0010] In one implementation of the first aspect, the steps of determining the three-dimensional coordinates of the center feature point of the concentric circles through the second image and reconstructing the three-dimensional coordinate data of the laser center point in the first image using the intrinsic parameters and the attitude transformation matrix between each calibration plate include: performing sub-pixel edge detection and iterative compensation for concentric circle eccentricity error on the second image; extracting the sub-pixel image coordinates of the center feature point of the concentric circles to obtain the three-dimensional coordinates of the center feature point of the concentric circles; fitting the calibration plate plane equation using the intrinsic parameters; extracting the laser center point; and reconstructing the three-dimensional coordinates of the center point according to the attitude transformation matrix between each calibration plate and the calibration plate plane equation.

[0011] In one implementation of the first aspect, the step of extracting the laser center point and reconstructing the three-dimensional coordinates of the center point based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate includes: selecting a matrix-based laser center point extraction method to extract the two-dimensional coordinates of the laser center point; the laser center point refers to the center point of the line structured light determined among several points on the line structured light plane projected by a laser stripe; and reconstructing the three-dimensional coordinates of the line structured light center point in the camera coordinate system using the two-dimensional coordinates of the laser center point based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate.

[0012] In one implementation of the first aspect, the step of fitting the line structured light plane of the first image with the three-dimensional coordinate data to determine the calibration equation of the line structured light plane includes: fitting the line structured light plane of the first image with the least squares method using the three-dimensional coordinate data to determine the calibration equation of the line structured light plane.

[0013] In one implementation of the first aspect, the step of fitting the line structured light plane of the first image using the least squares method to determine the calibration equation of the line structured light plane includes: constructing a general expression for each line structured light plane equation based on the first image; determining n points (n≥3) on the line structured light plane; fitting the line structured light plane equation using the n points; rewriting the expression of the three-dimensional coordinate data of the laser center point into a matrix form, solving the system of equations in the matrix to determine the zeroth parameter, the first parameter, and the second parameter; and determining the calibration equation of the line structured light plane based on the zeroth parameter, the first parameter, and the second parameter.

[0014] Secondly, this application provides a calibration system based on line structured light, characterized in that the system comprises: an image acquisition module configured to acquire a first image projected by line structured light and a second image projected by wireless structured light in the field of view of a camera with calibration plates placed thereon; a camera calibration module configured to perform camera calibration using the second image, determining the intrinsic parameters of the camera and the attitude transformation matrix between each calibration plate; a three-dimensional reconstruction module configured to determine the three-dimensional coordinates of the center feature points of concentric circles through the second image, and reconstruct the three-dimensional coordinate data of the laser center point in the first image using the intrinsic parameters and the attitude transformation matrix between each calibration plate; and a fitting calibration module configured to fit the line structured light plane of the first image with the three-dimensional coordinate data, determining the calibration equation of the line structured light plane.

[0015] Thirdly, this application provides an electronic device, the electronic device comprising: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the method described thereon.

[0016] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method described thereon.

[0017] As described above, the calibration method, system, device, and storage medium based on line structured light described in this application have the following advantages:

[0018] Beneficial effects:

[0019] This application proposes a line structure light calibration method based on multiple small calibration plates (e.g., four plates), utilizing these plates to calibrate the camera and line structure light. This application fully considers the impact of calibration plate size on the calibration accuracy of the camera and line structure light in large field-of-view environments, using multiple small calibration plates with different orientations to calibrate the camera. The laser center point is then extracted, and its 3D coordinates in the camera coordinate system are reconstructed. Then, using the transformation matrix between the coordinate systems of the four calibration plates, the 3D coordinates are merged into a large 3D data field, which is then used to calibrate the line structure light plane. This improves the calibration accuracy in large field-of-view environments. Attached Figure Description

[0020] Figure 1 The diagram shows an application scenario of the calibration method based on line structured light described in the embodiments of this application.

[0021] Figure 2 The diagram shown illustrates the principle flowchart of the calibration method based on line structured light described in this application embodiment.

[0022] Figure 3 The diagram shown is an image acquisition flowchart of the calibration method based on line structured light described in the embodiments of this application.

[0023] Figure 4 The diagram shown is a camera calibration flowchart of the calibration method based on line structured light described in the embodiments of this application.

[0024] Figure 5 The diagram shown is a camera calibration schematic of the calibration method based on line structured light described in the embodiments of this application.

[0025] Figure 6 The diagram shown is a flowchart of the three-dimensional coordinate reconstruction of the calibration method based on line structured light described in the embodiments of this application.

[0026] Figure 7 The diagram shown is a fitting and calibration flowchart of the calibration method based on line structured light described in the embodiments of this application.

[0027] Figure 8 The diagram shown is a schematic diagram of the calibration system based on line structured light as described in an embodiment of this application.

[0028] Figure 9 The diagram shown is a structural connection diagram of the electronic device described in an embodiment of this application.

[0029] Component designation explanation

[0030] 8. Calibration System Based on Line Structured Light

[0031] 81 Image Acquisition Module

[0032] 82 Camera Calibration Module

[0033] 83 Three-Dimensional Reconstruction Module

[0034] 84 Fitting and Calibration Module

[0035] 9 Electronic devices

[0036] 91 processor

[0037] 92 Memory

[0038] 921 Random Access Memory

[0039] 922 cache memory

[0040] 923 storage system

[0041] 924 Utilities

[0042] 925 Program Module

[0043] 93 bus

[0044] 94 External devices

[0045] 95 monitor

[0046] 96 I / O interfaces

[0047] 97 Network Adapter

[0048] Steps S21 to S24

[0049] Steps S211~S212

[0050] Steps S221~S222

[0051] Steps S231~S233

[0052] Steps S241~S244 Detailed Implementation

[0053] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0054] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the illustrations only show the components related to this application and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0055] The following embodiments of this application provide calibration methods, systems, devices, and storage media based on line structured light, including but not limited to... Figure 1 The application scenarios shown below will be used as examples. Figure 1 The application scenario shown is used as an example for description.

[0056] Please see Figure 1 The image shows a schematic diagram illustrating an application scenario of the calibration method based on line structured light described in this application embodiment. For example... Figure 1 As shown in the embodiment of this application, four small calibration plates are used as an example to calibrate the camera and line structured light. In the field of line structured light measurement, a measurement range exceeding 40cm is considered a large scene, thus requiring the use of a large calibration plate. This application considers the influence of calibration plate size on the calibration accuracy of the camera and line structured light in a large field of view or large scene environment, and uses four small calibration plates with different orientations to finally fit and calibrate the line structured light plane of the large calibration plate.

[0057] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0058] Please see Figure 2 The diagram shows the principle flowchart of the calibration method based on line structured light described in the embodiments of this application. Figure 2 As shown, this embodiment provides a calibration method based on line structured light, which specifically includes the following steps:

[0059] S21, in the field of view of the camera with the calibration plate already placed, acquire the first image projected by wired structured light and the second image projected by wireless structured light respectively.

[0060] The second image projected by wireless structured light is used for camera calibration, while the first image projected by wired structured light is used for three-dimensional coordinate reconstruction of the laser center point.

[0061] Please see Figure 3 The image acquisition flowchart is shown as an embodiment of the calibration method based on line structured light described in this application.

[0062] like Figure 3 As shown, step S21 includes:

[0063] S211, randomly changes the camera position a preset number of times.

[0064] Specifically, four identical small calibration targets are randomly placed within the camera's field of view. The combined area of ​​the four calibration targets should cover as much of the entire camera's field of view as possible to ensure calibration accuracy. Then, the camera position is randomly changed 20 times.

[0065] S212, at each position, acquires the first image of wired structured light projection and the second image of wireless structured light projection respectively.

[0066] Specifically, during 20 random camera position changes, images of both line structured light projection (e.g., eight parallel and evenly distributed line structured light rays) and wireless structured light projection are captured at each position. This yields 20 calibration images of the line structured light projection (first image) and 20 calibration images of the wireless structured light projection (second image).

[0067] S22, use the second image to perform camera calibration, and determine the intrinsic parameters of the camera and the attitude transformation matrix between each calibration plate.

[0068] Please see Figure 4 The diagram shows a camera calibration flowchart of the calibration method based on line structured light described in the embodiments of this application.

[0069] like Figure 4 As shown, step S22 includes:

[0070] S221, determine the linear solution of the camera intrinsic parameters. This application uses four calibration plates to calibrate the camera. The camera parameters are calculated based on the H matrix between each calibration plate and the image plane. Then, the feature points on each calibration plate are merged together through the transformation matrix between the coordinate systems of the four calibration plates to obtain a large three-dimensional data field.

[0071] Please see Figure 5 The image shown is a schematic diagram of camera calibration using the line structured light-based calibration method described in an embodiment of this application. Figure 5 As shown, O1-x1y1z1 is the coordinate system of the reference calibration plate, and R... Ti , t Ti (i = 1, 2, 3) represent the rotation matrices and translation vectors from other calibration plate coordinate systems to the reference calibration plate coordinate system. The mathematical model is as follows:

[0072]

[0073] in, and q Ti These are the 3D coordinates of the i-th small target feature point in the benchmark calibration board and other calibration boards, respectively.

[0074] Let Pi be a point on the reference calibration board. Its homogeneous coordinates in the pixel coordinate system and the calibration board coordinate system are pi = [ui, vi, 1]T and qi = [xi, yi, 1]T, respectively. R and t are the rotation matrix and translation vector from the reference calibration board coordinate system to the camera coordinate system. The transformation relationship between pi and qi is:

[0075] ρ1p i =K[r1 r2 t]q i =Hq i (2)

[0076] Where r1 and r2 are the first and second columns of R, H = [h1 h2 h3] is the homography matrix from the calibration board coordinate system to the pixel coordinate system, h k (k = 1, 2, 3) is the k-th column of H. According to the orthogonality of rotation matrices, we have:

[0077]

[0078] Since B is a symmetric matrix, the six-dimensional vector can be set as:

[0079] b = [B 11 B 12 B 22 B 13 B 23 B 33 ] T (5)

[0080] Therefore, formula (3) can be written as a homogeneous equation, where b is unknown:

[0081] vb = 0(6)

[0082] Where v is a 2×6 matrix, and the camera is randomly positioned at least twice to capture images of the calibration board. For the n calibration images captured by the camera:

[0083] Vb = 0(7)

[0084] Where V is a 2n×6 matrix.

[0085] When n≥3, b is obtained through the singular value decomposition (SVD) of V.

[0086] According to equation (4), K can be obtained using the Cholesky decomposition algorithm. -1 K -1 The inverse matrix K is the camera's intrinsic parameter matrix. The rotation matrix and translation vector R from the calibration board coordinate system to the camera coordinate system are also given. ic , t ic This can also be obtained accordingly.

[0087] The rotation matrices and translation vectors RTi,tTi (i = 1, 2, 3) from other calibration plate coordinate systems to the reference calibration plate coordinate system can be solved using the camera coordinate system as an intermediary. The transformation relationships are as follows:

[0088]

[0089] Where R,t is the rotation matrix and translation vector from the reference calibration plate coordinate system to the camera coordinate system.

[0090] S222, using the linear solution as the initial assumption, perform nonlinear optimization to determine the attitude transformation matrix between each calibration plate.

[0091] Specifically, assuming that at the j-th camera shooting position, the three-dimensional homogeneous coordinates of the i-th feature point P in the calibration board coordinate system and the reference calibration board coordinate system are q ij and In the pixel coordinate system, the distorted and undistorted homogeneous image coordinates of P are respectively p ij and The homogeneous projection coordinates of point P in the pixel coordinate system are: The specific conversions between the above parameters are as follows:

[0092]

[0093] Among them, R j , t j It is the rotation matrix and translation vector from the reference calibration plate coordinate system to the camera coordinate system.

[0094] Based on the camera's imaging characteristics, it is assumed that the image noise consists of a Gaussian distribution and independent distributions. The objective function for nonlinear optimization is established as follows:

[0095]

[0096] Where a=[K,k1,k1,R] Ti ,t Ti ,R j ,t j ], where n is the number of calibration plates and d(A,B) is the distance between A and B.

[0097] Using the results in Equation (8) as initial assumptions, the nonlinear optimization problem is solved by the Levenberg-Marquardt algorithm.

[0098] S23, determine the three-dimensional coordinates of the center feature point of the concentric circles through the second image, and reconstruct the three-dimensional coordinate data of the laser center point in the first image using the intrinsic parameters and the attitude transformation matrix between each calibration plate.

[0099] Please see Figure 6The diagram shows a flowchart of the three-dimensional coordinate reconstruction process of the calibration method based on line structured light described in this application embodiment. Figure 6 As shown, step S23 includes:

[0100] S231, perform subpixel edge detection and concentric circle eccentricity error iterative compensation on the second image, extract the subpixel image coordinates of the concentric circle center feature points, and then obtain the three-dimensional coordinates of the concentric circle center feature points.

[0101] Specifically, for any pair of calibration images (i.e. the first image projected by wired structured light), the calibration board image (i.e. the second image projected by wireless structured light) that does not contain light stripes at the camera position is taken to perform sub-pixel edge detection and concentric circle eccentricity error iterative compensation, extract the sub-pixel image coordinates of the feature points at the center of the concentric circles, and then obtain the three-dimensional coordinates of the feature points according to the camera calibration parameters.

[0102] S232, use the intrinsic parameters to fit the plane equation of the calibration plate.

[0103] Specifically, by combining the intrinsic parameters of the camera calibration, the extrinsic parameters of the camera at that location can be calculated. With the origin of the world coordinate system fixed at a corner of the calibration plate, the axis perpendicular to the plane of the calibration plate, and the plane coinciding with the plane of the calibration plate, the equation of the plane of the calibration plate can be expressed using its extrinsic parameters as follows:

[0104] r 13 x+r 23 y+r 33 z+r 13 t1+r 23 t2+r 33 t3=0(11)

[0105] S233, extract the laser center point, and reconstruct the three-dimensional coordinates of the center point based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate.

[0106] In one embodiment, the step of extracting the laser center point and reconstructing the three-dimensional coordinates of the center point based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate includes:

[0107] (1) Select a matrix-based laser center point extraction method to extract the two-dimensional coordinates of the laser center point; the laser center point refers to the center point of the line structure light determined among several points on the line structure light plane by a laser stripe.

[0108] Specifically, a laser center point extraction method based on the HESSION matrix is ​​selected to achieve high-precision extraction of the laser center point.

[0109] (2) Based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate, the three-dimensional coordinates of the line structure light center point in the camera coordinate system are reconstructed using the two-dimensional coordinates of the laser center point.

[0110] Specifically, line structure calibration requires obtaining the three-dimensional coordinates of the center points of these light stripes in the camera coordinate system. In order to solve the problem that the depth of the target point cannot be determined by a single camera, the equation of the calibration plate plane where the light stripe is located is calculated, and then the three-dimensional point is reconstructed by combining the results of large field of view single-target calibration.

[0111] From the camera imaging relationship, we can obtain:

[0112]

[0113] Substituting formula (12) into formula (11) yields the three-dimensional coordinates of the center point of the light stripe:

[0114]

[0115] S24, combine the three-dimensional coordinate data to fit the line structured light plane of the first image, and determine the calibration equation of the line structured light plane.

[0116] Please see Figure 7 The diagram shows the fitting and calibration flowchart of the calibration method based on line structured light described in the embodiments of this application. Figure 7 As shown, step S24 includes: combining the three-dimensional coordinate data, fitting the line structured light plane of the first image using the least squares method to determine the calibration equation of the line structured light plane. In one embodiment, step S24 specifically includes the following steps:

[0117] S241, Based on the first image, construct a general expression for the equations of each line structured light plane.

[0118] Specifically, in combination Figure 1 As shown, let the general expression of the plane equation for any linear structured light be:

[0119] Ax + By + Cz + D = 0 (C ≠ 0) (14)

[0120] remember:

[0121]

[0122] Substituting formula (15) into formula (14) yields:

[0123] z = a0x + a1y + a2 (16)

[0124] S242, determine n points (n≥3) on the line structured light plane; fit the line structured light plane equation using the n points.

[0125] Specifically, for a series of n points (n≥3) on the structured light plane; (x i y i , z i Given n points i = 0, 1, ..., n-1, to fit the plane equation using these n points, even if:

[0126]

[0127] S243, rewrite the expression of the three-dimensional coordinate data of the laser center point into matrix form, and solve the system of equations in the matrix to determine the zeroth parameter, the first parameter, and the second parameter.

[0128] Specifically, to minimize S, we should take the partial derivatives of both sides of equation (13) with respect to a0, a1, and a2, and set the partial derivatives to zero. That is:

[0129]

[0130] Rewritten in matrix form:

[0131]

[0132] Solving the system of equations (19) yields the zeroth parameter a0, the first parameter a1, and the second parameter a2.

[0133] S244, Determine the calibration equation of the line structured light plane based on the zeroth parameter, the first parameter, and the second parameter.

[0134] Specifically, the plane equation can be obtained by substituting the zeroth parameter a0, the first parameter a1, and the second parameter a2 into formula (16).

[0135] The scope of protection of the calibration method based on line structured light described in this application is not limited to the order of steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0136] This application also provides a calibration system based on line structured light. The calibration system based on line structured light can implement the calibration method based on line structured light described in this application. However, the implementation device of the calibration method based on line structured light described in this application includes, but is not limited to, the structure of the calibration system based on line structured light listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this application are included within the protection scope of this application.

[0137] Please see Figure 8The diagram shows the structural principle of the calibration system based on line structured light according to an embodiment of this application. This embodiment provides a calibration system based on line structured light, which includes: an image acquisition module 81, a camera calibration module 82, a three-dimensional reconstruction module 83, and a fitting calibration module 84.

[0138] The image acquisition module 81 is configured to acquire a first image projected by wired structured light and a second image projected by wireless structured light in the field of view of the camera with the calibration plate placed on it.

[0139] In one embodiment, the image acquisition module 81 is specifically configured to randomly change the position of the camera a preset number of times; and to acquire a first image projected by wired structured light and a second image projected by wireless structured light at each position.

[0140] The camera calibration module 82 is configured to perform camera calibration using the second image, and determine the camera's intrinsic parameters and the attitude transformation matrix between each calibration plate.

[0141] In one embodiment, the camera calibration module 82 is specifically configured to determine a linear solution of the camera intrinsic parameters; using the linear solution as an initial assumption, perform nonlinear optimization to determine the attitude transformation matrix between each calibration board.

[0142] The 3D reconstruction module 83 is configured to determine the 3D coordinates of the center feature point of the concentric circles through the second image, and reconstruct the 3D coordinate data of the laser center point in the first image using the intrinsic parameters and the attitude transformation matrix between each calibration plate.

[0143] In one embodiment, the three-dimensional reconstruction module 83 is specifically configured to perform sub-pixel edge detection and concentric circle eccentricity error iterative compensation on the second image, extract the sub-pixel image coordinates of the concentric circle center feature points to obtain the three-dimensional coordinates of the concentric circle center feature points; fit the calibration plate plane equation using the intrinsic parameters; extract the laser center point, and reconstruct the three-dimensional coordinates of the center point according to the attitude transformation matrix between each calibration plate and the calibration plate plane equation.

[0144] Furthermore, the three-dimensional reconstruction module 83 is specifically configured to select a matrix-based laser center point extraction method to extract the two-dimensional coordinates of the laser center point; the laser center point refers to the center point of the line structured light determined among several points on the line structured light plane projected by a laser stripe; based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate, the three-dimensional coordinates of the line structured light center point in the camera coordinate system are reconstructed using the two-dimensional coordinates of the laser center point.

[0145] The fitting and calibration module 84 is configured to fit the line structured light plane of the first image with the three-dimensional coordinate data to determine the calibration equation of the line structured light plane.

[0146] In one embodiment, the fitting and calibration module 84 is specifically configured to combine the three-dimensional coordinate data and use the least squares method to fit the line structured light plane of the first image to determine the calibration equation of the line structured light plane.

[0147] Furthermore, the fitting and calibration module 84 is specifically configured to: construct a general expression for each line structured light plane equation based on the first image; determine n points (n≥3) on the line structured light plane; fit the line structured light plane equation using the n points; rewrite the expression of the three-dimensional coordinate data of the laser center point into a matrix form; solve the system of equations in the matrix to determine the zeroth parameter, the first parameter, and the second parameter; and determine the calibration equation of the line structured light plane based on the zeroth parameter, the first parameter, and the second parameter.

[0148] In the embodiments provided in this application, it should be understood that the disclosed systems or methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or modules or units, and may be electrical, mechanical, or other forms.

[0149] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0150] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0151] This application provides an electronic device, including: a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the calibration method based on line structured light.

[0152] Please see Figure 9 The diagram shows the structural connections of the electronic device described in the embodiments of this application. Figure 9 As shown, electronic device 9 is represented in the form of a general-purpose computing device. The components of electronic device 9 may include, but are not limited to: one or more processors 91, memory 92, and bus 93 connecting different system components (including processors 91 and memory 92).

[0153] Bus 93 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. Examples of these architectures include, but are not limited to, the ISA (Industry Standard Architecture) bus, the MCA (MicroChannel Architecture) bus, the enhanced ISA bus, the VESA (Video Electronics Standards Association) local bus, and the PCI (Peripheral Component Interconnect) bus.

[0154] Electronic device 9 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 9, including volatile and non-volatile media, removable and non-removable media.

[0155] Memory 92 may include computer system readable media in the form of volatile memory, such as RAM (Random Access Memory) 921 and / or cache memory 922. Electronic device 9 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 923 may be used to read and write non-removable, non-volatile magnetic media, commonly referred to as "hard disk drives". Disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disk drives for reading and writing to removable non-volatile optical disks, such as CD-ROMs (Compact Disc Read-Only Memory), DVD-ROMs (Digital Video Discs), or other optical media, may be provided. In these cases, each drive may be connected to bus 93 via one or more data media interfaces. Memory 92 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0156] A program / utility 924 having a set (at least one) of program modules 925 may be stored, for example, in memory 92. Such program modules 925 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 925 typically perform the functions and / or methods described in the embodiments of the present invention.

[0157] The computer system can also communicate with one or more external devices 94 (e.g., keyboard, pointing device, display 95, etc.), and with one or more devices that enable a user to interact with the computer system, and / or with any device that enables the computer system to communicate with one or more other computing devices (e.g., network interface card, modem, etc.). This communication can be performed via input / output (I / O) interface 96. Furthermore, the computer system can communicate with one or more networks via network adapter 97, such as LAN (Local Area Network), WAN (Wide Area Network), and / or public networks, such as the Internet. Network adapter 97 communicates with other modules of the computer system via bus 93. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the computer system, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID (Redundant Arrays of Independent Disks) systems, tape drives, and data backup storage systems.

[0158] In practical applications, the electronic device may be a computer including all or some of its components such as memory, storage controller, one or more processing units (CPU), peripheral interfaces, RF circuitry, audio circuitry, speakers, microphones, input / output (I / O) subsystems, displays, other output or control devices, and external ports; the computer includes, but is not limited to, personal computers such as desktop computers, laptops, tablets, smartphones, smart TVs, and personal digital assistants (PDAs). In other embodiments, the electronic device may also be a server, which may be deployed on one or more physical servers depending on factors such as function and load, or it may be a cloud server composed of distributed or centralized server clusters; this embodiment does not impose any limitations.

[0159] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed, implements the calibration method based on line structured light.

[0160] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. This available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0161] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0162] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A calibration method based on line structured light, characterized in that, The method includes: In the field of view of the camera with the calibration plate already placed, the first image projected by wired structured light and the second image projected by wireless structured light are acquired respectively. The second image is used to perform camera calibration, and the intrinsic parameters of the camera and the attitude transformation matrix between each calibration plate are determined. The three-dimensional coordinates of the center feature point of the concentric circle are determined by the second image, and the three-dimensional coordinate data of the laser center point in the first image are reconstructed by the intrinsic parameters and the attitude transformation matrix between each calibration plate. By fitting the line structured light plane of the first image with the three-dimensional coordinate data, the calibration equation of the line structured light plane is determined.

2. The method according to claim 1, characterized in that, The steps of acquiring a first image projected by wired structured light and a second image projected by wireless structured light within the camera's field of view, where a calibration plate has been placed, include: The camera position is randomly changed a preset number of times; At each location, a first image of wired structured light projection and a second image of wireless structured light projection are acquired respectively.

3. The method according to claim 1, characterized in that, The steps of using the second image to perform camera calibration and determine the intrinsic parameters of the camera and the attitude transformation matrix between each calibration plate include: Determine a linear solution for the camera's intrinsic parameters; Using the linear solution as the initial assumption, nonlinear optimization is performed to determine the attitude transformation matrix between each calibration plate.

4. The method according to claim 1, characterized in that, The steps of determining the three-dimensional coordinates of the center feature point of the concentric circles from the second image, and reconstructing the three-dimensional coordinate data of the laser center point in the first image using the intrinsic parameters and the attitude transformation matrix between each calibration plate, include: Subpixel edge detection and concentric circle eccentricity error iterative compensation are performed on the second image to extract the subpixel image coordinates of the concentric circle center feature points and thus obtain the three-dimensional coordinates of the concentric circle center feature points. The intrinsic parameters are used to fit the plane equation of the calibration plate; The laser center point is extracted, and the three-dimensional coordinates of the center point are reconstructed based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate.

5. The method according to claim 4, characterized in that, The steps of extracting the laser center point and reconstructing the three-dimensional coordinates of the center point based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate include: A matrix-based laser center point extraction method is selected to extract the two-dimensional coordinates of the laser center point; the laser center point refers to the center point of the line structured light determined among several points on the line structured light plane projected by a laser stripe. Based on the attitude transformation matrix between each calibration plate and the plane equation of the calibration plate, the three-dimensional coordinates of the line structured light center point in the camera coordinate system are reconstructed using the two-dimensional coordinates of the laser center point.

6. The method according to claim 1, characterized in that, The step of fitting the line structured light plane of the first image with the three-dimensional coordinate data to determine the calibration equation of the line structured light plane includes: By combining the three-dimensional coordinate data, the line structured light plane of the first image is fitted using the least squares method to determine the calibration equation of the line structured light plane.

7. The method according to claim 6, characterized in that, The steps of fitting the line structured light plane of the first image using the least squares method to determine the calibration equation of the line structured light plane include: Based on the first image, construct general expressions for the plane equations of each line structure light; Determine n points (n≥3) on the line structured light plane; fit the equation of the line structured light plane using the n points; The expression for the three-dimensional coordinate data of the laser center point is rewritten in matrix form, and the zeroth parameter, the first parameter, and the second parameter are determined by solving the system of equations in the matrix. The calibration equation of the line structured light plane is determined based on the zeroth parameter, the first parameter, and the second parameter.

8. A calibration system based on line structured light, characterized in that, The system includes: The image acquisition module is configured to acquire a first image projected by wired structured light and a second image projected by wireless structured light in the field of view of the camera with the calibration plate placed on it. The camera calibration module is configured to perform camera calibration using the second image, and determine the intrinsic parameters of the camera and the attitude transformation matrix between each calibration plate; The 3D reconstruction module is configured to determine the 3D coordinates of the center feature point of the concentric circles through the second image, and reconstruct the 3D coordinate data of the laser center point in the first image using the intrinsic parameters and the attitude transformation matrix between each calibration plate. The fitting and calibration module is configured to fit the line structured light plane of the first image to the three-dimensional coordinate data, and determine the calibration equation of the line structured light plane.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the method described in any one of claims 1 to 7.

Images