Camera assisted calibration method, system, device and storage medium

Abstract

This invention discloses a camera-assisted calibration method, system, device, and storage medium, relating to the field of camera calibration. The method includes obtaining initial intrinsic parameters of the camera; determining the calibration orientation of the calibration board using simulation technology based on the calibration board; performing coordinate system transformation using a construction matrix based on the calibration orientation of the calibration board to obtain vertex coordinates in the image plane coordinate system; the construction matrix is ​​determined by the initial intrinsic parameters of the camera; the construction matrix includes a model matrix, a projection matrix, and an observation matrix; rendering virtual calibration information based on the vertex coordinates in the image plane coordinate system to obtain a rendering result; and calibrating the camera based on the rendering result to obtain the camera calibration result. This invention can improve the accuracy of calibration and reduce the workload in the calibration process.

Description

Technical Field

[0001] The present invention relates to the field of camera calibration, and in particular to a camera auxiliary calibration method, system, device and storage medium. Background Art

[0002] In a 3D vision system, all kinds of information about 3D objects are obtained from the image information acquired by the camera. In this process, the relationship between spatial points and camera pixels must be determined by camera calibration. The performance of many 3D computer vision algorithms directly depends on the quality of this calibration. Camera calibration is an important issue in the field of computer vision. Whether in image measurement or machine vision applications, the calibration of camera parameters is a very critical link. The accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera.

[0003] Camera calibration methods are generally divided into traditional camera calibration methods and camera self-calibration methods. Since the calibration accuracy of camera self-calibration methods is low, the robustness is poor, and a high-precision camera model cannot be established, the traditional camera calibration method is generally used in augmented reality devices. The traditional camera calibration method uses some landmarks with known sizes and specific rules. Through the relationship between the three-dimensional position of a point in space and the corresponding point on the image plane, the relative geometric posture relationship of the camera and the intrinsic distortion coefficient of the camera are calculated. At present, the most widely used is the Zhang Zhengyou calibration method. The Zhang Zhengyou calibration method uses a black and white plane chessboard, takes multiple groups of pictures from different positions and angles, and solves the camera parameters through these pictures, and then performs nonlinear optimization. However, the calibration accuracy is low, the operation is complicated, and the calibration consumes a long time. Therefore, seeking an efficient, accurate, and easy-to-operate camera calibration method has become a hot topic in current research.

[0004] Camera calibration is an important task in the field of computer vision and computer graphics, which is used to estimate the intrinsic parameters and extrinsic pose of the camera to achieve accurate image measurement and 3D reconstruction. Traditional camera calibration methods require precise measurements and calculations during the calibration process. However, this method is tedious and prone to errors.

[0005] With the continuous development of augmented reality technology, it has been widely used in games, education, medical treatment, manufacturing and other fields. Augmented reality technology can locate and track the user's position and posture more accurately, thereby realizing more realistic virtual information superposition. Augmented reality technology combines the interaction of virtual information with the real world, and can superimpose virtual information in the real environment, providing a more intuitive and immersive interactive experience.

[0006] Through the above analysis, the problems and defects of traditional calibration technology are: long calibration time consumption, poor robustness, and inability to complete calibration quickly. Among the traditional camera calibration methods, Zhang Zhengyou's calibration method has low calibration accuracy and cannot be applied to complex scenes. The traditional camera calibration method requires manual measurement of feature points on the calibration plate, which is cumbersome and has high calibration costs.

[0007] Therefore, it is necessary to design a camera calibration method based on augmented reality to solve the technical problems faced by the above-mentioned traditional camera calibration. Summary of the invention

[0008] The object of the present invention is to provide a camera-assisted calibration method, system, device and storage medium, which can improve the accuracy of calibration and reduce the workload during the calibration process.

[0009] To achieve the above object, the present invention provides the following solutions:

[0010] A camera-assisted calibration method, comprising:

[0011] Get the initial intrinsic parameters of the camera;

[0012] Determine the calibration posture of the calibration plate by using simulation technology according to the calibration plate;

[0013] The coordinate system is transformed using a construction matrix according to the calibration posture of the calibration plate to obtain vertex coordinates in an image plane coordinate system; the construction matrix is ​​determined by the initial intrinsic parameters of the camera; the construction matrix includes a model matrix, a projection matrix and an observation matrix;

[0014] Performing virtual calibration information rendering according to the vertex coordinates in the image plane coordinate system to obtain a rendering result;

[0015] The camera is calibrated according to the rendering result to obtain a camera calibration result.

[0016] Optionally, the coordinate system is transformed by using a construction matrix according to the calibration posture of the calibration plate to obtain the vertex coordinates in the image plane coordinate system, specifically including:

[0017] The calibration posture of the calibration plate is converted according to the model matrix to obtain the vertex coordinates in the world coordinate system;

[0018] Use the observation matrix to transform the vertex coordinates in the world coordinate system to the camera coordinate system;

[0019] The vertex coordinates in the camera coordinate system are two-dimensionally transformed according to the projection matrix to obtain the vertex coordinates in the image plane coordinate system; the projection matrix is ​​constructed according to the initial intrinsic parameters of the camera.

[0020] Optionally, performing virtual calibration information rendering according to the vertex coordinates in the image plane coordinate system to obtain a rendering result specifically includes:

[0021] The vertex coordinates in the image plane coordinate system are screen mapped to obtain a rendering result; the rendering result includes the pixel coordinates of each vertex of the calibration plate in the rendering window.

[0022] Optionally, calibrating the camera according to the rendering result to obtain a camera calibration result specifically includes:

[0023] Fitting the calibration plate to the rendering result;

[0024] Switch different calibration plate postures and collect calibration plate images;

[0025] The camera is calibrated according to the calibration plate image to obtain a camera calibration result.

[0026] The present invention also provides a camera-assisted calibration system, comprising:

[0027] The acquisition module is used to obtain the initial intrinsic parameters of the camera;

[0028] A simulation module, used to determine the calibration posture of the calibration plate by using simulation technology according to the calibration plate;

[0029] A coordinate system conversion module, used to perform coordinate system conversion using a construction matrix according to the calibration posture of the calibration plate to obtain vertex coordinates in an image plane coordinate system; the construction matrix is ​​determined by the initial intrinsic parameters of the camera; the construction matrix includes a model matrix, a projection matrix and an observation matrix;

[0030] A rendering module, used for performing virtual calibration information rendering according to the vertex coordinates in the image plane coordinate system to obtain a rendering result;

[0031] The calibration module is used to calibrate the camera according to the rendering result to obtain a camera calibration result.

[0032] Optionally, the coordinate system conversion module specifically includes:

[0033] A world coordinate system conversion unit, used to convert the calibration posture of the calibration plate according to the model matrix to obtain vertex coordinates in the world coordinate system;

[0034] A camera coordinate system conversion unit, used to convert vertex coordinates in the world coordinate system to the camera coordinate system using an observation matrix;

[0035] The two-dimensional conversion unit is used to perform two-dimensional conversion on the vertex coordinates in the camera coordinate system according to the projection matrix to obtain the vertex coordinates in the image plane coordinate system; the projection matrix is ​​constructed according to the initial intrinsic parameters of the camera.

[0036] Optionally, the rendering module specifically includes:

[0037] The rendering unit is used to perform screen mapping on the vertex coordinates in the image plane coordinate system to obtain a rendering result; the rendering result includes the pixel coordinates of each vertex of the calibration plate in the rendering window.

[0038] Optionally, the calibration module specifically includes:

[0039] A laminating unit, used for laminating the calibration plate with the rendering result;

[0040] A switching unit, used for switching different calibration plate postures and collecting calibration plate images;

[0041] The calibration unit is used to perform camera calibration according to the calibration plate image to obtain a camera calibration result.

[0042] The present invention also provides an electronic device, comprising:

[0043] one or more processors;

[0044] a storage device having one or more programs stored thereon;

[0045] When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method as described above.

[0046] The present invention also provides a computer storage medium on which a computer program is stored, wherein the computer program implements the method as described above when executed by a processor.

[0047] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] The present invention first determines the calibration posture of the calibration plate, and then performs coordinate system conversion on the calibration posture to obtain the vertex coordinates in the image plane coordinate system. The vertex coordinates in the image plane coordinate system are used to render virtual calibration information, and finally the calibration plate and the virtual calibration information are aligned to complete the camera calibration. Calibration directly through the rendering result can improve the accuracy of calibration and reduce the workload during the calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1A schematic diagram of a camera-assisted calibration method provided by the present invention;

[0051] Figure 2 Capture images for the camera that overlays the calibration plate rendering results;

[0052] Figure 3 The calibration plate images of different postures collected;

[0053] Figure 4 Extract result graph of landmark points during camera calibration process;

[0054] Figure 5 This is a flow chart of the camera-assisted calibration method provided by the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The object of the present invention is to provide a camera-assisted calibration method, system, device and storage medium, which can improve the accuracy of calibration and reduce the workload during the calibration process.

[0057] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0058] like Figure 1 and Figure 5 As shown, the present invention provides a camera-assisted calibration method, comprising:

[0059] Step 101: Obtain initial intrinsic parameters of the camera.

[0060] Prepare a checkerboard calibration plate and fix it on a flat plate, fix the camera on the experimental platform, connect the computer to the camera and start the camera. Prepare the initial intrinsic parameters of the camera. You can refer to the camera product manual or historical calibration results, or collect 4-5 target images in different poses and calibrate them to get a rough intrinsic parameter calibration result, which is used to construct the projection matrix used in the rendering process.

[0061] Step 102: Determine the calibration posture of the calibration plate using simulation technology according to the calibration plate.

[0062] The calibration plate posture determination includes:

[0063] Determine the appropriate size of the calibration plate based on the camera target size, lens focal length, working distance, etc.

[0064] During the calibration process, the calibration plate should always keep all the markers in the camera measurement space, which is determined by the camera lens field of view angle and the depth of field range of the lens.

[0065] A set of calibration plate poses are selected so that they are evenly distributed in the measurement space of the camera, with a total of 11 sets of calibration poses: first, the center point of the calibration plate is coincided with the point at the working distance of the camera optical axis from the optical center, the angles between the center axis of the calibration plate and the camera Z axis are {0, -30°, +30°}, and the angles between the center axis of the calibration plate and the camera X axis are {0, -30°, +30°}, for a total of 3×3=9 poses; in addition, 2 calibration poses are added, namely, the center axis of the calibration plate coincides with the camera optical axis, and the plane of the calibration plate coincides with the front depth of field and back depth of field of the camera lens, respectively.

[0066] The camera coordinate system is specified as follows: the positive direction of the Z axis is forward along the camera optical axis, the positive direction of the Y axis is vertically downward, and the X axis direction can be obtained by Y×Z, which conforms to the right-hand rule.

[0067] Step 103: According to the calibration posture of the calibration plate, coordinate system conversion is performed using the construction matrix to obtain vertex coordinates in the image plane coordinate system; the construction matrix is ​​determined by the initial intrinsic parameters of the camera; the construction matrix includes a model matrix, a projection matrix and an observation matrix. Matrix construction is performed using OpenGL to convert the vertex coordinates of the calibration plate in different calibration postures into vertex coordinates in the image plane coordinate system. The calibration posture is a set of calibration postures, each of which corresponds to a Matrix, that is, the model matrix.

[0068] Step 103 specifically includes: converting the calibration posture of the calibration plate according to the model matrix to obtain the vertex coordinates in the world coordinate system; using the observation matrix to convert the vertex coordinates in the world coordinate system to the camera coordinate system; performing a two-dimensional transformation on the vertex coordinates in the camera coordinate system according to the projection matrix to obtain the vertex coordinates in the image plane coordinate system; the projection matrix is ​​constructed according to the initial intrinsic parameters of the camera.

[0069] The calibration plate vertex coordinate transformation includes: using the model matrix to transform from the calibration plate model coordinate system O-XpYpZp to the world coordinate system O-XwYwZw, and then using the observation matrix to transform from the world coordinate system O-XwYwZw to the camera coordinate system O-XcYcZc, and then using the projection matrix to transform to the two-dimensional image plane coordinate system O-XY.

[0070] Among them, projection, view, They are projection matrix, observation matrix and model matrix respectively; position is the vertex coordinate of the calibration plate (calibration plate model coordinate system), the value of position has nothing to do with the calibration plate posture, but only with the number of rows, columns and point spacing of the calibration plate markers; norm_imagePt is the image point in the image plane coordinate system, The dimension coordinates are normalized to the interval [-1, 1]. Any coordinates outside the range will be discarded / cropped and will not be displayed in the rendering window. vec4(position, 1) represents position 3×1 The corresponding homogeneous coordinates. norm_imagePt is The homogeneous coordinate vector of .

[0071] The projection matrix and the internal parameters of the camera are related to the horizontal and vertical resolution of the rendered image and the set imaging depth range. The M matrix is ​​related to the position and posture of the calibration plate. N calibration plate positions correspond to N M matrices. For each M matrix of a posture, a set of calibration plate vertex coordinates in the image plane coordinate system can be obtained. After the OpenGL rendering pipeline, the rendered image of the calibration plate in this posture can be obtained.

[0072] The M matrix is ​​a 4×4 matrix and can be written in the form of a block matrix as follows, where is a 3×3 rotation matrix, is a 3×1 translation vector.

[0073]

[0074] To reduce storage space, Convert it to a 3×1 rotation vector for storage, where the modulus of the rotation vector is the rotation angle θ (radians), and the unit vector K corresponding to the rotation vector represents the direction vector of the rotation axis. Use the Rodrigues formula to convert the 3×1 rotation vector to a 3×3 rotation matrix :

[0075]

[0076] Where I represents The unit matrix of the observation matrix converts the world space coordinates of the vertex into the camera coordinates; the LookAt function of the glm library in OpenGL can be used to complete the calculation of the observation matrix and obtain the required viewing angle and observation effect. In fact, it is a three-dimensional space redefined with the observer as the origin. Since the world coordinate system and the camera coordinate system coincide with each other in the present invention, the observation matrix takes the unit matrix, and the corresponding parameters of the LookAt function are: pos=(0,0,0) T , target=(0, 0, 1) T , up=(0,-1,0) T. T is the matrix transpose.

[0077] The method of constructing the projection matrix is ​​as follows: the initial intrinsic parameters of the camera are represented by matrix A, f x , f y represent the normalized focal length in the x and y directions, respectively, c x , c y Respectively represent the x-coordinate and y-coordinate of the principal point position.

[0078]

[0079] Calculate the complete projection matrix:

[0080]

[0081] Among them, w and h are the number of pixels in the horizontal and vertical directions of the image, respectively, and f and n are the positions of the far clipping plane and the near clipping plane, respectively. The initial intrinsic parameters of the camera used in the construction of the Projection matrix can refer to the calibration values ​​or historical calibration results given in the product manual. If there are no such reference values, the camera intrinsic parameters can be roughly calibrated by collecting 4-5 calibration plate images with different postures using the camera calibration method based on the plane calibration plate.

[0082] The vertex coordinates of the calibration plate are transformed into the image plane coordinate system through the projection matrix projection, the observation matrix view and the model matrix M.

[0083] According to the 11 sets of pre-designed calibration plate poses, they are converted into a corresponding series of M matrices, and the calibration plate is rendered into the camera screen by running the rendering program. The calibration plate should be completely visible in the camera screen. The calibration personnel are guided to hold the calibration plate and align it with the virtual calibration plate in the camera screen. An image is collected for each pose and stored. The camera screen of the calibration test is as follows: Figure 3 shown.

[0084] Step 104: Rendering the virtual calibration information according to the vertex coordinates in the image plane coordinate system to obtain a rendering result. The calibration plate texture is rendered using an OpenGL vertex shader and a fragment shader.

[0085] Step 104 specifically includes: performing screen mapping on the vertex coordinates in the image plane coordinate system to obtain a rendering result; the rendering result includes the pixel coordinates of each vertex of the calibration plate in the rendering window.

[0086] Calibration plate pose rendering includes: That is, the x and y coordinate values ​​of norm_imagePt are mapped to the screen, and finally the pixel coordinates of each vertex of the calibration plate in the rendering window are obtained. Screen mapping depends on the final resolution of the window to be rendered. The coordinates of the screen space are obtained through screen mapping. In OpenGL, the pixel coordinates of the lower left corner of the screen space ViewPort are (0, 0), and the upper right corner is (pixelWidth, pixelHeight). The process of obtaining the pixel coordinates of the rendering window is to directly call the viewport transformation interface in OpenGL.

[0087] The mapping method is:

[0088]

[0089] Use the OpenGL image programming interface to render the calibration board graphics (simulating the real acquisition effect of the camera). The workflow is divided into the following steps:

[0090] (1) Define graphic data: First, Figure 4 As shown, the calibration plate is divided into multiple triangular facets with each marker point as a vertex, and the vertex coordinate data of each marker point on the calibration plate (calibration plate coordinate system) is stored in a vertex buffer object (VertexBuffer Object, VBO); the triangular facet vertex index data is stored in an element buffer object (Element BufferObject, EBO).

[0091] (2) Define the shader program: The shader is a program running on the graphics processing unit (GPU) to control the rendering process of the graphics. According to the graphics rendering pipeline processing flow, it mainly includes the vertex shader (Vertex Shader) and the fragment shader (Fragment Shader) program. The vertex shader is used to process vertex data and convert 3D vertex data into 2D coordinates. The fragment shader is used to calculate the color value of the pixel at the 2D coordinate. Among them, the projection\view\M matrix and vertex buffer data calculated above need to be passed into the vertex shader. The fragment shader sets the color used for calibration plate rendering. The rendering process requires input of the projection matrix, the observation matrix, the model matrix, and the vertex position in the calibration plate model coordinate system. norm_imagePt and the screen pixel coordinates are all completed through the rendering pipeline.

[0092] (3) Rendering graphics: After the graphics data and shader program are ready, call the OpenGL interface to render the graphics. During the rendering process, OpenGL processes the vertex data according to the state settings and shader program, generates graphics, and draws the graphics into the frame buffer memory area.

[0093] (4) Display graphics: Display the contents of the frame buffer on the screen. This is done by swapping the front and back frame buffers (in double buffer mode) or directly copying the contents of the frame buffer to the screen. Finally, the camera captured image with the calibration plate rendering result superimposed on it is obtained in the specified window on the screen, as shown in the attached figure. Figure 3 shown.

[0094] Step 105: Calibrate the camera according to the rendering result to obtain a camera calibration result.

[0095] Step 105 specifically includes: attaching the calibration plate to the rendering result; switching different calibration plate postures and collecting calibration plate images; performing camera calibration according to the calibration plate images to obtain camera calibration results. Figure 3 As shown, Figure 3 L1.bmp represents the calibration plate image of the first frontal upward posture. Figure 3 L2.bmp represents the calibration plate image of the second frontal upward posture. Figure 3 L3.bmp in the figure represents the calibration plate image of the frontal head-up posture. Figure 3 L4.bmp in the figure represents the calibration plate image of the third frontal upward posture. Figure 3 L5.bmp in the figure represents the calibration plate image of the second upward-looking posture in the lower right corner. Figure 3 L6.bmp in the figure represents the calibration plate image of the first upward-looking posture in the lower right corner. Figure 3 L7.bmp in the figure represents the calibration plate image of the third upward-looking posture in the lower left corner. Figure 3 L8.bmp in the figure represents the calibration plate image of the first upward-looking posture in the lower left corner. Figure 3 L9.bmp in the figure represents the calibration plate image of the second upward-looking posture in the lower left corner. Figure 3 In the figure, L10.bmp represents the calibration plate image of the third upward-looking posture in the lower right corner, where the angle of the first upward-looking posture is smaller than that of the second upward-looking posture, and the angle of the second upward-looking posture is smaller than that of the third upward-looking posture. The camera-collected image with the calibration plate rendering result superimposed is shown in Figure 1. Figure 2 shown.

[0096] Use Camera Calibration in MATLAB to calibrate and accurately calculate the camera's internal parameters, external parameters, and distortion parameters.

[0097] The present invention can provide a convenient and accurate camera calibration solution. The virtual calibration information is superimposed on the real-time image captured by the camera using augmented reality technology. The user only needs to align the calibration plate and the virtual calibration information and use the camera to shoot. Then the Camera Calibration toolbox in MATLAB is used for calibration. Through image processing and computer vision algorithms, the system can automatically detect the position and posture of the calibration plate and calculate the calibration parameters of the camera.

[0098] Experiments have shown that the present invention can effectively reduce the error caused by the unreasonable posture of the calibration plate, has high accuracy and can reduce the workload during the calibration process. Compared with the traditional calibration method, it greatly simplifies the calibration process, reduces the user's operating difficulty, and provides an intuitive and visual calibration method.

[0099] In summary, the camera-assisted calibration method provided by the present invention has broad application prospects in the field of camera calibration. It provides a novel and efficient solution for camera calibration and is expected to be widely used in the fields of computer vision, augmented reality, and virtual reality.

[0100] The present invention also provides a camera-assisted calibration system, comprising:

[0101] The acquisition module is used to obtain the initial intrinsic parameters of the camera.

[0102] The simulation module is used to determine the calibration posture of the calibration plate using simulation technology according to the calibration plate.

[0103] A coordinate system conversion module is used to perform coordinate system conversion using a construction matrix according to the calibration posture of the calibration plate to obtain vertex coordinates in an image plane coordinate system; the construction matrix is ​​determined by the initial intrinsic parameters of the camera; the construction matrix includes a model matrix, a projection matrix and an observation matrix.

[0104] The rendering module is used to render the virtual calibration information according to the vertex coordinates in the image plane coordinate system to obtain a rendering result.

[0105] The calibration module is used to calibrate the camera according to the rendering result to obtain a camera calibration result.

[0106] As an optional implementation, the coordinate system conversion module specifically includes:

[0107] The world coordinate system conversion unit is used to convert the calibration posture of the calibration plate according to the model matrix to obtain the vertex coordinates in the world coordinate system.

[0108] The camera coordinate system conversion unit is used to convert the vertex coordinates in the world coordinate system to the camera coordinate system using the observation matrix.

[0109] The two-dimensional conversion unit is used to perform two-dimensional conversion on the vertex coordinates in the camera coordinate system according to the projection matrix to obtain the vertex coordinates in the image plane coordinate system; the projection matrix is ​​constructed according to the initial intrinsic parameters of the camera.

[0110] As an optional implementation, the rendering module specifically includes:

[0111] The rendering unit is used to perform screen mapping on the vertex coordinates in the image plane coordinate system to obtain a rendering result; the rendering result includes the pixel coordinates of each vertex of the calibration plate in the rendering window.

[0112] As an optional implementation, the calibration module specifically includes:

[0113] A fitting unit is used to fit the calibration plate to the rendering result.

[0114] The switching unit is used to switch between different calibration plate postures and collect calibration plate images.

[0115] The calibration unit is used to perform camera calibration according to the calibration plate image to obtain a camera calibration result.

[0116] The present invention also provides an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described.

[0117] The present invention also provides a computer storage medium on which a computer program is stored, wherein the computer program implements the method as described above when executed by a processor.

[0118] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0119] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A camera-assisted calibration method, It is characterized in that include: Get the initial intrinsic parameters of the camera; Determine the calibration posture of the calibration plate by using simulation technology according to the calibration plate; According to the calibration posture of the calibration plate, a coordinate system conversion is performed using a construction matrix to obtain vertex coordinates in an image plane coordinate system; The construction matrix is ​​determined by the initial intrinsic parameters of the camera; the construction matrix includes a model matrix, a projection matrix and an observation matrix; Performing virtual calibration information rendering according to the vertex coordinates in the image plane coordinate system to obtain a rendering result; Calibrate the camera according to the rendering result to obtain a camera calibration result; The coordinate system is transformed by using a construction matrix according to the calibration posture of the calibration plate to obtain the vertex coordinates in the image plane coordinate system, specifically including: The calibration posture of the calibration plate is converted according to the model matrix to obtain the vertex coordinates in the world coordinate system; Use the observation matrix to transform the vertex coordinates in the world coordinate system to the camera coordinate system; Performing a two-dimensional transformation on the vertex coordinates in the camera coordinate system according to the projection matrix to obtain the vertex coordinates in the image plane coordinate system; the projection matrix is ​​constructed according to the initial intrinsic parameters of the camera; The method further comprises: performing virtual calibration information rendering according to the vertex coordinates in the image plane coordinate system to obtain a rendering result. The vertex coordinates in the image plane coordinate system are mapped to the screen to obtain a rendering result; the rendering result includes the pixel coordinates of each vertex of the calibration plate in the rendering window; There are 11 groups of calibration postures of the calibration plate, and the determination process includes: coinciding the center point of the calibration plate with the point at the working distance of the camera optical axis from the optical center, taking the angles between the center axis of the calibration plate and the Z axis of the camera as {0, -30°, +30°}, and taking the angles between the center axis of the calibration plate and the X axis of the camera as {0, -30°, +30°}, to obtain 9 groups of calibration postures; coinciding the center axis of the calibration plate with the optical axis of the camera, and coinciding the plane of the calibration plate with the front depth of field and the back depth of field of the camera lens, to obtain 2 groups of calibration postures; The model matrix is: The projection matrix is: The observation matrix is ​​a unit matrix, which is calculated using the LookAt function of the glm library in OpenGL; The screen mapping method is as follows: Among them, M represents the model matrix, R represents the rotation matrix, T represents the translation vector, projection represents the projection matrix, and f x and f y represent the normalized focal length in the x and y directions, respectively, c x and c y They represent the x- and y-coordinates of the principal point, respectively; w and h represent the number of pixels in the horizontal and vertical directions of the image, respectively; f and n represent the positions of the far clipping plane and the near clipping plane, respectively; (s x ,s y ) represents the pixel coordinates of the vertex of the calibration plate in the rendering window, (n x ,n y ) represents the vertex coordinates in the image plane coordinate system, and (pixelWidth, pixelHeight) represents the pixel coordinates of the upper right corner of the ViewPort in the screen space.

2. The camera-assisted calibration method according to claim 1, It is characterized in that Calibrate the camera according to the rendering result to obtain the camera calibration result, which specifically includes: Fitting the calibration plate to the rendering result; Switch different calibration plate postures and collect calibration plate images; The camera is calibrated according to the calibration plate image to obtain a camera calibration result.

3. A camera-assisted calibration system, It is characterized in that include: The acquisition module is used to obtain the initial intrinsic parameters of the camera; A simulation module, used to determine the calibration posture of the calibration plate by using simulation technology according to the calibration plate; A coordinate system conversion module, used to perform coordinate system conversion using a construction matrix according to the calibration posture of the calibration plate to obtain vertex coordinates in an image plane coordinate system; the construction matrix is ​​determined by the initial intrinsic parameters of the camera; the construction matrix includes a model matrix, a projection matrix and an observation matrix; A rendering module, used for performing virtual calibration information rendering according to the vertex coordinates in the image plane coordinate system to obtain a rendering result; A calibration module, used to calibrate the camera according to the rendering result to obtain a camera calibration result; The coordinate system conversion module specifically includes: A world coordinate system conversion unit, used to convert the calibration posture of the calibration plate according to the model matrix to obtain vertex coordinates in the world coordinate system; A camera coordinate system conversion unit, used to convert vertex coordinates in the world coordinate system to the camera coordinate system using an observation matrix; A two-dimensional conversion unit, used for performing a two-dimensional conversion on the vertex coordinates in the camera coordinate system according to a projection matrix to obtain the vertex coordinates in the image plane coordinate system; the projection matrix is ​​constructed according to the initial intrinsic parameters of the camera; The rendering module specifically includes: A rendering unit, used for performing screen mapping on the vertex coordinates in the image plane coordinate system to obtain a rendering result; the rendering result includes the pixel coordinates of each vertex of the calibration plate in the rendering window; There are 11 groups of calibration postures of the calibration plate, and the determination process includes: coinciding the center point of the calibration plate with the point at the working distance of the camera optical axis from the optical center, taking the angles between the center axis of the calibration plate and the Z axis of the camera as {0, -30°, +30°}, and taking the angles between the center axis of the calibration plate and the X axis of the camera as {0, -30°, +30°}, to obtain 9 groups of calibration postures; coinciding the center axis of the calibration plate with the optical axis of the camera, and coinciding the plane of the calibration plate with the front depth of field and the back depth of field of the camera lens, to obtain 2 groups of calibration postures; The model matrix is: The projection matrix is: The observation matrix is ​​a unit matrix, which is calculated using the LookAt function of the glm library in OpenGL; The screen mapping method is as follows: Among them, M represents the model matrix, R represents the rotation matrix, T represents the translation vector, projection represents the projection matrix, and f x and f y represent the normalized focal length in the x and y directions, respectively, c x and c y They represent the x- and y-coordinates of the principal point, respectively; w and h represent the number of pixels in the horizontal and vertical directions of the image, respectively; f and n represent the positions of the far clipping plane and the near clipping plane, respectively; (s x ,s y ) represents the pixel coordinates of the vertex of the calibration plate in the rendering window, (n x ,n y ) represents the vertex coordinates in the image plane coordinate system, and (pixelWidth, pixelHeight) represents the pixel coordinates of the upper right corner of the ViewPort in the screen space.

4. The camera-assisted calibration system according to claim 3, It is characterized in that The calibration module specifically includes: A laminating unit, used for laminating the calibration plate with the rendering result; A switching unit, used for switching different calibration plate postures and collecting calibration plate images; The calibration unit is used to perform camera calibration according to the calibration plate image to obtain a camera calibration result.

5. An electronic device, It is characterized in that include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 2.

6. A computer storage medium, It is characterized in that A computer program is stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

Images