Camera calibration system, method, device and storage medium based on stereo target
Through the camera calibration system with built-in patterns of three-dimensional targets and parallel light tubes, the problems of high difficulty and cost of camera calibration in the prior art are solved, and fast and high-precision camera calibration is achieved, which simplifies the calibration process.
Patent Information
- Application Number
- CN202210067141.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-01-20
AI Technical Summary
The existing camera calibration method requires moving the calibration plate or camera multiple times. Especially in vehicle-mounted cameras, the calibration plate needs to be large and costly, which increases the difficulty and time of calibration.
Using a camera calibration system based on stereo targets, using multiple parallel light tubes and built-in patterns, the three-dimensional target images are captured by the camera, the images and world coordinates of feature marking points are extracted, and the camera calibration parameters are obtained by fitting the perspective imaging model. Optimize algorithms to optimize parameters for fast and high-precision calibration.
The work space and time required for calibration is shortened, the target production cost is saved, the camera is quickly and highly accurate calibration is achieved, and the need to acquire multiple images and move the target or camera is avoided.
Smart Images

Figure CN114494457B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of camera calibration, and more specifically, to a camera calibration system, method, device and storage medium based on a stereo target. Background Art
[0002] Camera calibration is one of the key technologies in machine vision, photogrammetry, 3D imaging, and image geometry correction. Its main function is to estimate the internal and external parameters of the camera. The accuracy of the calibration results and the stability of the calibration algorithm directly affect the accuracy of subsequent work. In general calibration methods, multiple images need to be collected, so the calibration plate or camera needs to be moved manually. When the working distance of the calibrated camera is far, such as a car-mounted camera, a large calibration plate needs to be made during the calibration process and a large calibration space is required. In actual applications, it is not only time-consuming and labor-intensive, but also increases the production cost. Summary of the invention
[0003] The present application proposes a camera calibration system, method, device, electronic device and storage medium based on a stereo target to improve the above-mentioned problems.
[0004] In the first aspect, the embodiment of the present application provides a camera calibration system based on a stereo target, which includes a stereo target, a camera and an electronic device. The stereo target includes a plurality of collimators and a built-in pattern of the collimators, and the plurality of collimators are configured to correspond to the built-in pattern within the field of view of the camera; the collimator carrier is used to configure the plurality of collimators to correspond to the built-in pattern within the field of view of the camera; the camera is used to shoot the stereo target and collect a target image, and the target image includes an image formed by a combination of the stereo target and the built-in pattern of the collimators arranged on the stereo target; the electronic device is used to extract the characteristic marking points of the built-in pattern of the collimators in the target image according to the target image, and determine the image coordinates of the characteristic marking points in the target image; the electronic device is also used to obtain the longitude and latitude coordinates of each collimator relative to the reference collimator according to the relative position relationship between the plurality of collimators, with one of the collimators as the reference collimator. According to the latitude and longitude coordinates of each of the plurality of collimators and the preset distance corresponding to each collimator, the world coordinates of the feature mark point are obtained, wherein the preset distance is the distance from the preset point in the axial direction of the collimator to the camera; and the camera calibration parameters u0, v0, f, k1, k2, k3, k4, ..., k are obtained by fitting the image coordinates and the world coordinates according to the perspective imaging model of the camera according to the design parameters of the camera. n The electronic device is also used to use an optimization algorithm to optimize u0,v0,f,k1,k2,k3,k4,…,k according to the image coordinates and the world coordinates.n The initial values of are optimized to determine the parameters corresponding to the camera target imaging model.
[0005] In the second aspect, the embodiment of the present application provides a camera calibration method based on a stereo target, the method comprising: using the camera to shoot the stereo target and collect a target image, the target image comprising a stereo target and an image formed by a combination of a built-in pattern of a collimator arranged on the stereo target; using the electronic device to extract the characteristic marking points of the built-in pattern of the collimator in the target image according to the target image, and determine the image coordinates of the characteristic marking points in the target image; the electronic device is also used to obtain the longitude and latitude coordinates of each collimator relative to the reference collimator according to the relative position relationship between the multiple collimators and one of the collimators as a reference collimator. According to the longitude and latitude coordinates of each of the multiple collimators and the preset distance corresponding to each collimator, the world coordinates of the characteristic marking points are obtained, the preset distance being the distance from the preset point in the axial direction of the collimator to the camera; according to the design parameters of the camera or according to the image coordinates and the world coordinates, the camera calibration parameters u0, v0, f, k1, k2, k3, k4, ..., k n using the electronic device according to the image coordinates and the world coordinates, using an optimization algorithm to adjust u0,v0,f,k1,k2,k3,k4,…,k n The initial values of are optimized to determine the parameters corresponding to the camera target imaging model.
[0006] In a third aspect, an embodiment of the present application provides a camera calibration device based on a stereo target, the device comprising: an image coordinate acquisition module, used to acquire the target image captured by the camera, extract the characteristic marking points of the built-in pattern of the collimator in the target image, and determine the image coordinates of the characteristic marking points in the target image; a world coordinate acquisition module, used to obtain the world coordinates of the characteristic marking points according to the relative position relationship between the collimators; an initial value acquisition module, used to acquire the camera calibration parameters u0, v0, f, k1, k2, k3, k4, ..., k n The initial value of the calibration optimization module is used to calibrate u0,v0,f,k1,k2,k3,k4…,k according to the image coordinates and the world coordinates and the optimization algorithm. n The initial values are optimized to determine the parameters corresponding to the target imaging model.
[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code is configured to, when called by a processor, enable the processor to execute the above method.
[0008] The camera calibration system, method, device, electronic device and storage medium provided in the embodiments of the present application collect a target image of a three-dimensional target through a camera, wherein the target image includes an image formed by a combination of a three-dimensional target and a built-in pattern of a collimator arranged on the three-dimensional target; extract characteristic marking points of the built-in pattern of the collimator in the target image according to the target image, and determine the image coordinates of the characteristic marking points in the target image; obtain the world coordinates of the characteristic marking points according to the relative position relationship between the multiple collimators; obtain camera calibration parameters u0, v0, f, k1, k2, k3, k4, ..., k according to the design parameters of the camera or according to the image coordinates and the world coordinates by fitting according to the perspective imaging model of the camera n According to the image coordinates and the world coordinates, an optimization algorithm is used to optimize u0,v0,f,k1,k2,k3,k4,…,k n The initial value of is optimized to determine the parameters corresponding to the camera target imaging model. Through the stereo target, there is no need to move the target or the camera to collect multiple target images. Only one target image needs to be collected in a small space to achieve fast and high-precision calibration of the camera.
[0009] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0011] Figure 1 A schematic diagram of a camera calibration system based on a stereo target provided by an embodiment of the present application is shown.
[0012] Figure 2 A schematic diagram of a three-dimensional target provided by an embodiment of the present application is shown.
[0013] Figure 3 A schematic diagram of a built-in pattern of a collimator provided in one embodiment of the present application is shown.
[0014] Figure 4 A schematic diagram of longitude and latitude corresponding to characteristic marking points obtained by extracting built-in patterns of multiple parallel light tubes from a target image after acquiring the target image provided by an embodiment of the present application is shown.
[0015] Figure 5A flow chart of a camera calibration method based on a stereo target provided by an embodiment of the present application is shown.
[0016] Figure 6 A schematic diagram of coordinates provided by an embodiment of the present application is shown.
[0017] Figure 7 A structural block diagram of a camera calibration device based on a stereo target provided by an embodiment of the present application is shown.
[0018] Figure 8 A structural block diagram of an electronic device proposed in an embodiment of the present application for executing a camera calibration method based on a stereo target according to an embodiment of the present application is shown.
[0019] Fig. 9 A storage medium provided by an embodiment of the present application for storing or carrying a program code for implementing a camera calibration method based on a stereo target according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.
[0021] Camera calibration is one of the key technologies in machine vision, photogrammetry, 3D imaging, and image geometry correction. Its main function is to estimate the internal and external parameters of the camera. The accuracy of the calibration results and the stability of the calibration algorithm directly affect the accuracy of subsequent work. In recent years, vehicle-mounted module cameras have been widely used in panoramic vision, video surveillance, car navigation, virtual reality and other fields due to their ultra-large field of view and long working distance. However, the large field of view also brings serious image distortion, which affects the intuitive visual perception of the human eye and the use of image information. Therefore, the camera needs to be calibrated. On the other hand, due to the long working distance, the distance between the camera and the target needs to be controlled within the normal working distance of the camera, which results in a larger target size, which not only increases the production cost of the target, but also increases the difficulty of calibration.
[0022] The method that has been studied maturely is to use a plane target. There are tool software based on plane targets, such as Matlab toolbox and Opencv tool software. In these methods, a plane calibration plate is placed in different positions in front of the camera to collect multiple target images to obtain calibration raw data with a large distribution range. This method requires placing the calibration plate in different positions multiple times and collecting target images, or rotating the camera multiple times in different positions and collecting target images. These methods are not suitable for some occasions where the camera needs to be installed quickly and calibrated, such as production lines for mass production or assembly of cameras.
[0023] Therefore, the inventors proposed the camera calibration system, method, device, electronic device and storage medium based on a stereo target of the present application. The stereo target is photographed by the camera to collect a target image, wherein the target image is an image formed by a combination of the stereo target and a collimator built-in pattern arranged on the stereo target; based on the target image, characteristic marking points of the collimator built-in pattern in the target image are extracted, and the image coordinates of the characteristic marking points in the target image are determined; the world coordinates of the characteristic marking points are obtained based on the relative positional relationship between the multiple collimators; the camera calibration parameters u0, v0, f, k1, k2, k3, k4, ..., k are obtained by fitting the image coordinates and the world coordinates according to the perspective imaging model of the camera according to the design parameters of the camera or the camera perspective imaging model. n using the electronic device according to the image coordinates and the world coordinates, using an optimization algorithm to adjust u0,v0,f,k1,k2,k3,k4,…,k n The initial value of is optimized to determine the parameters corresponding to the camera target imaging model. The camera is calibrated using a three-dimensional target formed by a collimator, which greatly shortens the workspace required for calibration and saves the cost of making the target. At the same time, by collecting three-dimensional target images, there is no need to move the target or the camera to collect multiple target images. Only one target image is needed to achieve fast and high-precision calibration of the camera, which greatly reduces the time required for calibration and truly saves time and effort.
[0024] See also Figure 1 The embodiment of the present application provides a camera calibration system based on a stereo target, and the specific system may include a stereo target 100, a camera 110, and an electronic device 120. The stereo target 100 includes a plurality of collimators and a collimator built-in pattern. The plurality of collimators are configured to correspond to the built-in pattern within the field of view of the camera 110 through a collimator carrier. The stereo target 100 is photographed by the camera 110 to obtain a target image, and the target image is an image formed by the stereo target and the collimator built-in pattern set on the stereo target.
[0025] The camera 110 can send the captured target image to the electronic device 120. The camera 110 and the electronic device 120 can be one device or two devices. The electronic device 120 extracts the characteristic marking points of the built-in pattern of the collimator in the target image according to the target image, and determines the image coordinates of the characteristic marking points in the target image; according to the relative position relationship between the multiple collimators, one of the collimators is used as a reference collimator to obtain the longitude and latitude coordinates of each collimator relative to the reference collimator. According to the longitude and latitude coordinates of each of the multiple collimators and the preset distance corresponding to each collimator, the world coordinates of the characteristic marking points are obtained, and the preset distance is the distance from the preset point in the axial direction of the collimator to the camera; the camera calibration parameters u0, v0, f, k1, k2, k3, k4,…, k are obtained according to the design parameters of the camera or according to the image coordinates and the world coordinates by fitting according to the perspective imaging model of the camera. n The electronic device is also used to use an optimization algorithm to optimize u0,v0,f,k1,k2,k3,k4,…,k according to the image coordinates and the world coordinates. n The initial value of is optimized to determine the parameters corresponding to the camera target imaging model, thereby achieving calibration of the camera 110.
[0026] See also Figure 2 , wherein the three-dimensional target 100 includes a plurality of collimators 101 and a collimator built-in pattern 102. The plurality of collimators 101 are rigidly fixed together by a collimator carrier 103, or rigidly fixed together after fine-tuning, and the plurality of collimators are configured by the collimator carrier 103 so that the corresponding built-in pattern is within the field of view of the camera.
[0027] Furthermore, the built-in pattern of the collimator can be formed by circular spots, grids, BMW patterns, checkerboards, ChArUco and other customized patterns with certain characteristics, and the built-in patterns of multiple collimators can be the same or different. Figure 3 , Figure 3 A schematic diagram of a built-in pattern of a collimator provided in an embodiment of the present application is shown, wherein the built-in pattern 102 of the collimator can be displayed as a circular spot pattern 102A, a field pattern 102B, a BMW pattern 102C, a checkerboard pattern 102D and a ChArUco pattern 102E, and can also display other customized patterns with certain characteristics, which are not limited here.
[0028] The relative position relationship between the multiple collimators can be determined according to the field of view of the camera to be calibrated, so that the built-in pattern of the collimator can be distributed at the maximum field of view as much as possible. Take one of the multiple collimators as the reference collimator, and according to the relative position relationship between the multiple collimators, obtain the longitude and latitude coordinates of each collimator relative to the reference collimator. Specifically, the longitude and latitude values of the multiple collimators can be set as needed, for details, please refer to Figure 4 , shows a plan view of the distribution of multiple parallel light pipes in a three-dimensional target. The latitude line 104 is a circle in the figure, and the longitude line 105 is a straight line passing through the center of the circle in the figure. Each parallel light pipe 101 can be placed at the intersection of any latitude line and longitude line.
[0029] See also Figure 5 , Figure 5 FIG. 1 is a flow chart of a camera calibration method provided by an embodiment of the present application. In a specific embodiment, the camera calibration method is applied to Figure 7 The camera calibration device 700 and the electronic device 600 ( Figure 8 ). The following will take an electronic device as an example to illustrate the specific process of this embodiment. Of course, it can be understood that the electronic device used in this embodiment can be a terminal device with computing capabilities, such as a smart phone, a tablet computer, a desktop computer, a wearable electronic device, etc., which is not limited here. Figure 5 The process shown is described in detail, and the camera calibration method may specifically include the following steps:
[0030] Step S110: acquiring a target image, wherein the target image includes a three-dimensional target and an image formed by a combination of a built-in pattern of a collimator disposed on the three-dimensional target.
[0031] In this embodiment, an electronic device is used to obtain a target image. The target image is obtained by photographing a three-dimensional target with a camera so that the built-in patterns of multiple parallel light tubes are imaged on the image sensor to obtain a target image with multiple characteristic marking points.
[0032] Step S120: Determine the image coordinates of the characteristic marking points. Extract the characteristic marking points of the built-in pattern of the collimator in the target image, and determine the image coordinates of the characteristic marking points in the target image.
[0033] In this embodiment, the image coordinates of each characteristic marker point are determined according to the position of the image of the characteristic marker point in the target image, wherein the target image includes a plurality of characteristic marker points. To determine the image coordinates of the characteristic marker point, it is necessary to extract the center point of each characteristic marker point. The center point of the extracted characteristic marker point is obtained according to the characteristics of the characteristic marker point itself. For example, when the built-in pattern of the collimator is a circular spot, an ellipse fitting can be performed on the characteristic marker point in the target image, and the ellipse equation of the characteristic marker point can be obtained according to the boundary of the characteristic marker point and the ellipse fitting equation. According to the ellipse equation, the center coordinates of the characteristic marker point are determined as the image coordinates of the corresponding characteristic marker point; when the built-in pattern of the collimator is a checkerboard, the checkerboard corner points can be extracted as the center coordinates of the characteristic marker point, as the image coordinates of the corresponding characteristic marker point; the built-in patterns of other styles of collimators can obtain the center coordinates of the characteristic calibration points as image coordinates according to the pattern features, which will not be described in detail here.
[0034] Step S130: Calculate the world coordinates of each marking point according to the relative position relationship between the collimators.
[0035] In this embodiment, the relative position relationship between the multiple collimators is given by the hardware structure, and the multiple collimators are configured to correspond to the built-in pattern within the field of view of the camera through the collimator carrier, and the relative position relationship between the multiple collimators is guaranteed in the structure. One of the collimators is used as the reference collimator, and the longitude and latitude of the reference collimator is set to θ=0. Where θ represents latitude, Represents longitude. The longitude and latitude of each collimator is calculated based on the relative position relationship between the collimators. The longitude and latitude coordinates of the collimator are the longitude and latitude of the feature mark points extracted from the built-in pattern of the collimator. According to the conversion relationship between the rectangular coordinate system and the spherical coordinate system, a preset distance R is given. The preset distance is the distance from the preset point in the axial direction of each collimator to the camera. The point (R, θ, ) is converted into the midpoint of the rectangular coordinates (X, Y, Z) as follows: Z = Rsinθ. Among them, R, θ and The range is: R ≥ 0, -π / 2 ≤ θ ≤ π / 2, The coordinates (X, Y, Z) in the rectangular coordinate system are the world coordinates of the feature marker point. For details, see Figure 6 The embodiment of the present application is given. After the preset distance corresponding to each characteristic mark point in the stereo target is given, the world coordinates of each characteristic mark point in the stereo target can be determined according to the longitude and latitude coordinates of each characteristic mark point in the stereo target image and the preset distance corresponding to each characteristic mark point.
[0036] Step S140: Obtain camera calibration parameters u0, v0, f, k1, k2, k3, k4, ..., k n The initial value of .
[0037] In this embodiment, after the world coordinates and image coordinates of each feature mark point in the stereo target are obtained, the focal length, principal point and initial values of the distortion parameters of the camera can be calculated according to the perspective imaging model. In the perspective imaging model, there is no distortion parameter, so the initial value of the distortion parameter of the camera can be set to 0.
[0038] The perspective imaging model is also called the pinhole model. In the perspective imaging model, the world coordinates (X, Y, Z) of any point P in space and the image coordinates (u, v) of the image point of the point in the camera satisfy the perspective imaging formula:
[0039]
[0040] Where [X, Y, Z, 1] is the homogeneous coordinates of the spatial point in the world coordinate system, corresponding to the homogeneous pixel coordinates [u, v, 1] in the camera coordinate system. [R, T] is a pair of rotation matrices and translation matrices between the world coordinate system and the camera coordinate system. (f u ,f v ) represents the normalized focal length in the u direction (horizontal direction) and the v direction (vertical direction) in the camera coordinate system, which is the pixel focal length of the camera. (u0,v0) is the coordinate of the camera principal point. Substitute the world coordinates of all the marking points on the stereo target and the image coordinates corresponding to these marking points into the perspective imaging formula, and solve the simultaneous equations composed of these marking points into the perspective imaging formula to obtain the camera focal length (f u ,f v ) and the initial values of the principal point coordinates (u0,v0) under the perspective imaging model.
[0041] In some implementations, the image coordinates and the world coordinates are directly fitted to an imaging model required by the camera according to the design parameters of the camera to obtain initial values of the focal length, principal point, and distortion parameters of the camera.
[0042] In some embodiments, for a camera, point O is the optical center of the camera. A point P outside the optical center can be represented by X, Y, Z or R, θ, Indicates that if all the points in the world coordinates are on a sphere, we can also use only θ, When the camera is a pinhole camera without distortion, the image point on the image sensor is only at the focal length f. When the camera is a pinhole model with distortion, the image point on the image sensor is also related to the distortion parameter. The specific type of the camera is not limited here.
[0043] Step S160: optimizing the focal length, principal point and initial values of distortion parameters of the camera through an algorithm to determine parameters corresponding to the camera target imaging model.
[0044] In this embodiment, after obtaining the initial values of the focal length, principal point, and distortion parameters of the camera, an optimization algorithm can be used to optimize the initial values of the focal length, principal point, and distortion parameters of the camera to determine the parameters corresponding to the camera target imaging model.
[0045] In some embodiments, the sum of the squares of the differences between the projection values of the feature marker points on the stereo target and the measured values of the feature marker points on the stereo target is minimized by an optimization algorithm. The projection values of the feature marker points are calculated by using a projection model based on the world coordinates of each feature marker point in the stereo target, and the measured values refer to the coordinates of each feature marker point in the target image in the image coordinate system after the target image composed of the built-in pattern of multiple parallel light tubes is obtained by photographing the stereo target with a camera. Thus, the sum of the squares of the differences between the projection values of the feature marker points on the stereo target and the measured values of the feature marker points on the stereo target can be minimized by an optimization algorithm, and the focal length, principal point and initial values of the distortion parameters of the camera are continuously updated to obtain the optimal solution, so that the imaging model parameters required by the camera can be determined.
[0046] Furthermore, the optimization algorithm is used to minimize the sum of the squares of the differences between the projection values of the characteristic marking points on the stereo target and the measurement points of the characteristic marking points on the stereo target, even if the objective function The value of is the smallest, where N represents the number of characteristic marking points on the three-dimensional target. is the projection value of the feature marker point, that is, the pixel coordinate of the feature marker point calculated according to the projection model, m j It is the measured value of the feature marker point, that is, the pixel coordinate of the feature marker point measured from the image.
[0047] A camera calibration method provided by an embodiment of the present application obtains a target image by photographing a stereoscopic target with a camera. The stereoscopic target is photographed so that the built-in patterns of multiple collimators are imaged on the image sensor, and a target image composed of the built-in patterns of multiple collimators is obtained. The characteristic marking points of the built-in patterns of the collimators in the target image are extracted, and the image coordinates of the characteristic marking points in the target image are determined; according to the relative position relationship between the collimators, one of the collimators is used as a reference to calculate the longitude and latitude of each collimator, which is equivalent to obtaining the longitude and latitude of each marking point. Given a preset distance, the world coordinates corresponding to each characteristic marking point can be calculated. The camera calibration parameters u0, v0, f, k1, k2, k3, k4,…, k are obtained according to the design parameters of the camera or according to the image coordinates and the world coordinates, and fitting is performed according to the perspective imaging model of the camera. n According to the image coordinates and the world coordinates, an optimization algorithm is used to optimize u0,v0,f,k1,k2,k3,k4,…,k n The initial value of is optimized to determine the parameters corresponding to the camera target imaging model, so that when calibrating the camera, there is no need to move the target, and only one picture needs to be taken to complete the camera calibration. While reducing the difficulty of camera calibration, it can be achieved in a smaller calibration space to complete the fast and high-precision calibration of the camera.
[0048] See also Figure 7 , Figure 7 The module block diagram of the camera calibration device 700 based on the stereo target provided in the embodiment of the present application is shown in FIG. Figure 7 The camera calibration device 700 includes: an image coordinate acquisition module 710, a world coordinate acquisition module 720, an initial value acquisition module 730, and a calibration optimization module 740, wherein:
[0049] An image coordinate acquisition module 710 is used to acquire the target image captured by the camera, extract the characteristic marking points of the built-in pattern of the collimator in the target image, and determine the image coordinates of the characteristic marking points in the target image;
[0050] A world coordinate acquisition module 720, configured to obtain the world coordinates of the feature mark points according to the relative position relationship between the collimators;
[0051] Furthermore, the world coordinate determination module 720 includes: a feature marker point longitude and latitude coordinate determination submodule, wherein:
[0052] The submodule for determining the longitude and latitude coordinates of the characteristic marking points is used to obtain the longitude and latitude coordinates of each of the plurality of parallel light tubes based on the relative positional relationship between the plurality of parallel light tubes, taking one of the parallel light tubes as the reference light tube; the longitude and latitude coordinates of each parallel light tube are the longitude and latitude coordinates of the characteristic marking points corresponding to the built-in pattern of each parallel light tube.
[0053] Furthermore, the world coordinate determination module 720 includes: a feature marker point world coordinate determination submodule, wherein:
[0054] The feature mark point world coordinate determination submodule can determine the world coordinate of each feature mark point in the three-dimensional target according to the longitude and latitude coordinates of each feature mark point in the three-dimensional target and the preset distance corresponding to each parallel light tube.
[0055] The initial value acquisition module 730 is used to obtain the camera calibration parameters u0, v0, f, k1, k2, k3, k4, ..., k according to the design parameters of the camera or according to the image coordinates and the world coordinates and the perspective imaging model of the camera. n The initial value of .
[0056] The calibration optimization module 740 is used to calibrate u0, v0, f, k1, k2, k3, k4 ..., k according to the image coordinates and the world coordinates and the optimization algorithm. n The initial values are optimized to determine the parameters corresponding to the target imaging model.
[0057] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.
[0058] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.
[0059] Please refer to Figure 8 , which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 600 may be a mobile terminal capable of data processing, such as a mobile phone or a tablet computer.
[0060] The electronic device 600 in the present application may include one or more of the following components: a processor 610, a memory 620, and one or more applications, wherein the one or more applications may be stored in the memory 620 and configured to be executed by one or more processors 610, and the one or more programs are configured to execute the method as described in the aforementioned method embodiment.
[0061] The processor 610 may include one or more processing cores. The processor 610 uses various interfaces and lines to connect various parts of the entire electronic device 600, and executes various functions and processes data of the electronic device 600 by running or executing instructions, programs, code sets or instruction sets stored in the memory 620, and calling data stored in the memory 620. Optionally, the processor 610 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 610 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 610, but may be implemented separately through a communication chip.
[0062] The memory 620 may include a random access memory (RAM) or a read-only memory (ROM). The memory 620 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 600 during use.
[0063] Please refer to Figure 6 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.
[0064] The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 that performs any method step of the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.
[0065] In summary, the camera calibration system, method, device, electronic device and storage medium based on a stereo target provided in the embodiments of the present application. The stereo target is photographed by the camera to collect a target image, wherein the target image is an image formed by a combination of the stereo target and a collimator built-in pattern arranged on the stereo target; based on the target image, characteristic marking points of the collimator built-in pattern in the target image are extracted, and the image coordinates of the characteristic marking points in the target image are determined; the world coordinates of the characteristic marking points are obtained based on the relative positional relationship between the multiple collimators; the camera calibration parameters u0, v0, f, k1, k2, k3, k4, ..., k are obtained by fitting the image coordinates and the world coordinates according to the perspective imaging model of the camera according to the design parameters of the camera or the camera perspective imaging model. n using the electronic device according to the image coordinates and the world coordinates, using an optimization algorithm to adjust u0,v0,f,k1,k2,k3,k4,…,k n The initial value of is optimized to determine the parameters corresponding to the camera target imaging model. The camera is calibrated using a stereo target composed of multiple parallel light tubes, which greatly shortens the workspace used for calibration and saves the cost of making the target. At the same time, by collecting stereo target images, there is no need to move the target or the camera to collect multiple target images. Only one target image is needed to achieve fast and high-precision calibration of the camera, which greatly reduces the time required for calibration and truly saves time and effort.
[0066] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A camera calibration system based on a stereo target, characterized in that: Includes stereo targets, cameras and electronic equipment: The three-dimensional target includes a plurality of collimators and patterns built into the collimators, and the plurality of collimators are configured to correspond to the built-in patterns within the field of view of the camera; The camera is used to photograph the three-dimensional target and collect a target image, wherein the target image includes an image formed by a combination of the three-dimensional target and a built-in pattern of a collimator disposed on the three-dimensional target; The electronic device is used to extract the characteristic marking points of the built-in pattern of the collimator in the target image according to the target image, and determine the image coordinates of the characteristic marking points in the target image; The electronic device is also used to obtain the world coordinates of the characteristic marking point according to the relative position relationship between the collimators; The electronic device is also used to obtain the camera calibration parameters u0, v0, f, k1, k2, k3, k4, …, k n The initial value of The electronic device is further used to use an optimization algorithm to optimize u0, v0, f,k1, k2, k3, k4 ..., k according to the image coordinates and the world coordinates. n The initial values of are optimized to determine the parameters corresponding to the camera target imaging model.
2. The system according to claim 1, characterized in that The three-dimensional target further includes a collimator carrier, which is used to configure the plurality of collimators to correspond to the built-in patterns within the field of view of the camera.
3. The system according to claim 1, characterized in that The built-in patterns of the collimator include circular spots, grid patterns, BMW patterns, checkerboard patterns, or ChArUco.
4. A camera calibration method based on a stereo target, calibrating a camera based on the calibration system according to any one of claims 1 to 3, characterized in that: The method comprises: Acquire a target image captured by the camera, extract characteristic marking points of a built-in pattern of a collimator in the target image, and determine image coordinates of the characteristic marking points in the target image; Obtaining the world coordinates of the characteristic marking point according to the relative position relationship between the plurality of parallel light pipes; Get the camera calibration parameters u0, v0, f, k1, k2, k3, k4, …, k n The initial value of According to the image coordinates and the world coordinates, an optimization algorithm is used to optimize u0, v0, f, k1, k2, k3, k4 …,k n The initial values are optimized to determine the parameters corresponding to the target imaging model.
5. The method according to claim 4, characterized in that The step of obtaining the world coordinates of the feature mark points according to the relative position relationship between the plurality of collimators includes: Obtaining the longitude and latitude coordinates of each of the plurality of collimators according to the relative positional relationship between the plurality of collimators; The world coordinates of the characteristic marking point are obtained according to the latitude and longitude coordinates of each of the plurality of collimators and the preset distance corresponding to each collimator, wherein the preset distance is the distance from the preset point in the axial direction of the collimator to the camera.
6. The method according to claim 5, characterized in that The step of obtaining the longitude and latitude coordinates of each of the plurality of collimators according to the relative positional relationship between the collimators comprises: One of the plurality of collimators is used as a reference collimator, and the longitude and latitude coordinates of each collimator relative to the reference collimator are obtained according to the relative positional relationship between the plurality of collimators.
7. The method according to claim 4, characterized in that The camera calibration parameters u0, v0, f, k1, k2, k3, k4, …, k are obtained. n The initial value of includes: According to the image coordinates of the feature marker point in the target image and the world coordinates of the feature marker point, the camera calibration parameters u0, v0, f, k1, k2, k3, k4, …, k n The initial value of .
8. The method according to claim 4, characterized in that The camera calibration parameters u0, v0, f, k1, k2, k3, k4, …, k are obtained. n The initial value of includes: According to the design parameters of the camera, obtain the camera calibration parameters u0, v0, f, k1, k2, k3, k4, …, k n The initial value of .
9. A camera calibration device based on a stereo target, characterized in that: The device comprises: An image coordinate acquisition module is used to acquire a target image captured by a camera, extract characteristic marking points of a pattern built into a collimator in the target image, and determine image coordinates of the characteristic marking points in the target image; A world coordinate acquisition module, used for obtaining the world coordinates of the characteristic marking points according to the relative position relationship between the collimators; Initial value acquisition module, used to obtain the camera calibration parameters u0, v0, f, k1, k2, k3, k4, …, k n The initial value of A calibration optimization module is used to calibrate u0, v0, f,k1, k2, k3, k4 …, k according to the image coordinates and the world coordinates and the optimization algorithm. n The initial values are optimized to determine the parameters corresponding to the target imaging model.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, and the program code is configured to, when called by a processor, enable the processor to execute the system or method according to any one of claims 1 to 9.
