Camera calibration system and method

Abstract

The present application discloses a camera calibration system, method, device and electronic device. The system includes a target, a camera, a rotating mechanism, a servo driver and an electronic device, wherein the target includes a plurality of collimators and a pattern built into the collimators. The electronic device is used to obtain a plurality of target images obtained by the camera shooting the target during the rotation of the rotating mechanism and the position information of the rotating mechanism corresponding to each target image, superimpose the plurality of target images to obtain a target composite image, calculate the image coordinates of the imaging point of the marking point in the target composite image in the target composite image, determine the world coordinates of one or more virtual space points in the virtual stereo target corresponding to each marking point imaging point according to the position information of the rotating mechanism and the relative position relationship between the collimators in the target, and thereby calculate the parameters of the camera imaging model. This solution can complete the rapid and high-precision calibration of the camera in a relatively small calibration space.

Description

Technical Field

[0001] The present application relates to the technical field of camera calibration, and more specifically, to a camera calibration system and method. Background Art

[0002] Camera calibration is one of the key technologies in machine vision, photogrammetry, 3D imaging, and image geometry correction. Its main function is to estimate the internal and external parameters of the camera. The accuracy of the calibration results and the stability of the calibration algorithm directly affect the accuracy of subsequent work. In general calibration methods, since multiple images in various postures need to be collected, it is necessary to manually move the calibration plate or camera, and the distance between the target and the camera during calibration must also meet the working distance of the camera. In actual applications, this is not only time-consuming and labor-intensive, but also increases the production cost of the target. At the same time, it is difficult to achieve the calibration accuracy required by the application. Summary of the invention

[0003] In view of this, the present application proposes a camera calibration system and method to solve the above problems.

[0004] In a first aspect, an embodiment of the present application provides a camera calibration system, the system comprising a target, a camera, a rotating mechanism, a servo driver and an electronic device, the camera and the servo driver being electrically connected to the electronic device; the target comprising a plurality of collimators and a built-in pattern of the collimators, the relative position relationship between the collimators being determined, the built-in pattern of the collimators comprising at least one collimator marking point; the camera being mounted on the rotating mechanism, the camera being used to photograph the target to obtain a target image, the target image comprising a marking point imaging point of the collimator marking point in the target; the rotating mechanism comprising at least one rotating axis, the rotating mechanism moving according to a preset rotation position information, all targets at the rotation positions forming a virtual three-dimensional target; the servo driver being used to drive the rotating mechanism to rotate, and to move the rotating mechanism The electronic device is used for acquiring a plurality of target images obtained by the camera shooting the target during the rotation of the rotating mechanism and the position information of the rotating mechanism corresponding to each of the target images, superimposing the plurality of target images to obtain a target composite image, extracting the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image, determining the world coordinates of one or more virtual space points in the virtual three-dimensional target corresponding to each of the marking point imaging points in the target composite image according to the position information of the rotating mechanism and the relative position relationship between the parallel light tubes in the target, performing fitting according to the imaging model of the camera based on the image coordinates and the world coordinates, and obtaining the parameters of the camera imaging model through an optimization algorithm.

[0005] In a second aspect, an embodiment of the present application provides a camera calibration method, the method comprising: obtaining a plurality of target images obtained by the camera taking pictures of the target during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each of the target images, the target image comprising the imaging point of the collimator mark point of the collimator built-in pattern in the target in the camera; superimposing the plurality of target images to obtain a target composite image, extracting the image coordinates of each imaging point of the mark point in each of the target images as the image coordinates of the imaging point of the mark point in the target composite image; determining the world coordinates of the virtual space point in the virtual three-dimensional target corresponding to each imaging point of the mark point according to the rotation position information of the rotating mechanism and the relative position relationship between the collimators; fitting according to the imaging model of the camera based on the image coordinates and the world coordinates, and obtaining the parameters of the camera imaging model through an optimization algorithm.

[0006] In a third aspect, an embodiment of the present application provides a camera calibration device, which includes: a data acquisition module, used to acquire multiple target images obtained by the camera taking pictures of the target during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each of the target images, wherein the target image includes the imaging point of the collimator mark point of the collimator built-in pattern in the target in the camera; a data processing module, used to superimpose the multiple target images to obtain a target composite image, extract the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging point in the target composite image, and determine the world coordinates of the virtual space point in the virtual three-dimensional target corresponding to each of the marking point imaging points in the target composite image according to the rotation position information of the rotating mechanism and the relative position relationship between the collimators; a camera parameter calculation module, used to fit according to the imaging model of the camera based on the image coordinates and the world coordinates, and obtain the parameters of the camera imaging model through an optimization algorithm.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code is configured to, when called by a processor, enable the processor to execute the above method.

[0008] The camera calibration system and method provided in the embodiments of the present application obtain multiple target images obtained by the camera shooting the target during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each of the target images, superimpose the multiple target images to obtain a target composite image, extract the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image, determine the world coordinates of the virtual space point in the virtual three-dimensional target corresponding to each of the marking point imaging points, obtain the parameters of the imaging model of the camera through an optimization algorithm, and thus can achieve that when calibrating the camera, there is no need to move the target, only need to collect images during the rotation of the rotating mechanism, while reducing the difficulty of camera calibration, the camera can be calibrated quickly and with high precision in a smaller calibration space. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 A schematic diagram of a camera calibration system provided in an embodiment of the present application is shown;

[0011] Figure 2 A schematic diagram of a rotating mechanism and a target provided in an embodiment of the present application is shown;

[0012] Figure 3 A schematic diagram of a rotating mechanism and a target provided in another embodiment of the present application is shown;

[0013] Figure 4 A schematic diagram of a rotating mechanism and a target provided in another embodiment of the present application is shown;

[0014] Figure 5 A schematic diagram of a rotating mechanism and a target provided by another embodiment of the present application is shown;

[0015] Figure 6 A schematic diagram of a rotating mechanism and a target provided by another embodiment of the present application is shown;

[0016] Figure 7 A schematic diagram of a built-in pattern of a collimator provided in an embodiment of the present application is shown;

[0017] Figure 8 A schematic diagram showing the longitude and latitude distribution of the optical axis of the parallel light pipe in a virtual stereo target provided by an embodiment of the present application is shown;

[0018] Fig. 9 A schematic diagram showing the spatial distribution of control points of a virtual three-dimensional target provided by an embodiment of the present application is shown;

[0019] Fig.10 A schematic diagram showing the spatial distribution of control points of a virtual stereo target provided by another embodiment of the present application is shown;

[0020] Fig.11 A schematic diagram showing the spatial distribution of control points of a virtual stereo target provided by another embodiment of the present application is shown;

[0021] Fig.12 A schematic diagram showing the longitude and latitude distribution of the optical axis of the parallel light pipe in a virtual stereo target provided by another embodiment of the present application is shown;

[0022] Fig.13 A schematic diagram showing the longitude and latitude distribution of the optical axis of the parallel light pipe in the virtual stereo target provided in another embodiment of the present application is shown;

[0023] Fig.14 A schematic diagram showing the longitude and latitude distribution of the optical axis of the parallel light pipe in the virtual stereo target provided by another embodiment of the present application is shown;

[0024] Fig.15 A schematic diagram of a camera calibration method according to an embodiment of the present application is shown;

[0025] Fig.16 A schematic diagram showing coordinates provided by an embodiment of the present application is shown;

[0026] Fig.17 A structural block diagram of a camera calibration device provided by an embodiment of the present application is shown;

[0027] Fig.18 A structural block diagram of an electronic device for executing a camera calibration method according to an embodiment of the present application is shown;

[0028] Fig.19 A storage medium provided by an embodiment of the present application for storing or carrying a program code for implementing a camera calibration method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0030] Camera calibration is one of the key technologies in machine vision, photogrammetry, 3D imaging, and image geometry correction. Its main function is to estimate the internal and external parameters of the camera. The accuracy of the calibration result and the stability of the calibration algorithm directly affect the accuracy of subsequent work. Generally, the perspective camera can be represented by a pinhole model, and can be calibrated using perspective projection mapping and affine transformation.

[0031] The method that has been studied maturely is to use a plane target. There are tool software based on plane targets, such as Matlab toolbox and Opencv tool software. In these methods, a plane calibration plate is placed in different positions in front of the camera to collect multiple target images to obtain calibration raw data with a large distribution range. This method requires placing the calibration plate in different positions multiple times and collecting target images, or rotating the camera multiple times in different positions and collecting target images, so as to achieve camera calibration.

[0032] At present, in the process of calibrating a camera, it is necessary to consider the working distance of the calibrated camera during normal use. When calibrating a camera, the distance between the target and the camera should be consistent with the working distance of the camera during normal use, so as to ensure that the calibration result is accurate and reliable. The inventor found in the study that some cameras have a longer working distance during normal use, such as vehicle-mounted cameras. When calibrating a camera with a longer working distance, it is necessary to control the distance between the camera and the target to the working distance of the camera during normal use. Due to the longer working distance, the size of the target is larger, which not only increases the production cost of the target, but also increases the difficulty of calibration.

[0033] In view of the above problems, the inventors have discovered after long-term research and proposed the camera calibration system and method provided in the embodiments of the present application, so that when calibrating the camera, there is no need to move the target, and only needs to collect images during the rotation of the rotating mechanism, which reduces the difficulty of camera calibration and completes the rapid and high-precision calibration of the camera in a smaller calibration space. The specific camera calibration method is described in detail in the subsequent embodiments.

[0034] The following describes a camera calibration method that can be used in an embodiment of the present application.

[0035] See also Figure 1 , Figures 2 to 6 , an embodiment of the present application provides a camera calibration system. Specifically, the system may include a target 100, an electronic device 200, a camera 300, a rotating mechanism 400 and a servo driver 500. The camera 300 and the servo driver 500 are electrically connected to the electronic device 200, respectively. The camera 300 may include a camera with a long working distance such as a vehicle-mounted camera, which is not limited here.

[0036] The target 100 includes a plurality of collimators and a built-in pattern of the collimators, the relative position relationship between the collimators is determined, and the built-in pattern of the collimators includes at least one collimator marking point; the target 100 also includes a collimator carrier, and the collimator carrier is used to configure the plurality of collimators to correspond to the built-in pattern within the field of view of the camera, so that the collimator marking point can be imaged in the camera; the scheme and process of fixing the collimators by the collimator carrier determines the true value of the relative position relationship between the collimators, which can be obtained by measurement;

[0037] The collimator carrier may be in the shape of a dome, arc-shaped spokes, pillars, etc., and the collimator is connected to the dome or arc-shaped spokes or pillars through a fixing frame.

[0038] The camera 300 is mounted on the rotating mechanism 400 through the camera fixing bracket (not shown), and the camera 300 is used to photograph the target to obtain a target image, and the target image includes the imaging point of the marking point of the collimator marking point in the target. The rotating mechanism 400 includes one, two or three rotating axes, and the optical axis of the camera 300 is coaxial with one of the rotating axes. The servo driver 500 is used to drive the rotating mechanism 400 to rotate, and send the rotation position information of the rotating mechanism 400 to the electronic device 200. The electronic device 200 analyzes and processes the acquired target image and the rotation position information of the rotating mechanism 400 corresponding to the target image, determines the imaging model parameters of the camera 300, and thereby realizes the calibration of the camera 300.

[0039] See also Figure 2 , shows a schematic diagram of a rotating mechanism and a target provided in an embodiment of the present application. The target 100 includes a plurality of collimators and a collimator built-in pattern, the optical axes of the collimators are in a plane and fixed on a collimator carrier, the relative position relationship between the collimators is determined, and the collimator built-in pattern includes at least one collimator marking point;

[0040] The camera 300 is mounted on the rotating mechanism 400 via the camera fixing bracket (not shown). The camera 300 is used to photograph the target to obtain a target image. The target image includes a marking point imaging point of the collimator marking point in the target.

[0041] The rotating mechanism 400 includes a carrying platform 410 , a first rotating mechanism 420 , and a camera fixing frame. The first rotating mechanism 420 has a first rotating shaft 421 , and the first rotating mechanism 420 is installed on the carrying platform 410 .

[0042] The camera mounting bracket is fixed to the first rotating mechanism 420 and is used to mount the camera 300 . The camera mounting bracket is configured such that when the camera 300 is mounted on the camera mounting bracket, the optical axis of the camera 300 is coaxial with the first rotating axis 421 .

[0043] Specifically, the camera fixing frame is fixed to the first rotating mechanism 420. In some embodiments, the first rotating mechanism 420 may have a groove adapted to the camera fixing frame, and the groove is used to fix the camera fixing frame to the first rotating mechanism 420. In some embodiments, the camera fixing frame and the first rotating mechanism 420 both have spiral grooves of the same size, and the camera fixing frame can be fixed to the first rotating mechanism 420 by screws. In some embodiments, the camera fixing frame and the first rotating mechanism 420 can also be fixed by in-mold injection.

[0044] The camera fixing frame may be a camera clip, through which the camera is fixed on the camera fixing frame. The camera fixing frame may also be a groove, and the groove is provided with a knob for adjusting the size of the groove, and the size of the groove is adjusted by the knob so that the groove is adapted to the size of the camera, and the camera is mounted on the camera fixing frame by snapping the camera into the groove.

[0045] See also Figure 3 , shows a schematic diagram of a rotating mechanism and a target provided in an embodiment of the present application. The target 100 includes a plurality of collimators and a built-in pattern of collimators, the collimators are fixed on a collimator carrier, the optical axes of the collimators are spatially distributed and point to the same point, the relative positional relationship between the collimators is determined, and the built-in pattern of the collimators includes at least one collimator marking point;

[0046] The camera 300 is mounted on the rotating mechanism 400 via the camera fixing bracket (not shown). The camera 300 is used to photograph the target to obtain a target image. The target image includes a marking point imaging point of the collimator marking point in the target.

[0047] The rotating mechanism 400 includes a carrying platform 410, a first rotating mechanism 420 and a camera fixing frame. The first rotating mechanism 420 has a first rotating axis 421, which is mounted on the carrying platform 410, and the optical axis of the camera 300 is coaxial with the first rotating axis 421 of the first rotating mechanism 420.

[0048] See also Figure 4, shows a schematic diagram of a rotating mechanism and a target provided in an embodiment of the present application. The target 100 includes a plurality of collimators and a collimator built-in pattern, the optical axes of the collimators are in a plane and fixed on a collimator carrier, the relative position relationship between the collimators is determined, and the collimator built-in pattern includes at least one collimator marking point;

[0049] The camera 300 is mounted on the rotating mechanism 400 via the camera fixing bracket (not shown). The camera 300 is used to photograph the target to obtain a target image. The target image includes a marking point imaging point of the collimator marking point in the target.

[0050] The rotating mechanism 400 includes a carrying platform 410, a first rotating mechanism 420, a second rotating mechanism 430 and a camera fixing frame. The first rotating mechanism 420 has a first rotating axis 421, and the second rotating mechanism 430 has a second rotating axis 431. Specifically, the second rotating mechanism 430 is installed on the carrying platform 410, and the second rotating axis 431 is perpendicular to the carrying platform 410. The first rotating mechanism 420 is fixed to the second rotating mechanism 430, and the first rotating axis 421 is perpendicular to the second rotating axis 431 and parallel to the carrying platform 410. The optical axis of the camera 300 is coaxial with the first rotating axis 421.

[0051] As a method, the first rotating mechanism 420 can be directly fixed to the second rotating mechanism 430, wherein the first rotating mechanism 420 can be fixed to the second rotating mechanism by in-mold injection molding. The first rotating mechanism 420 can also be fixed to the second rotating mechanism 430 by screws, and the specific fixing method is not limited here.

[0052] As another embodiment, the rotating mechanism 400 may further include a connecting bracket 440, and the first rotating mechanism 420 and the second rotating mechanism 430 are fixed to the connecting bracket 440. The camera fixing frame is fixed to the first rotating mechanism 420 and is used to install the camera 300. The camera fixing frame is configured such that when the camera 300 is installed on the camera fixing frame, the intersection of the center lines of the first rotating axis 421 and the second rotating axis 431 coincides with the center of the camera 300, and the optical axis of the camera 300 is coaxial with the first rotating axis 421.

[0053] Specifically, the camera fixing frame is fixed to the first rotating mechanism 420. In some embodiments, the first rotating mechanism 420 may have a groove adapted to the camera fixing frame, and the groove is used to fix the camera fixing frame to the first rotating mechanism 420. In some embodiments, the camera fixing frame and the first rotating mechanism 420 both have spiral grooves of the same size, and the camera fixing frame can be fixed to the first rotating mechanism 420 by screws. In some embodiments, the camera fixing frame and the first rotating mechanism 420 can also be fixed by in-mold injection.

[0054] The camera fixing frame may be a camera clip, through which the camera is fixed on the camera fixing frame. The camera fixing frame may also be a groove, and the groove is provided with a knob for adjusting the size of the groove, and the size of the groove is adjusted by the knob so that the groove is adapted to the size of the camera, and the camera is mounted on the camera fixing frame by snapping the camera into the groove.

[0055] See also Figure 5 , shows a schematic diagram of a rotating mechanism and a target provided in an embodiment of the present application. The target 100 includes a plurality of collimators and a built-in pattern of the collimators, the collimators are fixed on a collimator carrier, the optical axes of the collimators are spatially distributed and point to the same point, the relative positional relationship between the collimators is determined, and the built-in pattern of the collimators includes at least one collimator marking point.

[0056] The camera 300 is mounted on the rotating mechanism 400 via the camera fixing bracket (not shown). The camera 300 is used to photograph the target to obtain a target image. The target image includes a marking point imaging point of the collimator marking point in the target.

[0057] The rotating mechanism 400 includes a carrying platform 410, a first rotating mechanism 420, a second rotating mechanism 430 and a camera fixing frame. The first rotating mechanism 420 has a first rotating axis 421, and the second rotating mechanism 430 has a second rotating axis 431. Specifically, the second rotating mechanism 430 is installed on the carrying platform 410, and the second rotating axis 431 is perpendicular to the carrying platform 410. The first rotating mechanism 420 is fixed to the second rotating mechanism 430, and the first rotating axis 421 is perpendicular to the second rotating axis 431 and parallel to the carrying platform 410. The optical axis of the camera 300 is coaxial with the first rotating axis 421.

[0058] See also Figure 6, shows a schematic diagram of a rotating mechanism and a target provided in an embodiment of the present application. The target 100 includes a plurality of collimators and a collimator built-in pattern, the optical axes of the collimators are in a plane and fixed on a collimator carrier, the relative position relationship between the collimators is determined, and the collimator built-in pattern includes at least one collimator marking point;

[0059] The camera 300 is mounted on the rotating mechanism 400 via the camera fixing bracket (not shown). The camera 300 is used to photograph the target to obtain a target image. The target image includes a marking point imaging point of the collimator marking point in the target.

[0060] The rotating mechanism 400 includes a carrying platform 410, a first rotating mechanism 420, a second rotating mechanism 430, a third rotating mechanism 450 and a camera fixing frame. The first rotating mechanism 420 has a first rotating axis 421, the second rotating mechanism 430 has a second rotating axis 431, and the third rotating mechanism 450 has a third rotating axis 451. Specifically, the third rotating mechanism 450 is installed on the carrying platform 410, the third rotating axis 451 is parallel to the carrying platform 410, the second rotating mechanism 430 is fixed to the third rotating mechanism 450, the second rotating axis 431 is perpendicular to the carrying platform 410, the first rotating mechanism 420 is fixed to the second rotating mechanism 430, the first rotating axis 421 is perpendicular to the second rotating axis 431 and parallel to the carrying platform 410. The optical axis of the camera 300 is coaxial with the first rotating axis 421.

[0061] As a method, the first rotating mechanism 420 can be directly fixed to the second rotating mechanism 430, and the second rotating mechanism 430 can be directly fixed to the third rotating mechanism 450, wherein the first rotating mechanism 420 can be fixed to the second rotating mechanism 430 by means of in-mold injection molding, and the first rotating mechanism 420 can be fixed to the second rotating mechanism 430 by means of screws, and the second rotating mechanism 430 can be fixed to the third rotating mechanism 450 by means of in-mold injection molding, and the second rotating mechanism 430 can be fixed to the second rotating mechanism 450 by means of screws, and the specific fixing method is not limited here.

[0062] As another embodiment, the rotating mechanism 400 may further include a connecting bracket 440 and a connecting bracket 460, the first rotating mechanism 420 and the second rotating mechanism 430 are fixed to the connecting bracket 440, and the second rotating mechanism 430 and the third rotating mechanism 450 are fixed to the connecting bracket 460. The camera fixing frame is fixed to the first rotating mechanism 420 and is used to mount the camera 300. The camera fixing frame is configured such that when the camera 300 is mounted on the camera fixing frame, the centerline intersection of the first rotating axis 421, the second rotating axis 431 and the second rotating axis 451 coincides with the center of the camera 300, and the optical axis of the camera 300 is coaxial with the first rotating axis 421.

[0063] Specifically, the camera fixing frame is fixed to the first rotating mechanism 420. In some embodiments, the first rotating mechanism 420 may have a groove adapted to the camera fixing frame, and the groove is used to fix the camera fixing frame to the first rotating mechanism 420. In some embodiments, the camera fixing frame and the first rotating mechanism 420 both have spiral grooves of the same size, and the camera fixing frame can be fixed to the first rotating mechanism 420 by screws. In some embodiments, the camera fixing frame and the first rotating mechanism 420 can also be fixed by in-mold injection.

[0064] The camera fixing frame may be a camera clip, through which the camera is fixed on the camera fixing frame. The camera fixing frame may also be a groove, and the groove is provided with a knob for adjusting the size of the groove, and the size of the groove is adjusted by the knob so that the groove is adapted to the size of the camera, and the camera is mounted on the camera fixing frame by snapping the camera into the groove.

[0065] See also Figure 2 , Figure 3 In the application embodiment, there is one servo driver 500. The servo driver 500 can send the rotation angle φ of the first rotating mechanism 420 as position information to the electronic device 200. The rotation position information is calculated based on the number of collimators of the target, the relative position relationship between the collimators, the number of rotation axes of the rotating mechanism, the field of view of the camera, and the distribution of the imaging points of the marking points in the target composite image.

[0066] See also Figure 4 , Figure 5In the application embodiment, the servo driver 500 may be one or two, and the servo driver 500 may be driven to rotate toward the rotation position information while driving the first rotating mechanism 420 to rotate, or the first rotating mechanism 420 and the second rotating mechanism 430 may be driven to rotate separately. For example, after driving the first rotating mechanism 420 to rotate one circle, the second rotating mechanism 430 is driven to rotate, or the second rotating mechanism 430 is driven to rotate first, and then the first rotating mechanism 420 is driven to rotate.

[0067] The servo driver 500 can send the rotation angle φ of the first rotating mechanism 420 and the rotation angle θ of the second rotating mechanism 430 as the true value of the rotation position information to the electronic device 200. The rotation position information is calculated based on the number of the collimators of the target, the relative position relationship between the collimators, the number of rotation axes of the rotating mechanism, the field of view of the camera, and the distribution of the imaging points of the marking points in the target composite image.

[0068] See also Figure 6 In the application embodiment, the servo driver 500 may be one, two or three, and may simultaneously drive the first rotating mechanism 420, the second rotating mechanism 430 and the third rotating mechanism 450 to rotate toward the rotation position information, or may separately drive the first rotating mechanism 420, the second rotating mechanism 430 and the third rotating mechanism 450 to rotate. For example, after driving the first rotating mechanism 420 to rotate to the position, the second rotating mechanism 430 and the third rotating mechanism 450 are driven to rotate.

[0069] The servo driver 500 can send the rotation angle φ of the first rotating mechanism 420, the rotation angle θ of the second rotating mechanism 430, and the rotation angle Ω of the third rotating mechanism 450 as position information to the electronic device 200. The rotation position information is calculated based on the number of collimators of the target, the relative position relationship between the collimators, the number of rotation axes of the rotating mechanism, the field of view of the camera, and the distribution of the imaging points of the marking points in the target composite image.

[0070] In another embodiment, the target contains only one parallel light tube, and the rotating mechanism 400 includes a carrying platform 410, a first rotating mechanism 420, a second rotating mechanism 430, a third rotating mechanism 450, and a camera fixing frame. The first rotating mechanism 420 has a first rotating axis 421, the second rotating mechanism 430 has a second rotating axis 431, and the second rotating mechanism 450 has a second rotating axis 451. Specifically, the third rotating mechanism 450 is installed on the carrying platform 410, and the third rotating axis 451 is parallel to the carrying platform 410. The second rotating mechanism 430 is fixed to the third rotating mechanism 450, and the second rotating axis 431 is perpendicular to the carrying platform 410. The first rotating mechanism 420 is fixed to the second rotating mechanism 430, and the first rotating axis 421 is perpendicular to the second rotating axis 431 and parallel to the carrying platform 410. The optical axis of the camera 300 is coaxial with the first rotation axis 421 , and the same virtual stereo target calibration effect as that achieved when the target includes a plurality of parallel light tubes is achieved by adding the rotation position information.

[0071] Furthermore, the collimator built-in pattern includes at least one collimator marking point, and the collimator built-in pattern is Figure 7 Any one or a combination of the figures shown in a, b, c, d, e, and f is not limited here.

[0072] The camera 300 rotates along with the first rotating mechanism 420 or the first rotating mechanism 420 and the second rotating mechanism 430 or the first rotating mechanism 420, the second rotating mechanism 430 and the third rotating mechanism 430, which is equivalent to the camera 300 not rotating, the target 100 rotates around the camera 300, the rotating mechanism rotates according to the preset rotation position information, and the targets at all the rotation positions form a virtual three-dimensional target. The light rays of all the collimator marking points in the target at all the rotation positions in the virtual three-dimensional target form a radial light cluster with the collimator marking point as the starting point and the center of the camera 300 as the end point; when the collimator built-in pattern contains only one collimator marking point, and the collimator marking point is on the collimator optical axis, the light rays corresponding to the collimator marking point coincide with the collimator optical axis; the collimator corresponding to the imaging point of any marking point is selected as the reference, and the longitude and latitude coordinates of the collimator optical axis are set to 0, and the relative position relationship between the collimators is calculated. The longitude and latitude coordinates are used to calculate the longitude and latitude coordinates of the optical axis of each parallel light tube in the target at each rotation position information according to the rotation angle of the rotating mechanism; when the built-in pattern of the parallel light tube contains multiple parallel light tube marking points or the parallel light tube marking points are not on the optical axis of the parallel light tube, there is a certain angle between the light corresponding to the parallel light tube marking point and the optical axis of the parallel light tube, and the relative position relationship between the parallel light tubes refers to the relative position relationship between the parallel light tube marking points, which needs to be obtained through calibration. This situation is equivalent to that each of the parallel light tube marking points corresponds to a virtual parallel light tube, and the light corresponding to the parallel light tube marking point coincides with the optical axis of the virtual parallel light tube.

[0073] The camera 300 captures the image of the collimator mark point. Each collimator mark point has the longitude and latitude coordinates of the light beam where it is located. When the multiple target images collected by the camera are superimposed, a target composite image containing the imaging point of the mark point can be obtained. The image coordinates of each imaging point of the mark point in each target image are extracted as the image coordinates of the imaging point of the mark point in the target composite image. Each imaging point of the mark point has its corresponding collimator mark point, so the longitude and latitude coordinates of the corresponding incident light beam can be obtained, and one or more virtual distances can be assigned to it. Combined with the longitude and latitude coordinates of the collimator optical axis corresponding to each imaging point of the mark point, one or more virtual space point coordinates are calculated as the world coordinates of the virtual space point in the virtual stereo target corresponding to the imaging point of the mark point. It can be understood that each imaging point of the mark point corresponds to a collimator mark point, a light beam, a longitude and latitude coordinates, and multiple virtual space point coordinates; the number of virtual space points is greater than or equal to the number of imaging points of the mark point.

[0074] See also Figure 8 , Figure 8 The latitude and longitude coordinates ( Figure 8 Right), each circle corresponds to a marker imaging point, corresponding to a longitude and latitude coordinate. Fig. 9 It shows the distribution of virtual space points in the virtual stereo target when the same virtual distance is specified for each marking point imaging point, forming a spherical surface. Fig.10 The figure shows the distribution of virtual space points in the virtual stereo target when the same two virtual distances are specified for each marker imaging point. Virtual space points with small virtual distances are drawn using "o", and virtual space points with large virtual distances are drawn using "*". Fig.10 The left is a top view. Fig.10 The right is a side view. Fig.11 The figure shows the distribution of virtual space points in the virtual stereo target when no less than two and no more than five randomly sized virtual distances are specified for each marking point imaging point, and the virtual space points belonging to the same longitude and latitude are connected by line segments.

[0075] See also Fig.12 , Fig.12 The latitude and longitude coordinates ( Fig.12 Right), each black dot corresponds to a marker imaging point, corresponding to a longitude and latitude coordinate.

[0076] See also Fig.13 , Fig.13 The latitude and longitude coordinates ( Fig.13 Right), each black dot corresponds to a marker imaging point, corresponding to a longitude and latitude coordinate.

[0077] See also Fig.14 , Fig.14 The latitude and longitude coordinates ( Fig.14Right), each black dot corresponds to a marker imaging point, corresponding to a longitude and latitude coordinate.

[0078] See also Fig.15 , Fig.15 FIG. 1 is a flow chart of a camera calibration method provided by an embodiment of the present application. In a specific embodiment, the camera calibration method is applied to Fig.17 The camera calibration device 700 and the electronic device 200 ( Fig.18 ). The following will take an electronic device as an example to illustrate the specific process of this embodiment. Of course, it can be understood that the electronic device used in this embodiment can be a terminal device with computing capabilities, such as a smart phone, a tablet computer, a desktop computer, a wearable electronic device, etc., which is not limited here. Fig.15 The process shown is described in detail, and the camera calibration method may specifically include the following steps:

[0079] Step S110: acquiring a plurality of target images captured by the camera during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each of the target images.

[0080] The target image includes the imaging point of the collimator mark point of the collimator built-in pattern in the target in the camera;

[0081] In this embodiment, the shooting of the target image and the rotation of the rotating mechanism driven by the servo driver are coordinated through electronic equipment control, so that the collection of multiple target images can be achieved quickly and stably.

[0082] In some embodiments, the electronic device may be connected to the camera and the servo driver so that data can be exchanged between the electronic device and the camera and the servo driver, wherein the data may include multiple target images taken by the camera and the rotation position information of the rotating mechanism corresponding to each target image. The camera and the servo driver may be connected to the electronic device via a wired or wireless method such as a ZigBee network, WiFi, and Bluetooth, and the specific connection method is not limited here.

[0083] As one way, the electronic device can be set to obtain the camera taking multiple target images during the rotation of the rotating mechanism and the position information of the rotating mechanism corresponding to each target image at regular intervals. As another way, the electronic device can also be set to obtain the camera taking multiple target images during the rotation of the rotating mechanism and the position information of the rotating mechanism corresponding to each target image when the rotating mechanism rotates a certain angle.

[0084] In some embodiments, the rotating mechanism having one rotating axis is used, and the servo driver is used to drive the first rotating mechanism to rotate, and the rotation angle φ of the first rotating mechanism is sent to the electronic device as the rotation position information.

[0085] In some embodiments, the rotating mechanism having two rotating axes is used, and the servo driver is used to drive the first rotating mechanism and the second rotating mechanism to rotate, and the rotation angle φ of the first rotating mechanism and the rotation angle θ of the second rotating mechanism are sent to the electronic device as rotation position information.

[0086] In some embodiments, the rotating mechanism having three rotating axes is used, and the servo drive is used to drive the first rotating mechanism, the second rotating mechanism, and the third rotating mechanism to rotate, and the rotation angle φ of the first rotating mechanism, the rotation angle θ of the second rotating mechanism, and the rotation angle Ω of the third rotating mechanism are sent to the electronic device as rotation position information.

[0087] In this embodiment, the electronic device acquires a target image, which is obtained by photographing the target with a camera when the rotating mechanism is at each installation position information, and the target image includes a marking point imaging point.

[0088] In some embodiments, see Figure 7 The built-in pattern of the collimator may include but is not limited to one or a combination of circular spots, field grids, BMW patterns, checkerboards, and ChArUco, which is not limited here.

[0089] Step S120: superimposing the multiple target images to obtain a target composite image, and extracting the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image.

[0090] In this embodiment, the collimator marking point is a characteristic point of the collimator built-in pattern, and the pixel coordinates of the imaging point of the collimator marking point in the target composite image are defined as the image coordinates of the marking point imaging point, wherein the target image includes a plurality of marking point imaging points. To determine the image coordinates of the marking point imaging point, it is necessary to extract the characteristic points of each collimator built-in pattern. If the collimator built-in pattern is a BMW pattern, the collimator marking point is the corner point of the collimator built-in BMW pattern. To extract the characteristic points, the image of the collimator built-in BMW pattern in the target image can be processed to extract the image corner points.

[0091] In some embodiments, the built-in pattern of the collimator is circular, and the feature point extraction can be performed by performing ellipse fitting on the edge of the image of the built-in circular pattern of the collimator in the target image to obtain the ellipse equation of the edge of the image of the built-in circular pattern of the collimator, and the center coordinates of the image of the inner circular pattern are determined according to the ellipse equation as the image coordinates of the imaging point corresponding to the collimator marking point.

[0092] In some embodiments, the built-in pattern of the collimator is a pattern other than the BMW pattern and the circle, and an appropriate method for extracting feature points needs to be selected according to the characteristics of the specific pattern.

[0093] Step S130: determining the world coordinates of the virtual space point in the virtual stereo target corresponding to each of the marking point imaging points according to the rotation position information of the rotation mechanism and the relative position relationship between the collimators.

[0094] In this embodiment, the calculation process of obtaining the world coordinates corresponding to the imaging point of the marking point is: obtaining the longitude and latitude coordinates of each imaging point of the marking point; specifying the virtual distance and calculating the world coordinates of the virtual space point in the virtual three-dimensional target.

[0095] In this embodiment, the longitude and latitude coordinates of each marking point are determined according to the rotational position information of the rotating mechanism and the relative position relationship between the parallel light tubes: wherein, the parallel light tube corresponding to the imaging point of any marking point is selected as a reference, and the longitude and latitude coordinates of the optical axis of the parallel light tube are set to 0; according to the relative position relationship between the parallel light tubes, the longitude and latitude coordinates of the optical axes of other parallel light tubes in the target are calculated; according to the rotation angle of the rotating mechanism, the longitude and latitude coordinates of the optical axis of each parallel light tube in the target at each rotational position information are calculated.

[0096] In some embodiments, a rotation mechanism having one rotation axis is used, and the rotation angle φ of the first rotation mechanism is used to calculate the longitude and latitude coordinates of the collimator optical axis corresponding to each of the marking point imaging points in the target composite image.

[0097] In some embodiments, a rotation mechanism with two rotation axes is used, and the rotation angle φ of the first rotation mechanism and the rotation angle θ of the second rotation mechanism are used to calculate the longitude and latitude coordinates of the parallel light tube optical axis corresponding to each of the marking point imaging points in the target composite image.

[0098] In some embodiments, a rotation mechanism having three rotation axes is used, and the rotation angle φ of the first rotation mechanism, the rotation angle θ of the second rotation mechanism, the rotation angle θ of the third rotation mechanism, and the rotation angle Ω of the third rotation mechanism are used to calculate the longitude and latitude coordinates of the parallel light tube optical axis corresponding to each of the marking point imaging points in the target composite image.

[0099] In this embodiment, one or more virtual distances corresponding to each of the marking point imaging points are specified, and combined with the longitude and latitude coordinates of the parallel light tube optical axis corresponding to each of the marking point imaging points, one or more virtual space point coordinates are calculated as the world coordinates of the virtual space point.

[0100] In some embodiments, the camera rotates with the rotating mechanism, which is equivalent to the camera's fixed target rotating around the camera. The targets at all rotational positions form a virtual three-dimensional target, and the light rays of all collimator marking points in the targets at all rotational positions in the virtual three-dimensional target form a radial light cluster with the collimator marking point as the starting point and the camera center as the end point. Since the camera sees the collimator marking point at infinity, one or more virtual distances are set for it to obtain the world coordinates of the virtual space point. Please refer to Fig.13 , Fig.13 A schematic diagram of coordinates provided in an embodiment of the present application is shown. If the rectangular coordinates coincide with the origin of the spherical coordinates, the relationship between the midpoint P (X, Y, Z) of the rectangular coordinates and the midpoint P (R, α, β) of the spherical coordinates is, X = Rcos α cos β, Y = Rcosα sin β, Z = Rsinα, wherein the ranges of R, α, and β are: R ≥ 0, -π / 2 ≤ α ≤ π / 2, -π ≤ β ≤ π, β is longitude, and α is latitude.

[0101] In some embodiments, the collimator pattern includes a collimator mark point, and when the rotating mechanism having a rotating axis is used and the central collimator is coaxial with the optical axis of the camera, a collimator pattern is formed as follows: Figure 8 The latitude and longitude coordinates of the imaging point of the marking point shown in FIG. 1 ; when the same virtual distance is specified for each imaging point of the marking point, the distribution of the virtual space points in the virtual stereo target is as follows: Fig. 9 As shown in FIG. 1 , a sphere is formed; when the same two virtual distances are specified for each marker imaging point, the distribution of virtual space points in the virtual stereo target is as follows: Fig.10 As shown in Figure 2, two concentric spheres with different radii are formed; when each marker imaging point specifies a virtual distance of no less than two and no more than five random sizes, the distribution of the virtual space points in the virtual stereo target is as follows: Fig.11 As shown, scattered points in space are formed, and virtual space points belonging to the same longitude and latitude are connected together by line segments; among them, if the origin of the rectangular coordinates coincides with the origin of the spherical coordinates, then the relationship between the rectangular coordinates and the spherical coordinates is the same as the above-mentioned coordinate relationship, which will not be repeated here.

[0102] In some embodiments, for a camera, point O is the optical center of the camera. A point P outside the optical center can be represented by X, Y, Z, or by R, φ, θ. If the points on the world coordinates are all on a sphere, they can also be represented by only φ and θ. More representation forms are not described here.

[0103] Step S140: Fitting the image coordinates and the corresponding world coordinates of the virtual space point in the virtual stereo target according to the imaging model of the camera, and obtaining the parameters of the imaging model of the camera through an optimization algorithm.

[0104] In this embodiment, after the image coordinates and world coordinates of the imaging point of each marking point in the virtual target are obtained, the imaging model parameters of the camera can be calculated according to the camera imaging model.

[0105] In some embodiments, a perspective imaging model may be selected, and the focal length, principal point, and initial values ​​of distortion parameters of the camera may be calculated according to the perspective imaging model. In the perspective imaging model, there are no distortion parameters, so the initial values ​​of the distortion parameters of the camera may be set to 0.

[0106] The perspective imaging model is also called the pinhole model. In the perspective imaging model, the world coordinates (X, Y, Z) of any point P in space and the image coordinates (u, v) of the image point of the point in the camera satisfy the perspective imaging formula:

[0107]

[0108] Where [X, Y, Z, 1] is the homogeneous coordinates of the spatial point in the world coordinate system, corresponding to the homogeneous pixel coordinates [u, v, 1] in the camera coordinate system. [R, T] is a pair of rotation matrices and translation matrices between the world coordinate system and the camera coordinate system. (f u , f v ) represents the normalized focal length in the u direction (horizontal direction) and the v direction (vertical direction) in the camera coordinate system, which is the pixel focal length of the camera. 0 , v 0 ) is the coordinate of the camera principal point. Substitute the world coordinates of all the marking points on the virtual target and the image coordinates corresponding to these marking points into the perspective imaging formula, and solve the simultaneous equations composed of these marking points into the perspective imaging formula to obtain the camera focal length (f u , f v ) and the initial values ​​of the principal point coordinates (u0, v0) under the perspective imaging model.

[0109] In some embodiments, the imaging model may be various imaging models of various cameras, such as a pinhole camera imaging model, a fisheye camera imaging model, a panoramic camera imaging model, etc. The specific type of the camera is not limited herein.

[0110] In some implementations, the initial value may be obtained by other methods, may be calculated through lens design information, may be obtained through other technical means, or may be a parameter of other cameras in the same batch. The specific acquisition method is not limited here.

[0111] In this embodiment, after obtaining the initial values ​​of the focal length, principal point, and distortion parameters of the camera, the Levenberg-Marquardt algorithm can be used to optimize the initial values ​​of the focal length, principal point, and distortion parameters of the camera to determine the imaging model parameters of the camera.

[0112] In some embodiments, the Levenberg-Marquardt algorithm is used to minimize the sum of the squares of the differences between the projection values ​​of the virtual space points corresponding to the corresponding marker imaging points in the virtual stereo target and the measured values ​​of the corresponding marker imaging points on the virtual stereo target. The projection values ​​of the virtual space points corresponding to the marker imaging points are image coordinates calculated based on the world coordinates of each virtual space point in the virtual stereo target according to the camera model described by the camera imaging parameters; the measured values ​​refer to the coordinates of the marker imaging points in the image coordinate system extracted after the target image is obtained by photographing the target with a camera. Thus, the Levenberg-Marquardt algorithm can be used to minimize the sum of the squares of the differences between the projection values ​​of the virtual space points on the virtual stereo target and the measured values ​​of the marker imaging points on the virtual stereo target, and the focal length, principal point and initial values ​​of the distortion parameters of the camera are continuously updated to obtain the optimal solution, so that the imaging model parameters of the camera can be determined.

[0113] Furthermore, the Levenberg-Marquardt algorithm is used to minimize the sum of squares of the differences between the projection values ​​of the virtual space points corresponding to the corresponding marking point imaging points in the virtual stereo target and the measured values ​​of the corresponding marking point imaging points on the virtual stereo target, that is, to minimize the value of the objective function. Where N represents the number of virtual space points on the virtual three-dimensional target, is the projection value of the virtual space point, that is, the pixel coordinate of the virtual space point calculated according to the camera model, m j It is the measured value of the imaging point of the marker point, that is, the pixel coordinate of the imaging point of the marker point extracted from the target image.

[0114] A camera calibration method provided by an embodiment of the present application obtains multiple target images obtained by the camera shooting the target during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each of the target images, superimposes the multiple target images to obtain a target composite image, extracts the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image, determines the world coordinates of the virtual space point in the virtual three-dimensional target corresponding to each of the marking point imaging points, obtains the parameters of the imaging model of the camera through an optimization algorithm, and thus can achieve that when calibrating the camera, there is no need to move the target, only to collect images during the rotation of the rotating mechanism, thereby reducing the difficulty of camera calibration and completing fast and high-precision calibration of the camera in a smaller calibration space.

[0115] See also Fig.17 , Fig.17 The module block diagram of the camera calibration device 700 provided in the embodiment of the present application is shown in FIG. Fig.17 The camera calibration device 700 includes: a data acquisition module 710, a data processing module 720 and a camera parameter calculation module 750, wherein:

[0116] The data acquisition module 710 is used to obtain multiple target images obtained by the camera taking pictures of the target during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each target image, wherein the target image includes the imaging point of the collimator mark point of the collimator built-in pattern in the target in the camera.

[0117] The data processing module 720 is used to obtain a target image, to superimpose the multiple target images to obtain a target composite image, to extract the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image; and to determine the world coordinates of the virtual space point in the virtual three-dimensional target corresponding to each of the marking point imaging points according to the rotation position information of the rotating mechanism and the relative position relationship between the parallel light tubes.

[0118] The camera parameter calculation module 730 fits the image coordinates and the corresponding world coordinates of the virtual space point in the virtual stereo target according to the imaging model of the camera, and obtains the parameters of the imaging model of the camera through an optimization algorithm.

[0119] Furthermore, the data processing module 720 includes: an image coordinate determination submodule, wherein:

[0120] The image coordinate determination submodule superimposes a plurality of the target images to obtain a target composite image, and extracts the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image.

[0121] Furthermore, the world coordinate determination module includes: a marker point imaging point latitude and longitude coordinate determination submodule and a marker point imaging point world coordinate determination submodule, wherein:

[0122] The submodule for determining the longitude and latitude coordinates of the imaging point of the marking point is used to determine the longitude and latitude coordinates of each marking point according to the rotation position information of the rotating mechanism and the relative position relationship between the collimators. The collimator corresponding to any imaging point of the marking point is selected as the reference, and the longitude and latitude coordinates of the optical axis of the collimator are set to 0; the longitude and latitude coordinates of the optical axes of other collimators in the target are calculated according to the relative position relationship between the collimators; the longitude and latitude coordinates of the optical axis of each collimator in the target at each rotation position information are calculated according to the rotation angle of the rotating mechanism.

[0123] The marker point world coordinate determination submodule is used to specify one or more virtual distances corresponding to each marker point imaging point, and to calculate one or more virtual space point coordinates in combination with the longitude and latitude coordinates of the parallel light tube optical axis corresponding to each marker point imaging point as the world coordinates of the virtual space point.

[0124] The camera calibration module 730 is used to fit the image coordinates and the world coordinates according to the camera imaging model to obtain imaging model parameter values ​​such as the focal length, principal point and distortion parameters of the camera.

[0125] Furthermore, the camera calibration module 730 includes: an imaging model parameter initial value determination submodule and an imaging model parameter optimization submodule, wherein:

[0126] The imaging model parameter initial value determination submodule is used to determine the initial values ​​of the imaging model parameters.

[0127] The imaging model parameter optimization submodule is used to use the initial values ​​of the imaging model parameters and the optimization algorithm to minimize the objective function related to the imaging model and obtain the optimal camera imaging model parameters.

[0128] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0129] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0130] See also Fig.18 , which shows a structural block diagram of an electronic device 200 provided in an embodiment of the present application. The electronic device 200 in the present application may include one or more of the following components: a processor 210, a memory 220, and one or more applications, wherein the one or more applications may be stored in the memory 220 and configured to be executed by one or more processors 210, and the one or more programs are configured to execute the method described in the foregoing method embodiment.

[0131] Among them, the processor 210 may include one or more processing cores. The processor 210 uses various interfaces and lines to connect various parts within the entire electronic device 200, and executes various functions and processes data of the electronic device 200 by running or executing instructions, programs, code sets or instruction sets stored in the memory 220, and calling data stored in the memory 220. Optionally, the processor 210 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 210 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 210, but may be implemented separately through a communication chip.

[0132] The memory 220 may include a random access memory (RAM) or a read-only memory (ROM). The memory 220 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 220 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 200 during use.

[0133] See also Fig.19 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0134] The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer readable storage medium 800 has storage space for program code 810 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in an appropriate form.

[0135] In summary, the camera calibration system and method provided in the embodiments of the present application obtain multiple target images obtained by the camera shooting the target during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each of the target images, superimpose the multiple target images to obtain a target composite image, extract the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image, determine the world coordinates of the virtual space point in the virtual three-dimensional target corresponding to each of the marking point imaging points, and obtain the parameters of the imaging model of the camera through an optimization algorithm, so that when calibrating the camera, there is no need to move the target, only to collect images during the rotation of the rotating mechanism, thereby reducing the difficulty of camera calibration and completing the fast and high-precision calibration of the camera in a smaller calibration space.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A camera calibration system, characterized in that: It includes a target, a camera, a rotating mechanism, a servo driver and an electronic device, wherein the camera and the servo driver are electrically connected to the electronic device; The target includes a plurality of collimators and a pattern built into the collimators, the relative position relationship between the collimators is determined, and the pattern built into the collimators includes at least one collimator marking point; The camera is installed on the rotating mechanism, and the camera is used to photograph the target to obtain a target image, wherein the target image includes a marking point imaging point of the collimator marking point in the target; The rotating mechanism comprises at least one rotating axis, and the rotating mechanism moves according to the preset rotating position information, and the targets at all the rotating positions form a virtual three-dimensional target; The servo driver is used to drive the rotating mechanism to rotate and send the position information of the rotating mechanism to the electronic device; The electronic device is used to obtain multiple target images obtained by the camera shooting the target during the rotation of the rotating mechanism and the position information of the rotating mechanism corresponding to each of the target images, superimpose the multiple target images to obtain a target composite image, extract the image coordinates of each of the marked point imaging points in each of the target images as the image coordinates of the marked point imaging points in the target composite image, determine the world coordinates of one or more virtual space points in the virtual three-dimensional target corresponding to each of the marked point imaging points in the target composite image according to the position information of the rotating mechanism and the relative position relationship between the parallel light tubes in the target, and based on the image coordinates and the world coordinates, perform fitting according to the imaging model of the camera, and obtain the parameters of the camera imaging model through an optimization algorithm.

2. The system according to claim 1, characterized in that The target further includes a collimator carrier, which is used to configure the plurality of collimators to correspond to the built-in pattern within the field of view of the camera.

3. The system according to claim 1, characterized in that The number of rotation axes of the rotating mechanism is calculated based on the number of collimators used by the target and the relative position relationship between the collimators; the rotation position information is calculated based on the number of collimators of the target, the relative position relationship between the collimators, the number of rotation axes of the rotating mechanism, the field of view of the camera and the distribution of the imaging points of the marking points in the composite image of the target.

4. The system according to claim 1, characterized in that When the rotating mechanism with one rotation axis is used, the optical axis of the camera is coaxial with the rotation axis of the rotating mechanism; when the rotating mechanism with two rotation axes is used, the first rotation axis and the second rotation axis are perpendicular to and intersect with each other, and the optical axis of the camera is coaxial with the first rotation axis of the rotating mechanism; when the rotating mechanism with three rotation axes is used, the first rotation axis, the second rotation axis and the third rotation axis are perpendicular to and intersect with each other, and the optical axis of the camera is coaxial with the first rotation axis of the rotating mechanism.

5. The system according to claim 4, characterized in that When the rotating mechanism having one rotating shaft is used, the servo driver is used to drive the rotating mechanism to rotate according to the rotating position information, and sends the rotation angle φ of the first rotating mechanism as the position information to the electronic device; When the rotating mechanism with two rotating axes is used, the servo driver is used to drive the first rotating mechanism and the second rotating mechanism to rotate according to the rotational position information, and the rotation angle φ of the first rotating mechanism and the rotation angle θ of the second rotating mechanism are sent to the electronic device as position information; when the rotating mechanism with three rotating axes is used, the servo driver is used to drive the first rotating mechanism, the second rotating mechanism and the third rotating mechanism to rotate according to the rotational position information, and the rotation angle φ of the first rotating mechanism, the rotation angle θ of the second rotating mechanism and the rotation angle Ω of the third rotating mechanism are sent to the electronic device as position information.

6. The system according to claim 1, characterized in that The marking points include one or a combination of marking point schemes such as circular spots, grids, BMW patterns, chessboards, and ChArUco.

7. A camera calibration method, characterized in that: A camera is calibrated based on the camera calibration system according to any one of claims 1 to 6, wherein the method is applied to an electronic device and comprises: Acquire a plurality of target images obtained by the camera taking the target during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each of the target images, wherein the target images include imaging points of the collimator marking points of the collimator built-in pattern in the target in the camera; Superimposing the multiple target images to obtain a target composite image, and extracting the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image; Determine the world coordinates of the virtual space point in the virtual three-dimensional target corresponding to each marking point imaging point in the target composite image according to the rotation position information of the rotation mechanism and the relative position relationship between the collimators; Based on the image coordinates and the world coordinates, fitting is performed according to the imaging model of the camera, and parameters of the camera imaging model are obtained through an optimization algorithm.

8. The method according to claim 7, characterized in that The method of determining the world coordinates of the virtual space point in the virtual stereo target corresponding to each of the marking point imaging points according to the rotation position information of the rotation mechanism and the relative position relationship between the collimators includes: Determine the longitude and latitude coordinates of each marking point according to the rotation position information of the rotating mechanism and the relative position relationship between the collimators: wherein, select the collimator corresponding to the imaging point of any marking point as a reference, and set the longitude and latitude coordinates of the optical axis of the collimator to be 0; calculate the longitude and latitude coordinates of the optical axes of other collimators in the target according to the relative position relationship between the collimators; calculate the longitude and latitude coordinates of each optical axis of the collimator in the target at each rotation position information according to the rotation angle of the rotating mechanism; When a rotating mechanism with one rotating axis is used, the rotation angle φ of the first rotating mechanism is used to calculate the longitude and latitude coordinates of the optical axis of the parallel light tube corresponding to each imaging point of the marking point in the target composite image; When a rotating mechanism with two rotating axes is used, the rotation angle φ of the first rotating mechanism and the rotation angle θ of the second rotating mechanism are used to calculate the longitude and latitude coordinates of the optical axis of the parallel light tube corresponding to each imaging point of the marking point in the target composite image; When a rotation mechanism with three rotation axes is used, the rotation angle φ of the first rotation mechanism, the rotation angle θ of the second rotation mechanism, the rotation angle θ of the third rotation mechanism, and the rotation angle Ω of the third rotation mechanism are used to calculate the longitude and latitude coordinates of the optical axis of the parallel light tube corresponding to each imaging point of the marking point in the target composite image; Specify one or more virtual distances corresponding to each of the imaging points of the marking points, combine the longitude and latitude coordinates of the parallel light tube optical axis corresponding to each of the imaging points of the marking points, calculate one or more virtual space point coordinates as the world coordinates of the virtual space point, and all virtual space points constitute the control points of the virtual three-dimensional target.

9. A camera calibration device, characterized in that: The camera is calibrated based on the camera calibration system according to any one of claims 1 to 6, the device comprising: a data acquisition module, used for acquiring a plurality of target images obtained by the camera taking the target during the rotation of the rotating mechanism and the rotation position information of the rotating mechanism corresponding to each of the target images, wherein the target images include imaging points of the collimator mark points of the collimator built-in pattern in the target in the camera; A data processing module, used for superimposing the multiple target images to obtain a target composite image, extracting the image coordinates of each of the marking point imaging points in each of the target images as the image coordinates of the marking point imaging points in the target composite image, and determining the world coordinates of the virtual space point in the virtual three-dimensional target corresponding to each marking point imaging point in the target composite image according to the rotation position information of the rotation mechanism and the relative position relationship between the collimators; The camera parameter calculation module is used to perform fitting according to the imaging model of the camera based on the image coordinates and the world coordinates, and obtain the parameters of the camera imaging model through an optimization algorithm.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code, and when the program code is called by a processor, the processor is configured to execute the method according to claim 7.

Images