Multi-camera calibration method, multi-camera calibration device and storage medium

Abstract

The present disclosure relates to a multi-camera calibration method, a multi-camera calibration device and a storage medium. The multi-camera calibration method comprises: obtaining an image after multiple cameras shoot and image a calibration pattern at a preset calibration distance, wherein the calibration pattern is a checkerboard array composed of at least two checkerboard patterns arranged in sequence, wherein each checkerboard pattern in the checkerboard array includes a border pattern and at least one center pattern nested with the border pattern at the same center, and in each checkerboard pattern, the size between two adjacent corner points of the center pattern is different from the size between two adjacent corner points of the border pattern; determining the focal length of the current camera to be calibrated in the multi-camera, and based on the focal length of the current camera to be calibrated, extracting the corner points included in the image after shooting and imaging; based on the extracted corner points, calibrating the camera to be calibrated. The present disclosure improves the calibration accuracy by extracting corner points that match the focal length of the camera to be calibrated through a single calibration pattern.

Description

Technical Field

[0001] The present disclosure relates to the field of terminal technology, and in particular to a multi-camera calibration method, a multi-camera calibration device and a storage medium. Background Art

[0002] Terminals with shooting functions (such as smartphones, smart vehicles, robots, etc.) are limited by the spatial structure and cannot achieve the effects of independent cameras in many aspects. For example, large aperture bokeh, optical zoom, etc. In order to meet user needs, many solutions using multiple cameras (sometimes referred to as multi-cameras below) have been derived to enhance the shooting capabilities of terminals.

[0003] In multi-camera solutions, multi-camera calibration is usually involved to determine some parameters of the camera module itself and some parameters of the positional relationship between the camera modules. Through these parameters, the image information collected by multiple camera modules can be matched accordingly to facilitate subsequent image processing.

[0004] In the related art, multi-camera calibration is usually performed based on a calibration method of a single-plane chessboard (chart). Among them, in the calibration method based on a single-plane chessboard, one implementation method is: using calibration patterns of different sizes for different calibration distances. Using calibration patterns of different sizes for different calibration distances can adapt to different calibration distances, but the types of shooting modules in terminals with shooting functions are diverse, such as telephoto, wide-angle, ultra-wide-angle and other combinations. The focal lengths and field of view angles of different shooting modules vary greatly, which means that at the same calibration distance, no matter which size of calibration pattern is selected, it is impossible to adapt well to multiple shooting modules with different properties, resulting in poor calibration accuracy and inaccurate calibration results for multiple cameras. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a multi-camera calibration method, a multi-camera calibration device and a storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, a multi-camera calibration method is provided, including:

[0007] An image is obtained after multiple cameras shoot and image a calibration pattern at a preset calibration distance, wherein the calibration pattern is a checkerboard array composed of at least two checkerboard patterns arranged in sequence, wherein each checkerboard pattern in the checkerboard array includes a border pattern and at least one center pattern co-centeredly nested with the border pattern, and in each checkerboard pattern, a size between two adjacent corner points of the center pattern is different from a size between two adjacent corner points of the border pattern; a focal length of a current camera to be calibrated in the multiple cameras is determined, and based on the focal length of the current camera to be calibrated, corner points included in the image after the image is shot; and based on the extracted corner points, the camera to be calibrated is calibrated.

[0008] In one embodiment, the at least one central pattern of each checkerboard pattern and adjacent patterns in the border pattern have the same shape and different colors.

[0009] In one embodiment, the at least two checkerboard patterns include a first checkerboard pattern and a second checkerboard pattern; the first checkerboard pattern includes a center pattern of a first color and a border pattern of a second color; the second checkerboard pattern includes a center pattern of a second color and a border pattern of the first color.

[0010] In one embodiment, based on the focal length of the current camera to be calibrated, the corner points included in the captured image are extracted, including: if the focal length of the current camera to be calibrated is greater than a first focal length threshold and the field of view angle is less than a first field of view angle threshold, then the corner points of the border pattern are extracted in the captured image.

[0011] In one embodiment, based on the focal length of the current camera to be calibrated, the corner points included in the captured image are extracted, including: if the focal length of the current camera to be calibrated is less than a second focal length threshold and the field of view angle is greater than a second field of view angle threshold, then the corner points of the central pattern are extracted in the captured image.

[0012] According to a second aspect of an embodiment of the present disclosure, a multi-camera calibration device is provided, including:

[0013] An acquisition unit is used to acquire an image after multiple cameras shoot and image a calibration pattern at a preset calibration distance, wherein the calibration pattern is a checkerboard array composed of at least two checkerboard patterns arranged in sequence, wherein each checkerboard pattern in the checkerboard array includes a border pattern and at least one center pattern nested with the border pattern at the same center, and in each checkerboard pattern, the size between two adjacent corner points of the center pattern is different from the size between two adjacent corner points of the border pattern; an extraction unit is used to determine the focal length of a current camera to be calibrated among the multiple cameras, and based on the focal length of the current camera to be calibrated, extract the corner points included in the image after the shooting; a calibration unit is used to calibrate the camera to be calibrated based on the extracted corner points.

[0014] In one embodiment, the at least one central pattern of each checkerboard pattern and adjacent patterns in the border pattern have the same shape and different colors.

[0015] In one embodiment, the at least two checkerboard patterns include a first checkerboard pattern and a second checkerboard pattern; the first checkerboard pattern includes a center pattern of a first color and a border pattern of a second color; the second checkerboard pattern includes a center pattern of a second color and a border pattern of the first color.

[0016] In one embodiment, the extraction unit extracts the corner points included in the captured image based on the focal length of the current camera to be calibrated in the following manner: if the focal length of the current camera to be calibrated is greater than a first focal length threshold and the field of view angle is less than a first field of view angle threshold, then the corner points of the border pattern are extracted in the captured image.

[0017] In one embodiment, the extraction unit extracts the corner points included in the captured image based on the focal length of the current camera to be calibrated in the following manner: if the focal length of the current camera to be calibrated is less than a second focal length threshold and the field of view angle is greater than a second field of view angle threshold, then the corner points of the central pattern are extracted in the captured image.

[0018] According to a third aspect of an embodiment of the present disclosure, a multi-camera calibration device is provided, including:

[0019] a processor; a memory for storing instructions executable by the processor;

[0020] Among them, the processor is configured to: execute the multi-camera calibration method described in the first aspect or any one of the embodiments of the first aspect.

[0021] According to the fourth aspect of the embodiments of the present disclosure, a non-temporary computer-readable storage medium is provided. When the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal can execute the multi-camera calibration method described in the first aspect or any one of the embodiments of the first aspect.

[0022] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: the calibration pattern in the embodiments of the present disclosure is a checkerboard array composed of at least two checkerboard patterns arranged in sequence. Each checkerboard pattern in the checkerboard array includes a border pattern and at least one center pattern nested with the border pattern at the same center. In each checkerboard pattern, the size between two adjacent corner points of the center pattern is different from the size between two adjacent corner points of the border pattern. When extracting corner points, based on the focal length of the current camera to be calibrated, the corner points included in the image after being captured are extracted, which can adapt to the focal length of the camera to be calibrated and improve the calibration accuracy.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] Figure 1 The figure is a schematic diagram of a multi-camera calibration scene according to an exemplary embodiment.

[0026] Figure 2 is a schematic diagram of a calibration pattern according to an exemplary embodiment.

[0027] Figure 3 The present invention is a flowchart of a multi-camera calibration method according to an exemplary embodiment.

[0028] Figure 4 The present invention is a block diagram of a multi-camera calibration device according to an exemplary embodiment.

[0029] Figure 5 The invention is a block diagram of a device for multi-camera calibration according to an exemplary embodiment. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0031] The multi-camera calibration method provided by the embodiment of the present disclosure is applied to the scenario of calibrating multiple camera modules in a terminal with a shooting function. Figure 1 A schematic diagram of a multi-camera calibration scenario in an exemplary embodiment of the present disclosure is shown. Figure 1 As shown, the multi-camera terminal takes a photo of the calibration pattern at a preset calibration distance (for example, 55cm±0.5cm from the center of the calibration drawing in the figure). For multi-camera terminals with different field of view angles and focal lengths, appropriate feature points on the drawing can be selected for calibration.

[0032] In the related art, multi-camera calibration is usually performed based on a calibration method of a single-plane chessboard (chart). For example, in an implementation scheme for calibrating a dual-camera module, a calibration pattern corresponding to the calibration distance is selected according to the calibration distance. The calibration pattern is laid flat on a plane. The calibration pattern is photographed and imaged at the calibration distance using a dual-camera module to obtain a photographed image. The image is preprocessed, and the dual-camera module is calibrated based on the preprocessed image. Among them, different calibration drawings are provided according to different calibration distances. For example, when the calibration distance is 80 cm, a calibration pattern with a chessboard array of 24 rows and 18 columns and a total length of 1.2 m*0.9 m is selected, or a calibration pattern with a chessboard array of 32 rows and 24 columns and a total length of 0.8 m*0.6 m is selected. When the calibration distance is 50 cm, a calibration pattern with a chessboard array of 32 rows and 24 columns and a total length of 0.8 m*0.6 m is selected. When the calibration distance is 30 cm, a calibration pattern with a checkerboard array of 25 rows and 18 columns and a total length of 0.4 m*0.3 m is selected.

[0033] In the related art, the implementation method of dual-camera module calibration is to use chessboards and calibration drawings of different sizes for different calibration distances. Although this method can adapt to different calibration distances, in the current multi-camera shooting field, there are various types of shooting modules, and various combinations such as telephoto, wide-angle, and ultra-wide-angle are emerging in an endless stream. The focal lengths and field of view of different shooting modules vary greatly, which means that at the same calibration distance, no matter which size of calibration drawing is selected, it is impossible to adapt the two shooting modules with different properties well, which will inevitably cause the calibration accuracy of a certain shooting module to deteriorate, resulting in inaccurate calibration results for multiple cameras. For example, when calibrating dual-camera modules with different focal lengths, there is a problem that the corner points of the chessboard shot by the telephoto lens are too sparse, and the corner points of the chessboard shot by the short-focus lens are too dense.

[0034] The disclosed embodiment provides a multi-camera calibration method, in which the calibration of multiple cameras is performed based on a single calibration pattern. The single calibration pattern includes a checkerboard array composed of at least two calibration patterns arranged in sequence. Each checkerboard pattern in the checkerboard array includes a border pattern and at least one center pattern nested with the border pattern at the same center. In each checkerboard pattern, the size between two adjacent corner points of the center pattern is different from the size between two adjacent corner points of the border pattern.

[0035] Among them, the multi-camera calibration method provided by the embodiment of the present disclosure is compatible with multiple focal lengths. For example, a shooting module with a short focal length and a large field of view angle can be calibrated. A shooting module with a long focal length and a small field of view angle can also be calibrated. For a shooting module with a short focal length and a large field of view angle, corner points with larger sizes between corner points in the captured image can be extracted. For a shooting module with a long focal length and a small field of view angle, corner points with smaller sizes between corner points in the captured image can be extracted. Therefore, the multi-camera calibration method provided by the embodiment of the present disclosure can, on the one hand, ensure that a camera with a short focal length and a large field of view will not have extraction errors due to too dense corner point information. On the other hand, it can ensure that a camera with a long focal length and a small field of view angle will not incorrectly calculate the camera model due to too little corner point information.

[0036] In the embodiment of the present disclosure, the calibration pattern used in the multi-camera calibration method provided in the embodiment of the present disclosure is first described.

[0037] The calibration pattern provided in the embodiment of the present disclosure includes a checkerboard array composed of at least two calibration patterns arranged in sequence. In the embodiment of the present disclosure, each checkerboard pattern in the checkerboard array includes a plurality of patterns with different sizes between corners. In one example, each checkerboard pattern in the checkerboard array includes a border pattern and at least one center pattern nested with the same center as the border pattern. In each checkerboard pattern, the size between two adjacent corners of the center pattern is different from the size between two adjacent corners of the border pattern. In one example, each checkerboard pattern has a border pattern and one or more center patterns. Among them, when each checkerboard pattern includes a border pattern and a center pattern, the center pattern is nested in the border pattern and is located on the inside of the border pattern. The border pattern is located on the outside relative to the center pattern. In another example, when each checkerboard pattern includes multiple center patterns, it can be understood that each checkerboard pattern includes at least two center patterns, for example, when two center patterns are included, it can be a center pattern and a secondary center pattern. The frame pattern, the sub-center pattern and the center pattern are nested in sequence, that is, the sub-center pattern is nested in the frame pattern, and the center pattern is nested in the sub-center pattern.

[0038] In the embodiment of the present disclosure, in order to distinguish multiple patterns with different sizes between corner points in the same checkerboard pattern, any two adjacent patterns in each checkerboard pattern may be set to different colors.

[0039] In one embodiment, in at least one center pattern and border pattern of the same checkerboard pattern, adjacent patterns have different colors.

[0040] Further, in the embodiment of the present disclosure, at least one center pattern and border pattern of the same checkerboard pattern have the same shape. For example, it can be a regular shape, such as a rectangle. In one embodiment, in the embodiment of the present disclosure, the center pattern and border pattern in the checkerboard pattern can be a square.

[0041] Furthermore, each checkerboard pattern in the checkerboard array in the embodiment of the present disclosure may also have the same shape. For example, it may be a regular shape, such as a rectangle. In one implementation, in the embodiment of the present disclosure, the checkerboard pattern is a square.

[0042] In the disclosed embodiment, in order to distinguish different checkerboard patterns in the checkerboard array, in one implementation manner, any two adjacent checkerboard patterns in the checkerboard array have different colors.

[0043] In one embodiment, any two adjacent checkerboard patterns with different colors in the embodiments of the present disclosure may be two or more checkerboard patterns with different colors. In one embodiment, the checkerboard array includes at least a first checkerboard pattern and a second checkerboard pattern.

[0044] In the embodiment of the present disclosure, the first checkerboard pattern and the second checkerboard pattern can be understood as two complementary checkerboard patterns.

[0045] In one embodiment, the first checkerboard pattern includes a center pattern of a first color and a border pattern of a second color. The second checkerboard pattern includes a center pattern of a second color and a border pattern of the first color.

[0046] In one example, the first color is white and the second color is black. The center pattern and the border pattern are squares. The first checkerboard pattern can be understood as a checkerboard with a white square in the center and a black square in the outer border. The second checkerboard pattern can be understood as a checkerboard with a black square in the center and a white square in the outer border. The first checkerboard pattern and the second checkerboard pattern are arranged adjacent to each other and are arranged in sequence to form a checkerboard array.

[0047] Figure 2 FIG. 1 is a schematic diagram of a calibration pattern shown in an exemplary embodiment of the present disclosure. Figure 2 As shown, the calibration pattern includes two checkerboard patterns. One checkerboard pattern includes a center pattern and a border pattern. The center pattern is a black square and the border pattern is a white square. Another checkerboard pattern also includes a center pattern and a border pattern. The center pattern is a black square and the border pattern is a white square. The two checkerboard patterns are arranged in sequence to form a checkerboard array. In the embodiment of the present disclosure, the size between two adjacent corner points of the center pattern is called the first size, and the size between two adjacent corner points of the border pattern is called the second size. Continue to refer to Figure 2 As shown, the first size and the second size are different, so it can be understood that two types of corner points are formed in the same chessboard pattern. Figure 2 The corner points of the middle border pattern can be understood as first-type corner points, and the corner points of the center pattern can be understood as second-type corner points.

[0048] In the disclosed embodiment, the first dimension between adjacent corner points of the center pattern of a single checkerboard pattern and the second dimension between two adjacent corner points of the border pattern can be set based on the dimensions often used in practice. In one example, the first dimension between adjacent corner points of the center pattern of a single checkerboard pattern is 10 mm, and the second dimension between two adjacent corner points of the border pattern is 20 mm. The border width between the center pattern and the border pattern is 5 mm.

[0049] Furthermore, in the disclosed embodiment, the size of the checkerboard array can be preset based on the calibration distance, field of view, etc. of the commonly used camera module to ensure that the image after being captured fills the entire field of view of the camera module. Figure 2 A calibration pattern with a checkerboard array size of 8 rows and 8 columns is shown in FIG.

[0050] When the calibration patterns mentioned above are used to calibrate multiple cameras in the embodiments of the present disclosure, the calibration patterns mentioned above in the embodiments of the present disclosure can be installed on a plane.

[0051] In one implementation, the calibration pattern can be laid flat on a plane in the disclosed embodiment, for example, the flatness error does not exceed 1 mm, and the calibration pattern is ensured to be free of scratches or dirt. In the specific implementation process, the pattern can be pasted on a flat hard board (such as glass), the flatness error (the height difference between the highest point of the plane and the lowest point of the plane) does not exceed 1 mm, and there must be no scratches, dirt, etc.

[0052] After the calibration pattern is fixed to the plane, a multi-camera terminal (such as a mobile phone camera) can be used to capture the calibration drawing at a preset calibration distance.

[0053] In the disclosed embodiment, a multi-camera terminal (eg, a dual-camera) with different focal lengths may be used to capture and image a calibration pattern at a preset calibration distance.

[0054] As an optional embodiment, in the embodiment of the present disclosure, a multi-camera terminal (e.g., a dual-camera) can be placed 55 cm in front of the calibration pattern, with a distance error of no more than ±0.5 cm. The multi-camera terminal (e.g., a dual-camera) is kept perpendicular to the plane where the calibration pattern is located, with an angle error of no more than ±3°. While ensuring that no images of other objects are reflected on the calibration pattern and that the uniformity of the light source is ensured, the multi-camera terminal (e.g., a dual-camera) can be controlled to capture and image the calibration pattern. During the imaging process, the center of the calibration pattern is ensured to appear in the center of the captured image.

[0055] In one embodiment, during the specific implementation process, when using a multi-camera terminal (such as a dual-camera) to capture and image the calibration drawing, it is necessary to ensure that no other objects are reflected into the calibration pattern, ensure the uniformity of the light source, prevent excessive darkness or overexposure, and ensure that the calibration drawing is firmly installed during the calibration process.

[0056] In the disclosed embodiment, the terminal may obtain an image obtained by capturing and imaging the calibration pattern with multiple cameras at a preset calibration distance, and select appropriate corner points on the calibration pattern for calibration.

[0057] Figure 3 is a flowchart of a multi-camera calibration method according to an exemplary embodiment. Figure 3 As shown, the multi-camera calibration method is used in a terminal and includes the following steps.

[0058] In step S11, an image is obtained after the calibration pattern is photographed and imaged by multiple cameras at a preset calibration distance.

[0059] In step S12, the focal length of the current camera to be calibrated among the multiple cameras is determined, and based on the focal length of the current camera to be calibrated, the corner points included in the captured image are extracted.

[0060] In step S13, the camera to be calibrated is calibrated based on the extracted corner points.

[0061] The calibration pattern involved in the embodiment of the present disclosure is a checkerboard array composed of at least two checkerboard patterns arranged in sequence. Each checkerboard pattern in the checkerboard array includes a border pattern and at least one center pattern co-centered with the border pattern, and in each checkerboard pattern, the size between two adjacent corner points of the center pattern is different from the size between two adjacent corner points of the border pattern.

[0062] In one embodiment, in at least one center pattern and border pattern of each checkerboard pattern, adjacent patterns have the same shape and different colors.

[0063] For example, the at least two checkerboard patterns include a first checkerboard pattern and a second checkerboard pattern; the first checkerboard pattern includes a center pattern of a first color and a border pattern of a second color; the second checkerboard pattern includes a center pattern of a second color and a border pattern of the first color.

[0064] It is understandable that the captured image obtained in the embodiment of the present disclosure is an image obtained by capturing and imaging the calibration pattern mentioned above. For any incomplete description of the calibration pattern in the embodiment of the present disclosure, please refer to the relevant description of the above embodiment.

[0065] In the embodiment of the present disclosure, when extracting the corner points included in the captured image, the focal length of the current camera to be calibrated among the multiple cameras may be determined, and then the corner points included in the captured image may be extracted based on the determined focal length.

[0066] In one implementation, one or more focal length thresholds and one or more field of view angle thresholds may be preset in the disclosed embodiment. According to the focal length interval of the current camera to be calibrated and the field of view angle interval, the corresponding corner points in the checkerboard pattern are determined to be extracted from the image after the image is captured;

[0067] In one example, a first focal length threshold and a second focal length threshold, a first field of view angle threshold and a second field of view angle threshold are set in an embodiment of the present disclosure.

[0068] Among them, in the embodiment of the present disclosure, the first focal length threshold and the second focal length threshold may be the same. The first field of view angle threshold and the second field of view angle threshold may be the same. In one embodiment, if the focal length of the current camera to be calibrated is greater than the first focal length threshold and the field of view angle is less than the first field of view angle threshold, the corner points of the border pattern are extracted in the image after the image is captured. In another embodiment, if the focal length of the current camera to be calibrated is less than the second focal length threshold and the field of view angle is greater than the second field of view angle threshold, the corner points of the center pattern are extracted in the image after the image is captured.

[0069] Further, in the embodiment of the present disclosure, the first focal length threshold and the second focal length threshold may also be different, and the first field of view angle threshold and the second field of view angle threshold may also be different. When the first focal length threshold and the second focal length threshold are different, and the first field of view angle threshold and the second field of view angle threshold are different, it can be applied to the case where each checkerboard pattern in the checkerboard array has a border pattern and at least two center patterns, and corner point extraction is performed. In one example, the at least two center patterns include a center pattern and a sub-center pattern. The border pattern, the sub-center pattern and the center pattern are nested in sequence, that is, the sub-center pattern is nested in the border pattern, and the center pattern is nested in the sub-center pattern. If the focal length of the current camera to be calibrated is greater than the first focal length threshold and the field of view angle is less than the first field of view angle threshold, the corner points of the border pattern are extracted in the image after the imaging is captured. If the focal length of the current camera to be calibrated is less than the first focal length threshold and greater than the second focal length threshold, and the field of view angle is greater than the first field of view angle threshold and less than the second field of view angle threshold, the corner points of the sub-center pattern are extracted in the image after the imaging is captured. If the focal length of the current camera to be calibrated is less than the second focal length threshold, and the field of view angle is greater than the second field of view angle threshold, then the corner points of the central pattern are extracted from the captured image.

[0070] In an example of the present disclosure, before extracting corner points from the captured image, the captured image may be preprocessed, for example, by adjusting the size and brightness of the image. Figure 2 As shown in the figure, when extracting corner points in the image after shooting, for the camera corner point detection with a long focal length and a small field of view, the first category of corner points with a first size between the corner points in the calibration pattern are extracted, so as to ensure that a sufficient number of corner point information is detected. For the camera corner point detection with a short focal length and a large field of view, the second category of corner points with a second size between the corner points in the calibration pattern are extracted, so as to ensure that the corner point intervals are not too dense to avoid misdetection.

[0071] In the disclosed embodiment, after the corresponding corner points are extracted for the cameras to be calibrated with different focal lengths, the image and corner point information can be used to perform distortion correction and dual-camera calibration.

[0072] The multi-camera calibration method provided by the embodiment of the present disclosure can calibrate a multi-camera terminal with different focal lengths, such as (a group of binocular cameras with different focal lengths), through a single calibration pattern. In this multi-camera calibration method, the calibration pattern is simple to make, and the calibration accuracy is improved while ensuring the calibration efficiency.

[0073] In an embodiment of the present disclosure, a calibration pattern is spread out on a plane, and a multi-camera terminal is used to photograph and image the calibration pattern at the calibration distance to obtain a photographed image; the multi-camera terminal with different focal lengths extracts different corner points of the image to calibrate the multi-camera module, thereby realizing a calibration method that is compatible with multiple cameras at multiple focal lengths. By optimizing the calibration pattern, the embodiment of the present disclosure can complete the calibration of multiple camera terminals with different focal lengths through a single calibration pattern, which helps production line workers to quickly master the dual-camera calibration process, and can quickly perform batch calibration, thereby improving the calibration effect while ensuring the calibration efficiency of the production line. Therefore, the problem of long time consumption and unsatisfactory effect when calibrating multiple camera terminals with different focal lengths in the related art is solved.

[0074] Based on the same concept, the embodiment of the present disclosure also provides a multi-camera calibration device.

[0075] It is understandable that in order to achieve the above functions, the multi-camera calibration device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of the examples disclosed in the embodiments of the present disclosure, the embodiments of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0076] Figure 4 is a block diagram of a multi-camera calibration device according to an exemplary embodiment. Figure 4 The multi-camera calibration device 100 includes an acquisition unit 101, an extraction unit 102 and a calibration unit 103.

[0077] The acquisition unit 101 is used to acquire an image after the calibration pattern is photographed and imaged by multiple cameras at a preset calibration distance, wherein the calibration pattern is a checkerboard array composed of at least two checkerboard patterns arranged in sequence, wherein each checkerboard pattern in the checkerboard array includes a border pattern and at least one center pattern nested with the border pattern at the same center, and in each checkerboard pattern, the size between two adjacent corner points of the center pattern is different from the size between two adjacent corner points of the border pattern.

[0078] The extraction unit 102 is used to determine the focal length of the current camera to be calibrated among the multiple cameras, and extract the corner points included in the captured image based on the focal length of the current camera to be calibrated.

[0079] The calibration unit 103 is used to calibrate the camera to be calibrated based on the extracted corner points.

[0080] In one embodiment, in at least one center pattern and border pattern of each checkerboard pattern, adjacent patterns have the same shape and different colors.

[0081] In one embodiment, the at least two checkerboard patterns include a first checkerboard pattern and a second checkerboard pattern. The first checkerboard pattern includes a center pattern of a first color and a border pattern of a second color. The second checkerboard pattern includes a center pattern of a second color and a border pattern of the first color.

[0082] In one embodiment, the extraction unit 102 extracts the corner points included in the captured image based on the focal length of the current camera to be calibrated in the following manner: if the focal length of the current camera to be calibrated is greater than a first focal length threshold and the field of view angle is less than a first field of view angle threshold, the corner points of the border pattern are extracted from the captured image.

[0083] In one embodiment, the following method is used to extract the corner points included in the captured image based on the focal length of the current camera to be calibrated: if the focal length of the current camera to be calibrated is less than a second focal length threshold and the field of view angle is greater than a second field of view angle threshold, the extraction unit 102 extracts the corner points of the central pattern in the captured image.

[0084] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0085] Figure 5 1 is a block diagram of a device 200 for multi-camera calibration according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0086] Reference Figure 5 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0087] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0088] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0089] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0090] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0091] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0092] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0093] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0094] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0095] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0096] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0097] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0098] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0099] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0100] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0101] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A multi-camera calibration method, characterized in that: include: Acquire an image after the calibration pattern is photographed and imaged by multiple cameras at a preset calibration distance, wherein the calibration pattern is a checkerboard array composed of at least two checkerboard patterns arranged in sequence, wherein each checkerboard pattern in the checkerboard array includes a frame pattern and at least one center pattern co-centeredly nested with the frame pattern, and in each checkerboard pattern, a size between two adjacent corner points of the center pattern is different from a size between two adjacent corner points of the frame pattern; Determine the focal length of the current camera to be calibrated among the multiple cameras, and extract corner points included in the captured image based on the focal length of the current camera to be calibrated; Based on the extracted corner points, the camera to be calibrated is calibrated.

2. The multi-camera calibration method according to claim 1, characterized in that: In the at least one central pattern and the border pattern of each checkerboard pattern, adjacent patterns have the same shape and different colors.

3. The multi-camera calibration method according to any one of claims 1 to 2, characterized in that: The at least two checkerboard patterns include a first checkerboard pattern and a second checkerboard pattern; The first checkerboard pattern includes a center pattern of a first color and a border pattern of a second color; The second checkerboard pattern includes a center pattern of a second color and a border pattern of a first color.

4. The multi-camera calibration method according to claim 1, characterized in that: Extracting corner points included in the captured image based on the focal length of the current camera to be calibrated, including: If the focal length of the current camera to be calibrated is greater than a first focal length threshold, and the field of view angle is less than a first field of view angle threshold, then the corner points of the frame pattern are extracted from the captured image.

5. The multi-camera calibration method according to claim 1, characterized in that: Extracting corner points included in the captured image based on the focal length of the current camera to be calibrated, including: If the focal length of the current camera to be calibrated is less than the second focal length threshold, and the field of view angle is greater than the second field of view angle threshold, then the corner points of the central pattern are extracted from the captured image.

6. A multi-camera calibration device, characterized in that: include: an acquisition unit, configured to acquire an image after the calibration pattern is photographed and imaged by multiple cameras at a preset calibration distance, wherein the calibration pattern is a checkerboard array composed of at least two checkerboard patterns arranged in sequence, wherein each checkerboard pattern in the checkerboard array includes a frame pattern and at least one center pattern co-centeredly nested with the frame pattern, and in each checkerboard pattern, a size between two adjacent corner points of the center pattern is different from a size between two adjacent corner points of the frame pattern; An extraction unit, used to determine the focal length of the current camera to be calibrated among the multiple cameras, and extract corner points included in the captured image based on the focal length of the current camera to be calibrated; The calibration unit is used to calibrate the camera to be calibrated based on the extracted corner points.

7. The multi-camera calibration device according to claim 6, characterized in that: In the at least one central pattern and the border pattern of each checkerboard pattern, adjacent patterns have the same shape and different colors.

8. The multi-camera calibration device according to any one of claims 6 to 7, characterized in that: The at least two checkerboard patterns include a first checkerboard pattern and a second checkerboard pattern; The first checkerboard pattern includes a center pattern of a first color and a border pattern of a second color; The second checkerboard pattern includes a center pattern of a second color and a border pattern of a first color.

9. The multi-camera calibration device according to claim 6, characterized in that: The extraction unit extracts corner points included in the captured image based on the focal length of the current camera to be calibrated in the following manner: If the focal length of the current camera to be calibrated is greater than a first focal length threshold, and the field of view angle is less than a first field of view angle threshold, then the corner points of the frame pattern are extracted from the captured image.

10. The multi-camera calibration device according to claim 6, characterized in that: The extraction unit extracts corner points included in the captured image based on the focal length of the current camera to be calibrated in the following manner: If the focal length of the current camera to be calibrated is less than the second focal length threshold, and the field of view angle is greater than the second field of view angle threshold, then the corner points of the central pattern are extracted from the captured image.

11. A multi-camera calibration device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the multi-camera calibration method described in any one of claims 1 to 5.

12. A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to execute the multi-camera calibration method described in any one of claims 1 to 5.

Images