A camera calibration method, apparatus, and readable storage medium

Abstract

This disclosure relates to a camera calibration method, apparatus, and storage medium. The calibration method is applied to a binocular camera and includes: performing stereo alignment on a first initial image captured by a first camera and a second initial image captured by a second camera based on preset calibration parameters to obtain a first corrected image and a second corrected image; obtaining a first estimated intrinsic parameter of the first camera and a second estimated intrinsic parameter of the second camera through vanishing point detection based on the first and second corrected images, and performing feature matching and homography estimation on the first and second corrected images to obtain an estimation result; determining a first target intrinsic parameter of the first camera and a second target intrinsic parameter of the second camera based on the first estimated intrinsic parameter, the second estimated intrinsic parameter, and the preset calibration parameters; and obtaining target extrinsic parameters between the first camera and the second camera based on the first target intrinsic parameter, the second target intrinsic parameter, and the homography estimation.

Description

Technical Field

[0001] This disclosure relates to the field of camera technology, and in particular to a camera calibration method, apparatus, and readable storage medium. Background Technology

[0002] RGB-D cameras are a combination of RGB cameras and depth cameras, and they have wide applications in human-computer interaction, visual positioning, 3D reconstruction, and visual navigation, such as SLAM, AR, VR, assisted driving, and autonomous driving. In related technologies, the real-time optimization scheme for the relative pose (extrinsic parameters) of RGB-D cameras is based on offline calibration data, without considering changes during the shooting process. Furthermore, during shooting, technologies such as autofocus and image stabilization can cause changes in the intrinsic parameters of the RGB-D camera. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a camera calibration method, apparatus, and readable storage medium.

[0004] According to a first aspect of the present disclosure, a camera calibration method is provided, applied to a binocular camera, the calibration method comprising:

[0005] Based on preset calibration parameters, the first initial image captured by the first camera and the second initial image captured by the second camera are stereo aligned to obtain the first corrected image and the second corrected image;

[0006] Based on the first and second corrected images, the first estimated intrinsic parameters of the first camera and the second estimated intrinsic parameters of the second camera are obtained through vanishing point detection. Feature matching and homography estimation are performed on the first and second corrected images to obtain the estimation results.

[0007] Based on the first estimated intrinsic parameters, the second estimated intrinsic parameters, and the preset calibration parameters, the first target intrinsic parameters of the first camera and the second target intrinsic parameters of the second camera are determined.

[0008] Based on the first target intrinsic parameter, the second target intrinsic parameter, and the estimation result, the target extrinsic parameter between the first camera and the second camera is obtained.

[0009] In an exemplary embodiment, the step of obtaining the first estimated intrinsic parameters of the first camera and the second estimated intrinsic parameters of the second camera by vanishing point detection based on the first corrected image and the second corrected image includes: performing vanishing point detection on the first corrected image to obtain the first estimated intrinsic parameters of the first camera, and performing vanishing point detection on the second corrected image to obtain the second estimated intrinsic parameters of the second camera.

[0010] In an exemplary embodiment, the preset calibration parameters include: a first preset calibration intrinsic parameter of the first camera, a second preset calibration intrinsic parameter of the second camera, and a preset calibration extrinsic parameter between the first camera and the second camera.

[0011] In an exemplary embodiment, determining the first target intrinsic parameter of the first camera and the second target intrinsic parameter of the second camera based on the first estimated intrinsic parameter, the second estimated intrinsic parameter, and the preset calibration parameters includes:

[0012] Based on the first preset variation range corresponding to the first preset calibration intrinsic parameter and the first estimated intrinsic parameter, the first target intrinsic parameter of the first camera is determined;

[0013] The second target intrinsic parameter of the second camera is determined based on the second preset variation range corresponding to the second preset calibration intrinsic parameter and the second estimated intrinsic parameter.

[0014] In an exemplary embodiment, determining the first target intrinsic parameter of the first camera based on the first preset variation range of the first preset calibration intrinsic parameter and the first estimated intrinsic parameter includes:

[0015] If the first estimated intrinsic parameter is within the first preset range of change, the first estimated intrinsic parameter is determined to be the first target intrinsic parameter;

[0016] If the first estimated intrinsic parameter is not within the first preset range of change, the first preset intrinsic parameter is determined to be the first target intrinsic parameter.

[0017] In an exemplary embodiment, determining the second target intrinsic parameter of the second camera based on the second preset variation range of the second preset intrinsic parameter and the second estimated intrinsic parameter includes:

[0018] If the second estimated intrinsic parameter is within the second preset range of variation, the second estimated intrinsic parameter is determined to be the second target intrinsic parameter;

[0019] If the second estimated intrinsic parameter is not within the second preset range of change, the second preset intrinsic parameter is determined to be the second target intrinsic parameter.

[0020] In one exemplary embodiment, the first camera is an RGB camera and the second camera is a depth camera.

[0021] According to a second aspect of the present disclosure, a camera calibration device is provided, applied to a binocular camera, the calibration device comprising:

[0022] The correction module is configured to perform stereo alignment on a first initial image captured by a first camera and a second initial image captured by a second camera based on preset calibration parameters to obtain a first corrected image and a second corrected image.

[0023] The estimation module is configured to obtain a first estimated intrinsic parameter of the first camera and a second estimated intrinsic parameter of the second camera by vanishing point detection based on the first and second corrected images, and to perform feature matching and homography estimation on the first and second corrected images to obtain the estimation result.

[0024] The first determining module is configured to determine the first target intrinsic parameter of the first camera and the second target intrinsic parameter of the second camera based on the first estimated intrinsic parameter, the second estimated intrinsic parameter and the preset calibration parameter;

[0025] The second determining module is configured to determine the target extrinsic parameters between the first camera and the second camera based on the first target intrinsic parameters, the second target intrinsic parameters, and the estimation results.

[0026] In one exemplary embodiment, the first camera is an RGB camera and the second camera is a depth camera.

[0027] According to a third aspect of the present disclosure, a camera calibration apparatus is provided, characterized in that it comprises:

[0028] processor;

[0029] Memory used to store processor-executable instructions;

[0030] The processor is configured to perform a camera calibration method as described in any of the first aspects of the embodiments of this disclosure.

[0031] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the readable storage medium are executed by a processor of a device, the device is enabled to perform any of the methods described in the first aspect of the present disclosure.

[0032] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: by vanishing point detection and homography estimation, the changes in intrinsic and extrinsic parameters caused by image stabilization and other technologies during the shooting process of the binocular camera are obtained, thereby optimizing the intrinsic and extrinsic parameters of the binocular camera and effectively improving the calibration accuracy.

[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0034] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0035] Figure 1 This is a flowchart illustrating a camera calibration method according to an exemplary embodiment.

[0036] Figure 2 This is a flowchart illustrating a camera calibration method according to an exemplary embodiment.

[0037] Figure 3 This is a block diagram of a camera calibration device according to an exemplary embodiment.

[0038] Figure 4 This is a block diagram illustrating a camera calibration device according to an exemplary embodiment. Detailed Implementation

[0039] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0040] In an exemplary embodiment of this disclosure, a camera calibration method is provided, applied to a binocular camera. Figure 1 This is a flowchart illustrating a camera calibration method according to an exemplary embodiment, such as... Figure 1 As shown, the calibration method includes steps S101-S104:

[0041] Step S101: Based on preset calibration parameters, perform stereo alignment on the first initial image captured by the first camera and the second initial image captured by the second camera to obtain the first corrected image and the second corrected image.

[0042] Step S102: Based on the first corrected image and the second corrected image, the first estimated intrinsic parameters of the first camera and the second estimated intrinsic parameters of the second camera are obtained through vanishing point detection, and feature matching and homography estimation are performed on the first corrected image and the second corrected image to obtain the estimation result;

[0043] Step S103: Based on the first estimated intrinsic parameter, the second estimated intrinsic parameter, and the preset calibration parameters, determine the first target intrinsic parameter of the first camera and the second target intrinsic parameter of the second camera;

[0044] Step S104: Based on the first target intrinsic parameters, the second target intrinsic parameters, and the estimation results, the target extrinsic parameters between the first camera and the second camera are obtained.

[0045] In this embodiment, the binocular camera is an RGB-D camera, which consists of an RGB camera and a depth camera.

[0046] In step S101, when the first camera is an RGB camera and the second camera is a depth camera, the first initial image captured by the first camera is the initial RGB image, denoted as I. 0 r The second initial image captured by the second camera is the initial depth image, denoted as I. 0 d Stereo alignment is performed on the initial RGB image and initial depth image. For example, stereo correction is used to correct the initial RGB image and initial depth image, which are not coplanarly aligned, to coplanar alignment, resulting in the first corrected image and the second corrected image, i.e., the corrected RGB image and the corrected depth image.

[0047] The preset calibration parameters are offline calibration parameters, obtained through Zhang Zhengyou's stereo calibration method. Offline calibration is performed during the camera's production process to calibrate the camera's intrinsic and extrinsic parameters; therefore, the offline calibration parameters include both the camera's intrinsic and extrinsic parameters.

[0048] In some implementations, the preset calibration parameters include: a first preset calibration intrinsic parameter of the first camera, a second preset calibration intrinsic parameter of the second camera, and a preset calibration extrinsic parameter between the first camera and the second camera. When the first camera is an RGB camera, the first preset calibration intrinsic parameter is the offline calibration intrinsic parameter of the RGB camera, denoted as C. 0 r When the second camera is a depth camera, the second preset calibration intrinsic parameter is the offline calibration intrinsic parameter of the depth camera, denoted as C. 0 d The preset calibration extrinsic parameters are the offline calibration extrinsic parameters between the RGB camera and the depth camera, including rotation and offset, denoted as E. 0 r&d .

[0049] In some possible implementations, step S102, based on the first and second corrected images, obtains the first estimated intrinsic parameters of the first camera and the second estimated intrinsic parameters of the second camera through vanishing point detection, including: performing vanishing point detection on the first corrected image to obtain the first estimated intrinsic parameters of the first camera, and performing vanishing point detection on the second corrected image to obtain the second estimated intrinsic parameters of the second camera.

[0050] The main application of vanishing point detection is to obtain the intrinsic parameters of the camera through images; its essence is a clustering algorithm. Vanishing point detection is performed on both the corrected RGB image and the corrected depth image to obtain the intrinsic parameters of the RGB camera and the depth camera, respectively. Specifically, vanishing point detection on the corrected RGB image yields the intrinsic parameters of the RGB camera, and vanishing point detection on the corrected depth image yields the intrinsic parameters of the depth camera. These intrinsic parameters are inferred from the corrected image using an algorithm, and are therefore the estimated intrinsic parameters of the RGB camera and the depth camera, denoted as C, respectively. 1 r and C 1 d When performing vanishing point detection on a depth image, the depth image can be converted to a grayscale image before performing vanishing point detection on the grayscale image.

[0051] To subsequently derive the extrinsic parameters between the two cameras based on the camera's intrinsic parameters, homography estimation is performed on the first and second corrected images. First, feature points, including SIFT, SURF, SAFT, and ORB feature points, are extracted from the corrected RGB and corrected depth images. By matching feature points, matching feature point pairs are found in the corrected RGB and corrected depth images. Homography estimation is then performed using the feature point coordinates to obtain the estimation result, which is the homography matrix. The homography matrix is ​​the transformation matrix between two images, that is, the transformation matrix that maps a point in one image to the corresponding point in another image, denoted as H.

[0052] In step S103, since the estimated intrinsic parameters are online calibration parameters obtained through images, it is necessary to make a reasonable judgment on them based on the preset calibration parameters. Therefore, the first target intrinsic parameter is determined according to the first estimated intrinsic parameter and the preset calibration parameters, and the second target intrinsic parameter is determined according to the second estimated intrinsic parameter and the preset calibration parameters.

[0053] In some implementations, the first target intrinsic parameter of the first camera is determined based on the first preset variation range and the first estimated intrinsic parameter corresponding to the first preset calibration intrinsic parameter; the second target intrinsic parameter of the second camera is determined based on the second preset variation range and the second estimated intrinsic parameter corresponding to the second preset calibration intrinsic parameter.

[0054] When assembling the first and second cameras to form a binocular camera, there are certain tolerances in the holes and shafts used for assembly and mating between the two camera hardware components. These tolerances, known as allowable clearances or interference variations, are called manufacturing assembly tolerances. The preset variation range is the manufacturing assembly tolerance, and the first preset variation is denoted as C. x r Then the first preset range of change is [C 0 r +Cx r C 0 r-Cx r The second preset change is denoted as Cx. d Then the second preset range of variation is [C 0 d +Cx d C 0 d -Cx d ].

[0055] In some implementations, when determining the first target intrinsic parameter of the first camera, if the first estimated intrinsic parameter is within a first preset range of variation, the first estimated intrinsic parameter is determined as the first target intrinsic parameter; if the first estimated intrinsic parameter is not within the first preset range of variation, the first preset intrinsic parameter is determined as the first target intrinsic parameter.

[0056] In one example, the first calibration intrinsic parameter is C. 0 r The first estimated internal reference is C. 1 r If C is satisfied 0 r +C x r <C 1 r <C 0 r -C x r Then the first target intrinsic parameter is C. 1 r If C is not satisfied 0 r +C x r <C 1 r <C 0 r -C x r Then the first target intrinsic parameter is C. 0 r .

[0057] In some implementations, when determining the second target intrinsic parameter of the second camera, if the second estimated intrinsic parameter is within a second preset range of change, the second estimated intrinsic parameter is determined as the second target intrinsic parameter; if the second estimated intrinsic parameter is not within the second preset range of change, the second preset intrinsic parameter is determined as the second target intrinsic parameter.

[0058] In one example, the second calibration intrinsic parameter is C. 0 d The first estimated internal reference is C. 1 d If C is satisfied 0 d +C x d<C 1 d <C 0 d -C x d Then the first target intrinsic parameter is C. 1 d If C is not satisfied 0 d +C x d <C 1 d <C 0 d -C x d Then the first target intrinsic parameter is C. 0 d .

[0059] In step S104, based on the first target intrinsic parameter, the second target intrinsic parameter, and the estimation result, the target extrinsic parameter between the first camera and the second camera is obtained, including: separating the intrinsic and extrinsic parameters of the estimation result based on the first target intrinsic parameter and the second target intrinsic parameter to obtain the target extrinsic parameter between the first camera and the second camera, denoted as E. 1 r&d .

[0060] In the exemplary embodiments of this disclosure, by vanishing point detection and homography estimation, the changes in intrinsic and extrinsic parameters caused by image stabilization and other technologies during the shooting process of the binocular camera are obtained, thereby optimizing the intrinsic and extrinsic parameters of the binocular camera and effectively improving the calibration accuracy.

[0061] In addition, applying assembly tolerances to the calibration process increases reliability and speeds up calibration.

[0062] This disclosure provides a camera calibration method applied to an RGB-D camera. Figure 2 This is a flowchart illustrating a camera calibration method according to an exemplary embodiment, such as... Figure 2 As shown, the calibration method includes steps S201-S206:

[0063] Step S201: Obtain preset calibration parameters;

[0064] The preset calibration parameters are offline calibration parameters, including: the offline calibration intrinsic parameter C of the RGB camera. 0 r Offline calibration intrinsics C of depth cameras 0 d Offline calibration extrinsic parameter E between RGB camera and depth camera 0 r&d .

[0065] Step S202, the initial RGB image I captured by the RGB camera 0 r Initial depth image I captured by the depth camera 0 d ;

[0066] Step S203: Based on preset calibration parameters, perform stereo alignment on the initial RGB image captured by the RGB camera and the initial depth image captured by the depth camera to obtain the corrected RGB image I. 1 r and corrected depth image I 1 d ;

[0067] Step S204: Perform vanishing point detection on the corrected RGB image to obtain the estimated intrinsic parameters C of the RGB camera. 1 r Vanishing point detection is performed on the corrected depth image to obtain the estimated intrinsic parameters C of the depth camera. 1 d Furthermore, feature matching and homography estimation are performed on the corrected RGB image and the corrected depth image to obtain the estimation result H;

[0068] Step S205: Determine the target intrinsic parameters of the RGB camera based on the preset intrinsic parameter variation range corresponding to the offline calibration intrinsic parameters of the RGB camera and the estimated intrinsic parameters of the RGB camera.

[0069] The preset intrinsic parameter variation range corresponding to the offline calibration intrinsic parameters of the RGB camera is [C]. 0 r +C x r C 0 r -C x r If the estimated intrinsic parameter C of the RGB camera 1 r Satisfy C 0 r +C x r <C 1 r <C 0 r -C x r The target intrinsic parameter of the RGB camera is determined to be C. 1 r If not satisfied, determine the target intrinsic parameter of the RGB camera as C. 0 r .

[0070] Step S206: Determine the target intrinsic parameters of the depth camera based on the preset intrinsic parameter variation range corresponding to the offline calibration intrinsic parameters of the depth camera and the estimated intrinsic parameters of the depth camera.

[0071] The preset intrinsic parameter variation range corresponding to the offline calibration intrinsic parameters of the depth camera is [C]. 0 d +C x d C 0 d -C x d If the estimated intrinsic parameter C of the depth camera 1 d Satisfy C 0 d +C x d <C 1 d <C 0 d -C x d The target intrinsic parameter of the depth camera is determined to be C. 1 d If not satisfied, determine the target intrinsic parameter of the depth camera as C. 0 d .

[0072] Steps S205 and S206 are the internal parameter rationality judgment process. There is no requirement for the order of execution; they can be performed simultaneously or sequentially.

[0073] Step S207: Based on the target intrinsic parameters of the RGB camera, the target intrinsic parameters of the depth camera, and the estimation result H, the target extrinsic parameters E between the RGB camera and the depth camera are obtained. 1 r&d .

[0074] In an exemplary embodiment of this disclosure, a camera calibration device is provided, which is applied to a binocular camera. Figure 3 This is a block diagram illustrating a camera calibration device according to an exemplary embodiment, such as... Figure 3 As shown, the calibration device includes:

[0075] The correction module 301 is configured to perform stereo alignment on a first initial image captured by a first camera and a second initial image captured by a second camera based on preset calibration parameters to obtain a first corrected image and a second corrected image.

[0076] The estimation module 302 is configured to obtain a first estimated intrinsic parameter of the first camera and a second estimated intrinsic parameter of the second camera by vanishing point detection based on the first corrected image and the second corrected image, and to perform feature matching and homography estimation on the first corrected image and the second corrected image to obtain the estimation result;

[0077] The first determining module 303 is configured to determine the first target intrinsic parameter of the first camera and the second target intrinsic parameter of the second camera based on the first estimated intrinsic parameter, the second estimated intrinsic parameter and the preset calibration parameter;

[0078] The second determining module 304 is configured to determine the target extrinsic parameters between the first camera and the second camera based on the first target intrinsic parameters, the second target intrinsic parameters, and the estimation results.

[0079] In an exemplary embodiment, the estimation module 302 is further configured to perform vanishing point detection on the first corrected image to obtain a first estimated intrinsic parameter of the first camera, and to perform vanishing point detection on the second corrected image to obtain a second estimated intrinsic parameter of the second camera.

[0080] In an exemplary embodiment, the preset calibration parameters include: a first preset calibration intrinsic parameter of the first camera, a second preset calibration intrinsic parameter of the second camera, and a preset calibration extrinsic parameter between the first camera and the second camera.

[0081] In an exemplary embodiment, the first determining module 303 is further configured to:

[0082] Based on the first preset variation range corresponding to the first preset calibration intrinsic parameter and the first estimated intrinsic parameter, the first target intrinsic parameter of the first camera is determined;

[0083] The second target intrinsic parameter of the second camera is determined based on the second preset variation range corresponding to the second preset calibration intrinsic parameter and the second estimated intrinsic parameter.

[0084] In an exemplary embodiment, the first determining module 303 is further configured to:

[0085] If the first estimated intrinsic parameter is within the first preset range of change, the first estimated intrinsic parameter is determined to be the first target intrinsic parameter;

[0086] If the first estimated intrinsic parameter is not within the first preset range of change, the first preset intrinsic parameter is determined to be the first target intrinsic parameter.

[0087] In an exemplary embodiment, the first determining module 303 is further configured to:

[0088] If the second estimated intrinsic parameter is within the second preset range of variation, the second estimated intrinsic parameter is determined to be the second target intrinsic parameter;

[0089] If the second estimated intrinsic parameter is not within the second preset range of change, the second preset intrinsic parameter is determined to be the second target intrinsic parameter.

[0090] In one exemplary embodiment, the first camera is an RGB camera and the second camera is a depth camera.

[0091] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0092] When the camera's calibration device is a terminal, Figure 4 This is a block diagram illustrating a camera calibration device 400 according to an exemplary embodiment. For example, device 400 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0093] Reference Figure 4 The device 400 may include one or more of the following components: a processing component 402, a memory 404, a power component 406, a multimedia component 408, an audio component 410, an input / output (I / O) interface 412, a sensor component 414, and a communication component 416.

[0094] Processing component 402 typically controls the overall operation of device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0095] Memory 404 is configured to store various types of data to support the operation of device 400. Examples of this data include instructions for any application or method operating on device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0096] Power supply component 406 provides power to various components of device 400. Power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 400.

[0097] Multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0098] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0099] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0100] Sensor assembly 414 includes one or more sensors for providing status assessments of various aspects of device 400. For example, sensor assembly 414 may detect the on / off state of device 400, the relative positioning of components such as the display and keypad of device 400, changes in the position of device 400 or a component of device 400, the presence or absence of user contact with device 400, the orientation or acceleration / deceleration of device 400, and temperature changes of device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0101] Communication component 416 is configured to facilitate wired or wireless communication between device 400 and other devices. Device 400 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0102] In an exemplary embodiment, the apparatus 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0103] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, which can be executed by a processor 420 of the device 400 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0104] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a device, enables the device to perform a camera calibration method, the method comprising any of the methods described above.

[0105] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0106] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0107] Industrial applicability

[0108] This paper uses vanishing point detection and homography estimation to determine the changes in intrinsic and extrinsic parameters of a binocular camera during shooting due to techniques such as image stabilization. This allows for the optimization of the binocular camera's intrinsic and extrinsic parameters, effectively improving calibration accuracy. Furthermore, applying assembly tolerances to the calibration process increases reliability while also improving calibration speed.

Claims

1. A method for calibrating a camera, applied to a binocular camera, comprising the steps of: The calibration method includes: Based on preset calibration parameters, the first initial image captured by the first camera and the second initial image captured by the second camera are stereo aligned to obtain the first corrected image and the second corrected image; Based on the first and second corrected images, the first estimated intrinsic parameters of the first camera and the second estimated intrinsic parameters of the second camera are obtained through vanishing point detection. Feature matching and homography estimation are performed on the first and second corrected images to obtain the estimation results. Based on the first estimated intrinsic parameters, the second estimated intrinsic parameters, and the preset calibration parameters, the first target intrinsic parameters of the first camera and the second target intrinsic parameters of the second camera are determined. Based on the first target intrinsic parameter, the second target intrinsic parameter, and the estimation result, the target extrinsic parameter between the first camera and the second camera is determined.

2. The camera calibration method of claim 1, wherein, The step of obtaining the first estimated intrinsic parameters of the first camera and the second estimated intrinsic parameters of the second camera through vanishing point detection based on the first and second corrected images includes: Vanishing point detection is performed on the first corrected image to obtain the first estimated intrinsic parameters of the first camera, and vanishing point detection is performed on the second corrected image to obtain the second estimated intrinsic parameters of the second camera.

3. The camera calibration method of claim 1, wherein, The preset calibration parameters include: the first preset calibration intrinsic parameter of the first camera, the second preset calibration intrinsic parameter of the second camera, and the preset calibration extrinsic parameter between the first camera and the second camera.

4. The camera calibration method according to claim 3, characterized in that, The step of determining the first target intrinsic parameter of the first camera and the second target intrinsic parameter of the second camera based on the first estimated intrinsic parameter, the second estimated intrinsic parameter, and the preset calibration parameters includes: Based on the first preset variation range corresponding to the first preset calibration intrinsic parameter and the first estimated intrinsic parameter, the first target intrinsic parameter of the first camera is determined; The second target intrinsic parameter of the second camera is determined based on the second preset variation range corresponding to the second preset calibration intrinsic parameter and the second estimated intrinsic parameter.

5. The camera calibration method according to claim 4, characterized in that, The step of determining the first target intrinsic parameter of the first camera based on the first preset variation range of the first preset calibration intrinsic parameter and the first estimated intrinsic parameter includes: If the first estimated intrinsic parameter is within the first preset range of change, the first estimated intrinsic parameter is determined to be the first target intrinsic parameter; If the first estimated intrinsic parameter is not within the first preset range of change, the first preset calibration intrinsic parameter is determined to be the first target intrinsic parameter.

6. The camera calibration method according to claim 4, characterized in that, The step of determining the second target intrinsic parameter of the second camera based on the second preset variation range of the second preset calibration intrinsic parameter and the second estimated intrinsic parameter includes: If the second estimated intrinsic parameter is within the second preset range of variation, the second estimated intrinsic parameter is determined to be the second target intrinsic parameter; If the second estimated intrinsic parameter is not within the second preset range of change, the second preset calibration intrinsic parameter is determined as the second target intrinsic parameter.

7. The camera calibration method according to claim 1, characterized in that, The first camera is an RGB camera, and the second camera is a depth camera.

8. A calibration device for a camera, applied to a binocular camera, characterized in that, The calibration device includes: The correction module is configured to perform stereo alignment on a first initial image captured by a first camera and a second initial image captured by a second camera based on preset calibration parameters to obtain a first corrected image and a second corrected image. The estimation module is configured to obtain a first estimated intrinsic parameter of the first camera and a second estimated intrinsic parameter of the second camera by vanishing point detection based on the first and second corrected images, and to perform feature matching and homography estimation on the first and second corrected images to obtain the estimation result. The first determining module is configured to determine the first target intrinsic parameter of the first camera and the second target intrinsic parameter of the second camera based on the first estimated intrinsic parameter, the second estimated intrinsic parameter and the preset calibration parameter; The second determining module is configured to determine the target extrinsic parameters between the first camera and the second camera based on the first target intrinsic parameters, the second target intrinsic parameters, and the estimation results.

9. A calibration device for a camera, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the camera calibration method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a device, enable the device to perform a camera calibration method as described in any one of claims 1-7.

Images