Camera extrinsic parameter calibration method, device, electronic device and storage medium

Abstract

The embodiments of the present disclosure disclose a camera extrinsic parameter calibration method, device, electronic device and storage medium, wherein the method includes: obtaining calibration images of a preset calibration plate taken by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different; identifying calibration feature points in each calibration image, and determining the coordinate information of calibration feature points in other calibration images other than the calibration image itself in each calibration image based on the identification of the feature points in the preset calibration plate; determining the extrinsic parameters of each camera to be calibrated based on the coordinate conversion relationship between the identified calibration feature points in each calibration image. The technical solution of the embodiments of the present disclosure solves the problem that the extrinsic parameter calibration of multiple cameras without common viewing conditions cannot be completed efficiently and at low cost, and provides a method for determining the extrinsic parameters of multiple cameras without common viewing conditions efficiently and without the need to incur additional costs for maintaining calibration objects.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of camera imaging and calibration technology, and in particular to a camera extrinsic parameter calibration method, device, electronic device, and storage medium. Background Art

[0002] Depth cameras based on methods such as structured light and time-of-flight (ToF) are widely used in various scenarios. Due to the limitations of their ranging methods, these depth cameras often have a limited field of view (FoV). In applications requiring a wide field of view, multiple cameras are often used to cover this wide field of view. Therefore, calibration of multiple cameras is necessary to determine the positional relationship between them, known as extrinsic parameters, which are used by subsequent algorithms.

[0003] However, when there is no common field of view between multiple camera modules, that is, when the FoVs of multiple cameras do not overlap sufficiently, conventional extrinsic calibration methods cannot be used for extrinsic calibration. If three-dimensional calibration objects are used for camera extrinsic calibration, the production and maintenance costs of the calibration objects are high. Therefore, existing calibration technologies cannot efficiently and cost-effectively complete the calibration of multi-camera extrinsic parameters without common view conditions. Summary of the Invention

[0004] The embodiments of the present disclosure provide a camera extrinsic parameter calibration method, apparatus, electronic device, and storage medium. These methods can determine the extrinsic parameters of multiple cameras that do not share a common viewing angle by performing coordinate conversion and analysis on feature points in an image of a preset calibration plate acquired by multiple cameras to be calibrated. This method is efficient and does not require the additional cost of maintaining calibration objects.

[0005] In a first aspect, an embodiment of the present disclosure provides a camera extrinsic parameter calibration method, comprising:

[0006] Acquire calibration images of a preset calibration plate captured by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different;

[0007] Identifying calibration feature points in each calibration image, and determining, based on the identifiers of the feature points in the preset calibration plate, coordinate information of calibration feature points in other calibration images other than the calibration image itself in each calibration image;

[0008] The extrinsic parameters of each camera to be calibrated are determined according to the coordinate transformation relationship between the identified calibration feature points and the calibration images.

[0009] In a second aspect, an embodiment of the present disclosure further provides a camera extrinsic parameter calibration device, comprising:

[0010] a calibration image acquisition module, configured to acquire calibration images of a preset calibration plate taken by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different;

[0011] a calibration feature point determination module, configured to identify calibration feature points in each calibration image and determine, based on the identifiers of the feature points in the preset calibration plate, the coordinate information of the calibration feature points in other calibration images other than the calibration image itself;

[0012] The extrinsic parameter calibration module is used to determine the extrinsic parameters of each camera to be calibrated based on the coordinate transformation relationship between the identified calibration feature points and the calibration images.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the camera extrinsic parameter calibration method as described in any one of the embodiments of the present disclosure.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute the camera extrinsic parameter calibration method as described in any one of the embodiments of the present disclosure.

[0018] The technical solution of the embodiment of the present disclosure obtains calibration images of a preset calibration plate taken by multiple cameras to be calibrated with different perspectives; further identifies calibration feature points in each calibration image, and determines the coordinate information of calibration feature points in other calibration images other than the calibration image itself in each calibration image based on the identification of the feature points in the preset calibration plate; then, determines the extrinsic parameters of each camera to be calibrated based on the coordinate conversion relationship between the identified calibration feature points in each of the calibration images. The technical solution of the embodiment of the present disclosure solves the problem that existing calibration technologies cannot efficiently and cost-effectively complete the calibration of multiple cameras without common viewing conditions. By performing coordinate conversion and analysis on the feature points in the preset calibration plate images obtained by the multiple cameras to be calibrated, a method is provided to determine the extrinsic parameters of multiple cameras without common viewing conditions in an efficient manner without the need for additional cost to maintain calibration objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A schematic diagram of a flow chart of a camera extrinsic parameter calibration method provided in the first embodiment of the present disclosure;

[0021] Figure 2 A schematic diagram of a camera capturing a calibration image provided in the first embodiment of the present disclosure;

[0022] Figure 3 A schematic flow chart of a camera extrinsic calibration method provided in the second embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram of a calibration plate provided in the second embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of the structure of a camera extrinsic calibration device provided in the third embodiment of the present disclosure;

[0025] Figure 6 This is a schematic diagram of the structure of an electronic device provided in the fourth embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0027] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0028] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0029] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0030] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0031] Example 1

[0032] Figure 1 This is a flowchart of a camera extrinsic calibration method provided in the first embodiment of the present disclosure. This embodiment of the present disclosure is applicable to scenarios where camera extrinsic parameters need to be calibrated, and is particularly applicable to extrinsic calibration of multiple groups of cameras that do not share a common viewing angle or have a small overlapping viewing angle. This method can be performed by a camera extrinsic calibration device, which can be implemented in software and / or hardware and configured in an electronic device, such as a mobile terminal, server device, or camera control device.

[0033] like Figure 1 As shown, the camera extrinsic parameter calibration method provided in this embodiment includes:

[0034] S110: Acquire calibration images of a preset calibration plate captured by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different.

[0035] Specifically, the multiple cameras to be calibrated can be a combination of camera modules configured to achieve a wider viewing angle. By stitching the images captured simultaneously by these multiple camera modules, a target image with a sufficiently wide viewing angle can be obtained. Therefore, a prerequisite for obtaining the target image is to calibrate the multiple camera modules and determine the positional relationship between the cameras, i.e., the extrinsic parameters, which are then used for relevant parameters in subsequent image processing.

[0036] During camera calibration, each camera first captures an image of a pre-set calibration plate, which serves as the calibration image. The pre-set calibration plate contains easily identifiable feature points, and the positional relationships between these feature points are known. Calibration images captured by each camera under calibration, with different viewing angles, will contain different feature points.

[0037] For example, Figure 2 As shown in the diagram of the calibration image captured by the camera, there are two cameras with different fields of view (FoVs) (Camera 1 and Camera 2), and a calibration plate with four feature points. Within the respective field of view of Camera 1 and Camera 2, the calibration images captured will have different feature points.

[0038] S120: Identify calibration feature points in each calibration image, and determine, in each calibration image, coordinate information of calibration feature points in other calibration images except the calibration image itself according to the identifiers of the feature points in the preset calibration plate.

[0039] Among them, the calibration point is another feature point in the calibration image, and the feature point can be a preset identifier, a coded graphic or a text. Figure 2 For example, in the calibration image taken by camera 1, calibration feature points including feature point 1 and feature point 2 can be identified, and in the calibration image taken by camera 2, calibration feature points including feature point 3 and feature point 4 can be identified.

[0040] Furthermore, the positional relationship between all calibration feature points identified in each calibration image can be determined based on the identification of each feature point in the preset calibration plate. Based on the calibration image (current calibration image) taken by camera 1, the coordinate positions of feature point 1 and feature point 2 in the coordinate system of camera 1 are determined. Then, based on the known positional relationship between feature point 1, feature point 2, feature point 3 and feature point 4 in the calibration plate, the coordinates of feature point 3 and feature point 4 (calibrated feature points identified in other calibration images) in the coordinate system of camera 1 can be determined by conversion according to the coordinate position. Similarly, the coordinates of feature point 1 and feature point 2 in the coordinate system of camera 2 can also be calculated. That is, the coordinates of feature point 1, feature point 2, feature point 3 and feature point 4 in different camera coordinate systems are obtained.

[0041] When the camera module to be calibrated also includes camera 3, camera 4, or even more camera modules, images can be selected in sequence from multiple calibration images as the current calibration image. Then, based on the identifiers of the calibration feature points in the current calibration image, the identifiers of the calibration feature points in other calibration images, and the identifiers of each feature point in the preset calibration plate, the positional relationship between the calibration feature points in the current calibration image and the calibration feature points in other calibration images is determined. Based on the coordinates of the calibration feature points in the current calibration image in the current coordinate system of the corresponding camera to be calibrated and the positional relationship, the coordinates of the calibration feature points in other calibration images in the current coordinate system are determined. The final result is to determine the coordinates of the calibration feature points identified in all calibration images in the coordinate system of each camera to be calibrated.

[0042] S130 : Determine the extrinsic parameters of each camera to be calibrated according to the coordinate transformation relationship between the identified calibration feature points and the calibration images.

[0043] Specifically, a reference camera can be determined from among the multiple cameras to be calibrated, and the coordinate system of the reference camera can be used as the reference coordinate system. Then, the conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system can be determined, and the conversion relationship can be used as the extrinsic parameter of the non-reference camera. By determining the conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system, a set of coordinate conversion equations can be established for the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system. Based on the theory of solving the set of equations, an appropriate number of equations in the set of equations can be set. Furthermore, the set of coordinate conversion equations can be solved using a preset algorithm to determine the conversion relationship between the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system. The preset algorithm can be a mathematical algorithm such as the least squares method that can obtain the optimal solution to the set of equations.

[0044] For example, the coordinate system of camera 1 can be set as the reference coordinate system, and then the parameters for transforming the coordinates of the feature points in camera 2 to the coordinate system of camera 1 are calculated to determine the external parameters of camera 2, including the rotation parameters and translation parameters in the three axes XYZ. The coordinates of all feature points in the coordinate system of camera 1 can be expressed as p1, p2, p3 and p4; the coordinates of all feature points in the coordinate system of camera 2 can also be expressed as p1', p2', p3' and p4'. Then, the external parameters obtained can be pn = Rt*pn'. Among them, n takes the value of 1, 2, 3, 4; Rt represents the external parameter matrix.

[0045] The technical solution of the embodiment of the present disclosure can be achieved by obtaining calibration images of a preset calibration plate taken by multiple cameras to be calibrated with different perspectives; further identifying calibration feature points in each calibration image, and determining the coordinate information of calibration feature points in other calibration images other than itself in each calibration image based on the identification of the feature points in the preset calibration plate; then, determining the extrinsic parameters of each camera to be calibrated based on the coordinate conversion relationship between the identified calibration feature points in each of the calibration images. The technical solution of the embodiment of the present disclosure solves the problem that the existing calibration technology cannot efficiently and cost-effectively complete the calibration of multiple cameras without common viewing conditions. By performing coordinate conversion and analysis on the feature points in the preset calibration plate images obtained by the multiple cameras to be calibrated, a method is provided to determine the extrinsic parameters of multiple cameras without common viewing conditions in an efficient manner without the need for additional cost to maintain calibration objects.

[0046] Example 2

[0047] The embodiments of the present disclosure can be combined with the various optional solutions in the camera extrinsic calibration method provided in the above embodiments. The camera extrinsic calibration method provided in this embodiment further describes the entire process of camera extrinsic calibration starting from the generation of a preset calibration plate.

[0048] Figure 3 This is a flow chart of a camera extrinsic calibration method provided in the second embodiment of the present disclosure. Figure 3 As shown, the camera extrinsic parameter calibration method provided in this embodiment includes:

[0049] S210 , generating checkerboard grids with the same grid size according to the preset size and the preset number of checkerboard grids of the calibration plate.

[0050] In this embodiment, the calibration plate may be a charuco calibration plate, and the image of the charuco calibration plate is obtained by embedding Aruco codes into the white grids of a checkerboard.

[0051] First, the size of the checkerboard grid can be determined based on the preset size of the calibration plate and the preset number of checkerboard grids. Figure 4 The calibration board shown is a 12x9 checkerboard, with each grid measuring 30mm.

[0052] S220, embedding different Aruco coding patterns into the white grids of the chessboard respectively, and setting an identification mark for each Aruco coding pattern, and generating a Charuco calibration plate as a target calibration plate.

[0053] Once the basic checkerboard pattern of the calibration plate is determined, an Aruco dictionary can be selected, such as DICTIONARY_5x5 (indicating that the Aruco coding pattern is a 5x5 black and white checkerboard), and all white squares in the checkerboard are filled from left to right and from top to bottom. The coding pattern in each white square is different and has a unique identifier. Then, a charuco graph is completed. Based on the identifier of each Aruco coding pattern in the graph, its specific position on the calibration plate can be determined, and the position of other Aruco coding patterns can also be inferred based on the identifier of one Aruco coding pattern.

[0054] The main advantage of the charuco calibration target is that all coded patterns are uniquely encoded and identifiable. This means that even partially occluded or non-ideal camera images can be used for calibration. For example, a strong ring light may produce uneven illumination of the calibration target (semi-specular reflection areas), which will cause ordinary checkerboard detection to fail. With charuco, the remaining saddle point (coding pattern feature point) detection can still be used. Saddle point localization can be refined using sub-pixel detection like the checkerboard. This generally leads to very good robustness when determining lens distortion parameters.

[0055] S230: Acquire calibration images of the target calibration plate captured by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different.

[0056] During camera calibration, each camera first captures an image of a Charuco calibration plate, which serves as the calibration image. The Charuco calibration plate contains easily identifiable feature points, known as the Aruco coding pattern, and the positional relationships between these feature points are known. Calibration images captured by each camera under calibration, with different viewing angles, will contain different feature points.

[0057] S240: Identify calibration feature points in each calibration image, and determine, in each calibration image, the coordinate information of calibration feature points in other calibration images except the calibration image itself according to the identifiers of the feature points in the preset calibration plate.

[0058] For multiple cameras to be calibrated, an image can be sequentially selected from multiple calibration images of the multiple cameras to be calibrated as the current calibration image. Then, based on the identifiers of the calibration feature points in the current calibration image, the identifiers of the calibration feature points in other calibration images, and the identifiers of each feature point in the preset calibration plate, the positional relationship between the calibration feature points in the current calibration image and the calibration feature points in other calibration images is determined. Based on the coordinates of the calibration feature points in the current calibration image in the current coordinate system of the corresponding camera to be calibrated and the positional relationship, the coordinates of the calibration feature points in other calibration images in the current coordinate system are determined. The final result is to determine the coordinates of the calibration feature points identified in all calibration images in the coordinate system of each camera to be calibrated.

[0059] S250 : Determine the extrinsic parameters of each camera to be calibrated according to the coordinate transformation relationship between the identified calibration feature points and the calibration images.

[0060] Specifically, a reference camera can be determined from among the multiple cameras to be calibrated, and the coordinate system of the reference camera can be used as the reference coordinate system. Then, the conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system can be determined, and the conversion relationship can be used as the extrinsic parameter of the non-reference camera. By determining the conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system, a set of coordinate conversion equations can be established for the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system. Based on the theory of solving the set of equations, an appropriate number of equations in the set of equations can be set. Furthermore, the set of coordinate conversion equations can be solved using a preset algorithm to determine the conversion relationship between the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system. The preset algorithm can be a mathematical algorithm such as the least squares method that can obtain the optimal solution to the set of equations.

[0061] The technical solution of the embodiment of the present disclosure generates a charuco calibration plate according to a preset calibration plate pattern generation process, and obtains calibration images of the charuco calibration plate taken by multiple cameras to be calibrated with different perspectives during the calibration process; and further identifies the calibration feature points in each calibration image, and determines the coordinate information of the calibration feature points in other calibration images other than itself in each calibration image based on the identification of the feature points in the charuco calibration plate; then, determines the extrinsic parameters of each camera to be calibrated based on the coordinate conversion relationship between the identified calibration feature points in each of the calibration images. The technical solution of the embodiment of the present disclosure solves the problem that the existing calibration technology cannot efficiently and cost-effectively complete the calibration of multiple cameras without common viewing conditions. By performing coordinate conversion and analysis on the feature points in the preset calibration plate images obtained by the multiple cameras to be calibrated, a method is provided to determine the extrinsic parameters of multiple cameras without common viewing conditions in an efficient manner without the need to incur additional costs for maintaining calibration objects.

[0062] Example 3

[0063] Figure 5 This is a schematic diagram of the structure of a camera extrinsic calibration device provided in Example 3 of the present disclosure. The camera extrinsic calibration device provided in this embodiment is suitable for scenarios where camera extrinsics are calibrated, and is particularly suitable for calibrating extrinsic parameters of multiple groups of cameras that do not have a common perspective or have a small overlapping area of ​​perspective.

[0064] like Figure 5 As shown, the camera extrinsic parameter calibration device includes: a calibration image acquisition module 310, a calibration feature point determination module 320 and an extrinsic parameter calibration module 330.

[0065] Among them, the calibration image acquisition module 310 is used to obtain calibration images of a preset calibration plate taken by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different; the calibration feature point determination module 320 is used to identify the calibration feature points in each of the calibration images, and determine the coordinate information of the calibration feature points in other calibration images other than the calibration image itself in each calibration image based on the identification of the feature points in the preset calibration plate; the extrinsic parameter calibration module 330 is used to determine the extrinsic parameters of each camera to be calibrated based on the coordinate transformation relationship between the identified calibration feature points in each of the calibration images.

[0066] The technical solution of the embodiment of the present disclosure obtains calibration images of a preset calibration plate taken by multiple cameras to be calibrated with different perspectives; further identifies calibration feature points in each calibration image, and determines the coordinate information of calibration feature points in other calibration images other than the calibration image itself in each calibration image based on the identification of the feature points in the preset calibration plate; then, determines the extrinsic parameters of each camera to be calibrated based on the coordinate conversion relationship between the identified calibration feature points in each of the calibration images. The technical solution of the embodiment of the present disclosure solves the problem that existing calibration technologies cannot efficiently and cost-effectively complete the calibration of multiple cameras without common viewing conditions. By performing coordinate conversion and analysis on the feature points in the preset calibration plate images obtained by the multiple cameras to be calibrated, a method is provided to determine the extrinsic parameters of multiple cameras without common viewing conditions in an efficient manner without the need for additional cost to maintain calibration objects.

[0067] In some optional implementations, the calibration feature point determination module 320 is specifically configured to:

[0068] Selecting images in sequence from the calibration image as current calibration images, and determining positional relationships between calibration feature points in the current calibration image and calibration feature points in other calibration images according to identifiers of calibration feature points in the current calibration image and identifiers of calibration feature points in other calibration images, as well as identifiers of each feature point in the preset calibration plate;

[0069] The coordinates of the calibration feature points in the other calibration images in the current coordinate system are determined according to the coordinates of the calibration feature points in the current calibration image in the current coordinate system of the corresponding camera to be calibrated and the positional relationship.

[0070] In some optional implementations, the extrinsic parameter calibration module 330 is specifically configured to:

[0071] Determine a reference camera among the multiple cameras to be calibrated, and use the coordinate system of the reference camera as a reference coordinate system;

[0072] Determine a conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system, and use the conversion relationship as an extrinsic parameter of the non-reference camera.

[0073] In some optional implementations, the extrinsic parameter calibration module 330 is specifically configured to:

[0074] Establishing coordinate conversion equations for the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system respectively;

[0075] The coordinate conversion equations are solved by a preset algorithm to determine the conversion relationship between the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system.

[0076] In some optional implementations, the preset calibration plate is a charuco calibration plate.

[0077] In some optional implementations, the camera extrinsic parameter calibration apparatus further includes a calibration plate pattern generation module configured to: generate a checkerboard pattern having the same grid size according to the size of the preset calibration plate and the number of grids of a preset checkerboard pattern before acquiring calibration images of the preset calibration plate taken by the plurality of cameras to be calibrated;

[0078] Different Aruco coding patterns are respectively embedded in the white grids of the preset chessboard, and an identification mark is set for each Aruco coding pattern to generate a charuco calibration plate as the preset calibration plate.

[0079] The camera extrinsic parameter calibration device provided in the embodiments of the present disclosure can execute the camera extrinsic parameter calibration method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0080] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0081] Example 4

[0082] Reference below Figure 6 , which shows an electronic device (eg Figure 6The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0083] like Figure 6 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 406 into a random access memory (RAM) 403. Various programs and data required for the operation of the electronic device 400 are also stored in the RAM 403. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0084] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0085] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 409, or installed from the storage device 406, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-described functions defined in the camera extrinsic calibration method of the embodiments of the present disclosure are performed.

[0086] The electronic device provided in the embodiment of the present disclosure and the camera extrinsic parameter calibration method provided in the above embodiment belong to the same disclosed concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0087] Example 5

[0088] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon. When the program is executed by a processor, the camera extrinsic parameter calibration method provided in the above embodiment is implemented.

[0089] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory (FLASH), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0090] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0091] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0092] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0093] Acquire calibration images of a preset calibration plate captured by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different;

[0094] Identifying calibration feature points in each calibration image, and determining, based on the identifiers of the feature points in the preset calibration plate, coordinate information of calibration feature points in other calibration images other than the calibration image itself in each calibration image;

[0095] The extrinsic parameters of each camera to be calibrated are determined according to the coordinate transformation relationship between the identified calibration feature points and the calibration images.

[0096] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0098] The units involved in the embodiments described in this disclosure may be implemented via software or hardware. The names of units and modules do not, in some cases, limit the units and modules themselves. For example, a data generation module may also be described as a "video data generation module."

[0099] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and the like.

[0100] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0101] According to one or more embodiments of the present disclosure, [Example 1] provides a camera extrinsic calibration method, the method comprising:

[0102] Acquire calibration images of a preset calibration plate captured by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different;

[0103] Identifying calibration feature points in each calibration image, and determining, based on the identifiers of the feature points in the preset calibration plate, coordinate information of calibration feature points in other calibration images other than the calibration image itself in each calibration image;

[0104] The extrinsic parameters of each camera to be calibrated are determined according to the coordinate transformation relationship between the identified calibration feature points and the calibration images.

[0105] According to one or more embodiments of the present disclosure, [Example 2] provides a camera extrinsic calibration method, further comprising:

[0106] In some optional implementations, determining, in each calibration image, coordinate information of calibration feature points in other calibration images other than the calibration image itself based on the position coding information of the feature points in the preset calibration plate includes:

[0107] Selecting images in sequence from the calibration image as current calibration images, and determining positional relationships between calibration feature points in the current calibration image and calibration feature points in other calibration images according to identifiers of calibration feature points in the current calibration image and identifiers of calibration feature points in other calibration images, as well as identifiers of each feature point in the preset calibration plate;

[0108] The coordinates of the calibration feature points in the other calibration images in the current coordinate system are determined according to the coordinates of the calibration feature points in the current calibration image in the current coordinate system of the corresponding camera to be calibrated and the positional relationship.

[0109] According to one or more embodiments of the present disclosure, [Example 3] provides a camera extrinsic calibration method, further comprising:

[0110] In some optional implementations, determining the extrinsic parameters of each camera to be calibrated based on the coordinate transformation relationship between the identified calibration feature points and the calibration images includes:

[0111] Determine a reference camera among the multiple cameras to be calibrated, and use the coordinate system of the reference camera as a reference coordinate system;

[0112] Determine a conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system, and use the conversion relationship as an extrinsic parameter of the non-reference camera.

[0113] According to one or more embodiments of the present disclosure, [Example 4] provides a camera extrinsic calibration method, further comprising:

[0114] In some optional implementations, determining a conversion relationship between the coordinates of each calibration feature point in a calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system includes:

[0115] Establishing coordinate conversion equations for the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system respectively;

[0116] The coordinate conversion equations are solved by a preset algorithm to determine the conversion relationship between the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system.

[0117] According to one or more embodiments of the present disclosure, [Example 5] provides a camera extrinsic calibration method, further comprising:

[0118] In some optional implementations, the preset calibration plate is a charuco calibration plate.

[0119] According to one or more embodiments of the present disclosure, [Example 6] provides a camera extrinsic calibration method, further comprising:

[0120] In some optional implementations, before obtaining calibration images of a preset calibration plate captured by multiple cameras to be calibrated, the method further includes:

[0121] Generating a checkerboard with the same grid size according to the size of the preset calibration plate and the number of grids of the preset checkerboard;

[0122] Different Aruco coding patterns are respectively embedded in the white grids of the preset chessboard, and an identification mark is set for each Aruco coding pattern to generate a charuco calibration plate as the preset calibration plate.

[0123] According to one or more embodiments of the present disclosure, [Example 7] provides a camera extrinsic calibration device, further comprising:

[0124] a calibration image acquisition module, configured to acquire calibration images of a preset calibration plate taken by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different;

[0125] a calibration feature point determination module, configured to identify calibration feature points in each calibration image and determine, based on the identifiers of the feature points in the preset calibration plate, the coordinate information of the calibration feature points in other calibration images other than the calibration image itself;

[0126] The extrinsic parameter calibration module is used to determine the extrinsic parameters of each camera to be calibrated based on the coordinate transformation relationship between the identified calibration feature points and the calibration images.

[0127] According to one or more embodiments of the present disclosure, [Example 8] provides a camera extrinsic calibration device, further comprising:

[0128] In some optional implementations, the calibration feature point determination module is specifically configured to:

[0129] Selecting images in sequence from the calibration image as current calibration images, and determining positional relationships between calibration feature points in the current calibration image and calibration feature points in other calibration images according to identifiers of calibration feature points in the current calibration image and identifiers of calibration feature points in other calibration images, as well as identifiers of each feature point in the preset calibration plate;

[0130] The coordinates of the calibration feature points in the other calibration images in the current coordinate system are determined according to the coordinates of the calibration feature points in the current calibration image in the current coordinate system of the corresponding camera to be calibrated and the positional relationship.

[0131] According to one or more embodiments of the present disclosure, [Example 9] provides a camera extrinsic calibration device, further comprising:

[0132] In some optional implementations, the external parameter calibration module is specifically used to:

[0133] Determine a reference camera among the multiple cameras to be calibrated, and use the coordinate system of the reference camera as a reference coordinate system;

[0134] Determine a conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system, and use the conversion relationship as an extrinsic parameter of the non-reference camera.

[0135] According to one or more embodiments of the present disclosure, [Example 10] provides a camera extrinsic calibration device, further comprising:

[0136] In some optional implementations, the external parameter calibration module is specifically used to:

[0137] Establishing coordinate conversion equations for the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system respectively;

[0138] The coordinate conversion equations are solved by a preset algorithm to determine the conversion relationship between the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system.

[0139] According to one or more embodiments of the present disclosure, [Example 11] provides a camera extrinsic calibration device, further comprising:

[0140] In some optional implementations, the preset calibration plate is a charuco calibration plate.

[0141] According to one or more embodiments of the present disclosure, [Example 12] provides a camera extrinsic calibration device, further comprising:

[0142] In some optional implementations, the camera extrinsic parameter calibration apparatus further includes a calibration plate pattern generation module configured to: generate a checkerboard pattern having the same grid size according to the size of the preset calibration plate and the number of grids of a preset checkerboard pattern before acquiring calibration images of the preset calibration plate taken by the plurality of cameras to be calibrated;

[0143] Different Aruco coding patterns are respectively embedded in the white grids of the preset chessboard, and an identification mark is set for each Aruco coding pattern to generate a charuco calibration plate as the preset calibration plate.

[0144] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0145] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0146] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A camera extrinsic calibration method, characterized in that: include: Obtaining calibration images of a preset calibration plate captured by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different, and no identical marking feature points exist in the calibration images captured within the multiple viewing angles corresponding to the multiple cameras to be calibrated; Identifying calibration feature points in each calibration image, and determining, in each calibration image, coordinate information of calibration feature points in calibration images other than the calibration image itself based on identifiers of feature points in the preset calibration plate, wherein the preset calibration plate includes a plurality of feature points with known positional relationships, and the positional relationships between the plurality of feature points are used to determine the positional relationships between all calibration feature points in each calibration image; The extrinsic parameters of each camera to be calibrated are determined according to the coordinate transformation relationship between the identified calibration feature points and the calibration images.

2. The method according to claim 1, characterized in that The step of determining, based on the position coding information of the feature points in the preset calibration plate, the coordinate information of the calibration feature points in other calibration images other than the calibration image itself in each calibration image includes: Selecting images in sequence from the calibration image as current calibration images, and determining positional relationships between calibration feature points in the current calibration image and calibration feature points in other calibration images according to identifiers of calibration feature points in the current calibration image and identifiers of calibration feature points in other calibration images, as well as identifiers of each feature point in the preset calibration plate; The coordinates of the calibration feature points in the other calibration images in the current coordinate system are determined according to the coordinates of the calibration feature points in the current calibration image in the current coordinate system of the corresponding camera to be calibrated and the positional relationship.

3. The method according to claim 1, characterized in that Determining the external parameters of each camera to be calibrated based on the coordinate transformation relationship between the identified calibration feature points and the calibration images includes: Determine a reference camera among the multiple cameras to be calibrated, and use the coordinate system of the reference camera as a reference coordinate system; Determine a conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system, and use the conversion relationship as an extrinsic parameter of the non-reference camera.

4. The method according to claim 3, characterized in that The determining of the conversion relationship between the coordinates of each calibration feature point in the calibration image of the non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system includes: Establishing coordinate conversion equations for the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system respectively; The coordinate conversion equations are solved by a preset algorithm to determine the conversion relationship between the coordinates of each calibration feature point in the calibration image of each non-reference camera and the coordinates of each calibration feature point in the reference coordinate system.

5. The method according to any one of claims 1 to 4, characterized in that: The preset calibration plate is a charuco calibration plate.

6. The method according to claim 5, characterized in that Before obtaining calibration images of a preset calibration plate captured by multiple cameras to be calibrated, the method further includes: Generating a checkerboard with the same grid size according to the size of the preset calibration plate and the number of grids of the preset checkerboard; Different Aruco coding patterns are respectively embedded in the white grids of the preset chessboard, and an identification mark is set for each Aruco coding pattern to generate a charuco calibration plate as the preset calibration plate.

7. A camera extrinsic parameter calibration device, characterized in that: include: A calibration image acquisition module is configured to acquire calibration images of a preset calibration plate captured by multiple cameras to be calibrated, wherein the viewing angles of the multiple cameras to be calibrated are different, and no identical marking feature points exist in the calibration images captured within the multiple viewing angles corresponding to the multiple cameras to be calibrated; a calibration feature point determination module, configured to identify calibration feature points in each calibration image and, based on the identifiers of the feature points in the preset calibration plate, determine, in each calibration image, the coordinate information of the calibration feature points in other calibration images other than the calibration image itself, wherein the preset calibration plate includes a plurality of feature points with known positional relationships, and the positional relationships between the plurality of feature points are used to determine the positional relationships between all calibration feature points in each calibration image; The extrinsic parameter calibration module is used to determine the extrinsic parameters of each camera to be calibrated based on the coordinate transformation relationship between the identified calibration feature points and the calibration images.

8. The device according to claim 7, characterized in that The external parameter calibration module is specifically used for: Determine a reference camera among the multiple cameras to be calibrated, and use the coordinate system of the reference camera as a reference coordinate system; Determine a conversion relationship between the coordinates of each calibration feature point in the calibration image of a non-reference camera among the multiple cameras to be calibrated and the coordinates of each calibration feature point in the reference coordinate system, and use the conversion relationship as an extrinsic parameter of the non-reference camera.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the camera extrinsic parameter calibration method as described in any one of claims 1 to 6.

10. A storage medium comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform the camera extrinsic parameter calibration method according to any one of claims 1 to 6.

Images