Camera calibration method, system and equipment of multi-camera system and medium

Abstract

The invention relates to a camera calibration method, system and device of a multi-camera system and a medium. The method comprises the following steps: fixing a plurality of cameras in a multi-camera system according to the position of a target shooting object in a target scene; estimating an internal parameter and an external parameter of each of the plurality of cameras; shooting a view based on the internal reference and the external reference of each of the plurality of cameras, wherein the plurality of cameras have overlapped view fields during view shooting; a world coordinate system is determined, and world coordinate system coordinates of target viewpoints in the overlapped view fields are converted into camera coordinate system coordinates for the overlapped view fields; and optimizing the images corresponding to the coordinates of all the camera coordinate systems to obtain an optimized transformation relation of transforming the target viewpoint in the overlapped view field from the world coordinate system to the camera coordinate system, and completing camera calibration of the multi-camera system according to the optimized transformation relation. According to the invention, the calibration cost is reduced, the calibration accuracy is improved, and the camera calibration efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a camera calibration method, system, device and medium for a multi-camera system. Background Art

[0002] In order to achieve large field of view imaging, the mainstream solutions in the prior art mostly use multiple cameras to expose simultaneously, and obtain the coordinate transformation relationship between different views based on camera calibration. Although the camera calibration method is very intuitive in principle, it is often difficult to implement in practical applications. In order to achieve accurate camera calibration, it is generally necessary to make special settings for the scene and design and manufacture specific calibration equipment. During calibration, it is necessary to collect a certain number of calibration equipment images according to the requirements of lens type, camera resolution, calibration accuracy, etc., and construct optimization equations based on mathematical models such as camera imaging model and distortion model to solve the camera imaging model and distortion model parameters.

[0003] However, on the one hand, in actual use, due to factors such as changes in ambient light during each calibration, operational differences, and assembly tolerances of camera modules in the same batch, it is impossible to guarantee that the results obtained from each calibration are completely consistent, resulting in low calibration efficiency. On the other hand, calibrating multiple cameras will cost a lot of money in the actual product production process. Summary of the invention

[0004] Based on the above technical problems, the present application aims to provide a camera calibration method, system, device and medium for a multi-camera system to solve at least one of the above technical problems.

[0005] A first aspect of the present application provides a camera calibration method for a multi-camera system, the method comprising:

[0006] Fixing a plurality of cameras in the multi-camera system according to the position of a target photographed object in a target scene;

[0007] estimating intrinsic parameters and extrinsic parameters of each of the plurality of cameras;

[0008] Based on the intrinsic parameters and extrinsic parameters of each camera in the multiple cameras, the multiple cameras have overlapping fields of view when capturing the views;

[0009] Determine a world coordinate system, and for the overlapping fields of view, transform the world coordinate system coordinates of the target viewpoint in the overlapping fields of view into camera coordinate system coordinates;

[0010] The images corresponding to the coordinates of all the camera coordinate systems are optimized to obtain an optimized transformation relationship of the target viewpoints in the overlapping fields of view from the world coordinate system to the camera coordinate system, and the camera calibration of the multi-camera system is completed according to the optimized transformation relationship.

[0011] In some embodiments of the present application, the step of transforming the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates includes:

[0012] Randomly select three cameras from the multiple cameras as the first camera, the second camera, and the third camera in sequence;

[0013] Taking the camera coordinate system of the second camera as the world coordinate system;

[0014] According to the internal parameters of the first camera, the second camera, and the third camera, the external parameters of the first camera relative to the second camera, and the external parameters of the third camera relative to the second camera, the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into camera coordinate system coordinates.

[0015] In some embodiments of the present application, the external parameters include a rotation matrix and a translation vector; according to the internal parameters of the first camera, the second camera, and the third camera, the external parameters of the first camera relative to the second camera, and the external parameters of the third camera relative to the second camera, the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into camera coordinate system coordinates, including:

[0016] Determining an imaging result of the second camera according to the internal parameters of the second camera and the three-dimensional coordinates of the target viewpoint in the overlapping field of view;

[0017] According to the imaging result of the second camera, the rotation matrix and translation vector of the first camera relative to the world coordinate system, transform the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates of the first camera;

[0018] According to the imaging result of the second camera, the rotation matrix and translation vector of the third camera relative to the world coordinate system, the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into the camera coordinate system coordinates of the third camera.

[0019] In some embodiments of the present application, the formula for the imaging result of the second camera is:

[0020]

[0021] Among them, X p , Yp , Z p are the coordinates of the target viewpoint in the overlapping field of view on the X-axis, Y-axis, and Z-axis, K2 is the internal parameter of the second camera, I is the unit matrix, s2 represents the distance from the second camera to the target viewpoint in the overlapping field of view, and x2 and y2 represent the coordinates of the target viewpoint in the overlapping field of view in the camera coordinate system of the second camera.

[0022] In some embodiments of the present application, according to the imaging result of the second camera, the rotation matrix and translation vector of the first camera relative to the world coordinate system, the formula for transforming the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates of the first camera is:

[0023]

[0024] Among them, K1 is the internal parameter of the first camera, T 12 is the translation vector of the first camera relative to the world coordinate system, R 12 is the rotation matrix of the first camera relative to the world coordinate system, s1 represents the distance from the first camera to the target viewpoint in the overlapping field of view, and x1 and y1 represent the coordinates of the target viewpoint in the overlapping field of view in the camera coordinate system of the first camera.

[0025] In some embodiments of the present application, the optimizing the images corresponding to all the camera coordinate system coordinates includes:

[0026] Intercepting a sub-image of a first preset size from all images whose coordinates in the camera coordinate system correspond to those taken by the first camera, and intercepting a sub-image of a second preset size from all images whose coordinates in the camera coordinate system correspond to those taken by the second camera, wherein the first preset size is not equal to the second preset size;

[0027] Obtaining a relative offset between the two sub-images based on a similarity between the sub-image of the first preset size and the sub-image of the second preset size;

[0028] The error generated when the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into the camera coordinate system coordinates is corrected according to the relative offset.

[0029] In some embodiments of the present application, the estimating the intrinsic parameter of each of the multiple cameras includes:

[0030] The intrinsic parameters of each of the plurality of cameras are estimated using a camera calibration tool, wherein the intrinsic parameters include a focal length and an optical center.

[0031] A second aspect of the present application provides a camera calibration system for a multi-camera system, the system comprising:

[0032] an estimation module, configured to estimate intrinsic parameters and extrinsic parameters of each of the plurality of cameras;

[0033] A plurality of cameras configured to capture views based on intrinsic parameters and extrinsic parameters of each of the plurality of cameras, wherein the plurality of cameras have overlapping fields of view when capturing the views;

[0034] a transformation module configured to determine a world coordinate system, and transform the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates for the overlapping field of view;

[0035] The optimization module optimizes the images corresponding to the coordinates of all the camera coordinate systems to obtain an optimized transformation relationship of the target viewpoints in the overlapping fields of view from the world coordinate system to the camera coordinate system, and completes the camera calibration of the multi-camera system according to the optimized transformation relationship.

[0036] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the camera calibration method for a multi-camera system described in each embodiment of the present application.

[0037] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon; the computer program is executed by a processor to implement the camera calibration method for a multi-camera system described in each embodiment of the present application.

[0038] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages:

[0039] The camera calibration method of the multi-camera system in each embodiment of the present application obtains an optimized transformation relationship from the world coordinate system to the camera coordinate system of the target viewpoints in the overlapping field of view by transforming the world coordinate system coordinates of the target viewpoints in the overlapping field of view into the camera coordinate system coordinates and optimizing the images corresponding to all the camera coordinate system coordinates, and completes the camera calibration of the multi-camera system according to the optimized transformation relationship, thereby reducing the complexity of the calibration process, reducing the calibration cost, and improving the calibration accuracy, thereby improving the calibration efficiency. Even if the ambient light changes, operation differences, assembly tolerances of the same batch of camera modules and other factors affect each calibration, the results obtained by each calibration can be consistent.

[0040] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0042] Figure 1 A schematic diagram of the steps of a camera calibration method for a multi-camera system in an exemplary embodiment of the present application is shown;

[0043] Figure 2 A schematic diagram showing the position relationship between multiple cameras and target viewpoints in an exemplary embodiment of the present application is shown;

[0044] Figure 3 A schematic diagram of the structure of a camera calibration system for a multi-camera system in an exemplary embodiment of the present application is shown;

[0045] Figure 4 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application is shown.

[0046] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. DETAILED DESCRIPTION

[0047] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0048] Camera calibration is a key step in constructing large field of view imaging. Although the coordinate transformation relationship obtained by the camera calibration method in the prior art is very intuitive in principle, it is often difficult to implement in practical applications. In order to achieve accurate camera calibration, it is generally necessary to make special settings for the scene and design and manufacture specific calibration equipment. During calibration, it is necessary to collect a certain number of calibration equipment images according to the requirements of lens type, camera resolution, calibration accuracy, etc., and construct optimization equations based on mathematical models such as camera imaging model and distortion model to solve the camera imaging model and distortion model parameters.

[0049] However, on the one hand, in actual use, due to factors such as changes in ambient light during each calibration, operational differences, and assembly tolerances of camera modules in the same batch, it is impossible to guarantee that the results obtained from each calibration are completely consistent, resulting in low calibration efficiency. At the same time, calibration generally requires taking multiple images of the calibration equipment in different poses, which affects the calibration efficiency. On the other hand, calibrating multiple cameras will cost a lot of money in the actual product production process.

[0050] Therefore, in some embodiments of the present application, a camera calibration method for a multi-camera system is provided, such as Figure 1 As shown, the method includes steps S1-S5.

[0051] S1. Fix multiple cameras in a multi-camera system according to the position of a target object in a target scene.

[0052] As the name implies, a multi-camera system includes multiple cameras (also called cameras). In this application, the so-called "multi" means at least 3. In actual application scenarios, it is necessary to plan the positions of multiple cameras and fix the multiple cameras according to the positions of the target objects. In one possible implementation, the position of each camera in the multi-camera system is carefully planned to ensure that they can cover the target area in the application scenario and capture the target object. Consider the possible movement paths and activity areas of the target object, and then determine the optimal position and angle of each camera. When installing cameras, make sure they are firmly fixed in the selected positions. During the specific installation process, a bracket, fixed base or other appropriate mounting equipment is required to ensure that the camera does not move or shake, so as to ensure that the camera lens is clearly aimed at the target area.

[0053] As a variable implementation method, the specific implementation may also require networking and configuration, connecting each camera to a central control system or network. This can be done through wired or wireless connections, depending on the type of camera and system. Configure the parameters of each camera, including resolution, frame rate, exposure time, and other settings to ensure that they capture the target object in the best way. Not only that, but optionally, calibration tools or software can be used to initially ensure that the field of view and image quality of all cameras are consistent, which will help to seamlessly integrate the images of multiple cameras into a system later.

[0054] S2. Estimate the intrinsic parameters and extrinsic parameters of each of the multiple cameras.

[0055] In a preferred implementation, a camera calibration tool is used to estimate the intrinsic parameters of each of the multiple cameras, wherein the intrinsic parameters include focal length and optical center, and the calibration tool is, for example, OpenCV. When using OpenCV to calibrate the camera intrinsic parameters, steps such as preparing a calibration plate, acquiring images, extracting corner points, coordinate transformation and calibration are included. When using a calibration plate, it is usually a plane in a checkerboard pattern, arranged in the shooting scene. The squares on the calibration plate must be of a known size for accurate calibration. When acquiring images, use each camera to capture multiple images containing the calibration plate, ensuring that the calibration plate has multiple different positions and angles in the image, because the best results usually come from different angles, distances and directions. When extracting corner points, use image processing software or OpenCV functions to automatically detect the corner point positions of the calibration plate on each image, which are key points for calibration. When establishing the correspondence between the world coordinate system and the image coordinate system, a coordinate system is usually established with a corner point of the calibration plate as the origin, and the grid size of the calibration plate as the unit. For each detected corner point, its coordinates in the world coordinate system and in the image coordinate system are recorded. Finally, the calibration operation is performed using the image of each camera and the corresponding correspondence between the world coordinate system and the image coordinate system.

[0056] It should be noted that when obtaining the internal parameters of each camera, these parameters include the focal length of the camera, the coordinates of the principal point (i.e., the optical center), and the radial and tangential distortion parameters, etc. However, this application omits obtaining the radial and tangential distortion parameters, that is, it is not necessary to consider the influence of lens distortion to improve the overall calibration efficiency, so as to reduce the time and material costs consumed in obtaining the internal parameters. The calibration quality is enhanced in subsequent steps, which improves the camera calibration efficiency overall.

[0057] S3. Capture a view based on the intrinsic parameters and extrinsic parameters of each of the multiple cameras, so that the multiple cameras have overlapping fields of view when capturing the view.

[0058] Ensuring that multiple cameras have overlapping fields of view (field of view overlap) is very important for many application scenarios, especially in multi-camera systems, to ensure complete capture and tracking of the target object. In a preferred implementation, reference Figure 2 Specifically, the cameras may be arranged crosswise at different angles and positions, which may be achieved by installing the cameras at different heights or angles to ensure that their fields of view overlap.

[0059] like Figure 2 As shown, three cameras are randomly selected from the plurality of cameras as the first cameras (i.e. Figure 2 Camera 1 in the image), the second camera (i.e. Figure 2 Camera 2 in the video), the third camera (i.e. Figure 2The camera 3 in the image processing unit forms an overlapping field of view, i.e., the overlapping field of view of the three cameras. Point P is a target object, i.e., a target viewpoint, in the overlapping field of view of the three cameras. It can be understood that there are many target viewpoints in the overlapping field of view. In the subsequent steps, the three-dimensional coordinates of the target viewpoints including point P in the overlapping field of view in the world coordinate system are mapped to the camera coordinate system. The coordinate system conversion relationship here plays a key role in the camera calibration of the multi-camera system, and is a low-cost and efficient camera calibration method.

[0060] As a convertible implementation method, when there are 5 cameras in the multi-camera system, these 5 cameras can also be arranged based on the positional relationship with the target viewpoint in the application scene, so that these 5 cameras also form overlapping fields of view, and then perform subsequent operations.

[0061] S4. Determine a world coordinate system, and for the overlapping fields of view, transform the world coordinate system coordinates of the target viewpoints in the overlapping fields of view into camera coordinate system coordinates.

[0062] In a preferred implementation, reference can still be made to Figure 2 , determine the imaging result of the second camera according to the internal parameters of the second camera and the three-dimensional coordinates of the target viewpoint in the overlapping field of view; according to the imaging result of the second camera, the rotation matrix and translation vector of the first camera relative to the world coordinate system, transform the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates of the first camera; according to the imaging result of the second camera, the rotation matrix and translation vector of the third camera relative to the world coordinate system, transform the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates of the third camera. Among them, the formula involved in the imaging process of the second camera is formula (1):

[0063]

[0064] Among them, X p , Y p , Z p are the coordinates of the target viewpoint in the overlapping field of view on the X-axis, Y-axis, and Z-axis, K2 is the internal parameter of the second camera, I is the unit matrix, s2 represents the distance from the second camera to the target viewpoint in the overlapping field of view, and x2 and y2 represent the coordinates of the target viewpoint in the overlapping field of view in the camera coordinate system of the second camera.

[0065] The formula involved in the imaging process of the first camera is formula (2):

[0066]

[0067] Furthermore, the following formula (3) can be obtained by fusing formula (1) and formula (2), that is, according to the imaging result of the second camera, the rotation matrix and translation vector of the first camera relative to the world coordinate system, the world coordinate system coordinates of the target viewpoints in the overlapping field of view are transformed into the camera coordinate system coordinates of the first camera. Formula (3) is:

[0068]

[0069] Among them, K1 is the internal parameter of the first camera, T 12 is the translation vector of the first camera relative to the world coordinate system, R 12 is the rotation matrix of the first camera relative to the world coordinate system, s1 represents the distance from the first camera to the target viewpoint in the overlapping field of view, and x1 and y1 represent the coordinates of the target viewpoint in the overlapping field of view in the camera coordinate system of the first camera.

[0070] Similarly, according to the above formula, the world coordinate system coordinates of the target viewpoint in the overlapping field of view can be transformed into the camera coordinate system coordinates of the third camera, that is, according to the imaging result of the second camera, the rotation matrix and translation vector of the third camera relative to the world coordinate system, the world coordinate system coordinates of the target viewpoint in the overlapping field of view can be transformed into the camera coordinate system coordinates of the third camera.

[0071] The above rotation matrix is ​​a mathematical tool used to describe the rotation of an object around a certain point or around a coordinate axis in three-dimensional space. Usually, it is a 3x3 matrix, which can be obtained based on the relative position and posture relationship between multiple cameras, for example, it can be calculated using the rotation angle and the rotation axis. Specifically, when selecting the rotation axis, determine which coordinate axis to rotate around, such as around the X-axis, Y-axis or Z-axis; when calculating the rotation angle, determine the angle of rotation, usually in radians; when constructing the rotation matrix, construct the corresponding rotation matrix based on the selected rotation axis and rotation angle. The above translation vector is used to describe the transformation of an object moving along the coordinate axis in three-dimensional space. It is usually represented by a vector containing three components (X, Y, Z). To obtain the translation vector, it is necessary to know the displacement of the object when it moves from one coordinate system to another. First determine the displacement components, and then construct the translation vector. Specifically, determine the displacement components of the object in the X, Y and Z directions, usually in meters (m), and then arrange the displacement components into three components of the vector according to the corresponding coordinate axes.

[0072] The above implementation only lists the coordinate transformation of point P, and according to the above steps, the mapping relationship from the world coordinate system to the camera coordinate system of all target viewpoints in the overlapping field of view is further completed, which will not be described in detail here.

[0073] S5. Optimize the images corresponding to the coordinates of all camera coordinate systems to obtain the optimized transformation relationship of the target viewpoints in the overlapping field of view from the world coordinate system to the camera coordinate system, and complete the camera calibration of the multi-camera system according to the optimized transformation relationship.

[0074] In a preferred implementation, images corresponding to all camera coordinate system coordinates are optimized, including: intercepting a sub-image of a first preset size in all camera coordinate system coordinates corresponding to images shot by a first camera, and intercepting a sub-image of a second preset size in all camera coordinate system coordinates corresponding to images shot by a second camera, wherein the first preset size is not equal to the second preset size; obtaining a relative offset between the two sub-images based on a similarity between the sub-image of the first preset size and the sub-image of the second preset size; and correcting errors generated when transforming the world coordinate system coordinates of the target viewpoints in the overlapping fields of view into camera coordinate system coordinates according to the relative offset.

[0075] Since the influence of lens distortion is not considered when obtaining the intrinsic parameters of each camera, the estimated intrinsic parameters can be considered as a rough estimate, so errors are inevitable, so it is necessary to optimize the mapping relationship or coordinate conversion relationship. In specific implementation, the coordinates obtained in the above embodiment can be combined. For example, according to the coordinates of (x2, y2), a sub-image with (x2, y2) as the center and length and width of w2 and h2 are intercepted in the image of camera two. Then, according to the coordinates of (x1, y1), a sub-image with (x1, y1) as the center and length and width of w1 and h1 are intercepted in the image of camera one. It should be noted that the values ​​of w2 and h2 are several pixels, and w1 and h1 can be set based on the maximum error range that may be caused by the above step S4. Then, the relative offset between the two sub-images is obtained based on the similarity between the sub-images, and the error generated when the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into the camera coordinate system coordinates is corrected according to the relative offset. Optionally, the image similarity calculation can select common methods according to actual application requirements, such as difference square sum matching, standardized difference square sum matching, correlation matching, standard correlation matching, correlation matching, standard correlation matching, etc. Finally, the camera calibration of the multi-camera system is completed according to the optimized transformation relationship. The present application transforms the world coordinate system coordinates of the target viewpoints in the overlapping field of view into camera coordinate system coordinates and optimizes the images corresponding to all the camera coordinate system coordinates to obtain the optimized transformation relationship of the target viewpoints in the overlapping field of view from the world coordinate system to the camera coordinate system, and completes the camera calibration of the multi-camera system according to the optimized transformation relationship, which reduces the complexity of the calibration process, reduces the calibration cost, and improves the calibration accuracy, thereby improving the calibration efficiency.

[0076] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application.

[0077] In some embodiments of the present application, a camera calibration system for a multi-camera system is further provided, which executes the camera calibration method for the multi-camera system in each embodiment. Figure 3 As shown, the camera calibration system includes: an estimation module 301, configured to estimate the intrinsic parameters and extrinsic parameters of each camera in a plurality of cameras; a plurality of cameras 302, configured to capture views based on the intrinsic parameters and extrinsic parameters of each camera in the plurality of cameras, wherein the plurality of cameras have overlapping fields of view when capturing the views; a transformation module 303, configured to determine a world coordinate system, and for overlapping fields of view, transform the world coordinate system coordinates of the target viewpoints in the overlapping fields of view into camera coordinate system coordinates; an optimization module 304, optimizing images corresponding to all camera coordinate system coordinates, obtaining an optimized transformation relationship of the target viewpoints in the overlapping fields of view from the world coordinate system to the camera coordinate system, and completing the camera calibration of the multi-camera system according to the optimized transformation relationship.

[0078] It is understandable that the camera calibration system may also include other necessary hardware devices and software programs. The camera calibration system reduces the complexity of the calibration process, reduces the calibration cost, and improves the calibration accuracy, thereby improving the calibration efficiency. Even if the ambient light changes, operation differences, assembly tolerances of camera modules in the same batch, and other factors affect each calibration, the results obtained each time can be consistent. It should be understood that the above general description and the detailed description below are only exemplary and explanatory and cannot limit the present application.

[0079] Please refer to the following Figure 4 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 4 As shown, the electronic device 2 includes: a processor 200, a memory 201, a bus 202 and a communication interface 203, and the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and when the processor 200 runs the computer program, it executes a camera calibration method for a multi-camera system as described in any one of the embodiments of the present application.

[0080] The memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 203 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0081] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store a program, and the processor 200 executes the program after receiving an execution instruction. The camera calibration method of the multi-camera system disclosed in any implementation of the embodiment of the present application may be applied to the processor 200, or implemented by the processor 200.

[0082] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 200. The above processor 200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the method in combination with its hardware.

[0083] The present application also provides a computer-readable storage medium corresponding to the camera calibration method for a multi-camera system provided in the aforementioned embodiment, on which a computer program is stored, and when the computer program is executed by a processor, the camera calibration method for a multi-camera system described in any of the aforementioned embodiments will be executed. In addition, examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which will not be described one by one here.

[0084] In addition, an embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the camera calibration method for a multi-camera system in any of the aforementioned embodiments.

[0085] Those skilled in the art will appreciate that the various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components of the virtual machine creation device according to the embodiments of the present application.

[0086] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A camera calibration method for a multi-camera system, characterized in that: The method comprises: Fixing a plurality of cameras in the multi-camera system according to the position of a target photographed object in a target scene; estimating intrinsic parameters and extrinsic parameters of each of the plurality of cameras; Based on the intrinsic parameters and extrinsic parameters of each camera in the multiple cameras, the multiple cameras have overlapping fields of view when capturing the views; Determine a world coordinate system, and for the overlapping fields of view, transform the world coordinate system coordinates of the target viewpoint in the overlapping fields of view into camera coordinate system coordinates; The images corresponding to the coordinates of all the camera coordinate systems are optimized to obtain an optimized transformation relationship of the target viewpoints in the overlapping fields of view from the world coordinate system to the camera coordinate system, and the camera calibration of the multi-camera system is completed according to the optimized transformation relationship.

2. The camera calibration method of a multi-camera system according to claim 1, characterized in that: The step of transforming the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates comprises: Randomly select three cameras from the multiple cameras as the first camera, the second camera, and the third camera in sequence; Taking the camera coordinate system of the second camera as the world coordinate system; According to the internal parameters of the first camera, the second camera, and the third camera, the external parameters of the first camera relative to the second camera, and the external parameters of the third camera relative to the second camera, the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into camera coordinate system coordinates.

3. The camera calibration method of a multi-camera system according to claim 2, characterized in that: The external parameters include a rotation matrix and a translation vector; according to the internal parameters of the first camera, the second camera, and the third camera, the external parameters of the first camera relative to the second camera, and the external parameters of the third camera relative to the second camera, the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into camera coordinate system coordinates, including: Determining an imaging result of the second camera according to the internal parameters of the second camera and the three-dimensional coordinates of the target viewpoint in the overlapping field of view; According to the imaging result of the second camera, the rotation matrix and translation vector of the first camera relative to the world coordinate system, transform the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates of the first camera; According to the imaging result of the second camera, the rotation matrix and translation vector of the third camera relative to the world coordinate system, the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into the camera coordinate system coordinates of the third camera.

4. The camera calibration method for a multi-camera system according to claim 3, characterized in that: The formula for the imaging result of the second camera is: Among them, X p , Y p , Z p are the coordinates of the target viewpoint in the overlapping field of view on the X-axis, Y-axis, and Z-axis, K2 is the internal parameter of the second camera, I is the unit matrix, s2 represents the distance from the second camera to the target viewpoint in the overlapping field of view, and x2 and y2 represent the coordinates of the target viewpoint in the overlapping field of view in the camera coordinate system of the second camera.

5. The camera calibration method for a multi-camera system according to claim 4, characterized in that: According to the imaging result of the second camera, the rotation matrix and translation vector of the first camera relative to the world coordinate system, the formulas for transforming the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates of the first camera are: Among them, K1 is the internal parameter of the first camera, T 12 is the translation vector of the first camera relative to the world coordinate system, R 12 is the rotation matrix of the first camera relative to the world coordinate system, s1 represents the distance from the first camera to the target viewpoint in the overlapping field of view, and x1 and y1 represent the coordinates of the target viewpoint in the overlapping field of view in the camera coordinate system of the first camera.

6. The camera calibration method for a multi-camera system according to claim 2, characterized in that: The optimizing the images corresponding to all the camera coordinate system coordinates includes: Intercepting a sub-image of a first preset size from all images whose coordinates in the camera coordinate system correspond to those taken by the first camera, and intercepting a sub-image of a second preset size from all images whose coordinates in the camera coordinate system correspond to those taken by the second camera, wherein the first preset size is not equal to the second preset size; Obtaining a relative offset between the two sub-images based on a similarity between the sub-image of the first preset size and the sub-image of the second preset size; The error generated when the world coordinate system coordinates of the target viewpoint in the overlapping field of view are transformed into the camera coordinate system coordinates is corrected according to the relative offset.

7. The camera calibration method for a multi-camera system according to claim 1, characterized in that: The estimating the internal parameter of each camera of the plurality of cameras comprises: The intrinsic parameters of each of the plurality of cameras are estimated using a camera calibration tool, wherein the intrinsic parameters include a focal length and an optical center.

8. A camera calibration system for a multi-camera system, characterized in that ,, the system comprises: an estimation module, configured to estimate intrinsic parameters and extrinsic parameters of each of the plurality of cameras; A plurality of cameras configured to capture views based on intrinsic parameters and extrinsic parameters of each of the plurality of cameras, wherein the plurality of cameras have overlapping fields of view when capturing the views; a transformation module configured to determine a world coordinate system, and transform the world coordinate system coordinates of the target viewpoint in the overlapping field of view into the camera coordinate system coordinates for the overlapping field of view; The optimization module optimizes the images corresponding to the coordinates of all the camera coordinate systems, obtains and completes the camera calibration of the multi-camera system according to the optimized transformation relationship.

9. An electronic device, comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the camera calibration method for a multi-camera system as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon; the computer program is executed by a processor to implement the camera calibration method for a multi-camera system as described in any one of claims 1-7.

Images