Parameter calibration method, device and equipment of annular camera array and storage medium

Abstract

The application provides a parameter calibration method, device and equipment of a ring-shaped camera array and a storage medium. The method comprises the following steps: determining the camera intrinsic parameters corresponding to each camera in the ring-shaped camera array based on the multiple frames of first images of the object to be measured corresponding to different angles collected by each camera; grouping all the cameras in the ring-shaped camera array in sequence to obtain multiple groups of cameras, and determining the initial camera extrinsic parameters corresponding to each group of cameras based on the multiple frames of second images of the object to be measured corresponding to different angles collected by each camera in each group of cameras; wherein each group of cameras comprises multiple cameras, and there are the same cameras in the adjacent two groups of cameras; performing joint optimization processing on the multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters corresponding to the ring-shaped camera array; and determining the joint calibration parameter results corresponding to the ring-shaped camera array according to the camera intrinsic parameters and the target camera extrinsic parameters. The method can accurately calibrate the camera parameters corresponding to the ring-shaped camera array.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for calibrating parameters of a circular camera array. Background Technology

[0002] Camera calibration is a fundamental step in recovering the three-dimensional geometric structure information of a test object from a two-dimensional image. Existing multi-camera calibration methods mainly include: camera self-calibration and calibration object-based camera calibration. The former does not use a calibration object; it calibrates the parameters of multiple cameras solely through the correspondence constraints between images. This method has a simple calibration process, but excessive error accumulation occurs in the data processing part during the entire calibration process, leading to inaccurate calibration results. The latter generally uses a calibration object with known geometric information, i.e., a three-dimensional calibration object. However, the fabrication process of the three-dimensional calibration object is difficult, and if the three-dimensional calibration object is not accurate enough, the calibration results of the multi-camera system will also be inaccurate.

[0003] In summary, existing multi-camera calibration methods have certain limitations, resulting in the inability to accurately calibrate the camera parameters corresponding to each camera. Summary of the Invention

[0004] This invention provides a parameter calibration method, apparatus, device, and storage medium for a circular camera array, which addresses the limitations of existing multi-camera parameter calibration methods, which result in the inability to accurately calibrate the camera parameters corresponding to multiple cameras. This invention enables accurate calibration of the camera parameters corresponding to a circular camera array.

[0005] This invention provides a parameter calibration method for a circular camera array, comprising:

[0006] Based on the first images of the object under test captured by each camera in the circular camera array at different angles, the camera intrinsic parameters corresponding to each camera are determined.

[0007] All cameras in the circular camera array are grouped sequentially to obtain multiple camera groups. Based on the multiple frames of second images of the object under test captured by each camera in each group at different angles, the initial camera extrinsic parameters corresponding to each camera group are determined. Each camera group includes multiple cameras, and there are identical cameras in adjacent camera groups.

[0008] The initial camera extrinsic parameters are jointly optimized to determine the target camera extrinsic parameters corresponding to the circular camera array.

[0009] Based on the intrinsic parameters of the camera and the extrinsic parameters of the target camera, the joint calibration parameter results corresponding to the ring camera array are determined.

[0010] According to a parameter calibration method for a ring camera array provided by the present invention, the method determines the initial camera extrinsic parameters corresponding to each group of cameras based on multiple frames of second images of the object under test captured by each camera in each group of cameras at different angles. The method includes: acquiring multiple frames of second images of the object under test captured by each camera in each group of cameras at different angles; determining a first frame of second image and a plurality of first feature points in the first frame of second image from the multiple frames of second images corresponding to each camera in each group of cameras; determining a measurement coordinate system corresponding to each group of cameras based on any one of the plurality of first feature points; and determining the initial camera extrinsic parameters corresponding to each group of cameras according to the measurement coordinate system.

[0011] According to a parameter calibration method for a circular camera array provided by the present invention, the method involves jointly optimizing multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters corresponding to the circular camera array. The method includes: transforming the camera coordinate system of each first camera in a first group of cameras to a first measurement coordinate system corresponding to that first group of cameras, obtaining a first coordinate system corresponding to each first camera, where the first group of cameras is any one of the multiple groups of cameras; transforming the camera coordinate system of each second camera in a second group of cameras to a second measurement coordinate system corresponding to that second group of cameras, obtaining a second coordinate system corresponding to each second camera, where the second group of cameras is adjacent to the first group of cameras; identifying a target camera from the second group of cameras that is identical to the first group of cameras; determining a transformation relationship based on the first coordinate system and the second coordinate system corresponding to the target camera; and transforming the second coordinate system corresponding to each second camera to the first measurement coordinate system based on the transformation relationship to obtain the target camera extrinsic parameters corresponding to the circular camera array.

[0012] According to a parameter calibration method for a ring camera array provided by the present invention, determining the transformation relationship based on a first coordinate system and a second coordinate system corresponding to the target camera includes: determining a relative transformation matrix corresponding to the target camera based on the first coordinate system and the second coordinate system corresponding to the target camera, the relative transformation matrix including a relative rotation matrix and a relative translation matrix; converting the relative rotation matrix into Euler angles; and determining the transformation relationship based on the relative translation matrix and the Euler angles.

[0013] According to the parameter calibration method of a ring camera array provided by the present invention, the method for determining the initial camera extrinsic parameters corresponding to each group of cameras based on the measurement coordinate system includes: determining the current camera extrinsic parameters corresponding to each group of cameras based on the measurement coordinate system; acquiring multiple frames of second images of the object under test at different angles, collected by at least two cameras in each group of cameras; determining the second feature points corresponding to each group of cameras based on the multiple frames of second images; and performing triangulation and bundle adjustment (BA) optimization processing on the second feature points using the current camera extrinsic parameters to obtain the initial camera extrinsic parameters corresponding to each group of cameras.

[0014] According to a parameter calibration method for a circular camera array provided by the present invention, the method involves grouping all cameras in the circular camera array sequentially to obtain multiple groups of cameras, including: labeling each camera in the circular camera array with a serial number, and grouping the labeled cameras in a preset order to obtain multiple groups of cameras, wherein the preset order includes a counterclockwise order or a clockwise order.

[0015] According to the present invention, a parameter calibration method for a circular camera array is provided. The method determines the camera intrinsic parameters corresponding to each camera based on multiple frames of first images of the object under test acquired by each camera in the circular camera array at different angles. The method includes: acquiring multiple frames of first images of the object under test acquired by each camera in the circular camera array at different angles; calibrating the multiple frames of first images using the Zhang Zhengyou calibration method to obtain the camera intrinsic parameters corresponding to each camera.

[0016] The present invention also provides a parameter calibration device for a ring camera array, comprising:

[0017] The processing module is used to determine the camera intrinsic parameters of each camera based on multiple frames of first images of the object under test captured by each camera in the circular camera array at different angles; to group all cameras in the circular camera array sequentially to obtain multiple groups of cameras, and to determine the initial camera extrinsic parameters of each group of cameras based on multiple frames of second images of the object under test captured by each camera in each group at different angles; wherein each group of cameras includes multiple cameras, and there are identical cameras in adjacent groups; to perform joint optimization processing on the multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters of the circular camera array; and to determine the joint calibration parameter results of the circular camera array based on the camera intrinsic parameters and the target camera extrinsic parameters.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the parameter calibration method of any of the above-described circular camera arrays.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the parameter calibration method for the circular camera array as described above.

[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the parameter calibration method for any of the above-described circular camera arrays.

[0021] The present invention provides a method, apparatus, device, and storage medium for calibrating parameters of a circular camera array. This method determines the intrinsic parameters of each camera based on multiple frames of first images of the object under test captured by each camera in the circular camera array at different angles. All cameras in the circular camera array are sequentially grouped to obtain multiple groups of cameras. Based on multiple frames of second images of the object under test captured by each camera in each group at different angles, initial extrinsic parameters are determined for each group of cameras. Each group of cameras includes multiple cameras, and adjacent groups may contain identical cameras. Joint optimization processing is performed on the multiple initial extrinsic parameters to determine the target extrinsic parameters of the circular camera array. Based on the intrinsic and target extrinsic parameters, a joint calibration parameter result for the circular camera array is determined. This method overcomes the limitations of existing multi-camera parameter calibration methods, which cannot accurately calibrate the camera parameters of multiple cameras, and enables accurate calibration of the camera parameters of a circular camera array. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a scene where a ring camera array provided by the present invention captures images of an object under test;

[0024] Figure 2 This is a flowchart illustrating the parameter calibration method for a ring camera array provided by the present invention.

[0025] Figure 3 This is a schematic diagram of a scene in which any camera in a circular camera array, provided by the present invention, acquires multiple frames of the first image;

[0026] Figure 4 This is a schematic diagram of the reconstruction of the object under test provided by the present invention;

[0027] Figure 5This is a schematic diagram of the parameter calibration device for the ring camera array provided by the present invention;

[0028] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] It should be noted that the circular camera array involved in the embodiments of the present invention refers to multiple cameras arranged in sequence. Optionally, the circular camera array may include a 360° circular camera array or a birdcage camera array, etc.

[0031] A camera is a device that acquires images of the object being measured. The camera can use a lens imaging model to increase the exposure and quickly acquire the image. However, the camera introduces distortion during the acquisition process, which makes the image less accurate.

[0032] The object to be tested refers to the same object that the circular camera array needs to photograph.

[0033] For example, such as Figure 1 The image shown is a schematic diagram of a scene where a ring camera array, provided by this invention, captures images of the object under test. Figure 1 In this system, the circular camera array can include 40 cameras, each of which can acquire a first image of the object under test, which is a checkerboard pattern.

[0034] Optionally, the checkerboard pattern can be used with a 7*5 calibration plate or other calibration plates; no specific limitation is made here.

[0035] It should be noted that the checkerboard pattern has the advantages of easy feature point extraction and allows electronic devices to accurately calibrate camera parameters for a circular camera array.

[0036] The electronic devices involved in the embodiments of the present invention may include: computers, mobile terminals, and wearable devices, etc.

[0037] Optionally, the electronic devices can be connected to each camera in the circular camera array via wireless communication technology, which may include, but is not limited to, one of the following: fourth-generation mobile communication technology (4G), fifth-generation mobile communication technology (5G), and wireless Fidelity (WiFi).

[0038] It should be noted that the execution subject involved in the embodiments of the present invention can be a parameter calibration device for a ring camera array or an electronic device. The embodiments of the present invention will be further described below using an electronic device as an example.

[0039] like Figure 2 The diagram shown is a flowchart illustrating the parameter calibration method for a ring camera array provided by this invention, which may include:

[0040] 201. Based on the first images of the object under test captured by each camera in the circular camera array at different angles, determine the camera intrinsic parameters corresponding to each camera.

[0041] Here, different angles refer to the camera shooting angles corresponding to the different positions of the object under test. For example, when the object under test is in the first position, any camera in the circular camera array takes a picture of the object under test at the first position to obtain the first frame of the first image; when the object under test is in the second position, any camera takes a picture of the object under test at the second position to obtain the second frame of the first image. The second position is different from the first position, and so on. The number of positions the object under test is in corresponds to the number of frames of the first image that any camera can collect. In other words, the object under test can correspond to one frame of the first image at one angle, and the object under test can correspond to multiple frames of the first image at different angles.

[0042] The first image refers to the color (Red Green Blue, RGB) image captured by the camera acquisition device in each camera of the object under test. That is, the first image may include the object under test.

[0043] Camera internal parameters refer to the camera's internal parameters. Optionally, these camera internal parameters may include the intrinsic parameter matrix K and / or distortion parameters, etc.

[0044] Optionally, the intrinsic parameter matrix K is used to project the image coordinates in the camera coordinate system to the imaging coordinate system. This intrinsic parameter matrix K may include: the camera's x-coordinate and focal length f.x The camera's vertical coordinate focal length f y The x-coordinate of the principal point of the imaging plane, C x and the ordinate C of the principal point of the imaging plane y wait;

[0045] Distortion parameters affect image quality and can include: first radial distortion k1, second radial distortion k2, third radial distortion k3, first tangential distortion p1, and second tangential distortion p2, etc.

[0046] The camera coordinate system refers to a three-dimensional rectangular coordinate system established with the camera's focal center as the origin and the camera's optical axis as the Z-axis. The X-axis of the camera coordinate system is parallel to the X-axis of the imaging coordinate system, and the Y-axis of the camera coordinate system is parallel to the Y-axis of the imaging coordinate system.

[0047] An imaging coordinate system, also known as an image coordinate system, refers to a two-dimensional rectangular coordinate system established with the center of the image plane as the origin. The X-axis and Y-axis of this imaging coordinate system are parallel to the two vertical sides of the image plane, respectively.

[0048] After each camera in the circular camera array acquires multiple first images of the object under test at different angles, it can send these multiple first images to an electronic device. Then, the electronic device can obtain the multiple first images sent by each camera and accurately determine the camera intrinsic parameters corresponding to each camera based on these multiple first images.

[0049] Optionally, the camera intrinsics for different cameras can be the same or different; no specific limitation is made here.

[0050] For example, such as Figure 3 The image shown is a schematic diagram of a scene where any camera in a circular camera array provided by this invention acquires multiple frames of the first image. Figure 3 In the diagram, any camera in the circular camera array is designated as camera A, and the target object is a 7x5 checkerboard pattern. Camera A maintains a constant shooting position and can capture N frames of the checkerboard pattern from different angles, where N ≥ 2. Based on these N frames, the electronic device can determine the intrinsic parameters of camera A.

[0051] In some embodiments, the electronic device determines the camera intrinsic parameters corresponding to each camera based on multiple frames of first images of the object under test captured by each camera in the circular camera array at different angles. This may include: the electronic device acquiring multiple frames of first images of the object under test captured by each camera in the circular camera array at different angles; the electronic device calibrating the multiple frames of first images using the Zhang Zhengyou calibration method to obtain the camera intrinsic parameters corresponding to each camera.

[0052] Among them, Zhang Zhengyou's calibration method refers to the conventional method used to calibrate the corresponding intrinsic parameters of a camera.

[0053] After acquiring multiple frames of first images of the object under test at different angles from each camera in the circular camera array, the electronic device can use the Zhang Zhengyou calibration method to perform parameter calibration processing on the multiple frames of first images acquired by each camera in order to accurately obtain the camera intrinsic parameters corresponding to each camera.

[0054] Optionally, the electronic device can use the calibration toolbox of scientific computing software (Matlab) to calibrate the camera intrinsic parameters corresponding to the camera.

[0055] Optionally, the electronic device acquires multiple frames of the first image of the object under test at different angles, captured by each camera in the circular camera array. This may include: the electronic device acquiring the...

[0056] The electronic device has a current remaining battery power; when the current remaining battery power is determined to be greater than a preset battery power threshold, the electronic device acquires multiple first images of the object under test from each camera in the circular camera array at different angles.

[0057] The current remaining power refers to the remaining power of the power supply device (e.g., battery) in the electronic device.

[0058] Optionally, the preset power threshold can be set before the electronic device leaves the factory or it can be user-defined; no specific limitation is made here.

[0059] After obtaining the current remaining battery power, the electronic device can...

[0060] The remaining battery level is compared with a preset battery threshold: if the remaining battery level is greater than the preset battery threshold, it indicates that the electronic device has sufficient battery power.

[0061] At this time, the electronic device can directly acquire multiple frames of first images of the object under test from different angles, captured by each camera in the circular camera array; if the electronic device determines that the current remaining power is less than or equal to the preset power threshold, it indicates that the electronic device has sufficient power.

[0062] In addition, the electronic device can output a first prompt message to remind the user to charge the electronic device.

[0063] For example, suppose the preset battery threshold is 20%. The electronic device obtains that its current remaining battery is 30%, which is greater than the preset battery threshold of 20%. At this time, the electronic device can directly acquire multiple frames of the first image of the object under test from each camera in the circular camera array at different angles.

[0064] 5.202. Group all cameras in the circular camera array sequentially to obtain multiple groups of phases.

[0065] The system uses a machine and, based on multiple frames of second images of the object under test captured by each camera in each group of cameras at different angles, determines the initial camera extrinsic parameters for each group of cameras.

[0066] Each group of cameras may include multiple cameras, and there may be the same cameras in two adjacent groups of cameras;

[0067] The second image refers to the RGB image of the object under test captured by the camera acquisition device in each camera in each group of cameras. The second image may also include the object under test. Multiple frames of the second image can also be called calibration data.

[0068] Camera external parameters refer to the external parameters of a camera. These parameters are used to transform image coordinates from the measurement coordinate system to the camera coordinate system. The measurement coordinate system is a three-dimensional Cartesian coordinate system, which can be used as a reference to accurately describe the relative spatial position between the camera and the object being measured.

[0069] Optionally, camera extrinsic parameters may include: rotation matrix R and / or translation matrix T, etc.

[0070] The rotation matrix R describes the orientation of the camera coordinate system's axes relative to the measurement coordinate system's axes; the translation matrix T describes the position of the camera coordinate system's origin in the measurement coordinate system.

[0071] For example, suppose the camera's intrinsic parameter matrix is... The rotation matrix of the camera is R. [3×3] The translation matrix of the camera is T. [1×3] The coordinates of a point in the measurement coordinate system are P(X, Y, Z).

[0072] Electronic devices can use the projection formula to project the coordinates P(X, Y, Z) onto the camera coordinate system to obtain p(x, y, z). The projection formula is p = K[R|T]P.

[0073] The electronic device can first group all the cameras in the circular camera array into multiple groups in sequence; then, each camera in each group can acquire multiple frames of second images of the object under test at different angles and send these multiple frames of second images to the electronic device; then, the electronic device can acquire the multiple frames of second images sent by each camera in each group and accurately determine the initial camera extrinsic parameters corresponding to each group of cameras based on these multiple frames of second images.

[0074] It should be noted that each group of cameras corresponds to one initial camera extrinsic parameter. In other words, the electronic device will acquire as many initial camera extrinsic parameters as there are groups of cameras, and the initial camera extrinsic parameters corresponding to different groups of cameras are different.

[0075] In this process, when each group of cameras is shooting the object under test, the position of the object under test should be distributed as evenly as possible within the shooting angle of each camera in the group. In this way, the initial camera extrinsic parameters obtained by the electronic device for each group of cameras are more accurate.

[0076] For example, each camera in each group can capture 30 frames of the second image.

[0077] It should be noted that the measurement coordinate systems corresponding to different groups of cameras are different.

[0078] In some embodiments, the electronic device may group all cameras in the circular camera array sequentially to obtain multiple groups of cameras. This may include: the electronic device labeling each camera in the circular camera array with a serial number and grouping the labeled cameras in a preset order to obtain multiple groups of cameras.

[0079] The preset order can include counterclockwise or clockwise order.

[0080] For example, a circular camera array may include 40 cameras. The electronic device can first label these 40 cameras with serial numbers, resulting in camera 1, camera 2, ..., camera 40. Then, the electronic device can group the 40 numbered cameras in a clockwise order to obtain multiple groups of cameras.

[0081] Optionally, these multiple camera groups can be nine groups, namely camera 1-camera 8, camera 5-camera 12, camera 9-camera 16, camera 13-camera 20, camera 17-camera 24, camera 21-camera 28, camera 25-camera 32, camera 29-camera 36 and camera 32-camera 40.

[0082] Optionally, these multiple camera groups can be 5 groups, namely camera 1-camera 15, camera 11-camera 25, camera 21-camera 30, camera 26-camera 35 and camera 31-camera 40.

[0083] In other words, the number of cameras in each group can be the same or different, and there is no specific limitation here. However, there will always be the same cameras in every two adjacent groups. This makes it easier for the electronic equipment to effectively map all cameras to the same measurement coordinate system.

[0084] In some embodiments, the electronic device determines the initial camera extrinsic parameters for each group of cameras based on multiple frames of second images of the object under test captured by each camera in each group of cameras at different angles. This can be achieved, but is not limited to, at least one of the following implementation methods:

[0085] Implementation Method 1: The electronic device acquires multiple frames of second images of the object under test captured by each camera in each group of cameras at different angles; the electronic device determines a first frame of the second image and multiple first feature points in the first frame of the second image from the multiple frames of the second image corresponding to each camera in each group of cameras; the electronic device determines the measurement coordinate system corresponding to each group of cameras based on any one of the multiple first feature points; the electronic device determines the initial camera extrinsic parameters corresponding to each group of cameras according to the measurement coordinate system.

[0086] After each camera in each group acquires multiple frames of second images of the object under test at different angles, it can send these multiple frames of second images to an electronic device. Then, based on each group of cameras, the electronic device determines the first frame of the second image acquired by each camera in that group from the multiple frames of second images. Next, the electronic device determines multiple first feature points corresponding to the object under test from the first frame of the second image, and randomly selects one of these first feature points as the center point of the measurement coordinate system. Then, based on the two vertical sides of the object under test, the electronic device determines the X-axis and Y-axis of the measurement coordinate system, thereby establishing the measurement coordinate system corresponding to each group of cameras. Since the object under test is different in the first frame of the second image acquired by each group of cameras, the measurement coordinate system corresponding to each group of cameras is also different. Finally, based on the measurement coordinate system, the electronic device can determine the initial camera extrinsic parameters corresponding to each group of cameras.

[0087] Optionally, when the object to be tested is a chessboard, any one of the multiple first feature points of the chessboard can be any corner point of the chessboard.

[0088] Optionally, the electronic device determines the initial camera extrinsic parameters corresponding to each group of cameras based on the measurement coordinate system. This may include: the electronic device using the solve Perspective-N-Point (solvePNP) function in the Open Source Computer Vision Library (OpenCV) to determine the initial camera extrinsic parameters corresponding to each group of cameras based on the measurement coordinate system.

[0089] It should be noted that the process by which the electronic device uses the solvePnP function to determine the initial camera extrinsic parameters can be viewed as an N-point perspective pose PNP problem solved by the electronic device for the first frame of the second image. For each group of cameras, the electronic device uses the coordinates of multiple first feature points in the measurement coordinate system and the perspective projection coordinates of the first frame of the second image in the imaging coordinate system to obtain the pose relationship between the camera coordinate system and the measurement coordinate system, thus obtaining the initial camera extrinsic parameters corresponding to each group of cameras.

[0090] Implementation Method 2: The electronic device determines the current camera extrinsic parameters corresponding to each group of cameras based on the measurement coordinate system; the electronic device acquires multiple frames of second images of the object under test at different angles collected by at least two cameras in each group of cameras; the electronic device determines the second feature points corresponding to each group of cameras based on the multiple frames of second images; the electronic device uses the current camera extrinsic parameters to perform triangulation and bundle adjustment (BA) optimization processing on the second feature points to obtain the initial camera extrinsic parameters corresponding to each group of cameras.

[0091] The electronic device can first determine the current camera extrinsic parameters corresponding to each group of cameras based on the measurement coordinate system. Since the current camera extrinsic parameters are not accurate enough, the electronic device needs to optimize them. Specifically, after acquiring multiple frames of second images of the object under test from at least two cameras in each group of cameras at different angles, the electronic device can extract the second feature points corresponding to each group of cameras based on these multiple frames of second images. The number of the second feature points is the sum of the number of feature points in each frame of the second image. Then, the electronic device uses the current camera extrinsic parameters to triangulate the second feature points to obtain their depth information, and then performs bundle adjustment (BA) optimization processing to accurately obtain the initial camera extrinsic parameters corresponding to each group of cameras.

[0092] For example, when the object to be tested is a checkerboard pattern, the number of second feature points is the number of frames in the second image multiplied by the number of sizes of the checkerboard pattern. Since the calibration board corresponding to the checkerboard pattern has a preset indexing method, the matching relationship between each second feature point is also preset.

[0093] Optionally, after determining the second feature point corresponding to each group of cameras, the method may further include: the electronic device saving the second feature point.

[0094] This allows electronic devices to directly acquire the saved second feature points later.

[0095] 203. Perform joint optimization processing on multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters corresponding to the ring camera array.

[0096] Since the measurement coordinate system of the initial camera extrinsic parameters corresponding to each group of cameras is different, it is not convenient for the electronic device to subsequently map all cameras to the same measurement coordinate system. Therefore, after obtaining multiple initial camera extrinsic parameters, the electronic device can perform joint optimization processing on these multiple initial camera extrinsic parameters to obtain the target camera extrinsic parameters corresponding to the circular camera array. In other words, the measurement coordinate system of the multiple initial camera extrinsic parameters after joint optimization processing is the same, which makes it convenient for the electronic device to subsequently effectively map all cameras to the same measurement coordinate system.

[0097] In some embodiments, the electronic device performs joint optimization processing on multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters corresponding to the circular camera array. This may include: the electronic device transforming the camera coordinate system of each first camera in a first group of cameras to a first measurement coordinate system corresponding to the first group of cameras, obtaining a first coordinate system corresponding to each first camera, wherein the first group of cameras is any one of multiple groups of cameras; the electronic device transforming the camera coordinate system of each second camera in a second group of cameras to a second measurement coordinate system corresponding to the second group of cameras, obtaining a second coordinate system corresponding to each second camera, wherein the second group of cameras is adjacent to the first group of cameras; the electronic device determining a target camera from the second group of cameras that is the same as the first group of cameras; the electronic device determining a transformation relationship based on the first coordinate system and the second coordinate system corresponding to the target camera; and the electronic device transforming the second coordinate system corresponding to each second camera to the first measurement coordinate system based on the transformation relationship to obtain the target camera extrinsic parameters corresponding to the circular camera array.

[0098] After acquiring the first coordinate system corresponding to each first camera and the second coordinate system corresponding to each second camera, the electronic device can determine the transformation relationship between the two sets of cameras based on the coordinate systems corresponding to the same cameras in the two sets of cameras, where the two sets of cameras are adjacent sets of cameras. Then, based on the transformation relationship, the electronic device can transform the second coordinate system corresponding to each second camera in the second set of cameras to the first measurement coordinate system corresponding to the first set of cameras. In this way, the measurement coordinate systems of the initial camera extrinsic parameters corresponding to the first set of cameras and the second set of cameras are the same, that is, the electronic device can accurately obtain the target camera extrinsic parameters corresponding to the circular camera array.

[0099] Optionally, the electronic device transforms the camera coordinate system of each first camera in the first group of cameras to the first measurement coordinate system corresponding to the first group of cameras, thereby obtaining the first coordinate system corresponding to each first camera. This may include: the electronic device converting the rotation matrix of the camera with the smallest serial number in the first group of cameras into an identity matrix and converting the translation matrix of the camera with the smallest serial number into an all-zero matrix; the electronic device then performs corresponding transformations on the cameras in the first group of cameras other than the camera with the smallest serial number, based on the transformation process of the camera with the smallest serial number.

[0100] This ensures that the relative camera extrinsics among all cameras in the first group remain constant.

[0101] It should be noted that the process by which the electronic device determines the second coordinate system corresponding to each second camera is similar to the process by which the electronic device determines the first coordinate system corresponding to each first camera, and will not be elaborated here.

[0102] Optionally, the electronic device determines the transformation relationship based on the first coordinate system and the second coordinate system corresponding to the target camera, which may include: the electronic device determining the transformation relationship based on a relative formula.

[0103] The relative formula is [R] i |T i ] 相对 =[R i |T i ] C -1 ·[R i |T i ] D ;

[0104] i represents the camera number corresponding to the same camera in the first group of cameras and the second group of cameras; [R i |T i ] 相对 This represents the transformation relationship, specifically the relative transformation matrix of camera i in the second coordinate system relative to the first coordinate system; [R] i |T i ] C Represents the transformation matrix corresponding to camera i in the first group of cameras; [R] i |T i ] D This represents the transformation matrix corresponding to camera i in the second group of cameras.

[0105] For example, the first group of cameras consists of cameras 1-8; the second group consists of cameras 5-12. The relative formula is [R i |T i ] 相对 =[R i |Ti ] 1-8 -1 ·[R i |T i ] 5-12 ;

[0106] i represents any camera number from camera 5 to camera 8; [R] i |T i ] 相对 This represents the relative transformation matrix of camera i from camera 5 to camera 12 relative to camera 1 to camera 8; [R] i |T i ] 1-8 Represents the transformation matrix corresponding to camera i in camera 1 through camera 8; [R i |T i ] 5-12 This represents the transformation matrix corresponding to camera i in camera 5 through camera 12.

[0107] In some embodiments, the electronic device determines the transformation relationship based on a first coordinate system and a second coordinate system corresponding to the target camera, which may include: the electronic device determining a relative transformation matrix corresponding to the target camera based on the first coordinate system and the second coordinate system corresponding to the target camera, the relative transformation matrix including a relative rotation matrix and a relative translation matrix; the electronic device converting the relative rotation matrix into Euler angles; and the electronic device determining the transformation relationship based on the relative translation matrix and the Euler angles.

[0108] The electronic device determines the relative transformation matrix corresponding to the same camera in the first group of cameras and the second group of cameras based on the coordinate system of the same camera. Then, the electronic device converts the relative rotation matrix in the relative transformation matrix into Euler angles, and determines the transformation relationship between the two groups of cameras based on the relative translation matrix in the relative transformation matrix and the Euler angles.

[0109] Optionally, the electronic device determines the relative transformation of the target camera based on the first coordinate system and the second coordinate system corresponding to the target camera. This may include: the electronic device determining the relative transformation of the target camera based on the average relative formula.

[0110] The relative transformation is [R] i |T i ] E =[R i |T i ] D ·[R i |T i ] 平均相对 -1 ;

[0111] i represents the camera number corresponding to the same camera in the first group of cameras and the second group of cameras; [R i |T i ] 平均相对 Represents the average relative transformation matrix of camera i in the second coordinate system to the first coordinate system; [R] i |T i ] E This represents the transformation matrix corresponding to camera i in the first and second groups of cameras after they are placed in a unified measurement coordinate system.

[0112] It should be noted that when there is only one target camera, the relative transformation is the relative transformation corresponding to that single camera; when there are multiple target cameras, the relative transformation is the average of the relative transformations corresponding to all of them.

[0113] For example, the first group of cameras consists of cameras 1-8; the second group consists of cameras 5-12. The relative transformation is [R...]. i |T i ] 1-12 =[R i |T i ] 5-12 -1 ·[R i |T i ] 平均相对 -1 ;

[0114] i represents any camera number from camera 5 to camera 8; [R] i |T i ] 平均相对 This represents the average relative transformation matrix of camera i from camera 5 to camera 12 relative to camera 1 to camera 8; [R] i |T i ] 1-12 This represents the transformation matrix corresponding to camera i in camera 1 through camera 12.

[0115] 204. Determine the joint calibration parameter results corresponding to the ring camera array based on the camera's intrinsic parameters and the target camera's extrinsic parameters.

[0116] It is understandable that the joint calibration parameter results obtained by the electronic device may include camera intrinsic parameters and target camera extrinsic parameters.

[0117] Optionally, after step 204, the method may also be implemented in at least one of the following ways:

[0118] Implementation Method 1: The electronic device determines the third feature point corresponding to the circular camera array based on multiple frames of third images; the electronic device uses the target camera extrinsic parameters to triangulate the third feature point to obtain the three-dimensional coordinate point corresponding to the third feature point, and then uses BA optimization to obtain the final camera extrinsic parameters of the circular camera array.

[0119] Optionally, the third image mentioned above can be a newly captured image by the camera or the second image mentioned above; no specific limitation is made here.

[0120] After determining the third feature point corresponding to the circular camera array based on multiple frames of third images, the electronic device can use the target camera extrinsic parameters to triangulate the third feature point to obtain the depth information of the third feature point. Then, the electronic device performs BA optimization processing on the depth information to accurately obtain the final camera extrinsic parameters of the circular camera array.

[0121] For example, such as Figure 4 The image shown is a schematic diagram of the reconstruction of the object under test provided by this invention. Figure 4 In this process, electronic devices can effectively obtain the complete three-dimensional structure of the object under test.

[0122] Implementation method 2: The electronic device outputs a second prompt message, which is used to inform the user that the electronic device has successfully obtained the joint calibration parameter results.

[0123] This allows users to promptly know that their electronic devices have successfully obtained the joint calibration parameter results.

[0124] In this embodiment of the invention, based on multiple frames of first images of the object under test captured by each camera in the circular camera array at different angles, the intrinsic parameters of each camera are determined. All cameras in the circular camera array are sequentially grouped to obtain multiple groups of cameras. Based on multiple frames of second images of the object under test captured by each camera in each group at different angles, the initial extrinsic parameters of each group of cameras are determined. Joint optimization processing is performed on the multiple initial extrinsic parameters to determine the target extrinsic parameters of the circular camera array. Based on the intrinsic and extrinsic parameters of the cameras, the joint calibration parameter results of the circular camera array are determined. This method addresses the limitations of existing multi-camera parameter calibration methods, which cannot accurately calibrate the camera parameters of multiple cameras, and enables accurate calibration of the camera parameters of a circular camera array.

[0125] Furthermore, the parameter calibration method for the ring camera array provided by this invention can effectively solve the problem that when the number of cameras to be jointly calibrated increases, all cameras may not be able to simultaneously view the corresponding image of the object under test (e.g., a checkerboard pattern) or some cameras may have very poor viewing angles.

[0126] The parameter calibration device for the ring camera array provided by the present invention will be described below. The parameter calibration device for the ring camera array described below and the parameter calibration method for the ring camera array described above can be referred to in correspondence.

[0127] like Figure 5 The diagram shown is a structural schematic of the parameter calibration device for a ring camera array provided by the present invention, which may include:

[0128] The processing module 501 is used to determine the camera intrinsic parameters of each camera based on multiple frames of first images of the object under test acquired by each camera in the circular camera array at different angles; to group all cameras in the circular camera array sequentially to obtain multiple groups of cameras, and to determine the initial camera extrinsic parameters of each group of cameras based on multiple frames of second images of the object under test acquired by each camera in each group at different angles; wherein each group of cameras includes multiple cameras, and there are identical cameras in adjacent groups; to perform joint optimization processing on the multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters of the circular camera array; and to determine the joint calibration parameter results of the circular camera array based on the camera intrinsic parameters and the target camera extrinsic parameters.

[0129] Optionally, the acquisition module 502 is used to acquire multiple frames of second images of the object under test at different angles, captured by each camera in each group of cameras.

[0130] The processing module 501 is specifically used to determine a first frame of second images and a plurality of first feature points in the first frame of second images from the multiple frames of second images corresponding to each camera in each group of cameras; determine the measurement coordinate system corresponding to each group of cameras based on any one of the plurality of first feature points; and determine the initial camera extrinsic parameters corresponding to each group of cameras according to the measurement coordinate system.

[0131] Optionally, the processing module 501 is specifically configured to: transform the camera coordinate system of each first camera in the first group of cameras to the first measurement coordinate system corresponding to the first group of cameras, thereby obtaining the first coordinate system corresponding to each first camera, wherein the first group of cameras is any one of the multiple groups of cameras; transform the camera coordinate system of each second camera in the second group of cameras to the second measurement coordinate system corresponding to the second group of cameras, thereby obtaining the second coordinate system corresponding to each second camera, wherein the second group of cameras is adjacent to the first group of cameras; determine a target camera that is the same as the first group of cameras from the second group of cameras; determine a transformation relationship based on the first coordinate system and the second coordinate system corresponding to the target camera; and transform the second coordinate system corresponding to each second camera to the first measurement coordinate system based on the transformation relationship to obtain the target camera extrinsic parameters corresponding to the circular camera array.

[0132] Optionally, the processing module 501 is specifically used to determine the relative transformation matrix corresponding to the target camera based on the first coordinate system and the second coordinate system corresponding to the target camera, the relative transformation matrix including the relative rotation matrix and the relative translation matrix; convert the relative rotation matrix into Euler angles; and determine the transformation relationship based on the relative translation matrix and the Euler angles.

[0133] Optionally, the processing module 501 is specifically used to determine the current camera extrinsic parameters corresponding to each group of cameras based on the measurement coordinate system;

[0134] The acquisition module 502 is specifically used to acquire multiple frames of second images of the object under test at different angles, captured by at least two cameras in each group of cameras.

[0135] The processing module 501 is specifically used to determine the second feature point corresponding to each group of cameras based on the multiple frames of the second image; and to perform triangulation and bundle adjustment (BA) optimization processing on the second feature point using the current camera extrinsic parameters to obtain the initial camera extrinsic parameters corresponding to each group of cameras.

[0136] Optionally, the processing module 501 is specifically used to label each camera in the circular camera array with a serial number, and group the labeled cameras according to a preset order to obtain multiple groups of cameras. The preset order includes a counterclockwise order or a clockwise order.

[0137] Optionally, the acquisition module 502 is specifically used to acquire multiple frames of first images of the object under test at different angles, captured by each camera in the circular camera array.

[0138] The processing module 501 is specifically used to calibrate the first multi-frame image using the Zhang Zhengyou calibration method to obtain the camera intrinsic parameters corresponding to each camera.

[0139] like Figure 6The diagram shown is a structural schematic of the electronic device provided by the present invention. The electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logic instructions in the memory 630 to execute a parameter calibration method for a ring camera array. This method includes: determining the intrinsic parameters of each camera based on multiple frames of first images of the object under test captured by each camera in the ring camera array at different angles; grouping all cameras in the ring camera array sequentially to obtain multiple groups of cameras, and determining the initial extrinsic parameters of each group of cameras based on multiple frames of second images of the object under test captured by each camera in each group at different angles; wherein each group of cameras includes multiple cameras, and adjacent groups contain identical cameras; performing joint optimization processing on the multiple initial extrinsic parameters to determine the target extrinsic parameters of the ring camera array; and determining the joint calibration parameter result of the ring camera array based on the intrinsic parameters and the target extrinsic parameters.

[0140] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the parameter calibration method for the ring camera array provided by the above methods. The method includes: determining the camera intrinsic parameters corresponding to each camera based on multiple frames of first images of the object under test acquired by each camera in the ring camera array at different angles; grouping all cameras in the ring camera array sequentially to obtain multiple groups of cameras, and determining the initial camera extrinsic parameters corresponding to each group of cameras based on multiple frames of second images of the object under test acquired by each camera in each group at different angles; wherein each group of cameras includes multiple cameras, and there are identical cameras in adjacent groups of cameras; performing joint optimization processing on the multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters corresponding to the ring camera array; and determining the joint calibration parameter result corresponding to the ring camera array based on the camera intrinsic parameters and the target camera extrinsic parameters.

[0142] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a parameter calibration method for a ring camera array provided by the methods described above. The method includes: determining camera intrinsic parameters corresponding to each camera based on multiple frames of first images of the object under test acquired by each camera in the ring camera array at different angles; grouping all cameras in the ring camera array sequentially to obtain multiple groups of cameras, and determining initial camera extrinsic parameters corresponding to each group of cameras based on multiple frames of second images of the object under test acquired by each camera in each group at different angles; wherein each group of cameras includes multiple cameras, and there are identical cameras in adjacent groups; performing joint optimization processing on the multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters corresponding to the ring camera array; and determining the joint calibration parameter result corresponding to the ring camera array based on the camera intrinsic parameters and the target camera extrinsic parameters.

[0143] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for calibrating parameters of a circular camera array, characterized in that, include: Based on the multiple first images of the object under test captured by each camera in the circular camera array at different angles, the camera intrinsic parameters corresponding to each camera are determined. All cameras in the circular camera array are sequentially grouped to obtain multiple groups of cameras. Based on the multiple frames of second images of the object under test captured by each camera in each group at different angles, the initial camera extrinsic parameters corresponding to each group of cameras are determined. Each group of cameras includes multiple cameras, and there are identical cameras in adjacent groups. The initial camera extrinsic parameters are jointly optimized to determine the target camera extrinsic parameters corresponding to the ring camera array. The step of jointly optimizing multiple initial camera extrinsics to determine the target camera extrinsics corresponding to the ring camera array includes: The camera coordinate system of each first camera in the first group of cameras is transformed to the first measurement coordinate system corresponding to the first group of cameras to obtain the first coordinate system corresponding to each first camera. The first group of cameras is any group of cameras in the plurality of camera groups. The camera coordinate system of each second camera in the second group of cameras is transformed to the second measurement coordinate system corresponding to the second group of cameras to obtain the second coordinate system corresponding to each second camera. The second group of cameras is adjacent to the first group of cameras. From the second group of cameras, identify the target camera that is the same as the first group of cameras; The transformation relationship is determined based on the first coordinate system and the second coordinate system corresponding to the target camera; Based on the transformation relationship, the second coordinate system corresponding to each second camera is transformed to the first measurement coordinate system to obtain the target camera extrinsic parameters corresponding to the ring camera array; Based on the intrinsic parameters of the camera and the extrinsic parameters of the target camera, the joint calibration parameter results corresponding to the ring camera array are determined.

2. The method of claim 1, wherein, The determination of initial camera extrinsic parameters for each camera group, based on multiple frames of second images of the object under test captured by each camera in each camera group at different angles, includes: Acquire multiple frames of second images of the object under test at different angles, captured by each camera in each group of cameras; From the multiple frames of second images corresponding to each camera in each group of cameras, determine the first frame of the second image and a plurality of first feature points in the first frame of the second image; Based on any one of the plurality of first feature points, determine the measurement coordinate system corresponding to each group of cameras; Based on the measurement coordinate system, determine the initial camera extrinsic parameters corresponding to each group of cameras.

3. The method of claim 1, wherein, Determining the transformation relationship based on the first coordinate system and the second coordinate system corresponding to the target camera includes: Based on the first coordinate system and the second coordinate system corresponding to the target camera, the relative transformation matrix corresponding to the target camera is determined, and the relative transformation matrix includes a relative rotation matrix and a relative translation matrix; Convert the relative rotation matrix into Euler angles; The transformation relationship is determined based on the relative translation matrix and the Euler angles.

4. The method of claim 2, wherein, The step of determining the initial camera extrinsic parameters corresponding to each group of cameras based on the measurement coordinate system includes: Based on the measurement coordinate system, determine the current camera extrinsic parameters corresponding to each group of cameras; Acquire multiple frames of second images of the object under test at different angles, captured by at least two cameras in each group of cameras; Based on the multiple frames of the second images, determine the second feature points corresponding to each group of cameras; Using the current camera extrinsic parameters, the second feature point is triangulated and bundled adjustment (BA) is performed to obtain the initial camera extrinsic parameters corresponding to each group of cameras.

5. The method according to any one of claims 1-4, characterized in that, The step involves sequentially grouping all cameras in the circular camera array to obtain multiple camera groups, including: Each camera in the circular camera array is numbered, and the numbered cameras are grouped according to a preset order to obtain multiple groups of cameras. The preset order includes counterclockwise or clockwise order.

6. The method according to any one of claims 1-4, characterized in that, The determination of camera intrinsic parameters for each camera based on multiple frames of first images of the object under test captured by each camera in the circular camera array at different angles includes: Acquire multiple frames of the first image of the object under test at different angles, captured by each camera in the circular camera array; The Zhang Zhengyou calibration method is used to calibrate the multiple frames of the first image to obtain the camera intrinsic parameters corresponding to each camera.

7. A device for calibrating parameters of an annular camera array, characterized in that, include: The processing module is used to determine the camera intrinsic parameters corresponding to each camera based on the multiple frames of first images of the object under test acquired by each camera in the circular camera array at different angles. All cameras in the circular camera array are sequentially grouped to obtain multiple camera groups. Based on multiple frames of second images of the object under test captured by each camera in each group at different angles, the initial camera extrinsic parameters corresponding to each camera group are determined. Each camera group includes multiple cameras, and adjacent groups may contain identical cameras. The multiple initial camera extrinsic parameters are jointly optimized to determine the target camera extrinsic parameters corresponding to the circular camera array. The joint optimization of the multiple initial camera extrinsic parameters to determine the target camera extrinsic parameters corresponding to the circular camera array includes: transforming the camera coordinate system of each first camera in the first camera group to the first measurement coordinate system corresponding to the first camera group, obtaining the first coordinate system corresponding to each first camera. The camera group is any one of the multiple camera groups; the camera coordinate system of each second camera in the second camera group is transformed to the second measurement coordinate system corresponding to the second camera group to obtain the second coordinate system corresponding to each second camera, and the second camera group is adjacent to the first camera group; from the second camera group, a target camera that is the same as the first camera group is determined; according to the first coordinate system and the second coordinate system corresponding to the target camera, a transformation relationship is determined; according to the transformation relationship, the second coordinate system corresponding to each second camera is transformed to the first measurement coordinate system to obtain the target camera extrinsic parameters corresponding to the circular camera array; according to the camera intrinsic parameters and the target camera extrinsic parameters, the joint calibration parameter results corresponding to the circular camera array are determined.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the parameter calibration method for the ring camera array as described in any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the parameter calibration method for the ring camera array as described in any one of claims 1 to 6.

Images