Camera calibration method and device, computer readable storage medium, terminal

Abstract

A camera calibration method and device, a computer readable storage medium and a terminal are provided. The method comprises: determining several reference cameras from a plurality of cameras to be calibrated, and the remaining cameras to be calibrated are referred to as non-reference cameras; for each non-reference camera, determining a first mapping matrix and a second mapping matrix of an image coordinate system of the current non-reference camera mapping to image coordinate systems of a previous adjacent reference camera and a next adjacent reference camera, respectively; determining a first reference mapping matrix and a second reference mapping matrix of the image coordinate systems of the previous adjacent reference camera and the next adjacent reference camera mapping to a world coordinate system, respectively; and taking a weighted operation result of a product of the first mapping matrix and the first reference mapping matrix and a product of the second mapping matrix and the second reference mapping matrix as a mapping matrix of the current non-reference camera mapping to the world coordinate system. In a large-scale camera calibration scenario, the above scheme can improve the efficiency and accuracy of calibration.

Description

Technical Field

[0001] This invention relates to the field of camera calibration technology, and in particular to a camera calibration method and apparatus, a computer-readable storage medium, and a terminal. Background Technology

[0002] In the context of vehicle-road cooperative technology, to enhance the construction of smart highways and maximize the value of vehicle-road cooperation, it is necessary to deploy a large-scale camera network (usually a large number of cameras) on the roadside to provide comprehensive visual perception information. However, to fully utilize roadside surveillance cameras and analyze key information such as the position and speed of targets from video or images, the cameras must be calibrated, meaning that a connection (or mapping relationship) has been established between the camera's image coordinate system and the world coordinate system.

[0003] In existing technologies, the calibration of cameras at intersections or along roadsides typically involves calibrating each camera individually, either on a single pole or individually. Specifically, for each camera, the mapping relationship between the camera's image coordinate system and the world coordinate system is determined based on a specific location in the image captured by the camera and its corresponding actual location in the world coordinate system. However, since the actual location in the world coordinate system is usually determined through manual surveying or map data analysis, calibrating each camera individually on a single pole or individually is costly and inefficient, making it difficult to meet the calibration requirements of a large number of roadside cameras (e.g., a batch of roadside cameras installed on highways). Summary of the Invention

[0004] The technical problem solved by the embodiments of the present invention is how to improve the efficiency and accuracy of camera calibration in large-scale camera calibration scenarios.

[0005] To achieve the above objectives, embodiments of the present invention provide a camera calibration method, comprising the following steps: determining a plurality of reference cameras from a plurality of cameras to be calibrated, and denoting the remaining cameras to be calibrated as non-reference cameras; for each non-reference camera, determining a first mapping matrix and a second mapping matrix for the current non-reference camera, wherein the first mapping matrix indicates the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the preceding adjacent reference camera, and the second mapping matrix indicates the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the following adjacent reference camera; and determining the first reference mapping matrix of the preceding adjacent reference camera of the current non-reference camera, and... The second reference mapping matrix of the next adjacent reference camera, wherein the first reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the previous adjacent reference camera, and the second reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the next adjacent reference camera; the product of the first mapping matrix and the first reference mapping matrix is ​​determined and denoted as the first matrix product, and the product of the second mapping matrix and the second reference mapping matrix is ​​determined and denoted as the second matrix product; a weighted operation is performed on the first matrix product and the second matrix product, and the weighted operation result is used as the mapping matrix of the current non-reference camera to the world coordinate system.

[0006] Optionally, determining a plurality of reference cameras from a plurality of cameras to be calibrated includes: determining a first reference camera from the plurality of cameras to be calibrated, and then, starting from the first reference camera, determining the next reference camera of the current reference camera in each round; wherein, in each round, determining the next reference camera of the current reference camera includes: determining a next pre-selected reference camera of the current reference camera, such that the number of cameras between the current reference camera and the next pre-selected reference camera meets a preset number of intermediate cameras; determining a first calibration error coefficient based on the current reference camera and the next pre-selected reference camera, and whenever the first calibration error coefficient is less than or equal to a first error threshold, increasing the number of intermediate cameras to redetermine subsequent pre-selected reference cameras and continuing to determine the first calibration error coefficient, until the first calibration error coefficient is greater than the first error threshold, and using the determined pre-selected reference camera as the next reference camera of the current reference camera; wherein, the next reference camera of the current reference camera in the current round is used as the current reference camera in the next round.

[0007] Optionally, determining the first calibration error coefficient based on the current reference camera and the next pre-selected reference camera includes: selecting a target camera between the current reference camera and the next pre-selected reference camera, and selecting a first preset number of target points from the images acquired by the target camera; determining the mapping points of each target point to the world coordinate system based on the mapping matrix from the target camera to the current reference camera and the mapping matrix from the current reference camera to the world coordinate system, denoted as a first mapping point set; determining the mapping points of each target point to the world coordinate system based on the mapping matrix from the target camera to the next pre-selected reference camera and the mapping matrix from the next pre-selected reference camera to the world coordinate system, denoted as a second mapping point set; determining the distance between each first mapping point in the first mapping point set and the corresponding second mapping point in the second mapping point set, and using the average of the determined distances as the first calibration error coefficient.

[0008] Optionally, the first error threshold is determined as a preset percentage of the weighted calculation result between the first distance between the target camera and the current reference camera and the second distance between the target camera and the next pre-selected reference camera.

[0009] Optionally, the target camera is a camera located at an intermediate position between the current reference camera and the next pre-selected reference camera.

[0010] Optionally, after determining several reference cameras from multiple cameras to be calibrated, and before determining the first mapping matrix and the second mapping matrix of each non-reference camera, the method further includes: for each non-reference camera between each pair of adjacent reference cameras, determining the mapping error coefficient of the non-reference camera based on the common viewing area between the non-reference camera and the previous camera; if the sum of the mapping error coefficients of each non-reference camera between the pair of adjacent reference cameras is greater than a preset mapping error threshold, then selecting one or more non-reference cameras between the pair of adjacent reference cameras as reference cameras.

[0011] Optionally, determining the mapping error coefficient of the reference camera based on the shared viewing area between the non-reference camera and the previous camera includes: mapping the shared viewing area in the image acquired by the non-reference camera to the image acquired by the previous camera based on the mapping matrix between the non-reference camera and the previous camera to obtain the corresponding mapped area; and determining the mapping error coefficient of the non-reference camera based on each pixel of the mapped area and the original shared viewing area in the image acquired by the previous camera.

[0012] Optionally, determining the mapping error coefficient of the non-reference camera based on each pixel of the mapped region and the original co-view region in the image captured by the previous camera includes: using the L1 norm, determining the L1 norm value based on the distance between the horizontal coordinates and the vertical coordinates of each pair of pixels in the mapped region and the original co-view region in the image captured by the previous camera, and using the ratio of the L1 norm value to the area of ​​the mapped region as the mapping error coefficient of the non-reference camera.

[0013] Optionally, the method further includes: for each pair of adjacent reference cameras, determining a second calibration error coefficient based on the mapping matrix of each non-reference camera between the pair of adjacent reference cameras to the world coordinate system; if the second calibration error coefficient is greater than a second error threshold, then selecting one or more non-reference cameras between the pair of adjacent reference cameras as reference cameras.

[0014] Optionally, determining the second calibration error coefficient based on the mapping matrix from each non-reference camera between the pair of adjacent reference cameras to the world coordinate system includes: for each non-reference camera located between the pair of adjacent reference cameras, selecting a second preset number of target points from the images acquired by the non-reference camera; determining the actual position in the world coordinate system corresponding to each selected target point; determining the mapping points of each target point to the world coordinate system according to the mapping matrix from the non-reference camera to the world coordinate system, denoted as a third mapping point set; determining the distance between each third mapping point in the third mapping point set and its corresponding actual position, and determining the average value of each distance, denoted as a first average calculation result; and using the average value of the first average calculation results corresponding to each non-reference camera located between the pair of adjacent reference cameras as the second calibration error coefficient.

[0015] Optionally, determining the first mapping matrix of the current non-reference camera includes: for each non-reference camera between the current non-reference camera and the previous adjacent reference camera, determining a mapping matrix that maps the image coordinate system of the non-reference camera to the image coordinate system of the previous camera; and taking the product of the determined mapping matrices as the first mapping matrix.

[0016] This invention also provides a camera calibration apparatus, comprising: a reference camera determination module, configured to determine a plurality of reference cameras from a plurality of cameras to be calibrated, wherein the remaining cameras to be calibrated are denoted as non-reference cameras; a camera mapping matrix determination module, configured to, for each non-reference camera, determine a first mapping matrix and a second mapping matrix for the current non-reference camera, wherein the first mapping matrix indicates the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the preceding adjacent reference camera, and the second mapping matrix indicates the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the following adjacent reference camera; and a reference mapping matrix determination module, configured to determine the first reference mapping matrix of the preceding adjacent reference camera for each non-reference camera. The system includes a first reference mapping matrix and a second reference mapping matrix for the next adjacent reference camera. The first reference mapping matrix indicates the mapping relationship between the image coordinate system of the previous adjacent reference camera and the world coordinate system. The second reference mapping matrix indicates the mapping relationship between the image coordinate system of the next adjacent reference camera and the world coordinate system. The system also includes a matrix multiplication module for determining the product of the first mapping matrix and the first reference mapping matrix, denoted as the first matrix product, and the product of the second mapping matrix and the second reference mapping matrix, denoted as the second matrix product. The system further includes a calibration result determination module for performing a weighted operation on the first matrix product and the second matrix product, and using the weighted operation result as the mapping matrix for mapping the current non-reference camera to the world coordinate system.

[0017] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the steps of the camera calibration method described above.

[0018] This invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of the camera calibration method described above when running the computer program.

[0019] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0020] Since determining the mapping relationship between the image coordinate systems of two cameras often only involves point processing between images, it is simpler and more efficient than determining the mapping relationship between the camera's image coordinate system and the world coordinate system (which involves mapping points in the image to points in real world space). Therefore, this implementation scheme uses the baseline mapping relationship (i.e., the mapping relationship between the image coordinate system and the world coordinate system of the baseline cameras) of several established reference cameras (the number of reference cameras only accounts for a small portion of the total number of cameras to be calibrated) as a basis, combined with the relative relationship between non-reference cameras and reference cameras (the mapping relationship between image coordinate systems), to determine the mapping matrix (i.e., the calibration result) of the image coordinate system to the world coordinate system of all non-reference cameras. Compared with the existing technology, which requires performing a complex mapping process between the image coordinate system and the world coordinate system for each camera to be calibrated, this implementation scheme can significantly reduce calibration costs and improve calibration efficiency.

[0021] Furthermore, if the number of cameras (or intermediate cameras) between the current non-reference camera and the previous adjacent reference camera is large, accumulated errors may occur during the process of determining the mapping relationship (i.e., the first mapping matrix) between the image coordinate system of the current non-reference camera and the image coordinate system of the previous adjacent reference camera. Therefore, if the current non-reference camera is calibrated using only the first mapping matrix between the current non-reference camera and the previous adjacent reference camera (i.e., single-sided cascade calibration), the accuracy of the calibration results will be reduced. To solve this problem, this embodiment adopts a double-sided cascade calibration method, which uses not only the mapping relationship between the current non-reference camera and the previous adjacent reference camera (i.e., the first mapping matrix), but also the mapping relationship between the current non-reference camera and the next adjacent reference camera (i.e., the second mapping matrix). Then, the weighted result of the product of the first mapping matrix and the first reference mapping matrix, and the product of the second mapping matrix and the second reference mapping matrix, is used as the final calibration result of the current reference camera. This can mitigate the aforementioned accumulated errors and improve the accuracy of the calibration results.

[0022] Furthermore, the step of determining several reference cameras from a plurality of cameras to be calibrated includes: determining a first reference camera from the plurality of cameras to be calibrated, and then, starting from the first reference camera, determining the next reference camera for the current reference camera in each round; wherein, in each round, by iteratively comparing the determined first calibration error coefficient with a first error threshold, the maximum number of intermediate cameras that meet the calibration error requirements is determined, thereby determining the next reference camera for the current reference camera in the current round; wherein, the next reference camera for the current reference camera in the current round is used as the current reference camera for the next round. Thus, by performing multiple rounds of the process of determining the next reference camera for the current reference camera, each determined reference camera can meet the preset calibration error requirements.

[0023] Compared to existing technologies that may randomly select several reference cameras, this implementation scheme avoids excessive errors caused by random selection of reference cameras, improving the accuracy of subsequent cascade calibration of various non-reference cameras based on the reference cameras. Furthermore, this implementation scheme can, while meeting error requirements, maximize the number of cameras between two adjacent reference cameras (i.e., reduce the total number of reference cameras). As discussed above, considering the more complex process of determining the reference mapping matrix for the reference cameras, this implementation scheme, by reducing the number of reference cameras, signifies a reduction in calibration costs and an increase in calibration efficiency.

[0024] Furthermore, after determining several reference cameras from multiple cameras to be calibrated, and before determining the first mapping matrix and second mapping matrix of each non-reference camera, the method further includes: for each non-reference camera between each pair of adjacent reference cameras, determining the mapping error coefficient of the non-reference camera based on the common viewing area between the non-reference camera and the previous camera; if the sum of the mapping error coefficients of each non-reference camera between the pair of adjacent reference cameras is greater than a preset mapping error threshold, then one or more non-reference cameras are selected as reference cameras between the pair of adjacent reference cameras. In this embodiment of the invention, by using the sum of the mapping error coefficients of each non-reference camera between the determined adjacent reference cameras (i.e., the cumulative mapping error coefficient), the number of determined reference cameras is evaluated to determine whether it is sufficient (the more reference cameras, the smaller the cumulative error), and when the cumulative error coefficient does not meet the requirements, the cumulative error can be weakened and the accuracy of the calibration results improved by increasing the number of reference cameras. Attached Figure Description

[0025] Figure 1 This is a flowchart of a camera calibration method according to an embodiment of the present invention;

[0026] Figure 2This is a schematic diagram illustrating the process of determining several reference cameras from multiple cameras to be calibrated in an embodiment of the present invention;

[0027] Figure 3 This is a partial flowchart of another camera calibration method in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of a camera calibration device according to an embodiment of the present invention. Detailed Implementation

[0029] To make the above-mentioned objectives, features and beneficial effects of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] Reference Figure 1 , Figure 1 This is a flowchart of a camera calibration method according to an embodiment of the present invention.

[0031] The method may include steps S11 to S15:

[0032] Step S11: Determine a number of reference cameras from the multiple cameras to be calibrated, and the remaining cameras to be calibrated are denoted as non-reference cameras;

[0033] Step S12: For each non-reference camera, determine the first mapping matrix and the second mapping matrix of the current non-reference camera respectively. The first mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the previous adjacent reference camera. The second mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the next adjacent reference camera.

[0034] Step S13: Determine the first reference mapping matrix of the previous adjacent reference camera and the second reference mapping matrix of the next adjacent reference camera for the current non-reference camera. The first reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the previous adjacent reference camera, and the second reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the next adjacent reference camera.

[0035] Step S14: Determine the product of the first mapping matrix and the first reference mapping matrix, denoted as the first matrix product, and determine the product of the second mapping matrix and the second reference mapping matrix, denoted as the second matrix product;

[0036] Step S15: Perform a weighted operation on the product of the first matrix and the product of the second matrix, and use the weighted operation result as the mapping matrix for mapping the current non-reference camera to the world coordinate system.

[0037] In this embodiment, the camera calibration process described above is also the process of determining the mapping matrix between the image coordinate system and the world coordinate system of each camera to be calibrated. The mapping matrix can be a homography matrix to represent the mapping relationship between two planes, and this mapping relationship can specifically include rotation and translation relationships.

[0038] The image coordinate system (Pixel Coordinate System, PCS) can also be called a pixel coordinate system. For example, a Cartesian coordinate system uv can be defined as the image coordinate system on the image captured by the camera. The coordinates of each pixel in the image captured by the camera can be represented by (u, v). Here, u represents the x-coordinate of the pixel in the image coordinate system, and v represents the x-coordinate of the pixel in the image coordinate system.

[0039] The World Coordinate System (WCS) is used in 3D scenes where there is usually more than one object. Objects need to be placed in the scene according to a specific arrangement. Each object is distributed in an appropriate position within the scene. The coordinate system of the entire scene can be called the World Coordinate System. Without limitation, the World Coordinate System can be a geocentric coordinate system (also known as the East-North-Sky coordinate system, or ENU coordinate system).

[0040] In the specific implementation of step S11, the plurality of cameras to be calibrated can be some or all of a batch of cameras deployed along the side of the road. The number of cameras to be calibrated can reach dozens or even hundreds. The road can be, for example, a highway, an elevated road, a race track, or other road type where the road surface has no obvious undulations over a long distance and can be approximated as a plane. For example, the height difference of the road surface remains within a preset height difference range. Without limitation, the preset height difference can be selected from [0cm, 20cm].

[0041] Specifically, the plurality of cameras to be calibrated can be selected from a large-scale deployment of cameras that have undergone distortion correction and whose installation pitch angle, intrinsic parameters (e.g., focal length), etc., are consistent or nearly consistent. Among them, there is often a common field of view (or field of view overlap) between each pair of adjacent cameras to be calibrated.

[0042] Furthermore, in step S11, determining a number of reference cameras from a plurality of cameras to be calibrated may specifically include: determining the first reference camera from the plurality of cameras to be calibrated, and then, starting from the first reference camera, determining the next reference camera after the current reference camera in turn.

[0043] In each round, determining the next reference camera for the current reference camera includes: determining a next pre-selected reference camera for the current reference camera, such that the number of cameras between the current reference camera and the next pre-selected reference camera satisfies a preset number of intermediate cameras m (m is a positive integer); determining a first calibration error coefficient q1 based on the current reference camera and the next pre-selected reference camera; whenever the first calibration error coefficient q1 is less than or equal to a first error threshold Q1, increasing the number of intermediate cameras m to redetermine the subsequent pre-selected reference camera and continuing to determine the first calibration error coefficient q1 until the first calibration error coefficient q1 is greater than the first error threshold Q1, and using the determined pre-selected reference camera as the next reference camera for the current reference camera; wherein, the next reference camera for the current reference camera in the current round is used as the current reference camera in the next round.

[0044] In specific implementation, whenever the first calibration error coefficient q1 is less than or equal to the first error threshold Q1, the number of intermediate cameras m can be increased by using a preset step size (e.g., increasing by 1 each time) to increase the value of m, or by using a preset linear function to increase the value of m. This embodiment of the invention does not limit the method of increasing the value of m during the iteration process.

[0045] Furthermore, determining the first calibration error coefficient q1 based on the current reference camera and the next pre-selected reference camera may specifically include: selecting a target camera between the current reference camera and the next pre-selected reference camera, and selecting a first preset number of target points from the images acquired by the target camera; determining the mapping points of each target point to the world coordinate system based on the mapping matrix from the target camera to the current reference camera and the mapping matrix from the current reference camera to the world coordinate system, denoted as a first mapping point set; determining the mapping points of each target point to the world coordinate system based on the mapping matrix from the target camera to the next pre-selected reference camera and the mapping matrix from the next pre-selected reference camera to the world coordinate system, denoted as a second mapping point set; determining the distance between each first mapping point in the first mapping point set and the corresponding second mapping point in the second mapping point set, and using the average of the determined distances as the first calibration error coefficient.

[0046] In specific implementation, the value of the first preset quantity should not be set too small, otherwise the amount of data involved in the calculation will be too small, resulting in low accuracy of the determined first calibration error coefficient; the value of the first preset quantity should also not be set too large, otherwise the amount of data involved in the calculation will be too large, resulting in excessive computational overhead. Without limitation, the first preset quantity can be selected from an appropriate value in the range [20, 2000]. In specific implementation, the first error threshold Q1 is determined as a preset percentage of the weighted calculation result between the first distance d1 between the target camera and the current reference camera, and the second distance d2 between the target camera and the next pre-selected reference camera.

[0047] The preset percentage can be appropriately determined based on the value of the first calibration error coefficient q1 obtained from multiple experiments. Non-limitingly, the preset percentage can be selected from an appropriate proportion within the range [0.4%, 0.6%], for example, 0.5% can be chosen as the preset percentage.

[0048] In this process, a first weight ratio can be used to perform a weighted calculation on the first distance d1 and the second distance d2. The first weight ratio can be determined based on the ratio of the first distance d1 and the second distance d2 (denoted as the distance ratio). The larger the distance ratio, the larger the first weight ratio.

[0049] In some non-limiting embodiments, the target camera may be a camera located at an intermediate position between the current reference camera and the next pre-selected reference camera. In this case, the first weight ratio may be set to 1:1, that is, a preset percentage of the average value (d1+d2) / 2 of the first distance d1 and the second distance d2 is used as the first error threshold Q1.

[0050] The method for determining the camera at the intermediate position between the current reference camera and the next pre-selected reference camera varies depending on the number of cameras. Specifically, the total number of the current reference camera, the next pre-selected reference camera, and the cameras located between them is x. The method for determining the intermediate position camera will be described in detail below using two different embodiments.

[0051] In one embodiment, x is an odd number, and the camera at the intermediate position is the (x+1) / 2th camera following the current reference camera. Without limitation, the total number of the current reference camera, the next pre-selected reference camera, and the cameras in between is 5 (i.e., x = 5), namely A1, A2, A3, A4, and A5. Then, the camera at the intermediate position between the current reference camera A1 and the next pre-selected reference camera A5 is the 3rd camera following the current reference camera A1, i.e., camera A3.

[0052] In another embodiment, where x is an even number, the camera at the middle position is either the x / 2th camera or the (x / 2+1)th camera following the current reference camera. Without limitation, the total number of the current reference camera, the next pre-selected reference camera, and the cameras in between is 6 (i.e., x = 6), designated B1, B2, B3, B4, B5, and B6. The camera at the middle position between the current reference camera B1 and the next pre-selected reference camera B6 is either the 3rd or 4th camera following the current reference camera B1, i.e., either camera B3 or camera B4.

[0053] Reference Figure 2 , Figure 2 This is a schematic diagram illustrating the process of determining several reference cameras from multiple cameras to be calibrated in an embodiment of the present invention.

[0054] As a non-limiting embodiment, the number of cameras to be calibrated is 15, namely A, B, C, D, E, F, G, H, I, J, K, L, M, N, O. The first camera A is selected as the first reference camera, and the initial value of the preset number of intermediate cameras m is 2.

[0055] In the first round, the following process is performed to determine the next reference camera after the current reference camera A: the current reference camera is the first camera A, and in order to make the number of cameras between the current reference camera A and the next pre-selected reference camera two, the next pre-selected reference camera is the fourth camera D; then, based on the current reference camera A and the next pre-selected reference camera D, the initial value of the first calibration error coefficient q1 is determined. Whenever q1 ≤ Q1 (the first error threshold), the value of the intermediate camera number m is increased to redetermine the subsequent pre-selected reference camera and continue to determine the value of the first calibration error coefficient q1 until q1 > Q1. The determined pre-selected reference camera is then used as the next reference camera after the current reference camera A (in this embodiment, it is assumed that when the value of m increases to 4, q1 > Q1 is satisfied, and the determined pre-selected reference camera is F, that is, F is used as the next reference camera after the current reference camera A in the first round).

[0056] In the second round, the reference camera F determined in the first round is used as the current reference camera, and the next reference camera F after the current reference camera F is determined by referring to the process of determining the next reference camera F after camera A in the first round (in this embodiment, it is assumed that the next reference camera after the determined camera F is camera I).

[0057] In the third round, the reference camera I determined in the second round is used as the current reference camera, and the next reference camera of the current reference camera I is determined by referring to the process of determining the next reference camera F of camera A in the first round (in this embodiment, it is assumed that the next reference camera of the determined camera I is camera M).

[0058] At this point, since there are two cameras remaining after camera M, if the initial value of the number of intermediate cameras preset in the third round is also 2, it is impossible to find a subsequent pre-selected reference camera such that the number of intermediate cameras between it and camera M is two. Therefore, the process of determining the next reference camera for the current reference camera in each round ends, and the reference cameras among the 15 cameras to be calibrated are determined to be camera A, camera F, camera I, and camera M.

[0059] It should be noted that the initial value of the preset number of intermediate cameras m can be the same or different in each round. The initial value of m can be appropriately set according to the requirements of the number of cameras to be calibrated, calibration efficiency, and computational overhead in the actual application scenario. The initial value of m should not be set too large, otherwise the initial value of the first calibration error coefficient q1 will be too large, and q1 > Q1 can be satisfied at the beginning, making the interval between the determined current reference camera and the next reference camera too large (too many intermediate cameras in the interval), thereby reducing the accuracy of the final calibration; the initial value of m should not be set too small either, otherwise the first calibration error coefficient q1 determined after multiple iterations will only satisfy q1 > Q1, thereby increasing unnecessary computational overhead and reducing calibration efficiency.

[0060] In the specific implementation of step S12, for each non-reference camera, the first mapping matrix and the second mapping matrix of the current non-reference camera are determined respectively.

[0061] Further, determining the first mapping matrix of the current non-reference camera in step S12 may include: for each non-reference camera between the current non-reference camera and the previous adjacent reference camera, determining a mapping matrix that maps the image coordinate system of the non-reference camera to the image coordinate system of the previous camera; and using the product of the determined mapping matrices as the first mapping matrix.

[0062] Further, determining the second mapping matrix of the current non-reference camera in step S12 may include: for each non-reference camera between the current non-reference camera and the next adjacent reference camera, determining a mapping matrix that maps the image coordinate system of the non-reference camera to the image coordinate system of the next camera; and using the product of the determined mapping matrices as the second mapping matrix.

[0063] For example, the current non-reference camera is denoted as C. k The preceding adjacent reference camera is denoted as C.k-n The next adjacent reference camera is denoted as C. k+n :

[0064] (1) The mapping matrix (i.e., the first mapping matrix) that maps the image coordinate system of the current non-reference camera to the image coordinate system of the previous adjacent reference camera can be determined using the following formula:

[0065] H k-n,k =H k-1,k *H k-2,k-1 *H k-3,k-2 …*H k-n,k-n-1 ;

[0066] Among them, H k-1,k Indicates the current non-reference camera record C k The image coordinate system is mapped to the previous camera C. k-1 The mapping matrix of the image coordinate system, H k-2,k-1 Indicates camera C k-1 The image coordinate system is mapped to the previous camera C. k-2 (that is, C) k-1 The mapping matrix of the image coordinate system of the previous camera, H k-3,k-2 Indicates camera C k-2 The image coordinate system is mapped to the previous camera C. k-3 (that is, C) k-2 The mapping matrix of the image coordinate system of the previous camera, H k-n,k-n-1 Indicates camera C k-n-1 The image coordinate system is mapped to the previous camera C. k-n (that is, C) k-n-1 The mapping matrix of the image coordinate system of the previous camera.

[0067] In the above formula, the product of the various mapping matrices can be understood as the result of the multiplication of the non-reference camera C. k To the previous adjacent reference camera C k-n The composite of the mapping relationships between every two adjacent cameras (or the composite result of linear mappings). The composite result of these linear mappings can be used to represent C. k With C k-n The mapping relationship between them.

[0068] (2) The mapping matrix (i.e., the second mapping matrix) that maps the image coordinate system of the current non-reference camera to the image coordinate system of the next adjacent reference camera can be determined using the following formula:

[0069] H k-n,k =H k+1,k *H k+2,k+1 *H k+3,k+2 …*Hk+n,k+n-1 ;

[0070] Among them, H k+1,k Indicates the current non-reference camera record C k The image coordinate system is mapped to the next camera C. k+1 The mapping matrix of the image coordinate system, H k+2,k+1 Indicates camera C k+1 The image coordinate system is mapped to the next camera C. k+2 (that is, C) k+1 The mapping matrix of the image coordinate system of the next camera, H k+3,k+2 Indicates camera C k+2 The image coordinate system is mapped to the next camera C. k+3 (that is, C) k+2 The mapping matrix of the image coordinate system of the next camera, H k+n,k+n-1 Indicates camera C k+n-1 The image coordinate system is mapped to the next camera C. k+n (that is, C) k+n-1 The mapping matrix of the image coordinate system of the next camera.

[0071] In the above formula, the product of the various mapping matrices can be understood as the result of the multiplication of the non-reference camera C. k To the next adjacent reference camera C k+n The composite of the mapping relationships between every two adjacent cameras (or the composite result of linear mappings). The composite result of these linear mappings can be used to represent C. k With C k+n The mapping relationship between them.

[0072] In the specific implementation of step S13, the first reference mapping matrix of the previous adjacent reference camera and the second reference mapping matrix of the next adjacent reference camera are determined respectively.

[0073] Specifically, the reference mapping matrix for each reference camera can be determined using the specific scheme described below:

[0074] If the road surface is approximately planar (that is, only the position of the target in the world coordinate system is considered, not its height) and the camera has no significant distortion, then for the road surface p1 and the image plane p2 of the image acquired by a reference camera C:

[0075] Establish an appropriate world coordinate system on the road surface p1 and an image coordinate system on the image plane p2; then select multiple pairs (usually more than 4 pairs) of non-collinear points (e.g., points represented by homogeneous coordinates) within these two coordinate systems to determine the reference mapping matrix H0 representing the mapping relationship between the coordinate systems of these two planes, where H0 satisfies: P W =H0*P I ;

[0076] Among them, P W P is used to represent several selected points in the world coordinate system where the road surface p1 is located. I This is used to represent the corresponding position selected in the image coordinate system where image plane p2 is located. Where P... W The various sites represented are related to P. I Each of the indicated sites corresponds one-to-one.

[0077] For a single reference camera C, the process of determining the reference mapping matrix H0, which maps the image coordinate system of image plane p2 to the world coordinate system of road surface p1, is the process of calibrating the reference camera C. The resulting reference mapping matrix H0 is the calibration result of the reference camera C.

[0078] It should be noted that before step S13, the above-described scheme for calibrating the reference cameras can be used to calibrate each of the selected reference cameras among the plurality of cameras to be calibrated, obtaining the reference mapping matrix of each reference camera mapped to the world coordinate system, and storing it. During the execution of step S13, the required first reference mapping matrix and second reference mapping matrix can be directly obtained from the pre-stored data to improve the calibration efficiency of non-reference cameras.

[0079] In the specific implementation of step S14, the product of the first mapping matrix and the first reference mapping matrix is ​​determined and denoted as the first matrix product, and the product of the second mapping matrix and the second reference mapping matrix is ​​determined and denoted as the second matrix product.

[0080] For example, the current non-reference camera is denoted as The previous adjacent reference camera is denoted as The first matrix product can be determined using the following formula:

[0081]

[0082] in, Indicates the current non-reference camera The reference mapping matrix that maps the image coordinate system to the world coordinate system (i.e., the product of the first matrix). Indicates the previous adjacent reference camera The reference mapping matrix mapped to the world coordinate system (i.e., the first reference mapping matrix), Indicates camera (Right now, The mapping matrix that maps the image coordinate system of the previous camera to the world coordinate system. Indicates camera Image coordinate system mapped to camera The mapping matrix of the image coordinate system.

[0083] Accordingly, determine the product of the second matrix. The process and principle of the method are described above in the specific scheme for determining the product of the first matrix, and will not be repeated here.

[0084] It should be noted that the first matrix product (which can be denoted as the product of the first calibration result) and the second matrix (This can be denoted as the second calibration result) can both represent the mapping matrix from the current non-reference camera to the world coordinate system. However, the difference between the two lies in the fact that the first calibration result represents the calibration result of cascaded calibration from the current non-reference camera to the previous adjacent reference camera. The second calibration result represents the calibration result of cascaded calibration from the current non-reference camera to the next adjacent reference camera.

[0085] In the specific implementation of step S15, the first matrix product is... Product with the second matrix Perform a weighted calculation, and use the result of the weighted calculation as a mapping matrix to map the current non-reference camera to the world coordinate system.

[0086] In specific implementation, the product of the first matrix... Product with the second matrix The weighting ratio for weighted calculations can be set appropriately based on the actual needs of the scenario.

[0087] In one embodiment, it can be set and The weighting ratio is 1:1, that is, and The average calculation result is used as the calibration result for the current non-reference camera.

[0088] In another embodiment, the position of the current non-reference camera can be determined based on its specific location between the previous adjacent reference camera and the next adjacent reference camera. and The weight ratio. Without limitation, if there are m1 intermediate cameras between the current non-reference camera and the previous adjacent reference camera, and m2 intermediate cameras between the current non-reference camera and the next adjacent reference camera, then the weight ratio can be set. and The weight ratio is m1:m2, or (m1+1):(m2+1).

[0089] Understandably, if the number of cameras (or intermediate cameras) between the current non-reference camera and the previous adjacent reference camera is large, accumulated errors may occur during the cascade calibration process from the current non-reference camera to the previous adjacent reference camera (this process can be called first-side cascade calibration). Similarly, if the number of cameras between the current non-reference camera and the next adjacent reference camera is large, accumulated errors may also occur during the cascade calibration process from the current non-reference camera to the next adjacent reference camera (this process can be called second-side cascade calibration).

[0090] The aforementioned errors may arise from factors such as excessive distance between two cameras, resulting in the absence of a shared field of view and thus failing to meet the conditions for cascade calibration locally; or from inappropriate selection of corresponding points during the determination of the mapping matrix between adjacent cameras, leading to inaccuracies in the determination of the mapping matrix between each pair of adjacent cameras. The more intermediate cameras there are, the greater the accumulated error becomes. Consequently, the accuracy and reliability of the calibration results decrease.

[0091] To address the aforementioned issues, this invention employs a two-sided cascaded calibration method. The results of the cascaded calibration from the current non-reference camera (specifically, the non-reference camera to be calibrated) to the first side (i.e., the first calibration result) and the results of the cascaded calibration from the current non-reference camera to the second side (i.e., the second calibration result) are weighted and fused. Specifically, not only is the mapping relationship between the current non-reference camera and the previous adjacent reference camera used, but also the mapping relationship between the current non-reference camera and the next adjacent reference camera is employed. Then, the weighted result of the product of the first mapping matrix and the first reference mapping matrix, and the product of the second mapping matrix and the second reference mapping matrix, is used as the final calibration result for the current reference camera. This effectively mitigates the aforementioned accumulated errors and improves the accuracy and reliability of the calibration results.

[0092] Furthermore, the method described in this embodiment may also include: for each pair of adjacent reference cameras, determining a second calibration error coefficient q2 based on the mapping matrix of each non-reference camera between the pair of adjacent reference cameras to the world coordinate system; if the second calibration error coefficient q2 is greater than the second error threshold Q2, then selecting one or more non-reference cameras between the pair of adjacent reference cameras as reference cameras.

[0093] Specifically, determining the second calibration error coefficient based on the mapping matrix of each non-reference camera between the pair of adjacent reference cameras to the world coordinate system may include: for each non-reference camera located between the pair of adjacent reference cameras, selecting a second preset number of target points from the images acquired by the non-reference camera; determining the actual position of each selected target point in the world coordinate system; determining the mapping points of each target point to the world coordinate system according to the mapping matrix of the non-reference camera to the world coordinate system, denoted as a third mapping point set; determining the distance between each third mapping point in the third mapping point set and its corresponding actual position, and determining the average value of each distance, denoted as a first average calculation result; and using the average value of the first average calculation results corresponding to each non-reference camera located between the pair of adjacent reference cameras as the second calibration error coefficient q2.

[0094] In practice, the second preset quantity should not be set too small, otherwise the amount of data involved in the calculation will be too small, resulting in low accuracy of the determined second calibration error coefficient; the second preset quantity should also not be set too large, otherwise the amount of data involved in the calculation will be too large, resulting in excessive computational overhead. Without limitation, the second preset quantity can be selected from an appropriate value in the range [20, 2000].

[0095] Continue to refer to Figure 2 For example, for adjacent reference cameras A and F, there are four non-reference cameras (B, C, D, and E) between these two reference cameras. By adopting... Figure 1 The calibration scheme described in the illustrated embodiment determines the mapping matrix (i.e., the calibration results of each non-reference camera) of B, C, D, and E to the world coordinate system, which is divided into H... B H C H D H E .

[0096] For non-reference camera B, a second preset number of target points can be selected from the images acquired by B, for example, four target points can be selected as b1, b2, b3, and b4; the actual positions c1, c2, c3, and c4 in the world coordinate system corresponding to each selected target point are determined; and the mapping matrix H from non-reference camera B to the world coordinate system is used. B(That is, the calibration result of non-reference camera B), determine the mapping points b1', b2', b3', b4' of each target point to the world coordinate system, and denot them as the third mapping point set; determine the distances between b1', b2', b3', b4' and their corresponding actual positions c1, c2, c3, c4 (b1' corresponds to c1, b2' corresponds to c2, b3' corresponds to c3, b4' corresponds to c4), and determine the average value of each distance, denoted as the first average calculation result; take the average value of the first average calculation results corresponding to each non-reference camera B, C, D, E located between the pair of adjacent reference cameras A and F as the second calibration error coefficient q2.

[0097] It is understandable that when q2 > Q2 (the second error threshold), it means that the calibration error using the calibration results of the currently determined non-reference cameras is still relatively large. Converting one or more non-reference cameras between two adjacent reference cameras into reference cameras (that is, increasing the number of reference cameras) can help improve the calibration accuracy.

[0098] Furthermore, after adding a reference camera between two adjacent reference cameras, it may also include: iteratively performing... Figure 1 In the embodiment shown, steps S12 to S14 are used to calibrate each non-reference camera. After determining the calibration result of each non-reference camera, the second calibration error coefficient q2 is determined until q2≤Q2. Then, the determined calibration result of each non-reference camera is taken as the final calibration result.

[0099] Reference Figure 3 , Figure 3 This is a partial flowchart of another camera calibration method according to an embodiment of the present invention. The other camera calibration method may include... Figure 1 Steps S11 to S15 shown may also include steps S31 to S34. Steps S31 to S34 may be performed after step S11 and before step S12.

[0100] In step S31, for each non-reference camera between each pair of adjacent reference cameras, the mapping error coefficient of the non-reference camera is determined based on the common viewing area between the non-reference camera and the previous camera.

[0101] Further, determining the mapping error coefficient of the reference camera based on the shared viewing area between the non-reference camera and the previous camera includes: mapping the shared viewing area in the image acquired by the non-reference camera to the image acquired by the previous camera based on the mapping matrix between the non-reference camera and the previous camera to obtain the corresponding mapped area; and determining the mapping error coefficient q3 of the non-reference camera based on each pixel of the mapped area and the original shared viewing area in the image acquired by the previous camera.

[0102] Furthermore, determining the mapping error coefficient q3 of the non-reference camera based on each pixel of the mapped region and the original co-view region in the image captured by the previous camera includes: using the L1 norm, determining the L1 norm value based on the distance between the horizontal and vertical coordinates of each pair of pixels in the mapped region and the original co-view region in the image captured by the previous camera, and using the ratio of the L1 norm value to the area of ​​the mapped region as the mapping error coefficient q3 of the non-reference camera.

[0103] In step S32, the sum of the mapping error coefficients of each non-reference camera between the pair of adjacent reference cameras is determined.

[0104] In step S33, it is determined whether the sum of the error coefficients is greater than a preset mapping error threshold.

[0105] The sum of the mapping error coefficients q3 of each non-reference camera between adjacent reference cameras can also be called the cumulative mapping error value. If the judgment result is yes, then proceed to step S34. Then continue execution. Figure 1 Step S12 in the illustrated embodiment.

[0106] If the result of the judgment is negative, then you can directly jump to execution. Figure 1 Step S12 in the illustrated embodiment.

[0107] In step S34, one or more non-reference cameras are selected as reference cameras between the pair of adjacent reference cameras.

[0108] Understandably, when the cumulative mapping error of each non-reference camera between adjacent reference cameras exceeds Q3 (the third error threshold), it means that the number of non-reference cameras between two adjacent reference cameras is still relatively large. This results in low accuracy of the mapping matrix between the non-reference cameras and the reference cameras, which in turn leads to insufficient accuracy in determining the mapping matrix from the reference cameras to the world coordinate system. Therefore, converting one or more non-reference cameras between two adjacent reference cameras into reference cameras (i.e., increasing the number of reference cameras) helps reduce the cumulative mapping error and improve the calibration accuracy.

[0109] Furthermore, after adding a reference camera between two adjacent reference cameras, it may also include: iteratively performing... Figure 3 In the illustrated embodiment, steps S31 to S34 evaluate the cumulative mapping error until the cumulative mapping error value is ≤ Q3, at which point the process continues. Figure 1 Step S12 of the illustrated embodiment.

[0110] Reference Figure 4 , Figure 4 This is a schematic diagram of a camera calibration device according to an embodiment of the present invention. The camera calibration device may include:

[0111] The reference camera determination module 41 is used to determine a number of reference cameras from a number of cameras to be calibrated, and the remaining cameras to be calibrated are denoted as non-reference cameras.

[0112] The inter-camera mapping matrix determination module 42 is used to determine, for each non-reference camera, a first mapping matrix and a second mapping matrix of the current non-reference camera. The first mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the previous adjacent reference camera. The second mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the next adjacent reference camera.

[0113] The reference mapping matrix determination module 43 is used to determine the first reference mapping matrix of the previous adjacent reference camera and the second reference mapping matrix of the next adjacent reference camera, respectively. The first reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the previous adjacent reference camera, and the second reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the next adjacent reference camera.

[0114] The matrix multiplication module 44 is used to determine the product of the first mapping matrix and the first reference mapping matrix, denoted as the first matrix product, and to determine the product of the second mapping matrix and the second reference mapping matrix, denoted as the second matrix product.

[0115] The calibration result determination module 45 is used to perform a weighted operation on the product of the first matrix and the product of the second matrix, and use the weighted operation result as the mapping matrix for mapping the current non-reference camera to the world coordinate system.

[0116] For details regarding the principle, implementation, and beneficial effects of this camera calibration device, please refer to the preceding text. Figure 1 , Figure 3 The description of the camera calibration method shown will not be repeated here.

[0117] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the above-described... Figure 1 , Figure 3 The steps of the camera calibration method are shown. The computer-readable storage medium may include non-volatile or non-transitory memory, and may also include optical discs, hard disk drives, solid-state drives, etc.

[0118] Specifically, in this embodiment of the invention, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0119] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0120] This invention also provides a terminal, including a memory and a processor. The memory stores a computer program that can run on the processor, and when the processor runs the computer program, it performs the above-described... Figure 1 , Figure 3 The steps of the camera calibration method are shown. The terminal may include, but are not limited to, mobile phones, computers, tablets, and other terminal devices, and may also be servers, cloud platforms, etc.

[0121] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0122] In the embodiments of this application, "multiple" refers to two or more.

[0123] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0124] It should be noted that the sequence number of each step in this embodiment does not represent a limitation on the execution order of each step.

[0125] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A camera calibration method, characterized in that, include: Several reference cameras are determined from a number of cameras to be calibrated, and the remaining cameras to be calibrated are denoted as non-reference cameras. For each non-reference camera, a first mapping matrix and a second mapping matrix are determined respectively. The first mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the previous adjacent reference camera. The second mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the next adjacent reference camera. The first reference mapping matrix of the previous adjacent reference camera and the second reference mapping matrix of the next adjacent reference camera are determined respectively. The first reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the previous adjacent reference camera, and the second reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the next adjacent reference camera. The product of the first mapping matrix and the first reference mapping matrix is ​​determined and denoted as the first matrix product, and the product of the second mapping matrix and the second reference mapping matrix is ​​determined and denoted as the second matrix product; A weighted operation is performed on the product of the first matrix and the product of the second matrix, and the weighted operation result is used as the mapping matrix for mapping the current non-reference camera to the world coordinate system.

2. The method according to claim 1, characterized in that, The process of determining several reference cameras from a plurality of cameras to be calibrated includes: The first reference camera is determined from the plurality of cameras to be calibrated, and then the next reference camera is determined round by round starting from the first reference camera. In each round, determining the next reference camera for the current reference camera includes: Determine the next pre-selected reference camera of the current reference camera, such that the number of cameras between the current reference camera and the next pre-selected reference camera meets a preset number of intermediate cameras. Based on the current reference camera and the next pre-selected reference camera, a first calibration error coefficient is determined. Whenever the first calibration error coefficient is less than or equal to the first error threshold, the number of intermediate cameras is increased to redetermine the subsequent pre-selected reference camera and the first calibration error coefficient is determined again until the first calibration error coefficient is greater than the first error threshold. The determined pre-selected reference camera is then used as the next reference camera of the current reference camera. In this round, the next reference camera after the current reference camera is used as the current reference camera for the next round.

3. The method according to claim 2, characterized in that, The step of determining the first calibration error coefficient based on the current reference camera and the next pre-selected reference camera includes: Select a target camera between the current reference camera and the next pre-selected reference camera, and select a first preset number of target points from the images captured by the target camera; Based on the mapping matrix from the target camera to the current reference camera, and the mapping matrix from the current reference camera to the world coordinate system, the mapping points of each target point to the world coordinate system are determined and denoted as the first mapping point set. Based on the mapping matrix from the target camera to the next pre-selected reference camera, and the mapping matrix from the next pre-selected reference camera to the world coordinate system, the mapping points of each target point to the world coordinate system are determined and denoted as the second mapping point set. The distances between each first mapping point in the first mapping point set and the corresponding second mapping point in the second mapping point set are determined respectively, and the average value of the determined distances is used as the first calibration error coefficient.

4. The method according to claim 3, characterized in that, The first error threshold is determined as a preset percentage of the weighted calculation result between the first distance between the target camera and the current reference camera and the second distance between the target camera and the next pre-selected reference camera.

5. The method according to claim 3 or 4, characterized in that, The target camera is the camera positioned between the current reference camera and the next pre-selected reference camera.

6. The method according to claim 1, characterized in that, After determining several reference cameras from a plurality of cameras to be calibrated, and before determining the first mapping matrix and the second mapping matrix for each non-reference camera, the method further includes: For each non-reference camera between each pair of adjacent reference cameras, the mapping error coefficient of the non-reference camera is determined based on the common viewing area between the non-reference camera and the previous camera. If the sum of the mapping error coefficients of each non-reference camera between the pair of adjacent reference cameras is greater than a preset mapping error threshold, then one or more non-reference cameras are selected as reference cameras between the pair of adjacent reference cameras.

7. The method according to claim 6, characterized in that, The determination of the mapping error coefficient of the non-reference camera based on the shared viewing area between the non-reference camera and the previous camera includes: Based on the mapping matrix between the non-reference camera and the previous camera, the common viewing region in the image acquired by the non-reference camera is mapped to the image acquired by the previous camera to obtain the corresponding mapped region; The mapping error coefficient of the non-reference camera is determined based on the individual pixels of the mapped region and the original co-view region in the image acquired by the previous camera.

8. The method according to claim 7, characterized in that, The step of determining the mapping error coefficient of the non-reference camera based on each pixel of the mapped region and the original co-view region in the image acquired by the previous camera includes: The L1 norm is used to determine the L1 norm value based on the distance between the horizontal and vertical coordinates of each pair of pixels in the mapped region and the original co-view region in the image acquired by the previous camera. The ratio of the L1 norm value to the area of ​​the mapped region is used as the mapping error coefficient of the non-reference camera.

9. The method according to claim 1, characterized in that, The method further includes: For each pair of adjacent reference cameras, the second calibration error coefficient is determined based on the mapping matrix between each non-reference camera in the pair of adjacent reference cameras to the world coordinate system. If the second calibration error coefficient is greater than the second error threshold, then one or more non-reference cameras are selected as reference cameras between the pair of adjacent reference cameras.

10. The method according to claim 9, characterized in that, The determination of the second calibration error coefficient based on the mapping matrix of each non-reference camera between adjacent reference cameras to the world coordinate system includes: For each non-reference camera located between the pair of adjacent reference cameras, a second preset number of target points are selected from the images acquired by the non-reference camera; Determine the actual position of each selected target point in the world coordinate system; Based on the mapping matrix from the non-reference camera to the world coordinate system, determine the mapping points of each target point to the world coordinate system, denoted as the third mapping point set; Determine the distance between each third mapping point in the third mapping point set and its corresponding actual position, and determine the average value of each distance, which is denoted as the first average calculation result; The average of the first average calculation results corresponding to each non-reference camera located between the pair of adjacent reference cameras is used as the second calibration error coefficient.

11. The method according to claim 1, characterized in that, Determining the first mapping matrix of the current non-reference camera includes: For each non-reference camera between the current non-reference camera and the previous adjacent reference camera, determine the mapping matrix that maps the image coordinate system of the non-reference camera to the image coordinate system of the previous camera. The product of the determined mapping matrices is taken as the first mapping matrix.

12. A camera calibration device, characterized in that, include: The reference camera determination module is used to determine several reference cameras from multiple cameras to be calibrated, and the remaining cameras to be calibrated are denoted as non-reference cameras. The inter-camera mapping matrix determination module is used to determine, for each non-reference camera, a first mapping matrix and a second mapping matrix of the current non-reference camera. The first mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the previous adjacent reference camera. The second mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system of the current non-reference camera and the image coordinate system of the next adjacent reference camera. The reference mapping matrix determination module is used to determine the first reference mapping matrix of the previous adjacent reference camera and the second reference mapping matrix of the next adjacent reference camera, respectively. The first reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the previous adjacent reference camera, and the second reference mapping matrix is ​​used to indicate the mapping relationship between the image coordinate system and the world coordinate system of the next adjacent reference camera. The matrix multiplication module is used to determine the product of the first mapping matrix and the first reference mapping matrix, denoted as the first matrix product, and to determine the product of the second mapping matrix and the second reference mapping matrix, denoted as the second matrix product. The calibration result determination module is used to perform a weighted operation on the product of the first matrix and the product of the second matrix, and use the weighted operation result as the mapping matrix for mapping the current non-reference camera to the world coordinate system.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it performs the steps of the camera calibration method according to any one of claims 1 to 11.

14. A terminal comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the camera calibration method according to any one of claims 1 to 11.

Images