A UWB and Camera Joint Calibration Method and System

Abstract

The present invention relates to a method and system for joint calibration of UWB and camera. The method for joint calibration of UWB and camera provided by the present invention introduces one UWB positioning tag on the basis of the Zhang Zhengyou camera calibration method, enabling joint calibration while calibrating the internal parameters of the camera, so as to estimate the complete conversion relationship between the camera coordinate system and the UWB coordinate system, thereby providing a technical basis for achieving more accurate positioning and image-UWB data fusion.

Description

Technical Field

[0001] The present invention relates to the technical field of indoor positioning, and particularly to a method and system for jointly calibrating UWB and a camera. Background Art

[0002] Ultra Wide Band (UWB) technology is a wireless carrier communication technology and has wide applications in the field of indoor positioning. However, the accuracy of UWB positioning is still not high enough. By analyzing the objects captured by a camera, accurate three-dimensional positions can be obtained. Combining the positioning results of the two can improve the positioning accuracy, and at the same time enable the system to obtain rich semantic information of the images. However, this requires aligning the coordinate system of the camera with the coordinate system of the UWB positioning system, that is, jointly calibrating the UWB and the camera. The common method is to place the positioning tag at the position of the camera, but this can only obtain the two-dimensional coordinates of the camera in the UWB coordinate system, and cannot establish the complete relationship between the camera coordinates and the UWB coordinates, and cannot achieve more accurate positioning. Summary of the Invention

[0003] To solve the above problems existing in the prior art, the present invention provides a method and system for jointly calibrating UWB and a camera.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A method for jointly calibrating UWB and a camera includes:

[0006] Obtaining a first captured image; the first captured image is an image obtained by capturing a checkerboard calibration board placed on the ground by a camera at an initial moment;

[0007] Obtaining a second captured image; the second captured image is an image obtained by capturing a checkerboard calibration board with a UWB tag placed on the ground by a camera at different moments; the placement positions of the checkerboard calibration boards with UWB tags on the ground are different at different moments; the UWB tag is fixed at a blank position on the checkerboard calibration board; the fixed positions of the UWB tag on the checkerboard calibration board are different at different moments;

[0008] Obtaining the estimated position of the UWB tag in the second captured image;

[0009] Processing the first captured image and the second captured image by using the Zhang Zhengyou calibration processing method to obtain the camera internal parameter matrix and the camera external parameters;

[0010] Obtaining the marked position of the UWB tag in the second captured image;

[0011] Project the marked position of the UWB tag into the world coordinate system of the checkerboard calibration board in the first image based on the external camera parameters to obtain the world coordinate position;

[0012] Determine the corresponding relationship between the estimated position of the UWB tag and the world coordinate position;

[0013] Determine the two-dimensional rigid body transformation between the UWB coordinate system and the world coordinate system of the checkerboard calibration board in the first image according to the corresponding relationship.

[0014] Preferably, use the formula Project the marked position of the UWB tag into the world coordinate system of the checkerboard calibration board in the first image based on the external camera parameters to obtain the world coordinate position;

[0015] In the formula, is the world coordinate position, R1 is the first external camera parameter at the first shooting, and R i is the first external camera parameter at the i-th shooting, is the marked position of the UWB tag in the second captured image at the i-th shooting, and T i is the second external camera parameter at the i-th shooting, and T1 is the second external camera parameter at the first shooting, where i = 2, 3,..., N.

[0016] Preferably, the determination of the corresponding relationship between the estimated position of the UWB tag and the world coordinate position specifically includes:

[0017] Determine the first position average value based on the estimated position of the UWB tag;

[0018] Determine the second position average value based on the world coordinate position;

[0019] Determine the first coordinate transformation matrix according to the estimated position of the UWB tag and the first position average value;

[0020] Determine the second coordinate transformation matrix according to the world coordinate position and the second position average value;

[0021] Determine the third coordinate transformation matrix according to the first coordinate transformation matrix and the second coordinate transformation matrix;

[0022] Perform SVD decomposition on the third coordinate transformation matrix to obtain the corresponding relationship.

[0023] Preferably, the third coordinate transformation matrix is C: C = BA T ;

[0024] In the formula, B is the second coordinate transformation matrix, and A T is the transpose of the first coordinate transformation matrix.

[0025] Preferably, the corresponding relationship is: C = USV T ;

[0026] wherein, C is the third coordinate transformation matrix, U is the left singular matrix, S is the singular matrix, V is the right singular matrix, and V T is the transpose of V.

[0027] Preferably, determining the two-dimensional rigid body transformation between the UWB coordinate system and the world coordinate system of the checkerboard calibration board in the first image according to the corresponding relationship specifically includes:

[0028] Using the formula R' = Udiag(1, det(UV T ))V T to determine the first rigid body transformation parameter R';

[0029] According to the first rigid body transformation parameter, the first position average value and the second position average value, using the formula to determine the second rigid body transformation parameter T';

[0030] wherein, diag(*) is the symbol for constructing a diagonal matrix, det(*) is the symbol for calculating the determinant, is the first position average value, is the second position average value.

[0031] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0032] The UWB and camera joint calibration method provided by the present invention, on the basis of the Zhang Zhengyou camera calibration method, introduces 1 UWB positioning tag, enabling joint calibration while calibrating the camera internal parameters, so as to estimate the complete conversion relationship between the camera coordinate system and the UWB coordinate system, and further providing a technical basis for realizing more accurate positioning and image-UWB data fusion.

[0033] Corresponding to the above-provided UWB and camera joint calibration method, the present invention also provides a UWB and camera joint calibration system, and the system includes:

[0034] A first image acquisition module, configured to acquire a first captured image; the first captured image is an image obtained by using a camera to capture a checkerboard calibration board placed on the ground at an initial moment;

[0035] A second image acquisition module for acquiring a second captured image; the second captured image is an image obtained by capturing a checkerboard calibration board with UWB tags placed on the ground at different times; the placement positions of the checkerboard calibration board with UWB tags on the ground are different at different times; the UWB tags are fixed at blank positions on the checkerboard calibration board; the fixed positions of the UWB tags on the checkerboard calibration board are different at different times;

[0036] A first position acquisition module for acquiring the estimated position of the UWB tag in the second captured image;

[0037] A camera parameter determination module for processing the first captured image and the second captured image using the Zhang Zhengyou calibration method to obtain the camera internal parameter matrix and the camera external parameters;

[0038] A second position acquisition module for acquiring the marked position of the UWB tag in the second captured image;

[0039] A coordinate position projection module for projecting the marked position of the UWB tag into the world coordinate system of the checkerboard calibration board in the first image based on the camera external parameters to obtain the world coordinate position;

[0040] A correspondence determination module for determining the correspondence between the estimated position of the UWB tag and the world coordinate position;

[0041] A two-dimensional rigid body transformation module for determining the two-dimensional rigid body transformation between the UWB coordinate system and the world coordinate system of the checkerboard calibration board in the first image according to the correspondence;

[0042] Preferably, the correspondence determination module includes:

[0043] A first position averaging unit for determining a first position average value based on the estimated position of the UWB tag;

[0044] A second position averaging unit for determining a second position average value based on the world coordinate position;

[0045] A first matrix determination unit for determining a first coordinate transformation matrix according to the estimated position of the UWB tag and the first position average value;

[0046] A second matrix determination unit for determining a second coordinate transformation matrix according to the world coordinate position and the second position average value;

[0047] A third matrix determination unit for determining a third coordinate transformation matrix according to the first coordinate transformation matrix and the second coordinate transformation matrix;

[0048] A corresponding relationship determination unit is configured to perform SVD decomposition on the third coordinate transformation matrix to obtain the corresponding relationship.

[0049] Since the technical effects achieved by the UWB and camera joint calibration system provided by the present invention are the same as those achieved by the UWB and camera joint calibration method provided above, no further description will be given here. Description of the Drawings

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0051] Figure 1 It is a flowchart of the UWB and camera joint calibration method provided by the present invention;

[0052] Figure 2 It is a diagram showing the position transformation of the checkerboard calibration board during the calibration process provided by an embodiment of the present invention;

[0053] Figure 3 It is a schematic structural diagram of the UWB and camera joint calibration system provided by the present invention. Detailed Embodiments

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0055] The purpose of the present invention is to provide a UWB and camera joint calibration method and system, which can perform joint calibration while calibrating the internal parameters of the camera, thereby achieving more accurate positioning.

[0056] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0057] As Figure 1 shown, the UWB and camera joint calibration method provided by the present invention includes:

[0058] Step 100: Obtain a first captured image. The first captured image is an image obtained by using a camera to capture a checkerboard calibration board placed on the ground at an initial moment, denoted as I1.

[0059] Step 101: Obtain the second captured image. The second captured image is an image obtained by capturing, at different times, a checkerboard calibration board with UWB tags placed on the ground using a camera, denoted as I2,..., I N . As Figure 2 shown, the placement positions of the checkerboard calibration board with UWB tags on the ground are different at different times. The UWB tags are fixed at blank positions on the checkerboard calibration board. The fixed positions of the UWB tags on the checkerboard calibration board are different at different times. In the actual operation process, the second captured image is essentially an image set, which is obtained by repeatedly capturing (for example, N - 1 times) the checkerboard calibration board with UWB tags, changing the position and orientation of the checkerboard calibration board each time, and simultaneously changing the fixed position of the UWB tags, to obtain a captured data set wherein, is the estimated position of the UWB tag obtained at the i-th capture, i = 2, 3,..., N.

[0060] Step 102: Obtain the estimated positions of the UWB tags in the second captured image.

[0061] Step 103: Process the first captured image and the second captured image using the Zhang Zhengyou calibration method to obtain the camera internal parameter matrix and the camera external parameters. Among them, the camera internal parameter matrix is denoted as K, and the camera external parameters are denoted as R1, T1, R2, T2,..., R N , T N . R1 is the camera external parameter at the first capture, R i is the i-th camera external parameter at the i-th capture, T i is the camera external parameter at the i-th capture, and T1 is the camera external parameter at the first capture.

[0062] Step 104: Obtain the marked positions of the UWB tags in the second captured image, denoted as

[0063] Step 105: Project the marked positions of the UWB tags onto the world coordinate system of the checkerboard calibration board in the first image based on the camera external parameters to obtain the world coordinate positions, denoted as For example, using the formula Project the marked positions of the UWB tags onto the world coordinate system of the checkerboard calibration board in the first image based on the camera external parameters to obtain the world coordinate positions.

[0064] In the formula,[[]] is the world coordinate position, R1 is the first camera external parameter at the first capture, R i is the first camera external parameter at the i-th capture, is the marked position of the UWB tag in the second captured image at the i-th capture, T iis the external parameters of the second camera during the i-th shooting, and T1 is the external parameters of the second camera during the first shooting.

[0065] Step 106: Determine the correspondence between the estimated position of the UWB tag and the world coordinate position. Specifically, it includes:

[0066] Determine the first position average value based on the estimated position of the UWB tag. Among them,

[0067] Determine the second position average value based on the world coordinate position. Among them,

[0068] Determine the first coordinate transformation matrix according to the estimated position of the UWB tag and the first position average value.

[0069] Determine the second coordinate transformation matrix according to the world coordinate position and the second position average value.

[0070] Determine the third coordinate transformation matrix according to the first coordinate transformation matrix and the second coordinate transformation matrix. The third coordinate transformation matrix is C: C = BA T . In the formula, B is the second coordinate transformation matrix, and A T is the transpose of the first coordinate transformation matrix.

[0071] Perform SVD decomposition on the third coordinate transformation matrix to obtain the correspondence, and the correspondence is: C = USV T .

[0072] In the formula, C is the third coordinate transformation matrix, U is the left singular matrix, S is the singular matrix, V is the right singular matrix, and V T is the transpose of V.

[0073] Step 107: Determine the two-dimensional rigid body transformation between the UWB coordinate system and the world coordinate system of the checkerboard calibration board in the first image, denoted as R', T', and complete the calibration.

[0074] Specifically, use the formula R' = Udiag(1, det(UV T ))V T to determine the first rigid body transformation parameter R'.

[0075] According to the first rigid body transformation parameter, the first position average value and the second position average value, use the formula to determine the second rigid body transformation parameter T'.

[0076] In the formula, diag(*) is the symbol for constructing a diagonal matrix, det(*) is the symbol for calculating the determinant, is the first position average value, is the second position average value.

[0077] Corresponding to the above-provided UWB and camera joint calibration method, the present invention also provides a UWB and camera joint calibration system, as Figure 3 shown, the system includes:

[0078] A first image acquisition module 300, configured to acquire a first captured image. The first captured image is an image obtained by using a camera to capture a checkerboard calibration board placed on the ground at an initial moment.

[0079] A second image acquisition module 301, configured to acquire a second captured image. The second captured image is an image obtained by using a camera to capture a checkerboard calibration board with a UWB tag placed on the ground at different moments. The placement positions of the checkerboard calibration boards with UWB tags on the ground are different at different moments. The UWB tag is fixed at a blank position on the checkerboard calibration board. The fixed positions of the UWB tags on the checkerboard calibration board are different at different moments.

[0080] A first position acquisition module 302, configured to acquire the estimated position of the UWB tag in the second captured image.

[0081] A camera parameter determination module 303, configured to process the first captured image and the second captured image by using the Zhang Zhengyou calibration method to obtain an internal camera matrix and external camera parameters.

[0082] A second position acquisition module 304, configured to acquire the marked position of the UWB tag in the second captured image.

[0083] A coordinate position projection module 305, configured to project the marked position of the UWB tag into the world coordinate system of the checkerboard calibration board in the first image based on the external camera parameters to obtain a world coordinate position.

[0084] A correspondence determination module 306, configured to determine the correspondence between the estimated position of the UWB tag and the world coordinate position.

[0085] A two-dimensional rigid body transformation module 307, configured to determine a two-dimensional rigid body transformation between the UWB coordinate system and the world coordinate system of the checkerboard calibration board in the first image according to the correspondence.

[0086] Among them, the correspondence determination module 306 can also be set to include:

[0087] A first position averaging unit, configured to determine a first position average value based on the estimated position of the UWB tag.

[0088] A second position averaging unit, configured to determine a second position average value based on the world coordinate position.

[0089] A first matrix determination unit, configured to determine a first coordinate transformation matrix according to the estimated position of the UWB tag and the first position average value.

[0090] A second matrix determination unit, configured to determine a second coordinate transformation matrix according to the world coordinate position and the second position average value.

[0091] A third matrix determination unit, configured to determine a third coordinate transformation matrix according to the first coordinate transformation matrix and the second coordinate transformation matrix.

[0092] A correspondence determination unit, configured to perform SVD decomposition on the third coordinate transformation matrix to obtain a correspondence.

[0093] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method part.

[0094] Specific examples are used in this article to elaborate on the principles and implementation manners of the present invention. The descriptions of the above embodiments are only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A UWB and camera joint calibration method, characterized in that, Including: Obtain a first captured image; the first captured image is an image obtained by using a camera to capture a checkerboard calibration board placed on the ground at an initial moment; Obtain a second captured image; the second captured image is an image obtained by using a camera to capture a checkerboard calibration board with a UWB tag placed on the ground at different moments; the placement positions of the checkerboard calibration boards with UWB tags on the ground are different at different moments; the UWB tag is fixed at a blank position on the checkerboard calibration board; the fixed positions of the UWB tag on the checkerboard calibration board are different at different moments; The placement positions of the checkerboard calibration boards with UWB tags on the ground are different at different moments; the UWB tag is fixed at a blank position on the checkerboard calibration board; the fixed positions of the UWB tag on the checkerboard calibration board are different at different moments; Obtain the estimated position of the UWB tag in the second captured image; Process the first captured image and the second captured image by using the Zhang Zhengyou calibration method to obtain the camera internal parameter matrix and the camera external parameters; Obtain the marked position of the UWB tag in the second captured image; Project the marked position of the UWB tag into the world coordinate system of the checkerboard calibration board in the first captured image based on the camera external parameters to obtain the world coordinate position; Determine the correspondence between the estimated position of the UWB tag and the world coordinate position; Determine the two-dimensional rigid body transformation between the UWB coordinate system and the world coordinate system of the checkerboard calibration board in the first captured image according to the correspondence; 2. The UWB and camera joint calibration method according to claim 1, wherein Using the formula Project the marked position of the UWB tag into the world coordinate system of the checkerboard calibration board in the first captured image to obtain the world coordinate position; In the formula, is the world coordinate position, R1 is the first camera extrinsic parameter at the first shooting, and R i is the first camera extrinsic parameter at the i-th shooting, is the marked position of the UWB tag in the second captured image at the i-th shooting, T i is the second camera extrinsic parameter at the i-th shooting, T1 is the second camera extrinsic parameter at the first shooting, and i = 2, 3,..., N.

3. The UWB and camera joint calibration method according to claim 1, characterized in that, The determination of the correspondence between the estimated position of the UWB tag and the world coordinate position specifically includes: Determine the first position average value based on the estimated position of the UWB tag; Determine the second position average value based on the world coordinate position; Determine the first coordinate transformation matrix according to the estimated position of the UWB tag and the first position average value; Determine the second coordinate transformation matrix according to the world coordinate position and the second position average value; Determine the third coordinate transformation matrix according to the first coordinate transformation matrix and the second coordinate transformation matrix; Perform SVD decomposition on the third coordinate transformation matrix to obtain the correspondence; 4. The UWB and camera joint calibration method according to claim 3, wherein The third coordinate transformation matrix is C: C = BA T ; where B is the second coordinate transformation matrix, and A T is the transpose of the first coordinate transformation matrix.

5. The UWB and camera joint calibration method according to claim 3, wherein The corresponding relationship is: C = USV T ; where C is the third coordinate transformation matrix, U is the left singular matrix, S is the singular matrix, V is the right singular matrix, and V T is the transpose of V.

6. The UWB and camera joint calibration method according to claim 5, wherein, The determination of the two-dimensional rigid body transformation between the UWB coordinate system and the world coordinate system of the checkerboard calibration board in the first captured image according to the correspondence specifically includes: Use the formula \(R' = U\mathrm{diag}(1,\det(UV T ))V T to determine the first rigid body transformation parameter \(R'\); According to the first rigid body transformation parameter, the first average position, and the second average position, use the formula to determine the second rigid body transformation parameter T'; wherein, diag(*) is the symbol for constructing a diagonal matrix, and det(*) is the symbol for calculating the determinant, is the average value of the first position, is the average value of the second position.

7. A UWB and camera joint calibration system, characterized in that Including: A first captured image acquisition module, configured to obtain a first captured image; the first captured image is an image obtained by using a camera to capture a checkerboard calibration board placed on the ground at an initial moment; A second image acquisition module, configured to obtain a second captured image; the second captured image is an image obtained by using a camera to capture a checkerboard calibration board with a UWB tag placed on the ground at different moments; the placement positions of the checkerboard calibration boards with UWB tags on the ground are different at different moments; the UWB tag is fixed at a blank position on the checkerboard calibration board; the fixed positions of the UWB tag on the checkerboard calibration board are different at different moments; A first position acquisition module, configured to obtain the estimated position of the UWB tag in the second captured image; A camera parameter determination module, configured to process the first captured image and the second captured image by using the Zhang Zhengyou calibration method to obtain the camera internal parameter matrix and the camera external parameters; A second position acquisition module, configured to acquire the marked position of the UWB tag in the second captured image; A coordinate position projection module, configured to project the marked position of the UWB tag into the world coordinate system of the checkerboard calibration board in the first captured image based on the external camera parameters to obtain a world coordinate position; A correspondence determination module, configured to determine the correspondence between the estimated position of the UWB tag and the world coordinate position; A two-dimensional rigid body transformation module, configured to determine a two-dimensional rigid body transformation between the UWB coordinate system and the world coordinate system of the checkerboard calibration board in the first captured image according to the correspondence; 8. The UWB and camera joint calibration system according to claim 7, wherein The correspondence determination module includes: A first position averaging unit, configured to determine a first position average value based on the estimated position of the UWB tag; A second position averaging unit, configured to determine a second position average value based on the world coordinate position; A first matrix determination unit, configured to determine a first coordinate transformation matrix according to the estimated position of the UWB tag and the first position average value; A second matrix determination unit, configured to determine a second coordinate transformation matrix according to the world coordinate position and the second position average value; A third matrix determination unit, configured to determine a third coordinate transformation matrix according to the first coordinate transformation matrix and the second coordinate transformation matrix; A correspondence determination unit, configured to perform SVD decomposition on the third coordinate transformation matrix to obtain the correspondence.

Images