External parameter calibration method of camera-AOA base station
By introducing the AprilTag calibration board and IMU sensor into the camera-AoA base station system, combined with a hierarchical optimization algorithm and gravity constraints, the problems of slow calibration speed and large error in the existing technology are solved, and fast and high-precision extrinsic parameter calibration is achieved.
Patent Information
- Application Number
- CN202510953204.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-10
- Publication Date
- 2025-10-17
AI Technical Summary
In existing technologies, the external parameter calibration of cameras and AoA base stations is slow and has large errors, which cannot meet the perception needs in complex environments.
Using the AprilTag calibration plate and IMU sensor, the robot is controlled to move with the calibration plate to collect AprilTag pose, tag polar coordinates, and gravity vector data. Combined with the hierarchical optimization algorithm and gravity constraints, the optimal extrinsic parameters are quickly solved.
The calibration rate and accuracy are significantly improved, with a calibration time of 42±3s, a position error of 0.021±0.008m, and an angle error of 1.2±0.5°.
Smart Images
Figure CN120807654A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of robots, in particular to a camera-AOA base station external parameter calibration method. BACKGROUND
[0002] With the rapid development of mobile robot technology, a single sensor cannot meet the sensing needs in complex environments. Cameras provide rich visual information but lack depth perception, and AoA base stations provide accurate distance and angle measurements but cannot identify target features. The fusion of the two requires accurate external parameter calibration, i.e., determining the rigid transformation relationship between the camera coordinate system (CCS) and the AoA base station coordinate system (UCS).
[0003] In the prior art, the external parameter calibration scheme for cameras and AOA base stations often has the problems of slow calibration speed and large error. SUMMARY
[0004] In order to solve the problems in the prior art, the present application provides a camera-AOA base station external parameter calibration method to improve the calibration rate and accuracy.
[0005] To achieve the above-mentioned purpose, the present application provides a camera-AOA base station external parameter calibration method, comprising:
[0006] Fixing a UWB tag under the center of an AprilTag calibration board, and integrating an IMU sensor on an AOA base station;
[0007] Controlling a robot to move with the AprilTag calibration board, and collecting AprilTag poses, tag polar coordinates, and gravity vector data;
[0008] Based on the AprilTag poses, tag polar coordinates, and gravity vector data, performing coordinate unification to obtain UWB tag coordinates, UCS virtual point coordinates, and converted gravity vectors;
[0009] Based on the UWB tag coordinates, UCS virtual points, and converted gravity vectors, solving to obtain optimal external parameters.
[0010] Further, the specific steps of controlling the robot to move with the AprilTag calibration board and collecting AprilTag poses, tag polar coordinates, and gravity vector data include:
[0011] Controlling the robot to move with the AprilTag calibration board according to a spiral-ladder trajectory, and collecting a plurality of sets of spatio-temporally aligned AprilTag poses, tag polar coordinates, and gravity vector data.
[0012] Furthermore, the specific steps of performing coordinate unification based on the AprilTag pose, tag polar coordinates and gravity vector data to obtain UWB tag coordinates, UCS virtual point coordinates and converting the gravity vector are as follows: ; ; ; in, is the center coordinate of the AprilTag calibration plate, is the rotation matrix of the AprilTag calibration plate relative to the camera, is the vertical offset of the label, For the The coordinates of the UWB tag in the camera coordinate system in the data, For the The coordinates of the UCS virtual point in the camera coordinate system; For the The measurement distance of the AOA base station of the data, For the The horizontal angle of the AOA base station for each data point, is the camera-based gravity vector, is the original gravity vector of the IMU sensor, is the rotation transformation matrix.
[0013] Furthermore, the specific steps of solving the optimal external parameters based on the UWB tag coordinates, UCS virtual point and converted gravity vector include: Based on the preset distance consistency threshold, outliers are filtered and the initial value of the external parameter is determined. The formula is as follows: ; in, is the preset distance consistency threshold; Based on the initial values of the extrinsic parameters, the Ceres precision optimization method is used in conjunction with gravity constraints to optimize the initial values of the extrinsic parameters until convergence, and the optimal extrinsic parameters are obtained.
[0014] Furthermore, the SolvePnPRansac algorithm is used to determine the initial extrinsic parameters.
[0015] Furthermore, the method further comprises: Based on the errors detected in real time, the optimal extrinsic parameters are incrementally optimized.
[0016] Furthermore, the IMU sensor is rigidly connected to the AoA base station, and the installation spacing is less than 10 mm.
[0017] To achieve the above object, the present application further provides an electronic device, comprising: a processor; a memory having one or more computer program instructions stored thereon and running on the processor; When the processor runs the computer program instructions, the camera-AOA base station external parameter calibration method is executed.
[0018] To achieve the above object, the present application further provides a computer readable storage medium having computer instructions stored thereon, when the computer instructions are run by a processor, the steps of the camera-AOA base station external parameter calibration method are executed.
[0019] The camera-AOA base station external parameter calibration method provided by the present application can embed gravity constraints when using a layered optimization algorithm, quickly obtain optimal external parameters, and significantly improve the calibration rate and calibration accuracy.
[0020] Other features and advantages of the present application will be described in the following description, and some will become apparent from the description, or will be understood from the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS
[0021] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, and are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation on the present application. In the drawings:
[0022] Figure 1 The flowchart of the camera-AOA base station external parameter calibration method of the present application embodiment 1. DETAILED DESCRIPTION
[0023] The embodiments of the present application will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, on the contrary, these embodiments are provided to make the present application more thorough and complete. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes, and are not intended to limit the scope of protection of the present application.
[0024] It should be understood that each step described in the method embodiment of the present application can be executed in different order and / or in parallel. In addition, the method embodiment can include additional steps and / or omit the execution of the steps shown. The scope of the present application is not limited in this respect.
[0025] As used herein, the term "includes" and its variants are to be read to be analogous to "comprises," or "comprising." The term "based on" is to be read as "based, at least in part, on." The term "one embodiment" means "at least one embodiment." The term "another embodiment" means "at least one additional embodiment." The term "some embodiments" means "at least some embodiments." Related definitions are given throughout the description below.
[0026] It should be noted that the use of "a," "an," "the" and similar referents in the application is intended to be illustrative of the application and is not a limitation on the application of the application. It will be understood by those skilled in the art that, unless otherwise specified, "a," "an," "the," and the like, are used in the context of the application to mean one or more of something. The use of the term "plurality" means two or more.
[0027] Embodiments of the application will be described below in detail with reference to the accompanying drawings.
[0028] Embodiment 1 In one embodiment of the application, a camera-AOA base station extrinsic parameter calibration method is provided, which will be described below with reference to Figure 1 The camera-AOA base station extrinsic parameter calibration method of the application will be described in detail as follows, as shown in the accompanying drawings, which includes: Figure 1 Step S101: Fix the UWB tag under the center of the AprilTag calibration board, and integrate the IMU sensor on the AOA base station; In this embodiment, an AprilTag 36h11 pattern is printed on an 80x80mm aluminum alloy substrate to form an AprilTag calibration board; a DWM1001 UWB tag is installed at 20±0.1mm below the center of the AprilTag using CNC precise positioning technology.
[0029] In this embodiment, the IMU sensor (ICM-42688) is rigidly connected to the AoA base station with an installation distance of <10mm.
[0030] Step S102: Control the robot to move with the AprilTag calibration board, and collect AprilTag pose, tag polar coordinate and gravity vector data; In this embodiment, the robot is controlled to move with the AprilTag calibration board according to a spiral-ladder trajectory, and 30 sets of spatiotemporally aligned AprilTag pose, tag polar coordinate and gravity vector data are collected.
[0031] For example, a spiral-ladder trajectory is generated according to the following formula:
[0032]
[0033]
[0034] wherein t is time, x, y, z are three-dimensional coordinates.
[0035] Step S103: based on the AprilTag pose, label polar coordinates and gravity vector data, coordinate unification is carried out to obtain UWB label coordinates, UCS virtual point coordinates and converted gravity vector; The specific steps use the following formula: ; ; ; wherein, is the center coordinate of the AprilTag calibration board, is the rotation matrix of the AprilTag calibration board relative to the camera, is the label vertical offset, is the coordinate of the UWB label in the camera coordinate system in the first data, is the coordinate of the UCS virtual point in the camera coordinate system in the first data; is the measured distance of the AOA base station in the first data, is the horizontal angle of the AOA base station in the first data, is the gravity vector of the camera system, is the original gravity vector of the IMU sensor, is the rotation transformation matrix.
[0036] Step S104: based on the UWB label coordinates, UCS virtual point and converted gravity vector, the optimal external parameter is solved; In this embodiment, a hierarchical parameter optimization strategy is used to solve the external parameter: Firstly, based on the preset distance consistency threshold, filter out abnormal points, the formula is as follows: ; wherein, is the preset distance consistency threshold; Then the SolvePnPRansac algorithm is used to determine the initial external parameter, the formula is as follows: ; wherein, .
[0037] Then, the Ceres fine optimization method is used to optimize the initial value of the external parameter to convergence, and the optimal external parameter is obtained, and the formula is as follows: ; ; wherein, is a cost loss function, is a gravity constraint, is a translation vector to be optimized, is a rotation matrix to be optimized, is a camera Z-axis unit vector, and p(s) is a Cauchy loss function. Finally, the optimization is optimized to convergence, and an exemplary optimization is terminated when the residual change rate is <1e-6, and the optimal external parameter is obtained .
[0038] In the embodiment, the method further comprises: Based on the real-time detected error, the optimal external parameter is incrementally optimized, and the formula is as follows: ; ; wherein, is an average re-projection error; M is a sliding window size, and the default is 5, is an original optimal external parameter matrix, is a regularization weight, is a real-time detected UCS virtual point in the first data in the camera coordinate system.
[0039] After detection test, the calibration time of the scheme of the embodiment of the application is 42±3s, the calibration position error is 0.021±0.008m, and the angle error is 1.2±0.5°.
[0040] Embodiment 2 In the embodiment, an electronic device is also provided, comprising a processor and a memory. The memory is used to store non-transitory computer readable instructions. The processor is used to run the non-transitory computer readable instructions, and the non-transitory computer readable instructions can execute one or more steps of the camera-AOA base station external parameter calibration method described above when executed by the processor. The memory and the processor can be interconnected through a bus system and / or other forms of connection mechanism.
[0041] For example, the processor can be a central processing unit (CPU), a digital signal processor (DSP), or other form of processing unit having data processing and / or program execution capabilities, such as a field-programmable gate array (FPGA), and the like; for example, the central processing unit (CPU) can be of X86 or ARM architecture, or the like.
[0042] For example, the memory can include any combination of one or more computer program products. The computer program product can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. For example, the volatile memory can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory can include read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), USB memory, flash memory, and / or the like. One or more computer program modules can be stored on the computer-readable storage medium. The processor can execute one or more computer program modules to implement various functions of the electronic device. Various application programs and various data used and / or generated by the application programs can also be stored in the computer-readable storage medium.
[0043] It should be noted that in the embodiments of the present application, the specific functions and technical effects of the electronic device can refer to the description of the camera-AOA base station extrinsic calibration method in the foregoing, which will not be repeated here.
[0044] Embodiment 3 In this embodiment, a computer-readable storage medium is also provided, and the storage medium is used to store non-transitory computer-readable instructions. For example, when the non-transitory computer-readable instructions are executed by a computer, one or more steps of the camera-AOA base station extrinsic calibration method described above can be performed.
[0045] For example, the storage medium can be applied in the electronic device described above. For example, the storage medium can be the memory in the electronic device of embodiment 2. For example, the related description of the storage medium can refer to the corresponding description of the memory in the electronic device of embodiment 2, which will not be repeated here.
[0046] It should be noted that the storage medium (computer-readable medium) of the present application can be a computer-readable signal medium or a non-transitory computer-readable storage medium or any combination thereof. The non-transitory computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the non-transitory computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0047] In the present application, the non-transitory computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the non-transitory computer-readable storage medium, which can send, propagate or transmit a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency) or the like, or any suitable combination thereof.
[0048] The above computer-readable medium can be contained in the above electronic device; or can exist separately and not be assembled into the electronic device.
[0049] Computer program code for carrying out operations of the present application can be written in one or more programming languages or combinations of languages including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer and partially on a remote computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
[0050] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.
[0051] The units described in the embodiments of the present application can be implemented by software, or by hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.
[0052] The functions described above can be performed at least in part by one or more hardware logic components. For example, non-limiting examples of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.
[0053] The above description is merely illustrative of the application of technical principles of the present application. Those skilled in the art should understand that the scope of the disclosure involved in the present application is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by any combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by replacing the above-described features with technical features having similar functions disclosed in the present application (but not limited to) can be formed.
[0054] Moreover, while operations are depicted in a particular order, this should not be understood as requiring such an order unless otherwise specifically stated. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, while a number of potentially novel implementations have been described, such a number of implementations is merely a subset of implementations that are not only possible but also conceivable given the above description. Many variations are possible given these teachings, which have been described in the context of the presently preferred, but not necessarily all, embodiments. Alterations and further modifications of the inventive features illustrated herein, and additional implementations not described specifically herein, will occur to one skilled in the art. To the extent that certain novel features of this invention have been described above, it should be understood that they are presented by way of example only and not limitation. As such, numerous other embodiments can be devised that do not differ from the basic implementation of the invention as described above, while still further embodiments can arrive at the same or similar results achieved by the embodiments described above. Accordingly, applicants reserve the right running the full scope of the appended claims, and also to claim such other and further embodiments, with their full scope as described throughout this disclosure.
[0055] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Claims
1. A method for extrinsic parameter calibration of a camera-AOA base station, characterized in that: include: Fix the UWB tag below the center of the AprilTag calibration board and integrate the IMU sensor on the AOA base station; Control the robot to move with the AprilTag calibration plate and collect the AprilTag pose, tag polar coordinates, and gravity vector data; Based on the AprilTag pose, tag polar coordinates and gravity vector data, coordinate unification is performed to obtain UWB tag coordinates, UCS virtual point coordinates and converted gravity vector; Based on the UWB tag coordinates, the UCS virtual point and the converted gravity vector, the optimal extrinsic parameters are solved.
2. The extrinsic parameter calibration method of the camera-AOA base station according to claim 1, characterized in that: The specific steps of controlling the robot to move with the AprilTag calibration plate and collecting the AprilTag pose, tag polar coordinates, and gravity vector data include: The robot carrying the AprilTag calibration plate is controlled to move along a spiral-step trajectory to collect several sets of spatiotemporally aligned AprilTag poses, tag polar coordinates, and gravity vector data.
3. The extrinsic parameter calibration method of the camera-AOA base station according to claim 1, characterized in that: The specific steps of performing coordinate unification based on the AprilTag pose, tag polar coordinates and gravity vector data to obtain UWB tag coordinates, UCS virtual point coordinates and converting the gravity vector are as follows: ； ; ; in, is the center coordinate of the AprilTag calibration plate, is the rotation matrix of the AprilTag calibration plate relative to the camera, is the vertical offset of the label, For the The coordinates of the UWB tag in the camera coordinate system in the data, For the The coordinates of the UCS virtual point in the camera coordinate system; For the The measurement distance of the AOA base station of the data, For the The horizontal angle of the AOA base station for each data point, is the camera-based gravity vector, is the original gravity vector of the IMU sensor, is the rotation transformation matrix.
4. The extrinsic parameter calibration method of the camera-AOA base station according to claim 1, characterized in that: The specific steps of solving the optimal external parameters based on the UWB tag coordinates, the UCS virtual point and the converted gravity vector include: Based on the preset distance consistency threshold, outliers are filtered and the initial value of the external parameter is determined. The formula is as follows: ； in, is the preset distance consistency threshold; Based on the initial values of the extrinsic parameters, the Ceres precision optimization method is used in conjunction with gravity constraints to optimize the initial values of the extrinsic parameters until convergence, and the optimal extrinsic parameters are obtained.
5. The extrinsic parameter calibration method of the camera-AOA base station according to claim 4, characterized in that: The SolvePnPRansac algorithm is used to determine the initial external parameters.
6. The extrinsic parameter calibration method of the camera-AOA base station according to claim 4, characterized in that: The method further comprises: Based on the errors detected in real time, the optimal extrinsic parameters are incrementally optimized.
7. The extrinsic parameter calibration method of the camera-AOA base station according to claim 4, characterized in that: The IMU sensor is rigidly connected to the AoA base station, and the installation spacing is less than 10 mm.
8. An electronic device, characterized in that: include: processor; a memory having stored thereon one or more computer program instructions executed on the processor; When the processor runs the computer program instructions, it executes the camera-AOA base station extrinsic parameter calibration method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that Computer instructions are stored thereon, and when the computer instructions are executed by the processor, the steps of the camera-AOA base station external parameter calibration method according to any one of claims 1 to 7 are executed.