Double-radar online calibration method, device and equipment and storage medium

Through the online calibration method, the dual radar calibration process is simplified by using timestamp registration and error function iterative calculation, solving the problem of time-consuming and labor-intensive traditional calibration, real-time consistency and accuracy of dual radar data, and meeting the real-time perception process. Sexual requirements.

CN118980993APending Publication Date: 2024-11-19UBTECH ROBOTICS CORP LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411393483.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-29
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The traditional radar calibration method needs to be carried out offline, which is time-consuming and labor-intensive, and it is difficult to meet the real-time requirements in the perception process.

Method used

By obtaining the data of lidar and millimeter wave radar in the same scenario, performing time stamp registration, and iteratively calculate the rotation matrix and offset based on the error function to realize dual radar online calibration, simplifying the calibration process and reducing time consumption.

Benefits of technology

Real-time calibration of dual radars online combined use is realized, which improves calibration efficiency and performance, ensures data consistency and accuracy, and meets the real-time requirements in the perception process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118980993A_ABST
    Figure CN118980993A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of radar calibration, and discloses a double-radar online calibration method and device, equipment and a storage medium. The method comprises the steps that first radar data and second radar data obtained when the same target object is measured in the same scene are acquired, and point pairs are established between the first radar data and the second radar data under the same timestamp; and based on the constructed error function, iteratively calculating a point pair distance sum of every two composition points in all the group point pairs, determining an optimal rotation matrix and an optimal offset according to the point pair distance sum obtained each time, and further carrying out online calibration on the first radar and the second radar. Calibration is completed in real time in the double-radar combined online use process, calibration time consumption is reduced, calibration efficiency is improved, the performance of double-radar combined online use is improved, and consistency and accuracy of double-radar data in the online use process are guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of radar calibration technology, and in particular to a dual-radar online calibration method, device, equipment and storage medium. Background Art

[0002] Lidar and millimeter-wave radar are two commonly used types of sensors, which play an important role in autonomous driving, robot navigation, intelligent transportation and other fields.

[0003] In order to ensure the consistency and accuracy of radar data, dual radars need to be calibrated to eliminate the errors between different radars. Among them, traditional calibration methods usually need to be performed offline. The offline calibration process includes steps such as manual selection of feature points and identification of calibration plates. These processes are time-consuming and laborious, and it is difficult to meet the real-time requirements of the perception process. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a dual-radar online calibration method, device, equipment and storage medium, which can effectively solve the problem that the joint calibration process of lidar and millimeter-wave radar is time-consuming and labor-intensive and difficult to meet the real-time requirements in the perception process.

[0005] In a first aspect, the present application provides a dual radar online calibration method, comprising:

[0006] Acquire first radar data and second radar data obtained when measuring the same target object in the same scene;

[0007] Performing time stamp registration on the first radar data and the second radar data to establish point pairs between the first radar data and the second radar data at the same time stamp, wherein each group of the point pairs includes two component points representing the first radar data and the second radar data respectively;

[0008] Based on the constructed error function, iteratively calculate the point pair distance between two component points in each group of the point pair, add the point pair distances of all groups to obtain the sum of the point pair distances, and determine the optimal rotation matrix and the optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time;

[0009] The first radar and the second radar are calibrated online according to the optimal rotation matrix and the optimal offset.

[0010] In an optional implementation, the constructed error function is based on iteratively calculating the point pair distance between two constituent points in each group of the point pair, adding the point pair distances of all groups to obtain the sum of the point pair distances, and determining the optimal rotation matrix and the optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time, including:

[0011] Set a rotation matrix and offset to construct the error function;

[0012] Iteratively solving the squared Euclidean distance between two constituent points in each group of point pairs corresponding to different values ​​of the rotation matrix and different values ​​of the offset through the error function, and adding the squared Euclidean distances of all groups to obtain the sum of the point pair distances of all groups;

[0013] Determine whether the error function is currently converged according to the sum of the distances of the point pairs obtained each time;

[0014] The rotation matrix and the offset corresponding to when the error function converges are used as the optimal rotation matrix and the optimal offset.

[0015] In an optional implementation, before iteratively solving the squared Euclidean distance between two constituent points in each corresponding point pair when the rotation matrix takes different values ​​and the offset takes different values ​​through the error function, the method further includes:

[0016] Acquire an initial rotation matrix and an initial offset; wherein the initial rotation matrix and the initial offset are obtained by offline calibration of the first radar and the second radar;

[0017] Determining adjacent value ranges corresponding to the initial rotation matrix and the initial offset respectively;

[0018] Any value is selected from the adjacent value range as the corresponding value of the rotation matrix and the offset.

[0019] In an optional embodiment, in the process of iteratively solving the squared Euclidean distances between two constituent points in each group of point pairs corresponding to different values ​​of the rotation matrix and different values ​​of the offset, and adding the squared Euclidean distances of all groups to obtain the sum of the point pair distances, the termination conditions of the multiple iterations include:

[0020] Condition 1: The number of iterations is equal to the preset maximum number of iterations;

[0021] Condition 2: The difference between the sums of the distances of the point pairs obtained from any two adjacent iterations is less than a preset difference threshold.

[0022] In an optional embodiment, it also includes:

[0023] If the corresponding termination condition when the current iteration terminates only satisfies condition 1, jump to the step of obtaining the first radar data and the second radar data obtained when measuring the same target object in the same scene to enter the iteration again;

[0024] Or, if the termination condition corresponding to the termination of the current iteration satisfies condition 2, it is determined that the error function converges, and any one of the rotation matrices and offsets of two adjacent iterations corresponding to the triggering of the termination of the iteration is used as the optimal rotation matrix and the optimal offset.

[0025] In an optional implementation, the online calibration of the first radar and the second radar according to the optimal rotation matrix and the optimal offset includes:

[0026] Calculating the product between the optimal rotation matrix and the second radar data in the target point pair;

[0027] The sum of the product and the optimal offset is used as the corresponding value of the first radar data of the target point pair in the second radar coordinate system, and then the first radar data and the second radar data are fused into the same second radar coordinate system to obtain a conversion relationship between the first radar coordinate system and the second radar coordinate system.

[0028] In an optional implementation, the performing timestamp registration on the first radar data and the second radar data to establish a point pair between the first radar data and the second radar data at the same timestamp includes:

[0029] respectively calculating a timestamp difference between each of the first radar data and each of the second radar data;

[0030] If the timestamp difference is within a preset threshold range, taking the timestamp of the first radar data and the timestamp of the second radar data corresponding to the corresponding timestamp difference as the same timestamp;

[0031] The first radar data and the second radar data at the same timestamp are taken as a group of point pairs.

[0032] In a second aspect, the present application provides a dual radar online calibration device, comprising:

[0033] An acquisition module, used to acquire first radar data and second radar data when measuring the same target object in the same scene;

[0034] a registration module, configured to perform time stamp registration on the first radar data and the second radar data, so as to establish a point pair between the first radar data and the second radar data at the same time stamp, wherein the point pair includes two component points representing the first radar data and the second radar data respectively;

[0035] an iterative module, configured to iteratively calculate the point pair distance between two constituent points in each group of point pairs based on the constructed error function, add the point pair distances of all groups to obtain a sum of the point pair distances, and determine an optimal rotation matrix and an optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time;

[0036] A calibration module is used to perform online calibration on the first radar and the second radar according to the optimal rotation matrix and the optimal offset.

[0037] In a third aspect, the present application provides a terminal device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the aforementioned dual radar online calibration method.

[0038] In a fourth aspect, the present application provides a computer storage medium storing a computer program, which, when executed on a processor, implements the aforementioned dual radar online calibration method.

[0039] The embodiments of the present application have the following beneficial effects:

[0040] The present application provides a dual radar online calibration method, including: obtaining the first radar data and the second radar data obtained when measuring the same target in the same scene; aligning the timestamps of the first radar data and the second radar data to establish a point pair between the first radar data and the second radar data at the same timestamp, each group of point pairs includes two component points representing the first radar data and the second radar data respectively; based on the constructed error function, iteratively calculating the point pair distance between the two component points in each group of point pairs, adding the point pair distances of all groups to obtain the sum of the point pair distances, and determining the optimal rotation matrix and the optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time; and calibrating the first radar and the second radar online according to the optimal rotation matrix and the optimal offset. This embodiment completes the calibration in real time during the joint online use of the dual radars, while reducing the calibration time, improving the calibration efficiency, thereby improving the performance of the dual radars when used online, ensuring the consistency and accuracy of the dual radar data during online use, and meeting the real-time requirements of the perception process. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A first schematic flow chart of a dual radar online calibration method in an embodiment of the present application is shown;

[0043] Figure 2 A second schematic flow chart of the dual radar online calibration method in an embodiment of the present application is shown;

[0044] Figure 3 A third process diagram of the dual radar online calibration method in an embodiment of the present application is shown;

[0045] Figure 4 A structural schematic diagram of a dual radar online calibration device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0047] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0048] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0049] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0050] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0051] In actual scenarios involving environmental perception and obstacle detection, in order to improve the perception performance and reliability of the system, generally speaking, millimeter-wave radar and lidar are used simultaneously on the same device, and the detection data of the dual radars are combined to improve reliability. However, in order to ensure data consistency and accuracy, the calibration problem between them needs to be solved first. This calibration refers to determining the exact relationship between two or more sensors so that the data they obtain can be fused or processed collaboratively. Traditional calibration methods usually need to be performed offline, and the calibration process is time-consuming and labor-intensive, making it difficult to meet real-time requirements.

[0052] Based on this, this embodiment provides a dual-radar online calibration method to simplify the calibration process in the offline calibration process, thereby reducing the calibration time, so as to improve the efficiency and performance of the joint application between millimeter-wave radar and lidar.

[0053] The dual radar online calibration method is described below in conjunction with some specific embodiments.

[0054] For example, Figure 1 As shown, the dual radar online calibration method includes the following steps:

[0055] S100, obtaining first radar data and second radar data obtained when measuring the same target object in the same scene.

[0056] In this embodiment, when the first radar and the second radar are calibrated online, the point cloud data obtained when the first radar and the second radar measure the same target object online in the same scene are first collected, and the point cloud data is used as the first radar data (i.e., P radar , where each point cloud data is p i ) and the second radar data (i.e. P lidar , where each point cloud data is q i). Wherein, the first radar is a millimeter wave radar, and the first radar data corresponds to the millimeter wave radar data; the second radar is a laser radar, and the second radar data corresponds to the laser radar data.

[0057] S200: aligning the timestamps of the first radar data and the second radar data to establish a point pair between the first radar data and the second radar data at the same timestamp.

[0058] In actual scenarios, since the first radar and the second radar are installed at different positions on the same device, the collected first radar data and the second radar data are in different coordinate systems respectively, and therefore it is necessary to use complex mathematical transformations to convert the data in the two different coordinate systems into the same coordinate system for fusion.

[0059] Furthermore, this embodiment performs nearest neighbor processing on the collected second radar data and first radar data, and selects the second radar data and the first radar data that can be considered to be at the same time for registration.

[0060] In one example, if Figure 2 As shown, the above S200 may specifically include the following steps:

[0061] S210 , respectively calculating a timestamp difference between each first radar data and each second radar data.

[0062] S220: If the timestamp difference is within a preset threshold range, the timestamp of the first radar data and the timestamp of the second radar data corresponding to the corresponding timestamp difference are taken as the same timestamp.

[0063] S230: Take the first radar data and the second radar data at the same timestamp as a group of point pairs.

[0064] In actual scenarios, due to equipment hardware issues, the second radar data and the first radar data with the same timestamp are generally not obtained. In this embodiment, the nearest neighbor data pair is found from the second radar data and the first radar data, and then the registration is performed based on the nearest neighbor data pair.

[0065] In this embodiment, a threshold range is preset. When the timestamp difference between the second radar data and the first radar data falls within the preset threshold range, it can be considered that the two are data at the same timestamp and can be aligned. That is, when the timestamp difference between the second radar data and the first radar data satisfies the following formula, it means that the two are data at the same timestamp:

[0066] |T lidar -T radar | <THRESHOLD;

[0067] Among them, the above formula T lidar is the timestamp of the second radar data; T radar is the timestamp of the first radar data; THRESHOLD is the preset threshold range, and its specific value can be set accordingly according to actual needs.

[0068] Furthermore, the second radar data and the first radar data corresponding to the timestamp difference falling within the preset threshold range are taken as a set of point pairs (q i , p i ), and similarly, multiple groups of point pairs at different time stamps can be obtained. That is, each group of point pairs includes two constituent points representing the first radar data and the second radar data respectively.

[0069] S300, based on the constructed error function, iteratively calculate the point pair distance between two component points in each group of point pairs, add the point pair distances of all groups to obtain the sum of the point pair distances, and determine the optimal rotation matrix and the optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time.

[0070] After obtaining the point pairs at different timestamps, this embodiment uses an iterative closest point (ICP) algorithm to iterate the point pair distance between every two constituent points in all point pairs for multiple times, and determines the optimal rotation matrix and the optimal offset between the first radar coordinate system and the second radar coordinate system, so as to achieve high-precision three-dimensional point cloud data alignment between the first radar and the second radar.

[0071] Since the first radar data has no height information, that is, there is no z coordinate information, in order to achieve data alignment in two different coordinate systems, this embodiment sets all z coordinates of the first radar data to 0, and then uses the ICP algorithm with initial values ​​to iteratively solve the optimal spatial transformation relationship between all point pairs (including rotation and translation, that is, calculate the optimal rotation matrix and the optimal offset) to minimize the distance error between the first radar data and the second radar data.

[0072] For example, Figure 3 As shown, the above S300 may include the following steps:

[0073] S310, setting a rotation matrix and an offset to construct an error function.

[0074] S320, through the error function, iteratively solves the square Euclidean distance between the two constituent points in each corresponding group of point pairs when the rotation matrix takes different values ​​and the offset takes different values, and adds the square Euclidean distances of all groups to obtain the total point pair distance.

[0075] S330, determining whether the error function is currently converged according to the sum of the distances of the point pairs obtained each time.

[0076] S340, taking the rotation matrix and offset corresponding to when the error function converges as the optimal rotation matrix and optimal offset.

[0077] In this embodiment, an error function is constructed by using the rotation matrix and the offset as parameters, and then all corresponding point pairs (q i , p i ) is the sum of the distances between the point pairs between ; if the error function converges, it means that the current corresponding rotation matrix and offset are the optimal rotation matrix and the optimal offset, that is, the optimal rotation matrix and the optimal offset are determined by the convergence of the error function.

[0078] Among them, the expression of the error function is:

[0079]

[0080] In the formula, E represents the error value, R represents the rotation matrix, and t represents the offset; i =Rp i +t) means the value corresponding to the first radar data in the second radar coordinate system.

[0081] It can be understood that, according to the set rotation matrix and offset, this embodiment places the first radar data and the second radar data in the same second radar coordinate system, and continuously sets the values ​​of the rotation matrix and the offset, respectively solves the square Euclidean distance between the two constituent points in each group of point pairs under different values ​​of the rotation matrix and the offset, and synchronously calculates the sum of the square Euclidean distances of all groups of point pairs to obtain the sum of the point pair distances. Among them, the value ranges corresponding to the rotation matrix and the offset are different, and the value ranges of the rotation matrix and the offset can be set accordingly according to actual needs, and this embodiment does not limit this.

[0082] That is, a value of the rotation matrix and a value of the offset are randomly set, and the values ​​of the rotation matrix and the offset are substituted into the above-mentioned error function to calculate the square Euclidean distance between the two constituent points in each group of point pairs and the sum of the square Euclidean distances of all group point pairs under the current value. The sum of the square Euclidean distances of all group point pairs is the error value obtained in the above-mentioned error function under the current value of the rotation matrix and the offset. Then, the rotation matrix and the offset with different values ​​can be substituted into the above-mentioned error function to calculate multiple error values, so as to determine whether the current error function converges according to the changes of these multiple error values, and the values ​​of the rotation matrix and the offset corresponding to the convergence of the error function are used as the optimal rotation matrix and the optimal offset.

[0083] It should be noted that since the random value range of the rotation matrix and the offset is too large, in order to further reduce the number of calculations and the workload to improve the calculation efficiency, an initial value range or initial reference value of the rotation matrix and the offset can be set in advance as a reference for the initial value and subsequent multiple values; among which, the setting method of the initial value range or the initial reference value is not limited here.

[0084] For reference, at each iteration, a rotation matrix and an offset value (i.e., an initial value) may be initially set to calculate the square Euclidean distance of each group of point pairs in the same second radar coordinate system according to the rotation matrix and the offset value, and then the square Euclidean distances of all groups are added together to obtain the sum of the point pair distances; furthermore, each time the rotation matrix and the offset value are set, the sum of the point pair distances of all groups of point pairs may be obtained to detect whether the error function converges during the iteration process to determine the optimal rotation matrix and the optimal offset.

[0085] Optionally, the rotation matrix and offset corresponding to the minimum value in the sum of the distances of multiple point pairs can be used as the optimal rotation matrix and optimal offset. That is, the minimum value is found from the sum of the distances of multiple point pairs obtained, so as to determine the optimal rotation matrix and optimal offset according to the rotation matrix and offset corresponding to the minimum value.

[0086] In one example, to ensure computational efficiency and reliability, the initial value of this embodiment can be set accordingly according to the historical offline calibration results; for example, the optimal rotation matrix and the optimal offset obtained by the dual radar in the historical offline calibration process are used as the initial rotation matrix (R o ) and initial offset (t o ), wherein the specific offline calibration process is not limited here.

[0087] Then, the adjacent value ranges corresponding to the initial rotation matrix and the initial offset are determined respectively; then, when setting different values ​​of the rotation matrix and the offset respectively, any value can be selected from the corresponding adjacent value range, corresponding to the current value of the rotation matrix and the offset; then, the sum of the point pair distances of all groups of point pairs corresponding to any value in the adjacent value range is iteratively solved to detect whether the error function converges during the iteration process. That is, according to the initial rotation matrix and the initial offset obtained by the offline calibration, the value ranges of the rotation matrix and the offset in the online calibration process are respectively divided, and the value range includes the initial value of the rotation matrix or the offset and the value of the corresponding adjacent range.

[0088] It can be understood that during the online calibration and offline calibration processes, the transformation relationship between the coordinate systems of the two radars is often relatively close. Therefore, the value range of the rotation matrix and the offset in the online calibration process can be set by using the initial rotation matrix and the initial offset obtained by the offline calibration, so as to quickly find the optimal rotation matrix and the optimal offset for the error function to converge, thereby reducing the number of iterations and the amount of calculation while realizing online calibration.

[0089] In one embodiment, the termination conditions of multiple iterations can be set in the process of iteratively solving the sum of point pair distances between all groups of point pairs corresponding to different values ​​of the rotation matrix and the offset: Condition 1, the number of iterations is equal to the preset maximum number of iterations; Condition 2, the difference between the sum of the point pair distances obtained from any two adjacent iterations is less than the preset difference threshold (that is, the error function changes little and the error function converges).

[0090] It can be understood that the number of iterations is counted while performing iterations to detect whether the number of iterations currently completed is equal to the preset maximum number of iterations, and to detect whether the difference between the sum of the distances of the point pairs obtained in the current iteration and the previous iteration is less than the preset difference threshold; further, when it is determined that the current iteration satisfies any of the above termination conditions, the current iteration is terminated. The maximum number of iterations and the preset difference threshold can be set accordingly according to actual needs, and this embodiment does not limit this.

[0091] Among them, if the corresponding termination condition only satisfies condition 1 when the current iteration terminates, jump to the step of obtaining the first radar data and the second radar data to enter the iteration again; that is, if the iteration is terminated only when the preset maximum number of iterations is reached, jump to S100 and re-execute S100-S400, that is, re-acquire the first radar data and the second radar data to iterate again.

[0092] If the termination condition corresponding to the termination of the current iteration satisfies condition 2, the current error function is determined to converge, and any one of the rotation matrices and offsets of the two adjacent iterations corresponding to the triggering of the termination of the iteration is used as the optimal rotation matrix and the optimal offset.

[0093] It can be understood that if the corresponding situations when the current iteration terminates are that the termination condition only satisfies condition 2, or the termination condition satisfies condition 1 and condition 2 at the same time, it can be determined that the error function corresponding to the current iteration converges, and the rotation matrix and offset corresponding to any of the two adjacent iterations corresponding to the triggering iteration termination can be used as the optimal rotation matrix and the optimal offset respectively. In addition, as an optional implementation, the mean of the rotation matrix and the offset corresponding to the two adjacent iterations corresponding to the triggering iteration termination can also be calculated respectively, and the mean value can be used as the optimal rotation matrix and the optimal offset.

[0094] S400: Online calibration of the first radar and the second radar is performed according to the optimal rotation matrix and the optimal offset.

[0095] This embodiment can perform online calibration of the dual radars based on the obtained optimal rotation matrix and optimal offset. The product between the optimal rotation matrix and the second radar data in the target point pair is first calculated, and the sum of the obtained product and the optimal offset is used as the corresponding value of the first radar data in the target point pair in the second radar coordinate system, and then the first radar data and the second radar data can be fused into the same second radar coordinate system to obtain the conversion relationship between the second radar coordinate system and the first radar coordinate system.

[0096] That is, the optimal rotation matrix and the optimal offset can represent the corresponding conversion relationship when the first radar coordinate system is converted to the second radar coordinate system, and then according to the optimal rotation matrix and the optimal offset, the corresponding value obtained when the first radar data of each point is converted to the second radar coordinate system is calculated as p i =Rp i +t, thereby obtaining the data values ​​of the two radars in the same second radar coordinate system and realizing the online calibration of the dual radars.

[0097] This embodiment sets an ICP algorithm with initial values ​​(i.e., iteratively obtains the optimal rotation matrix and the optimal offset through the adjacent value range corresponding to the initial rotation matrix and the initial offset obtained by offline calibration), and then places the first radar data and the second radar data in the same second radar coordinate system when the point pairs at the same timestamp are obtained by alignment, and iteratively calculates the optimal rotation matrix and the optimal offset for fusing the first radar data into the second radar coordinate system, thereby completing the online calibration. It can be seen that this embodiment does not need to perform steps such as manually selecting feature points and identifying the calibration plate in the traditional offline calibration process, simplifies the calibration process, reduces the calibration time, and can complete the calibration in real time during the joint online use of the millimeter wave radar and the laser radar, so as to improve the calibration efficiency, and then improve the performance of the dual radars when they are used together online, ensuring the consistency and accuracy of the dual radar data during the online use process, so as to meet the real-time requirements in the perception process.

[0098] like Figure 4 As shown, an embodiment of the present application provides a dual radar online calibration device, which exemplarily includes:

[0099] An acquisition module 410 is used to acquire first radar data and second radar data obtained when measuring the same target object in the same scene;

[0100] a registration module 420, configured to perform time stamp registration on the first radar data and the second radar data to establish point pairs between the first radar data and the second radar data at the same time stamp, wherein each group of the point pairs includes two component points representing the first radar data and the second radar data respectively;

[0101] an iterative module 430, configured to iteratively calculate a point pair distance between two constituent points in each group of the point pairs based on the constructed error function, add the point pair distances of all groups to obtain a sum of the point pair distances, and determine an optimal rotation matrix and an optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time;

[0102] The calibration module 440 is used to perform online calibration on the first radar and the second radar according to the optimal rotation matrix and the optimal offset.

[0103] It can be understood that the device of this embodiment corresponds to the dual radar online calibration method of the above embodiment, and the optional items in the above embodiment are also applicable to this embodiment, so they will not be repeated here.

[0104] The present application also provides a terminal device. Exemplarily, the terminal device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the terminal device to execute the functions of each module in the above-mentioned dual-radar online calibration method or the above-mentioned dual-radar online calibration device.

[0105] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU) and a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0106] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0107] The present application also provides a computer storage medium for storing the computer program used in the above-mentioned terminal device, wherein when the computer program is executed on a processor, the dual radar online calibration method of the above-mentioned embodiment is implemented.

[0108] It can be understood that the options in the dual radar online calibration method of the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.

[0109] The computer storage medium may be a non-volatile storage medium or a volatile storage medium. For example, the computer storage medium may include, but is not limited to, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0112] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

[0113] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A dual radar online calibration method, characterized in that: include: Acquire first radar data and second radar data obtained when measuring the same target object in the same scene; Performing time stamp registration on the first radar data and the second radar data to establish point pairs between the first radar data and the second radar data at the same time stamp, wherein each group of the point pairs includes two component points representing the first radar data and the second radar data respectively; Based on the constructed error function, iteratively calculate the point pair distance between two component points in each group of the point pair, add the point pair distances of all groups to obtain the sum of the point pair distances, and determine the optimal rotation matrix and the optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time; The first radar and the second radar are calibrated online according to the optimal rotation matrix and the optimal offset.

2. The dual radar online calibration method according to claim 1, characterized in that: The method of iteratively calculating the point pair distance between two constituent points in each group of the point pair based on the constructed error function, adding the point pair distances of all groups to obtain the sum of the point pair distances, and determining the optimal rotation matrix and the optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time, includes: Set a rotation matrix and offset to construct the error function; Iteratively solving the squared Euclidean distance between two constituent points in each group of point pairs corresponding to different values ​​of the rotation matrix and different values ​​of the offset through the error function, and adding the squared Euclidean distances of all groups to obtain the sum of the point pair distances of all groups; Determine whether the error function is currently converged according to the sum of the distances of the point pairs obtained each time; The rotation matrix and the offset corresponding to when the error function converges are used as the optimal rotation matrix and the optimal offset.

3. The dual radar online calibration method according to claim 2, characterized in that: Before iteratively solving the squared Euclidean distance between two constituent points in each corresponding point pair when the rotation matrix takes different values ​​and the offset takes different values ​​through the error function, the method further includes: Acquire an initial rotation matrix and an initial offset; wherein the initial rotation matrix and the initial offset are obtained by offline calibration of the first radar and the second radar; Determining adjacent value ranges corresponding to the initial rotation matrix and the initial offset respectively; Any value is selected from the adjacent value range as the corresponding value of the rotation matrix and the offset.

4. The dual radar online calibration method according to claim 2 or 3, characterized in that: In the process of iteratively solving the squared Euclidean distances between two constituent points in each group of point pairs corresponding to different values ​​of the rotation matrix and different values ​​of the offset, and adding the squared Euclidean distances of all groups to obtain the sum of the point pair distances, the termination conditions of multiple iterations include: Condition 1: The number of iterations is equal to the preset maximum number of iterations; Condition 2: The difference between the sums of the distances of the point pairs obtained from any two adjacent iterations is less than a preset difference threshold.

5. The dual radar online calibration method according to claim 4, characterized in that: Also includes: If the corresponding termination condition when the current iteration terminates only satisfies condition 1, jump to the step of obtaining the first radar data and the second radar data obtained when measuring the same target object in the same scene to enter the iteration again; Or, if the termination condition corresponding to the termination of the current iteration satisfies condition 2, it is determined that the error function converges, and any one of the rotation matrices and offsets of two adjacent iterations corresponding to the triggering of the termination of the iteration is used as the optimal rotation matrix and the optimal offset.

6. The dual radar online calibration method according to claim 1, characterized in that: The online calibration of the first radar and the second radar according to the optimal rotation matrix and the optimal offset includes: Calculating the product between the optimal rotation matrix and the second radar data in the target point pair; The sum of the product and the optimal offset is used as the corresponding value of the first radar data of the target point pair in the second radar coordinate system, and then the first radar data and the second radar data are fused into the same second radar coordinate system to obtain a conversion relationship between the first radar coordinate system and the second radar coordinate system.

7. The dual radar online calibration method according to claim 1, characterized in that: The performing timestamp alignment on the first radar data and the second radar data to establish a point pair between the first radar data and the second radar data at the same timestamp includes: respectively calculating a timestamp difference between each of the first radar data and each of the second radar data; If the timestamp difference is within a preset threshold range, taking the timestamp of the first radar data and the timestamp of the second radar data corresponding to the corresponding timestamp difference as the same timestamp; The first radar data and the second radar data at the same timestamp are taken as a group of point pairs.

8. A dual radar online calibration device, characterized in that: include: An acquisition module, used to acquire first radar data and second radar data when measuring the same target object in the same scene; a registration module, configured to perform time stamp registration on the first radar data and the second radar data, so as to establish a point pair between the first radar data and the second radar data at the same time stamp, wherein the point pair includes two component points representing the first radar data and the second radar data respectively; an iterative module, configured to iteratively calculate the point pair distance between two constituent points in each group of point pairs based on the constructed error function, add the point pair distances of all groups to obtain a sum of the point pair distances, and determine an optimal rotation matrix and an optimal offset between the first radar coordinate system and the second radar coordinate system according to the sum of the point pair distances obtained each time; A calibration module is used to perform online calibration on the first radar and the second radar according to the optimal rotation matrix and the optimal offset.

9. A terminal device, characterized in that: The terminal device comprises a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the dual-radar online calibration method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: It stores a computer program, which, when executed on a processor, implements the dual radar online calibration method according to any one of claims 1 to 7.

Citation Information

Cited By

  • Multi-target tracking and behavior identification method and monitoring device for dual millimeter wave radars

    CN122362362A