Joint calibration method and apparatus

Abstract

The embodiment provides a joint calibration method and device. The embodiment calibrates the calibration parameters between multiple radar sensors and a global coordinate system, so that when the same target object is perceived by the multiple radar sensors, the error of the coordinate mapping of the target object on a global coordinate system, such as a map, is small, the trajectory of the target object on the global coordinate system, such as the map, is more distinguishable from the trajectories of other target objects, the global optimization is facilitated, and the subsequent multi-radar sensor signal fusion accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of sensor calibration, and in particular to a joint calibration method and apparatus. Background Technology

[0002] In applications such as traffic holographic sensing, when the same target object is detected by multiple sensors, such as millimeter-wave radar sensors operating in the millimeter-wave band, it is necessary to first map the sensing results of each sensor to the same coordinate system (denoted as the global coordinate system, such as the GPS coordinate system on a map), and then fuse the mapping results to obtain accurate measurement results of the target object, such as a moving vehicle, such as speed, distance, and angle. Here, when millimeter-wave radar sensors are involved, they are, for example, three-dimensional (3D) millimeter-wave radar sensors (involving three dimensions: speed, distance, and angle), 4D millimeter-wave radar sensors (involving four dimensions: speed, distance, angle, and height), etc.

[0003] However, current sensor calibration methods all involve the calibration of a single sensor and cannot be applied to joint calibration in multi-sensor application scenarios. Summary of the Invention

[0004] This application provides a joint calibration method and apparatus to achieve joint calibration in multi-sensor application scenarios.

[0005] This application provides a joint calibration method applied to a multi-radar sensor scenario where at least two radar sensors are deployed. Each radar sensor is pre-calibrated to achieve time synchronization between them. The method includes:

[0006] The target trajectory is compensated based on the motion speed output by the target radar sensor and associated with the target trajectory, so as to eliminate the differences in the sampling interval and / or sampling start point set by each target radar sensor; the target trajectory is selected from the trajectories sensed by each radar sensor within a specified time period; each radar sensor samples the trajectory according to the set sampling interval.

[0007] Based on the target objects associated with the target trajectories perceived by each target radar sensor, a global identifier ID is determined for the target trajectories perceived by each target radar sensor, so that the global ID of the target trajectories of the same target object perceived by different target radar sensors is the same.

[0008] Using the topology required by the specified space corresponding to the global coordinate system, the target trajectory perceived by each target radar sensor within at least one specified time period, and the global ID of the target trajectory, a first loss function and / or a second loss function and / or a third loss function are established to obtain the calibration parameters of each radar sensor through the first loss function and / or the second loss function and / or the third loss function; the calibration parameters of any radar sensor are used to map the position in the trajectory perceived by the radar sensor to the global coordinate system;

[0009] The first loss function is set based on the principle of minimizing the projection error of the same target object within the intersection area of ​​multiple radar sensors, and the first loss function involves the calibration parameters of each radar sensor; the second loss function is set based on the principle that the topology obtained after mapping the trajectory perceived by each radar sensor to the global coordinate system using the calibration parameters of each radar sensor is similar to the topology required by the specified space; the third loss function is set based on the principle that the topology obtained after mapping the trajectory corresponding to the same global ID perceived by each radar sensor in the same time period to the global coordinate system using the calibration parameters of each radar sensor is similar.

[0010] This application provides a joint calibration device applied in a multi-radar sensor scenario where at least two radar sensors are deployed. Each radar sensor is pre-calibrated to achieve time synchronization between them. The device includes:

[0011] The compensation module is used to compensate the target trajectory based on the motion speed output by the target radar sensor and associated with the target trajectory, so as to eliminate the differences in the sampling interval and / or sampling start point of each target radar sensor; the target trajectory is selected from the trajectories sensed by each radar sensor within a specified time period; each radar sensor samples the trajectory according to the set sampling interval.

[0012] The global identifier ID determination module is used to determine the global identifier ID for the target trajectory perceived by each target radar sensor based on the target object associated with the target trajectory perceived by each target radar sensor, so that the global ID of the target trajectory of the same target object perceived by different target radar sensors is the same.

[0013] The calibration module is used to establish a first loss function and / or a second loss function and / or a third loss function using the topology required by the specified space corresponding to the global coordinate system, the target trajectory perceived by each target radar sensor within at least one specified time period, and the global ID of the target trajectory. The calibration parameters of each radar sensor are used to map the position in the trajectory perceived by the radar sensor to the global coordinate system.

[0014] The first loss function is set based on the principle of minimizing the projection error of the same target object within the intersection area of ​​multiple radar sensors, and the first loss function involves the calibration parameters of each radar sensor; the second loss function is set based on the principle that the topology obtained after mapping the trajectory perceived by each radar sensor to the global coordinate system using the calibration parameters of each radar sensor is similar to the topology required by the specified space; the third loss function is set based on the principle that the topology obtained after mapping the trajectory corresponding to the same global ID perceived by each radar sensor in the same time period to the global coordinate system using the calibration parameters of each radar sensor is similar.

[0015] This application provides an electronic device, which includes a processor and a memory; wherein the memory is used to store machine-executable instructions; and the processor is used to read and execute the machine-executable instructions stored in the memory to implement the method described above.

[0016] In summary, this embodiment, by jointly calibrating the calibration parameters between multiple radar sensors and the global coordinate system, ensures that when the same target object is perceived by multiple radar sensors, the error of its coordinate mapping on the global coordinate system, such as a map, is small. This makes it easier to distinguish the trajectory of other target objects mapped on the global coordinate system, such as a map, which is conducive to achieving global optimization and improving the accuracy of subsequent multi-radar sensor signal fusion.

[0017] Furthermore, this embodiment, by jointly calibrating the calibration parameters between multiple radar sensors and the global coordinate system, ensures that when the same target object is perceived by multiple radar sensors (e.g., relay sensing), the relay trajectory mapped onto the global coordinate system, such as a map, is continuous and smooth, resulting in better visualization.

[0018] Furthermore, this embodiment enables the joint calibration of calibration parameters between multiple radar sensors and the global coordinate system without manual intervention, thus improving efficiency. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the method in an embodiment of this application;

[0020] Figure 2 This is a structural diagram illustrating a method example from an embodiment of this application;

[0021] Figure 3 This is a structural diagram of the device shown in the embodiments of this application;

[0022] Figure 4 This is a structural diagram of an electronic device shown in an embodiment of this application. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0024] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0025] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0026] The multi-millimeter-wave sensor calibration method provided in this embodiment enables joint calibration of multiple millimeter-wave radars and maps to improve calibration accuracy. An example is described below:

[0027] See Figure 1 , Figure 1 This is a flowchart illustrating a method provided in an embodiment of this application. This process can be applied to electronic devices, such as servers or terminals, but this embodiment does not specifically limit the application to these devices.

[0028] As an example, Figure 1 The method shown can be applied to multi-radar sensor scenarios. In this scenario, at least two radar sensors are deployed.

[0029] In a multi-radar sensor scenario, the time of multiple radar sensors is first synchronized to ensure that the system time of each radar sensor is consistent, thus achieving time synchronization between the radar sensors.

[0030] like Figure 1 As shown, the process may include the following steps:

[0031] Step 101: Based on the motion speed output by the target radar sensor and associated with the target trajectory, compensate the target trajectory to eliminate the differences in the sampling intervals set by each target radar sensor; the target trajectory is selected from the trajectories sensed by each radar sensor within a specified time period; each radar sensor samples the trajectory according to the set sampling interval.

[0032] As an example, this embodiment can obtain relatively reliable trajectories measured by each radar sensor through pre-set trajectory filtering logic.

[0033] Optionally, in this embodiment, the trajectories perceived by each radar sensor within a specified time period and their confidence levels can be obtained first; here, the trajectory confidence level is determined by the radar sensor for the trajectory it perceives. Then, the trajectory whose confidence level meets the confidence requirement (e.g., greater than a set confidence threshold) is selected as the target trajectory. That is, the selected target trajectory can be a relatively reliable trajectory. Correspondingly, in this embodiment, the radar sensor that outputs the target trajectory is also designated as the target radar sensor.

[0034] In this embodiment, each radar sensor samples the trajectory according to a set sampling interval. For example, one radar sensor starts sampling the trajectory every second from the 1st second of 9:10, while another radar sensor starts sampling every second from the 1.5th second of 9:10, and so on. It can be observed that the sampling starting points of different radar sensors are different, and there may even be differences in sampling intervals. To address this, this embodiment compensates for the target trajectory based on the motion speed output by the target radar sensor and associated with the target trajectory, thereby eliminating the differences in the set sampling intervals and / or sampling starting points of the various target radar sensors.

[0035] For example, one radar sensor (denoted as sensor a1) starts sampling the trajectory every second from the 1st second of 9:10, while another radar sensor (denoted as sensor a2) starts sampling the trajectory every second from the 1.5th second of 9:10. Based on the motion velocity output by sensor a2 that is associated with the target trajectory, the position of sensor a2 at the 1st second of 9:10 can be inferred. The trajectory sensed by sensor a2 can be adjusted based on the inferred position to align the starting time points of the trajectories sensed by each target radar sensor.

[0036] Step 102: Based on the target objects associated with the target trajectories perceived by each target radar sensor, determine a global identifier ID for the target trajectory perceived by each target radar sensor, so that the global ID of the target trajectory of the same target object perceived by different target radar sensors is the same.

[0037] As an example, in step 102, determining a global identifier ID for the target trajectory perceived by each target radar sensor based on the target object associated with the target trajectory perceived by each target radar sensor can be implemented in many ways. For example, for any position in the target trajectory perceived by each target radar sensor, where the position is perceived by the target radar sensor at a point in time within the specified time period, the target object and its feature identifier acquired by the visual camera associated with the target radar sensor at that point in time are obtained; the position is mapped from the radar coordinate system corresponding to the target radar sensor to the visual coordinate system corresponding to the visual camera to obtain a mapping point; it is checked whether the mapping point is on the target object acquired by the visual camera; if so, the position is determined to meet the matching requirements; if at least N positions in the target trajectory perceived by the target radar sensor meet the matching requirements, the target object feature identifier is assigned to the target trajectory as the global identifier ID of the target trajectory.

[0038] Optionally, in this embodiment, a vehicle ReID model is used to determine the global identifier ID of the target trajectory. For example, each radar sensor has a corresponding vision camera, and it is generally assumed that their installation positions overlap. The radar sensor and vision camera are automatically calibrated (this embodiment does not specifically limit how they are calibrated). The position perceived by the radar sensor can be mapped from the radar coordinate system to the corresponding camera coordinate system of the vision camera. This mapping can be called the RV calibration parameter. Taking a vehicle as the target object, in this embodiment, the vision camera uses a built-in vehicle target detection and license plate recognition algorithm to obtain the vehicle (i.e., the target object) and its associated license plate number (i.e., the target object feature identifier) ​​in each visual image over a period of time. During the same time period, based on the RV calibration parameter, each position in the trajectory perceived by the radar sensor is mapped from the radar coordinate system to the camera coordinate system to obtain the corresponding mapping point. If the mapping point at at least one time point happens to fall on the vehicle in the visual image acquired by the vision camera at that time point, then the license plate number can be used as the global ID of the trajectory. Optionally, this embodiment can also determine the license plate number by using the global ID of the trajectory sensed by the radar sensor in a previous time period and the trajectory sensed in a subsequent time period. This ultimately achieves the goal of determining the global ID of trajectories sensed by different radar sensors.

[0039] It should be noted that, optionally, this embodiment may also use a neural network to extract vehicle image features (which may be license plate numbers or other features) by referring to the vehicle ReID model. This embodiment is not specifically limited.

[0040] Step 103: Using the topology required by the specified space corresponding to the global coordinate system, the target trajectory perceived by each target radar sensor within at least one specified time period, and the global ID of the target trajectory, establish a first loss function and / or a second loss function and / or a third loss function to obtain the calibration parameters of each radar sensor through the first loss function and / or the second loss function and / or the third loss function; the calibration parameters of any radar sensor are used to map the position in the trajectory perceived by the radar sensor to the global coordinate system.

[0041] As an example, the position in the trajectory sensed by any radar sensor can be represented by coordinates in a 2D Cartesian coordinate system, as shown in the following formula.

[0042]

[0043] Where N represents the total number of target objects, i represents the target object with global ID i, R represents the total number of target radar sensors, and r represents the r-th target radar sensor, (x r,i ,y r,i ) represents the position of the target object with global ID i sensed by the r-th target radar sensor (represented by coordinate information in the radar coordinate system), θ r The calibration parameters of the r-th target radar sensor are used to map the position in the trajectory sensed by the radar sensor to the global coordinate system (such as the map coordinate system). This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i sensed by the r-th target radar sensor in the target trajectory, and f represents the mapping function (such as 2D rotation or translation).

[0044] In this embodiment, on the one hand, it is necessary to consider that after the position in the trajectory perceived by each radar sensor is mapped to a global coordinate system such as a map coordinate system, the coordinate difference of the same target object at the same time point is minimized. On the other hand, it is necessary to maintain the matching between the motion trajectory perceived by each radar sensor and the topology required by the specified space corresponding to the global coordinate system such as the map coordinate system. Under this premise, this embodiment needs to introduce a first loss function and / or a second loss function and / or a third loss function.

[0045] As an example, the first loss function is set based on the principle of minimizing the projection error of the same target object within the intersection area of ​​multiple radar sensors. The first loss function involves the calibration parameters of each radar sensor.

[0046] Optionally, in specific implementation, the first loss function is expressed by the following formula:

[0047]

[0048] Where N represents the total number of target objects, i represents the target object with global ID i, R represents the total number of target radar sensors, and r represents the r-th target radar sensor. Determined based on the coordinate information of each mapping point in the global coordinate system. For example... This can be expressed by the following formula: Each mapping point refers to the position of the target object with global ID i, as perceived by each target radar sensor, in the target trajectory, mapped to the global coordinate system.

[0049] In this embodiment, This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the r-th target radar sensor; This can be expressed by the following formula: f represents the mapping function, θ r This represents the calibration parameters of the radar sensor for the r-th target.

[0050] It should be noted that, in order to achieve the optimization effect of minimizing the projection error of the same target object within the intersection area of ​​multiple radar sensors, the above-mentioned first loss function can also be expressed by the following formula:

[0051]

[0052] in, This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the p-th target radar sensor. This can be expressed by the following formula: f represents the mapping function, θ p This represents the calibration parameters of the radar sensor for the p-th target; This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the q-th target radar sensor. This can be expressed by the following formula: f represents the mapping function, θ q This represents the calibration parameters of the radar sensor for the q-th target.

[0053] By using the first loss function L′1, the coordinate error of mapping the trajectory position points of the same target object perceived by any two radar sensors at the same time point to the global coordinate system, such as the map coordinate system, can be minimized (equivalent to minimizing the projection error of the same target object in the intersection area of ​​multiple radar sensors).

[0054] In this embodiment, the topological structure within the specified space corresponding to the global coordinate system conforms to the requirements of the specified space. Taking the global coordinate system as the map coordinate system as an example, the topological structure in the map can be represented by parametric equations of map elements (such as straight lines, arcs, Bézier curves, etc.), topological maps, gradients of unit coordinates (direction of motion), etc. To ensure the consistency between the trajectory perceived by the radar sensor and the topological structure required by the specified space corresponding to the global coordinate system, this embodiment introduces a second loss function.

[0055] Optionally, the second loss function is set according to the principle that the topology obtained after mapping the trajectory perceived by each radar sensor to the global coordinate system using the calibration parameters of each radar sensor is similar to the topology required by the specified space.

[0056] As an example, the second loss function is expressed by the following formula:

[0057] Where N represents the total number of target objects, R represents the total number of target radar sensors, r represents the r-th target radar sensor, and i represents the target object with global ID i. This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i detected by the r-th target radar sensor in the target trajectory. This means that after the target trajectory of the target object with global ID i sensed by the r-th target radar sensor is mapped to the global coordinate system, the topology transformation function τ is used to convert the mapped target trajectory into a topological representation in a specified space; T m This represents the standard topology corresponding to the topology representation in the specified space; This can be expressed by the following formula: f represents the mapping function, θ r This represents the calibration parameters of the radar sensor for the r-th target.

[0058] By minimizing the second loss function L2, the topological structure corresponding to the trajectory perceived by each radar sensor can be made similar to the topological structure required by the specified space corresponding to the global coordinate system. This manifests as the trajectory perceived by each radar sensor being mapped onto a global coordinate system, such as a map coordinate system, and following a route, such as a lane, on a map within the specified space corresponding to the global coordinate system. The second loss function also allows for the inference of calibration parameters between radar sensors in topologically similar regions (e.g., long straight road segments) when the topological structure is similar in the global coordinate system (e.g., long straight road segments). This avoids the problem of multiple local solutions by using radar sensors in topologically dissimilar regions (e.g., curves in a road segment or intersections connecting to the road segment) to infer the calibration parameters of radar sensors in topologically similar regions.

[0059] In this embodiment, the trajectories corresponding to the same global ID perceived by each radar sensor within the same time period, after being mapped to the global coordinate system using the calibration parameters of each radar sensor, result in similar topologies. Leveraging this characteristic, this embodiment introduces a third loss function. This third loss function is set according to the principle that the topologies obtained after mapping the trajectories corresponding to the same global ID perceived by each radar sensor within the same time period to the global coordinate system using the calibration parameters of each radar sensor are similar. This third loss function ensures that the trajectories of the same target object perceived by multiple radar sensors, when projected onto a global coordinate system such as a map, are continuous, smooth, or approximately approximate.

[0060] Alternatively, the third loss function can be expressed as follows:

[0061]

[0062] Where N represents the total number of target objects, i represents the target object with global ID i, R represents the total number of target radar sensors, and r represents the r-th target radar sensor. This indicates that the target trajectory of the p-th target radar sensor, with global ID i, after being mapped to the global coordinate system, is transformed into the first topological structure in the specified space by the topological transformation function τ. f represents the mapping function, θ p This represents the calibration parameters of the radar sensor for the p-th target; This indicates that the target trajectory of the target object with global ID i, sensed by the q-th target radar sensor, after being mapped to the global coordinate system, is transformed into a second topological structure in a specified space by the topological transformation function τ. This can be expressed by the following formula: f represents the mapping function, θ q This represents the calibration parameters of the radar sensor for the q-th target.

[0063] As an example, the topology transformation function τ can be a function that fits a set of trajectory points into a K-order spline curve; this example is not specifically limited.

[0064] It should be noted that the first and third loss functions address the scenario where the same target object can be detected by multiple radar sensors simultaneously. If a target object cannot be detected by multiple radar sensors at the same time, its trajectory may not be included in the calculation of the first and third loss functions. All trajectories detected by radar sensors are included in the calculation of the second loss function.

[0065] As an example, in this embodiment, with at least two loss functions involved, the values ​​obtained from each loss function are weighted and fused to obtain the calibration parameters of each radar sensor. For example, by weighting and summing the above three loss functions, the optimization objective for joint calibration of multiple radar sensors with a global coordinate system such as a map is obtained: L = L1 + αL2 + βL3; where α and β are preset weight parameters. This can be easily solved using existing optimizers (such as gradient descent, LM, RANSAC, and other robust optimizers). Finally, the calibration parameters of each radar sensor are calculated.

[0066] This concludes the process. Figure 1 The process is shown. Using... Figure 1 The process shown is as follows: Figure 2 The example structure is shown using the global coordinate system as the map coordinate system.

[0067] pass Figure 1 As can be seen from the process shown, this embodiment, by jointly calibrating the calibration parameters between multiple radar sensors and the global coordinate system, makes the error of the coordinate mapping of the same target object on the global coordinate system, such as a map, smaller when the same target object is perceived by multiple radar sensors. This makes it easier to distinguish the trajectory of other target objects mapped on the global coordinate system, such as a map, which is conducive to achieving global optimization and improving the accuracy of subsequent multi-radar sensor signal fusion.

[0068] Furthermore, this embodiment, by jointly calibrating the calibration parameters between multiple radar sensors and the global coordinate system, ensures that when the same target object is perceived by multiple radar sensors (e.g., relay sensing), the relay trajectory mapped onto the global coordinate system, such as a map, is continuous and smooth, resulting in better visualization.

[0069] Furthermore, this embodiment enables the joint calibration of calibration parameters between multiple radar sensors and the global coordinate system without manual intervention, thus improving efficiency.

[0070] The methods provided in the embodiments of this application have been described above. The apparatus provided in the embodiments of this application is described below:

[0071] See Figure 3 , Figure 3 This is a structural diagram of a device provided in an embodiment of this application. The device is applied in a multi-radar sensor scenario, wherein at least two radar sensors are deployed; each radar sensor is pre-calibrated to achieve time synchronization between them; the device includes:

[0072] The compensation module is used to compensate the target trajectory based on the motion speed output by the target radar sensor and associated with the target trajectory, so as to eliminate the differences in the sampling interval and / or sampling start point of each target radar sensor; the target trajectory is selected from the trajectories sensed by each radar sensor within a specified time period; each radar sensor samples the trajectory according to the set sampling interval.

[0073] The global identifier ID determination module is used to determine the global identifier ID for the target trajectory perceived by each target radar sensor based on the target object associated with the target trajectory perceived by each target radar sensor, so that the global ID of the target trajectory of the same target object perceived by different target radar sensors is the same.

[0074] The calibration module is used to establish a first loss function and / or a second loss function and / or a third loss function using the topology required by the specified space corresponding to the global coordinate system, the target trajectory perceived by each target radar sensor within at least one specified time period, and the global ID of the target trajectory. The calibration parameters of each radar sensor are used to map the position in the trajectory perceived by the radar sensor to the global coordinate system.

[0075] The first loss function is set based on the principle of minimizing the projection error of the same target object within the intersection area of ​​multiple radar sensors, and the first loss function involves the calibration parameters of each radar sensor; the second loss function is set based on the principle that the topology obtained after mapping the trajectory perceived by each radar sensor to the global coordinate system using the calibration parameters of each radar sensor is similar to the topology required by the specified space; the third loss function is set based on the principle that the topology obtained after mapping the trajectory corresponding to the same global ID perceived by each radar sensor in the same time period to the global coordinate system using the calibration parameters of each radar sensor is similar.

[0076] As an example, the target trajectory is selected through the following steps:

[0077] Obtain the trajectories sensed by each radar sensor within a specified time period and the trajectory confidence level;

[0078] The trajectory whose confidence level meets the confidence level requirement is selected as the target trajectory.

[0079] As one embodiment, determining a global identifier ID for the target trajectory perceived by each target radar sensor based on the target object associated with the target trajectory perceived by each target radar sensor includes:

[0080] For any position in the target trajectory perceived by each target radar sensor, where the position is perceived by the target radar sensor at a point in time within the specified time period, the target object and its feature identifier acquired by the vision camera associated with the target radar sensor at that point in time are obtained; the position is mapped from the radar coordinate system corresponding to the target radar sensor to the visual coordinate system corresponding to the vision camera to obtain the mapping point; it is checked whether the mapping point is on the target object acquired by the vision camera; if so, the position is determined to meet the matching requirements.

[0081] If at least N positions in the target trajectory sensed by the target radar sensor meet the matching requirements, then the target object feature identifier is assigned to the target trajectory as the global identifier ID of the target trajectory.

[0082] As an example, the first loss function is expressed by the following formula:

[0083]

[0084] Where N represents the total number of target objects, i represents the target object with global ID i, R represents the total number of target radar sensors, and r represents the r-th target radar sensor. Determined based on the coordinate information of each mapping point in the global coordinate system, where each mapping point refers to the position of the target object with global ID i sensed by each target radar sensor in the target trajectory mapped to the global coordinate system; This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the r-th target radar sensor; This can be expressed by the following formula: f represents the mapping function, θ r This represents the calibration parameters of the radar sensor for the r-th target;

[0085] Alternatively, the first loss function can be expressed as follows:

[0086]

[0087] in, This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the p-th target radar sensor. This can be expressed by the following formula: f represents the mapping function, θ p This represents the calibration parameters of the radar sensor for the p-th target; This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the q-th target radar sensor. This can be expressed by the following formula: f represents the mapping function, θ q This represents the calibration parameters of the radar sensor for the q-th target.

[0088] As an example, This can be expressed by the following formula:

[0089]

[0090] As an example, the second loss function is expressed by the following formula:

[0091] Where N represents the total number of target objects, R represents the total number of target radar sensors, r represents the r-th target radar sensor, and i represents the target object with global ID i. This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i detected by the r-th target radar sensor in the target trajectory. This means that after the target trajectory of the target object with global ID i sensed by the r-th target radar sensor is mapped to the global coordinate system, the topology transformation function τ is used to convert the mapped target trajectory into a topological representation in a specified space; T m This represents the standard topology corresponding to the topology representation in the specified space; This can be expressed by the following formula: f represents the mapping function, θ r This represents the calibration parameters of the radar sensor for the r-th target.

[0092] As an example, the third loss function is expressed by the following formula:

[0093]

[0094] Where N represents the total number of target objects, i represents the target object with global ID i, R represents the total number of target radar sensors, and r represents the r-th target radar sensor. This indicates that the target trajectory of the p-th target radar sensor, with global ID i, after being mapped to the global coordinate system, is transformed into the first topological structure in the specified space by the topological transformation function τ. f represents the mapping function, θp This represents the calibration parameters of the radar sensor for the p-th target; This indicates that the target trajectory of the target object with global ID i, sensed by the q-th target radar sensor, after being mapped to the global coordinate system, is transformed into a second topological structure in a specified space by the topological transformation function τ. This can be expressed by the following formula: f represents the mapping function, θ q This represents the calibration parameters of the radar sensor for the q-th target.

[0095] As one embodiment, obtaining the calibration parameters of each radar sensor through a first loss function and / or a second loss function and / or a third loss function includes:

[0096] Given at least two loss functions, the values ​​obtained from each loss function are weighted and fused to obtain the calibration parameters of each radar sensor.

[0097] This concludes the process. Figure 3 Structural description of the device shown.

[0098] Correspondingly, embodiments of this application also provide Figure 3 The hardware structure diagram of the device shown is as follows: Figure 4 As shown, the electronic device can be a device implementing the above-described method. Figure 4 As shown, the hardware architecture includes a processor and memory.

[0099] The memory is used to store machine-executable instructions;

[0100] The processor is configured to read and execute machine-executable instructions stored in the memory to implement the method embodiment shown above.

[0101] As one embodiment, the memory can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, the memory can be volatile memory, non-volatile memory, or similar storage media. Specifically, the memory can be RAM (Random Access Memory), flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0102] Based on the same inventive concept, this embodiment also provides a computer-readable storage medium. This computer-readable storage medium is used to store a computer program; when executed by a processor, the computer program implements the method embodiment described above.

[0103] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0104] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention filed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0105] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0106] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A joint calibration method, characterized in that, This method is applied to multi-radar sensor scenarios where at least two radar sensors are deployed. Each radar sensor is pre-calibrated to achieve time synchronization between them; The method includes: The target trajectory is compensated based on the motion speed output by the target radar sensor and associated with the target trajectory, so as to eliminate the differences in the sampling interval and / or sampling start point set by each target radar sensor; the target trajectory is selected from the trajectories sensed by each radar sensor within a specified time period; each radar sensor samples the trajectory according to the set sampling interval. Based on the target objects associated with the target trajectories perceived by each target radar sensor, a global identifier ID is determined for the target trajectories perceived by each target radar sensor, so that the global ID of the target trajectories of the same target object perceived by different target radar sensors is the same. Using the topology required by the specified space corresponding to the global coordinate system, the target trajectory perceived by each target radar sensor within at least one specified time period, and the global ID of the target trajectory, a first loss function and / or a second loss function and / or a third loss function are established to obtain the calibration parameters of each radar sensor through the first loss function and / or the second loss function and / or the third loss function; the calibration parameters of any radar sensor are used to map the position in the trajectory perceived by the radar sensor to the global coordinate system; The first loss function is set based on the principle of minimizing the projection error of the same target object within the intersection area of ​​multiple radar sensors, and the first loss function involves the calibration parameters of each radar sensor; the second loss function is set based on the principle that the topology obtained after mapping the trajectory perceived by each radar sensor to the global coordinate system using the calibration parameters of each radar sensor is similar to the topology required by the specified space; the third loss function is set based on the principle that the topology obtained after mapping the trajectory corresponding to the same global ID perceived by each radar sensor in the same time period to the global coordinate system using the calibration parameters of each radar sensor is similar.

2. The method according to claim 1, characterized in that, The target trajectory is selected through the following steps: Obtain the trajectories sensed by each radar sensor within a specified time period and the trajectory confidence level; Select the trajectory whose confidence level meets the confidence level requirement as the target trajectory.

3. The method according to claim 1, characterized in that, The process of determining a global identifier ID for the target trajectory perceived by each target radar sensor based on the target object associated with the target trajectory perceived by each target radar sensor includes: For any position in the target trajectory perceived by each target radar sensor, where the position is perceived by the target radar sensor at a point in time within the specified time period, the target object and its feature identifier acquired by the vision camera associated with the target radar sensor at that point in time are obtained; the position is mapped from the radar coordinate system corresponding to the target radar sensor to the visual coordinate system corresponding to the vision camera to obtain the mapping point; it is checked whether the mapping point is on the target object acquired by the vision camera; if so, the position is determined to meet the matching requirements. If at least N positions in the target trajectory sensed by the target radar sensor meet the matching requirements, then the target object feature identifier is assigned to the target trajectory as the global identifier ID of the target trajectory.

4. The method according to claim 1, characterized in that, The first loss function is expressed by the following formula: Where N represents the total number of target objects, i represents the target object with global ID i, R represents the total number of target radar sensors, and r represents the r-th target radar sensor. Determined based on the coordinate information of each mapping point in the global coordinate system, where each mapping point refers to the position of the target object with global ID i sensed by each target radar sensor in the target trajectory mapped to the global coordinate system; This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the r-th target radar sensor; This can be expressed by the following formula: f represents the mapping function, θ r This represents the calibration parameters of the radar sensor for the r-th target; Alternatively, the first loss function can be expressed as follows: in, This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the p-th target radar sensor. This can be expressed by the following formula: f represents the mapping function, θ p This represents the calibration parameters of the radar sensor for the p-th target; This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i in the target trajectory sensed by the q-th target radar sensor. This can be expressed by the following formula: f represents the mapping function, θ q This represents the calibration parameters of the radar sensor for the q-th target.

5. The method according to claim 4, characterized in that, This can be expressed by the following formula:

6. The method according to claim 1, characterized in that, The second loss function is expressed by the following formula: Where N represents the total number of target objects, R represents the total number of target radar sensors, r represents the r-th target radar sensor, and i represents the target object with global ID i. This represents the coordinate information of the mapping point in the global coordinate system to the position of the target object with global ID i detected by the r-th target radar sensor in the target trajectory. This means that after the target trajectory of the target object with global ID i sensed by the r-th target radar sensor is mapped to the global coordinate system, the topology transformation function τ is used to convert the mapped target trajectory into a topological representation in a specified space; T m This represents the standard topology corresponding to the topology representation in the specified space; This can be expressed by the following formula: f represents the mapping function, θ r This represents the calibration parameters of the radar sensor for the r-th target.

7. The method according to claim 1, characterized in that, The third loss function is expressed by the following formula: Where N represents the total number of target objects, i represents the target object with global ID i, R represents the total number of target radar sensors, and r represents the r-th target radar sensor. This indicates that the target trajectory of the target object with global ID i, sensed by the p-th target radar sensor, after being mapped to the global coordinate system, is transformed into the first topological structure in the specified space by the topological transformation function τ. f represents the mapping function, θ p This represents the calibration parameters of the radar sensor for the p-th target; This indicates that the target trajectory of the target object with global ID i, sensed by the q-th target radar sensor, after being mapped to the global coordinate system, is transformed into a second topological structure in a specified space using the topological transformation function τ. This can be expressed by the following formula: f represents the mapping function, θ q This represents the calibration parameters of the radar sensor for the q-th target.

8. The method according to claim 1, characterized in that, The process of obtaining the calibration parameters of each radar sensor through a first loss function and / or a second loss function and / or a third loss function includes: Given at least two loss functions, the values ​​obtained from each loss function are weighted and fused to obtain the calibration parameters of each radar sensor.

9. A joint calibration device, characterized in that, This device is used in multi-radar sensor scenarios where at least two radar sensors are deployed. Each radar sensor is pre-calibrated to achieve time synchronization between them; The device includes: The compensation module is used to compensate the target trajectory based on the motion speed output by the target radar sensor and associated with the target trajectory, so as to eliminate the differences in the sampling interval and / or sampling start point of each target radar sensor; the target trajectory is selected from the trajectories sensed by each radar sensor within a specified time period; each radar sensor samples the trajectory according to the set sampling interval. The global identifier ID determination module is used to determine the global identifier ID for the target trajectory perceived by each target radar sensor based on the target object associated with the target trajectory perceived by each target radar sensor, so that the global ID of the target trajectory of the same target object perceived by different target radar sensors is the same. The calibration module is used to establish a first loss function and / or a second loss function and / or a third loss function using the topology required by the specified space corresponding to the global coordinate system, the target trajectory perceived by each target radar sensor within at least one specified time period, and the global ID of the target trajectory. The calibration parameters of each radar sensor are used to map the position in the trajectory perceived by the radar sensor to the global coordinate system. The first loss function is set based on the principle of minimizing the projection error of the same target object within the intersection area of ​​multiple radar sensors, and the first loss function involves the calibration parameters of each radar sensor; the second loss function is set based on the principle that the topology obtained after mapping the trajectory perceived by each radar sensor to the global coordinate system using the calibration parameters of each radar sensor is similar to the topology required by the specified space; the third loss function is set based on the principle that the topology obtained after mapping the trajectory corresponding to the same global ID perceived by each radar sensor in the same time period to the global coordinate system using the calibration parameters of each radar sensor is similar.

10. An electronic device, characterized in that, Electronic devices include: processors and memory; The memory is used to store machine-executable instructions; The processor is configured to read and execute machine-executable instructions stored in the memory to implement the method as described in any one of claims 1 to 8.

Images