Vehicle-mounted sensor calibration method and system, electronic equipment and storage medium

Abstract

The invention provides a vehicle-mounted sensor calibration method and system, electronic equipment and a storage medium, and relates to the field of intelligent driving, the method comprises the following steps: obtaining detection data of an inertial measurement unit, the detection data comprising vibration acceleration and three-axis acceleration data; when the vibration acceleration is greater than the preset acceleration, carrying out vibration spectrum analysis based on the three-axis acceleration data; calculating a compensation matrix of the vehicle-mounted sensor based on the result of the vibration spectrum analysis; and updating the external parameter matrix of the vehicle-mounted sensor according to the compensation matrix. According to the method provided by the invention, the calibration precision of the vehicle-mounted sensor can be improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a vehicle sensor calibration method, system, electronic device and storage medium. Background Technology

[0002] Vehicle sensors mainly include lidar, millimeter-wave radar, and cameras. In related technologies, the calibration of vehicle sensors mostly adopts mechanical fixture time-sharing calibration, which is time-consuming and depends on a fixed platform. It cannot compensate for vehicle vibration errors, resulting in low calibration accuracy of vehicle sensors. Summary of the Invention

[0003] This application provides a method, system, electronic device, and storage medium for calibrating vehicle-mounted sensors, which helps to improve the calibration accuracy of vehicle-mounted sensors.

[0004] Firstly, this application provides a method for calibrating an on-board sensor, including: Acquire detection data from the inertial measurement unit, including vibration acceleration and triaxial acceleration data; When the vibration acceleration is greater than the preset acceleration, vibration spectrum analysis is performed based on triaxial acceleration data; The compensation matrix of the vehicle-mounted sensor is calculated based on the results of vibration spectrum analysis. Update the extrinsic parameter matrix of the vehicle sensor based on the compensation matrix.

[0005] One possible implementation involves vibration spectrum analysis based on triaxial acceleration data, including: The power spectral density is obtained by performing noise filtering and Fourier transform on the triaxial acceleration data. Calculate vibration energy in the vibration frequency band based on power spectral density; Resonance peak detection is performed based on power spectral density to identify resonance frequencies.

[0006] In one possible implementation, the compensation matrix includes rotational and translational compensation amounts. The compensation matrix for the on-board sensor is calculated based on the results of vibration spectrum analysis, including: The rotational compensation amount is calculated based on the perturbation mapping relationship between the resonant frequency and the angle. Calculate translation compensation based on vibration energy in the vibration frequency band; The compensation matrix of the vehicle-mounted sensor is calculated based on the rotation compensation and translation compensation.

[0007] In one possible implementation, after updating the extrinsic parameter matrix of the vehicle sensor according to the compensation matrix, the method further includes: The weighted vibration energy of the vibration frequency band is calculated based on the power spectral density and the weighting function of the vibration frequency band. The time offset of the inertial measurement unit relative to the on-board sensor is updated based on the weighted vibration energy and vibration acceleration amplitude of the vibration frequency band.

[0008] One possible implementation of the method also includes: Acquire temperature information from onboard sensors, including the rate of temperature change and the amount of temperature change. When the rate of temperature change is greater than the preset rate of temperature change, temperature compensation is performed on the external parameter matrix of the vehicle sensor based on the amount of temperature change.

[0009] One possible implementation of the method also includes: Receive fault codes, including first fault code, second fault code and third fault code; When the fault code is the first fault code, the fault type is determined to be insufficient laser power, and the corresponding solution is to check the power supply or replace the laser module. When the fault code is the second fault code, the fault type is determined to be inertial measurement unit data out of tolerance. The corresponding solution is to recalibrate or replace the inertial measurement unit. When the fault code is the third fault code, the fault type is determined to be communication timeout, and the corresponding solution is to check the CAN termination resistor.

[0010] One possible implementation involves vehicle-mounted sensors including cameras, lidar, and millimeter-wave radar.

[0011] Secondly, this application provides an electronic device, including: a processor and a memory, wherein the memory is used to store a computer program; and the processor is used to run the computer program to implement the vehicle sensor calibration method as described in the first aspect.

[0012] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to implement the vehicle sensor calibration method of the first aspect.

[0013] Fourthly, this application provides an on-board sensor calibration system, including: an inertial measurement unit, an active calibration target integrating checkerboard coding and a laser array, and electronic equipment as shown in the second aspect.

[0014] The beneficial effects of this application are as follows: This application provides a vehicle-mounted sensor calibration method, system, electronic device, and storage medium. The method involves acquiring detection data from an inertial measurement unit, including vibration acceleration and triaxial acceleration data. When the vibration acceleration exceeds a preset acceleration, vibration spectrum analysis is performed based on the triaxial acceleration data. A compensation matrix for the vehicle-mounted sensor is calculated based on the results of the vibration spectrum analysis. The extrinsic parameter matrix of the vehicle-mounted sensor is updated according to the compensation matrix to eliminate calibration errors caused by vibration and improve the calibration accuracy of the vehicle-mounted sensor. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the structure of the vehicle sensor calibration system provided in the embodiments of this application; Figure 2 A timing diagram of the calibration process provided in the embodiments of this application; Figure 3 This is a timing diagram of the calibration in vehicle dynamic mode provided in an embodiment of this application; Figure 4 A schematic flowchart illustrating the vehicle sensor calibration method provided in this application embodiment; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0016] In this embodiment of the application, unless otherwise stated, the character " / " indicates that the preceding and following objects are in an OR relationship. For example, A / B can represent A or B. "AND / OR" describes the relationship between the associated objects, indicating that three relationships can exist. For example, A AND / OR B can represent: A existing alone, A and B existing simultaneously, and B existing alone.

[0017] It should be noted that the terms "first" and "second" used in the embodiments of this application are used only for distinguishing descriptive purposes and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor should they be construed as indicating or implying order.

[0018] In the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. Furthermore, "at least one of the following" or similar expressions refer to any combination of these items, which may include any combination of a single item or a plurality of items. For example, at least one of A, B, or C can represent: A, B, C, A and B, A and C, B and C, or A, B, and C. Each of A, B, and C can be an element itself or a set containing one or more elements.

[0019] In this application, terms such as "exemplary," "in some embodiments," and "in another embodiment" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the term "exemplary" is intended to present the concept in a concrete manner.

[0020] In the embodiments of this application, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction, their meanings are consistent. Similarly, in the embodiments of this application, "communication" and "transmission" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction, their meanings are consistent. For example, transmission can include sending and / or receiving, and can be a noun or a verb.

[0021] In the embodiments of this application, the term "equal to" can be used in conjunction with "greater than" to apply to technical solutions employing the condition of "greater than", and can also be used in conjunction with "less than" to apply to technical solutions employing the condition of "less than". It should be noted that when "equal to" is used with "greater than", it cannot be used with "less than"; and when "equal to" is used with "less than", it cannot be used with "greater than".

[0022] In related technologies, the calibration of vehicle-mounted sensors mostly adopts time-sharing calibration using mechanical fixtures, which is time-consuming and relies on a fixed platform. It cannot compensate for vehicle vibration errors, resulting in low calibration accuracy of vehicle-mounted sensors.

[0023] Based on the above problems, this application proposes a vehicle sensor calibration method, system, electronic device, and storage medium, which helps to improve the calibration accuracy of vehicle sensors.

[0024] Figure 1 This is a schematic diagram of the structure of the vehicle sensor calibration system provided in the embodiments of this application, as shown below. Figure 1 As shown, the vehicle-mounted sensor calibration system includes an active calibration target 1, an IMU (Inertial Measurement Unit) 2, and electronic equipment 3.

[0025] In this application, the electronic device may be an on-board computer or other electronic device used to implement the on-board sensor calibration method. The electronic device includes a calibration master controller and a multi-sensor interface. The calibration master controller is used to implement the on-board sensor calibration method, and the multi-sensor interface is used to connect to multiple on-board sensors wirelessly or via wired means to receive data collected by the multiple on-board sensors. The calibration master controller of the electronic device can be connected to an active calibration target and an IMU via a CAN FD bus.

[0026] The active calibration target can be a laser calibration target, integrating a checkerboard pattern and a laser array. The laser array can be a 905nm laser array, and its wavelength and power must meet the following conditions:

[0027] Where λ represents the laser wavelength; d represents the upper limit of the laser peak power (unit: milliwatt); d represents the measurement distance (unit: meter), specifically the straight-line distance from the human eye (or the target being measured) to the laser emission point.

[0028] This expression describes the constraints on the laser wavelength and peak laser power, and is divided into two parts.

[0029] 1. Wavelength constraint (λ) The laser wavelength (λ) is 905 nanometers (nm), and the allowable manufacturing or operating error range is ±10 nm (i.e., the actual wavelength may be between 895 nm and 915 nm).

[0030] Among them, 905 nm is a common near-infrared laser wavelength, which is suitable for LiDAR (good atmospheric penetration and safe for human eyes); ±10 nm is the wavelength tolerance of the laser, which ensures that the system works stably within the allowable range.

[0031] 2. Peak power constraint ( ) Meaning: The peak power of the laser must satisfy the inverse square attenuation relationship, that is, the peak power of the laser is inversely proportional to the square of the measurement distance. For example, when d=1 meter, the power limit is 1 milliwatt; when d=10 meters, the power limit drops to 0.01 milliwatts.

[0032] By constraining the laser wavelength and peak power as described above, damage to the human eye from high-power lasers at close range is avoided (complying with laser safety standards such as IEC 60825), ensuring eye safety. Simultaneously, during long-distance measurements, laser energy weakens due to atmospheric scattering and distance attenuation, thus allowing for a suitable increase in power to compensate for signal attenuation. In practical operation, the active calibration target needs to dynamically adjust the laser power according to the actual measurement distance.

[0033] Optionally, this application also includes a self-diagnostic system that automatically alarms when laser power attenuation exceeds 5%.

[0034] The IMU has a sampling rate of 2000 Hz and a zero-bias stability of 0.1° / h. Optionally, the IMU is a MEMS-IMU (Micro-Electro-Mechanical Systems Inertial Measurement Unit), which includes a 6-DOF motion compensation module. This application also provides a modular design that supports hot-swappable IMU module replacement (replacement time < 2 minutes).

[0035] The application scenarios of this application include onboard autonomous calibration of autonomous taxis (less than once a month), automatic calibration of armored vehicles while in motion, and calibration of port AGVs (Automated Guided Vehicles) in high humidity environments.

[0036] Figure 2 The calibration process timing diagram provided for the embodiments of this application is as follows: Figure 2 As shown, the device (vehicle) sends a ready signal to the main controller (i.e., the calibration main controller), and the main controller sends an instruction to activate the calibration mode to the laser target; the laser target sensor (camera / LiDAR / millimeter-wave radar) sends an coded pattern; the sensors synchronously collect data, and in order to give the collected data an accurate timestamp, the sensors request time synchronization from the IMU; the IMU returns spatiotemporal parameters to the main controller; the main controller receives the image, point cloud, and timestamp, performs visual corner point recognition and point cloud feature extraction, and performs extrinsic parameter calibration between multiple sensors, calculating the extrinsic parameter matrix between the sensors; the main controller writes the calibration results to the ECU, and the calibration results include the extrinsic parameter matrix between the sensors.

[0037] Extrinsic parameter calibration refers to calculating the relative poses (i.e., rotation matrix R and translation vector t) between multiple sensors. Taking LiDAR and camera as vehicle sensors as an example, the extrinsic parameter calibration between LiDAR and camera is as follows: The calculation formula is as follows:

[0038] Where R is the rotation matrix and t is the translation vector. The first in the lidar coordinate system i One point, The first in camera coordinate system i One point, n To match the number of point pairs, This represents the Euclidean norm.

[0039] This formula describes a point cloud registration problem, often referred to as rigid registration or rigid body transformation estimation. Its objective is to optimize the rotation matrix R and translation vector t to ensure that the corresponding sets observed by LiDAR and the camera are aligned. and The goal is to minimize the Euclidean distance between the points. Specifically, to find the optimal values ​​of R and t such that a point in the camera coordinate system, after rigid body transformation (rotation and translation), is as close as possible to its corresponding point in the lidar coordinate system. This formula automatically calibrates the spatial relative pose (rotation and translation) of the lidar and camera observation points by minimizing the alignment error, thus achieving spatiotemporal parameter calibration of the multi-sensor system.

[0040] Figure 3 The calibration timing diagram in the vehicle dynamic mode provided in the embodiments of this application is as follows: Figure 3 As shown, the IMU sends detection data to the main controller at a sampling rate of 2000Hz. The main controller sends a command to the laser calibration target to activate the compensation mode. The laser calibration target sensor projects a dynamic pattern. The sensor sends the collected data directly to the main controller, or the sensor sends the data to its corresponding ECU, which then aggregates the data and sends it to the main controller. The main controller calculates the compensation matrix based on the detection data collected by the IMU and writes the compensation matrix into the ECU to update the sensor's extrinsic parameter matrix. The main controller acquires a new vibration spectrum from the IMU every 100 ms and updates the compensation matrix accordingly. By using the IMU's detection data to dynamically compensate for the extrinsic parameter matrix, calibration errors caused by vibration are eliminated, improving the calibration accuracy of the vehicle-mounted sensors.

[0041] like Figure 4 As shown, Figure 4 This is a flowchart illustrating the vehicle sensor calibration method provided in an embodiment of this application, which specifically includes the following steps: Step S41: Obtain the detection data of the inertial measurement unit, including vibration acceleration and triaxial acceleration data.

[0042] Specifically, the main controller acquires the IMU's detection data via the CAN FD bus. The IMU's detection data includes, but is not limited to: (1) Triaxial acceleration (ax, ay, az): represents the linear acceleration of the measuring vehicle (i.e., the vehicle) in three directions (unit: m / s²). 2 Or g).

[0043] (2) Triaxial angular velocity (ωx, ωy, ωz): represents the rotational angular velocity of the measuring vehicle about the three axes (unit: rad / s or ° / s).

[0044] (3) Vibration acceleration: After filtering and separating the triaxial acceleration to obtain the triaxial vibration signal, the vibration acceleration is synthesized based on the triaxial vibration signal.

[0045] (4) Timestamp (t): Represents the precise time of data collection.

[0046] (5) Temperature data: Represents the internal temperature of the IMU.

[0047] By acquiring the detection data from the IMU, the vehicle's original motion information is provided for subsequent vibration analysis and dynamic compensation.

[0048] Step S42: When the vibration acceleration is greater than the preset acceleration, perform vibration spectrum analysis based on triaxial acceleration data.

[0049] When the vibration acceleration exceeds a preset acceleration, for example, when the vibration acceleration exceeds 0.5g (0.5g represents 0.5 times the acceleration due to gravity), this threshold is typically used to determine whether a vehicle is in a strong mechanical vibration environment. In other embodiments, a preset acceleration can also be set according to actual conditions, which is not limited in this application.

[0050] Vibration spectrum analysis is performed on the triaxial acceleration detected by the IMU. The purpose of vibration spectrum analysis is to dynamically correct the sensor extrinsic parameter matrix based on the results, thereby improving the accuracy of multi-sensor fusion in vibration environments.

[0051] In some embodiments, vibration spectrum analysis based on triaxial acceleration data includes: performing noise filtering and Fourier transform on the triaxial acceleration data to obtain the power spectral density; calculating the vibration energy of the vibration frequency band based on the power spectral density; and detecting resonance peaks based on the power spectral density to identify the resonance frequency.

[0052] Specifically, a low-pass filter (such as Butterworth) is used to remove high-frequency noise (e.g., >50Hz) from the triaxial acceleration (typically sampled at 100Hz~1kHz). A Fourier transform (e.g., Fast Fourier Transform) is then performed on the filtered acceleration signal to convert the time-domain acceleration signal to the frequency domain, obtaining the vibration spectrum, and subsequently the power spectral density (PSD). The vibration energy of the vibration frequency band is then calculated based on the power spectral density. In this application, the vibration frequency band refers to a frequency band sensitive to vibration, such as 5~50Hz. In other embodiments, the vibration frequency band can be set according to actual conditions; this application does not impose any limitations.

[0053] The specific formula for calculating the vibration energy in the vibration frequency band is as follows: =

[0054] in, Power spectral density, i.e., the distribution of vibrational energy at different frequencies f, is expressed in g. 2 / Hz, (m / s) 2 ) 2 / Hz; It represents the vibration energy within a vibration frequency band.

[0055] Resonance peak detection is performed based on power spectral density to identify peak frequencies in the spectrum (which may correspond to mechanical resonance points), and then the resonance frequencies are identified.

[0056] Step S43: Calculate the compensation matrix of the vehicle sensor based on the results of vibration spectrum analysis.

[0057] In this application, the vibration spectrum analysis results may include the main vibration frequency components, energy intensity, energy distribution, resonance frequency, vibration energy in the vibration frequency band, etc.

[0058] In some embodiments, the compensation matrix includes rotational compensation and translational compensation. The compensation matrix of the vehicle sensor is calculated based on the results of vibration spectrum analysis, including: calculating rotational compensation based on the disturbance mapping relationship between resonance frequency and angle; calculating translational compensation based on the vibration energy of the vibration frequency band; and calculating the compensation matrix of the vehicle sensor based on the rotational compensation and translational compensation.

[0059] Specifically, vibration at the resonant frequency may cause structural torsion, resulting in a phase delay at the resonant frequency; vibration also causes gyroscope output offset through the g-sensitivity coefficient, forming gyroscope vibration coupling error. A perturbation mapping relationship between the resonant frequency and angles (i.e., α, β, γ) is established through vibration table testing. The resonant frequency is obtained from the above vibration spectrum analysis results, and the corresponding angles α, β, γ are obtained according to the perturbation mapping relationship between the resonant frequency and angles (i.e., α, β, γ), where α, β, γ are the Euler angle changes caused by vibration (applicable to cases where the angle change is <5°).

[0060] Therefore, if vibration causes angular deviation, the rotational compensation amount is calculated as follows, using a small-angle approximation correction: R comp =R0 ΔR in, R represents the rotational compensation amount. comp R0 represents the rotation matrix after compensation, and R0 represents the rotation matrix before compensation.

[0061] The formula for calculating the translation compensation is as follows: , t comp = t0 + Δt1 Where k is the displacement coefficient vector (a 3×1 vector), calibrated experimentally, and Δt1 represents the translational compensation amount. The vibrational energy represented by t in the vibrational frequency band. comp Let t0 represent the translation matrix after compensation, and t0 represent the translation matrix before compensation.

[0062] Compensation matrix T of vehicle-mounted sensors comp Represented as: T comp = [R comp , t comp ; 0, 0, 0, 1] Step S44: Update the extrinsic parameter matrix of the vehicle sensor according to the compensation matrix.

[0063] Specifically, the compensation matrix T comp Write to the sensor calibration file and update the extrinsic parameter matrix of the vehicle sensor.

[0064] In other embodiments, the time offset of the IMU relative to the vehicle-mounted sensor can also be updated, or filter parameters (such as the process noise covariance of a Kalman filter) can be updated. The update can be performed using filter smoothing or dynamic loading. Filter smoothing refers to gradually fusing new parameters using a Kalman filter to avoid abrupt changes; dynamic loading refers to publishing new parameters in a real-time system (such as ROS, Robot Operating System) via topics or services.

[0065] In this application, by acquiring the detection data of the IMU, performing vibration spectrum analysis based on the detection data, calculating the compensation matrix, and updating the sensor extrinsic parameter matrix based on the compensation matrix, the calibration error caused by vibration can be eliminated, and the accuracy of multi-sensor fusion can be improved.

[0066] In some embodiments, after updating the extrinsic parameter matrix of the vehicle sensor according to the compensation matrix, the method provided in this application further includes: calculating the weighted vibration energy of the vibration frequency band based on the power spectral density and the weighting function of the vibration frequency band; and updating the time offset of the inertial measurement unit relative to the vehicle sensor based on the weighted vibration energy and vibration acceleration amplitude of the vibration frequency band.

[0067] Specifically, the formula for calculating the weighted vibration energy in the vibration frequency band is as follows:

[0068] in: The weighted vibration energy (dimensionless or unit-weighted vibration energy, used for subsequent compensation or decision-making) represents the vibration frequency band and can also be called the vibration influence coefficient or vibration acceleration compensation coefficient.

[0069] The weighting function (dimensionless) representing the vibration frequency band, also known as the frequency response function, is used to emphasize the impact of vibration in a specific frequency band on the system (e.g., the sensor's resonant frequency or sensitive frequency band). The integration interval is from 5 Hz to 50 Hz (typical mechanical vibration frequency band or vibration-sensitive frequency band), usually covering the resonant frequency range of the IMU or mechanical structure.

[0070] This formula describes a vibration-weighted integral parameter, which is typically used to assess the impact of vibration on the performance of sensors such as IMUs or lidar. It quantifies the overall impact of vibration energy on the system through frequency-domain weighted integration.

[0071] The weighting function H(f) for the vibration frequency band can be obtained based on the sensor's frequency response or mechanical resonance characteristics. For example, if the sensor resonates at 20 Hz, H(f) may have a large value near 20 Hz. Or, if vibrations in certain frequency bands have a greater impact on the system (such as causing IMU drift), H(f) will amplify the contribution of these frequency bands.

[0072] The formula for calculating the time offset of the IMU relative to the onboard sensor is as follows:

[0073] in, This indicates the time offset of the IMU relative to other sensors; This represents the timestamp of the IMU at the k-th moment; This represents the timestamp of other sensors (such as cameras / LiDAR) at time k; m represents the number of samples used to calculate the time offset. This represents the weighted vibration energy in the vibration frequency band, used to adjust the contribution of vibration to the time shift; The amplitude of vibration acceleration (used to measure the intensity of system vibration) is usually measured by the accelerometer of an IMU (such as filtered or energy in a specific frequency band). It can be an instantaneous value or a sliding window statistic (such as root mean square, standard deviation, peak value).

[0074] in, This represents the average deviation of the timestamps from the IMU and other sensors, used to estimate fixed delay. This indicates that the additional time shift caused by vibration is taken into account; the stronger the vibration (i.e., The larger the value, the greater the potential time synchronization error. Control the degree of influence of vibration on time shift.

[0075] This formula describes a time synchronization calibration (or time offset estimation) problem to compensate for the time delay (Δt) between the IMU and other sensors (such as cameras or lidar), while taking into account the effects of vibration.

[0076] In this application, when performing spatiotemporal joint calibration of multiple sensors, the weighted vibration energy of the vibration frequency band is calculated based on the power spectral density and the weighting function of the vibration frequency band. The time offset of the inertial measurement unit relative to the vehicle-mounted sensor is updated based on the weighted vibration energy and vibration acceleration amplitude of the vibration frequency band. Considering the impact of vibration on time calibration, the time delay is adjusted according to the calculated time offset. Combined with spatial compensation (i.e., rotation compensation and translation compensation), the corrected rotation matrix and translation vector, as well as the adjusted time delay, are used to update the sensor parameters, which helps to eliminate calibration errors caused by vibration and improve the accuracy of multi-sensor fusion.

[0077] In some embodiments, the method provided in this application further includes: acquiring temperature information of an on-board sensor, the temperature information including temperature change rate and temperature change amount; when the temperature change rate is greater than a preset temperature change rate, performing temperature compensation on the external parameter matrix of the on-board sensor based on the temperature change amount.

[0078] In this step, when the temperature change rate is greater than a preset temperature change rate (e.g., temperature change rate > 10℃ / hour), extrinsic parameter calibration based on temperature compensation is performed. Specifically, temperature compensation is applied to the extrinsic parameter matrix of the vehicle-mounted sensor based on the temperature change.

[0079] The formula for calculating temperature compensation is as follows:

[0080] in, This represents the extrinsic parameter matrix after temperature compensation, for example, the rigid body transformation matrix from sensor A to sensor B. It is usually a 4x4 homogeneous transformation matrix that includes rotation matrices and translation vectors. This represents the calibration matrix at the reference temperature (or calibration temperature), which is the initial calibration matrix (i.e., the external parameter matrix before temperature compensation). It represents the coefficient of thermal expansion or temperature coefficient of a material, used to assess the sensitivity of the extrinsic parameter matrix to temperature changes; It represents the change in temperature, that is, the difference between the current temperature and the calibrated temperature.

[0081] This formula describes a temperature-compensated extrinsic parameter calibration method to correct sensor extrinsic parameter drift caused by temperature changes. When the temperature changes... At times, the sensor's mechanical structure may undergo slight deformation, causing the extrinsic parameter matrix to become inaccurate. This can be addressed by introducing a temperature coefficient. By dynamically adjusting the transformation matrix, a more accurate extrinsic parameter matrix can be obtained. .

[0082] Understandably, this formula assumes that temperature changes affect... The effect is linear (i.e.) (as a linear correction term), in other cases, It can be a matrix (e.g.) ∈R4×4), used to indicate that different degrees of freedom (rotation, translation) have different sensitivities to temperature.

[0083] This application performs temperature compensation on the extrinsic parameter matrix of the vehicle sensor based on the temperature change, which helps to obtain a more accurate extrinsic parameter matrix and improve the calibration accuracy of the vehicle sensor.

[0084] In some embodiments, the method provided in this application further includes: receiving a fault code and determining the fault type and corresponding solution based on the fault code.

[0085] Optionally, the fault codes include a first fault code, a second fault code, and a third fault code. When the fault code is the first fault code, the fault type is determined to be insufficient laser power, and the corresponding solution is to check the power supply or replace the laser module. When the fault code is the second fault code, the fault type is determined to be inertial measurement unit data out of tolerance, and the corresponding solution is to recalibrate or replace the inertial measurement unit. When the fault code is the third fault code, the fault type is determined to be communication timeout, and the corresponding solution is to check the CAN terminal resistor. Specifically, during the calibration process, anomaly detection is performed on each device or apparatus in the vehicle sensor calibration system. When different faults occur, the main controller receives different fault codes. Each fault code corresponds to a fault type and a solution, and the main controller can determine the fault type and the corresponding solution based on the fault code. For example, the first fault code, E101, indicates insufficient laser power, and the corresponding solution is to check the power supply or replace the laser module; the second fault code, E205, indicates IMU data out of tolerance (data out of tolerance refers to excessive error, high dispersion, or unreasonable data itself), and the corresponding solution is to recalibrate or replace the IMU; the third fault code, E303, indicates communication timeout, and the corresponding solution is to check the CAN termination resistor. It is understood that fault codes include, but are not limited to, the above three types, and the fault types and solutions corresponding to the fault codes can be set according to actual engineering needs; this application does not impose any limitations on this.

[0086] By detecting anomalies and determining the fault type and corresponding solution based on the received fault codes, it is possible to handle faults in a timely manner, improve calibration accuracy, and ensure the safety and stability of the calibration process.

[0087] In this application, the calibration modes of the vehicle-mounted sensor include a factory pre-calibration mode, a vehicle-mounted dynamic mode, and an emergency recovery mode. The factory pre-calibration mode is performed on the vehicle assembly line, with a calibration accuracy of ±0.1° / ±1ms. The vehicle-mounted dynamic mode is triggered when the vibration acceleration exceeds a preset acceleration, with a calibration accuracy of ±0.3° / ±5ms. The emergency recovery mode is triggered after the sensor is updated, with a calibration accuracy of ±0.5° / ±10ms.

[0088] In the factory pre-calibration mode, rapid calibration is performed by first installing the calibration device on the vehicle calibration fixture (positioning accuracy ±0.05mm), followed by mechanical installation verification, such as verifying the theoretical relative positions of the lidar and millimeter-wave radar, and the theoretical relative positions of the IMU and camera. Then, calibration is initiated, activating the laser target mode, and automatically completing multi-sensor synchronous calibration, such as radar and vision extrinsic parameter calibration (time <3 minutes) and lidar and inertial measurement unit (IMU) time synchronization (error <1ms).

[0089] In vehicle dynamic mode, vibration acceleration is obtained based on IMU detection data. When the vibration acceleration exceeds 0.5g, dynamic compensation is performed on the extrinsic parameter calibration between multiple sensors. Specifically, the compensation matrix is ​​updated based on the acquired IMU detection data, which can be done every 100ms. Then, the extrinsic parameter matrix between the multiple sensors is updated based on the compensation matrix. The compensation matrix is ​​a set of parameters used to correct sensor data deviations caused by vibration, including rotational and translational compensation. The purpose of dynamic compensation is to maintain the consistency of multi-sensor data in a vibration environment, thereby reducing errors caused by vibration.

[0090] Optionally, when performing dynamic calibration in vehicle dynamic mode, the vehicle speed must be less than 5 km / h. This helps to achieve high-precision calibration and also ensures the safety and operational feasibility of the calibration process.

[0091] When the system detects that a sensor (such as a lidar) has been replaced, it activates the emergency recovery mode to perform single-point lidar calibration and IMU-assisted alignment. Through simple single-point calibration, the basic extrinsic parameter matrix is ​​quickly calculated and written to the ECU. In emergency recovery mode, the vehicle's speed is limited, such as to <40km / h, to reduce calibration errors. After sensor hardware replacement, the emergency recovery mode provides the vehicle with basic extrinsic parameters in the fastest and simplest way, supporting safe low-speed driving and creating conditions for subsequent high-precision calibration.

[0092] In the specific calibration operation, the calibration device is started after calibration. First, the calibration mode of the vehicle sensor is selected, choosing one of the following: factory pre-calibration mode, vehicle dynamic mode, or emergency recovery mode. In factory pre-calibration mode, laser target activation and multi-sensor synchronous calibration are performed. In vehicle dynamic mode, when the vibration acceleration is >0.5g, dynamic compensation is performed, the compensation matrix is ​​calculated, and the sensor parameters are updated. In emergency recovery mode, simplified calibration and key parameter reset are performed. The calibration accuracy of each of the three modes is verified. If the accuracy is met, the data is written to the Electronic Control Unit (ECU); if the accuracy is not met, an alarm is triggered.

[0093] In summary, this application has the following technical effects: (1) Improve calibration efficiency The laser target synchronously projects radar corner reflection points (905nm) and a visual checkerboard pattern (visible light). Specifically, a regular array of corner reflection points is projected by a 905nm infrared laser, while a standard checkerboard pattern is projected by a visible light laser (or LED), achieving parallel calibration. Furthermore, the IMU monitors vibration in real time at 2000Hz, dynamically adjusting the calibration sequence to achieve automatic feedback and improve calibration efficiency.

[0094] (2) Improve calibration accuracy This application utilizes detection data sampled at a high frequency of 2000Hz using an IMU to perform vibration compensation on the sensor's extrinsic parameter matrix, eliminating the influence of mechanical vibration. Furthermore, it performs temperature compensation on the extrinsic parameter matrix of the vehicle-mounted sensor based on temperature changes, improving the calibration accuracy of the vehicle-mounted sensor.

[0095] (3) Dynamic environmental adaptability This application can update the compensation matrix every 100ms to perform online compensation of the extrinsic parameter matrix, and perform temperature compensation on the extrinsic parameter matrix of the vehicle sensor based on the temperature change, so as to realize the adaptability of the calibration process to the dynamic environment.

[0096] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods provided in the embodiments shown in this application.

[0097] The following is combined Figure 5 The exemplary electronic devices provided in the embodiments of this application are further described. Figure 5 A schematic diagram of the structure of electronic device 5000 is shown.

[0098] The aforementioned electronic device 5000 may include: at least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor can execute the vehicle sensor calibration method provided in the embodiments shown in this application by calling the program instructions.

[0099] Figure 5 A block diagram is shown of an exemplary electronic device 5000 suitable for implementing embodiments of this application. Figure 5 The electronic device 5000 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0100] like Figure 5 As shown, the electronic device 5000 is presented in the form of a general-purpose computing device. The components of the electronic device 5000 may include, but are not limited to: one or more processors 5010, memory 5020, communication bus 5040 connecting different system components (including memory 5020 and processor 5010), and communication interface 5030.

[0101] The 5040 communication bus represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0102] Electronic devices 5000 typically include a variety of computer system-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0103] Memory 5020 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Although Figure 5As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the communication bus 5040 via one or more data media interfaces. The memory 5020 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0104] A program / utility having a set (at least one) of program modules may be stored in memory 5020. Such program modules include, but are not limited to, an operating system, one or more applications, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules typically perform the functions and / or methods described in the embodiments of this application.

[0105] Electronic device 5000 can also communicate with one or more external devices (e.g., keyboard, pointing device, display, etc.), and with one or more devices that enable a user to interact with the electronic device, and / or with any device that enables the electronic device to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through communication interface 5030. Furthermore, electronic device 5000 can also communicate through a network adapter ( Figure 5 (Not shown) communicates with one or more networks (e.g., Local Area Network (LAN), Wide Area Network (WAN), and / or public networks, such as the Internet). The aforementioned network adapter can communicate with other modules of the electronic device via the communication bus 5040. It should be understood that, although... Figure 5 As not shown, other hardware and / or software modules can be used in conjunction with the electronic device 5000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Drives (RAID) systems, tape drives, and data backup storage systems.

[0106] The processor 5010 executes various functional applications and data processing by running programs stored in the memory 5020, such as implementing the methods provided in the embodiments of this application.

[0107] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 5000. In other embodiments of this application, the electronic device 5000 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0108] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for calibrating an on-board sensor, characterized in that, The method includes: Acquire detection data from the inertial measurement unit, including vibration acceleration and triaxial acceleration data; When the vibration acceleration is greater than the preset acceleration, vibration spectrum analysis is performed based on the triaxial acceleration data; The compensation matrix of the vehicle-mounted sensor is calculated based on the results of the vibration spectrum analysis. The extrinsic parameter matrix of the vehicle sensor is updated based on the compensation matrix.

2. The method according to claim 1, characterized in that, Vibration spectrum analysis based on the triaxial acceleration data includes: The power spectral density is obtained by performing noise filtering and Fourier transform on the triaxial acceleration data. The vibration energy in the vibration frequency band is calculated based on the power spectral density. Resonance peaks are detected based on the power spectral density to identify the resonance frequency.

3. The method according to claim 2, characterized in that, The compensation matrix includes rotational compensation and translational compensation. The calculation of the compensation matrix for the vehicle-mounted sensor based on the vibration spectrum analysis results includes: The rotational compensation amount is calculated based on the perturbation mapping relationship between the resonant frequency and the angle; The translation compensation amount is calculated based on the vibration energy in the vibration frequency band. The compensation matrix of the vehicle-mounted sensor is calculated based on the rotation compensation amount and the translation compensation amount.

4. The method according to claim 2, characterized in that, After updating the extrinsic parameter matrix of the vehicle sensor according to the compensation matrix, the method further includes: The weighted vibration energy of the vibration frequency band is calculated based on the power spectral density and the weighting function of the vibration frequency band. The time offset of the inertial measurement unit relative to the vehicle-mounted sensor is updated based on the weighted vibration energy and vibration acceleration amplitude of the vibration frequency band.

5. The method according to claim 1, characterized in that, The method further includes: Acquire the temperature information from the vehicle-mounted sensor, the temperature information including the temperature change rate and the temperature change amount; When the temperature change rate is greater than the preset temperature change rate, temperature compensation is performed on the external parameter matrix of the vehicle sensor based on the temperature change amount.

6. The method according to claim 1, characterized in that, The method further includes: Receive fault codes, including a first fault code, a second fault code, and a third fault code; When the fault code is the first fault code, the fault type is determined to be insufficient laser power, and the corresponding solution is to check the power supply or replace the laser module. When the fault code is the second fault code, the fault type is determined to be inertial measurement unit data out of tolerance, and the corresponding solution is to recalibrate or replace the inertial measurement unit. When the fault code is the third fault code, the fault type is determined to be communication timeout, and the corresponding solution is to check the CAN termination resistor.

7. The method according to any one of claims 1-6, characterized in that, The vehicle-mounted sensors include cameras, lidar, and millimeter-wave radar.

8. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program; the processor being used to run the computer program to implement the vehicle sensor calibration method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the vehicle sensor calibration method as described in any one of claims 1-6.

10. A vehicle-mounted sensor calibration system, characterized in that, The system includes: An inertial measurement unit, an active calibration target integrating checkerboard coding and a laser array, and the electronic device as described in claim 8.

Images