A monitoring method and device for sensor external parameter calibration

By differential analysis and correction of the positioning results of unmanned driving equipment, the positioning inaccurate problem caused by the calibration error of the external parameter of the sensor is solved, and the positioning accuracy and stability of the equipment are improved.

CN114460550BActive Publication Date: 2024-11-22BEIJING SANKUAI ONLINE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210009327.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-06
Publication Date
2024-11-22
Estimated Expiration
2042-01-06

AI Technical Summary

Technical Problem

In unmanned driving equipment, errors in sensor external parameter calibration may lead to inaccurate positioning, and it is difficult for the prior art to monitor and correct these errors in real time.

Method used

By acquiring the positioning results of the unmanned driving equipment, the first positioning result is converted to the coordinate system of the second positioning result using pre-calibrated sensor external parameter data, the difference between the two is calculated, and whether it is necessary to correct the outer layer is determined based on the difference. The parameter data specifically includes correcting the translation or rotation matrix according to the type of travel path.

Benefits of technology

Real-time error monitoring and correction of sensor external parameter data is realized, the positioning accuracy of unmanned driving equipment is improved, and accurate positioning is ensured under different driving path conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114460550B_ABST
    Figure CN114460550B_ABST
Patent Text Reader

Abstract

The present specification discloses a monitoring method and device for sensor external parameter calibration, which relates to the field of unmanned driving. The method and device can obtain the positioning result of the unmanned driving equipment, which includes a first positioning result obtained by positioning through a first sensor and a second positioning result obtained by positioning through a second sensor. Then, based on the external parameter data between the first sensor and the second sensor that are pre-calibrated, the first positioning result is converted to the positioning coordinate system corresponding to the second sensor to obtain the converted first positioning result, and the difference between the converted first positioning result and the second positioning result is determined. Based on the difference, it is determined whether the external parameter data needs to be corrected. If so, the external parameter data is corrected according to the type of driving path when the unmanned driving equipment is driving at a location corresponding to the above positioning result, thereby performing online error monitoring for external parameter calibration and making accurate corrections.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This specification relates to the field of unmanned driving, and in particular to a monitoring method and device for calibrating external parameters of a sensor. Background Art

[0002] Currently, in the field of unmanned driving, unmanned driving equipment can use a variety of sensors for fusion positioning to accurately position itself. However, fusion positioning through a variety of sensors requires calibration of the external parameters between the sensors, that is, the conversion relationship between the sensors, so that more accurate positioning can be obtained when using sensor positioning fusion.

[0003] In the prior art, the calibration of sensors is usually limited to offline manual pre-calibration of sensors. However, since sensors are hardware devices, the screws used to fix the sensors or the screws inside the sensors may become loose during the driving of the unmanned driving device. This may cause errors in the results of the pre-calibration of the sensors, thereby causing the unmanned driving device to inaccurately position itself.

[0004] Therefore, how to determine whether there is an error in the calibration results of the sensors in the unmanned driving equipment is an urgent problem to be solved. Summary of the invention

[0005] This specification provides a monitoring method and device for sensor external parameter calibration to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This manual provides a monitoring method for sensor external parameter calibration, including:

[0008] Acquire a positioning result of the unmanned driving device, wherein the positioning result includes a first positioning result obtained by positioning through a first sensor and a second positioning result obtained by positioning through a second sensor;

[0009] Based on the pre-calibrated external parameter data between the first sensor and the second sensor, the first positioning result is converted into a positioning coordinate system corresponding to the second sensor to obtain a converted first positioning result;

[0010] Determining a difference between the converted first positioning result and the second positioning result, and determining whether the external reference data needs to be corrected according to the difference;

[0011] If so, the external parameter data is corrected according to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result.

[0012] Optionally, determining whether the external reference data needs to be corrected according to the difference specifically includes:

[0013] If it is determined that the difference between the converted first positioning result and the second positioning result exceeds a first preset threshold, the external parameter data needs to be corrected. If it is determined that the difference between the converted first positioning result and the second positioning result does not exceed the first preset threshold, the external parameter data does not need to be corrected.

[0014] Optionally, obtaining the positioning result of the unmanned driving device includes:

[0015] Obtaining the positioning result of the unmanned driving device and the positioning results of other unmanned driving devices;

[0016] Based on the pre-calibrated external parameter data between the first sensor and the second sensor, before converting the first positioning result to a positioning coordinate system corresponding to the second sensor, and obtaining the converted first positioning result, the method further includes:

[0017] For each of the unmanned driving device and the other unmanned driving devices, determining external reference data between a first sensor and a second sensor of the unmanned driving device as the external reference data corresponding to the unmanned driving device;

[0018] Based on the pre-calibrated external parameter data between the first sensor and the second sensor, converting the first positioning result into a positioning coordinate system corresponding to the second sensor to obtain the converted first positioning result specifically includes:

[0019] Based on the external reference data corresponding to the unmanned driving device, convert the first positioning result of the unmanned driving device into the positioning coordinate system corresponding to the second sensor to obtain the converted first positioning result corresponding to the unmanned driving device;

[0020] Determining a difference between the converted first positioning result and the second positioning result, and determining whether the external reference data needs to be corrected according to the difference, specifically includes:

[0021] Determine a difference between a converted first positioning result corresponding to the unmanned driving device and a second positioning result corresponding to the unmanned driving device as a difference corresponding to the unmanned driving device;

[0022] According to the differences corresponding to the unmanned driving devices, the unmanned driving devices are sorted to obtain a sorting result;

[0023] It is determined that the external parameter data of the unmanned driving equipment that is ahead of the set ranking in the sorting results needs to be corrected.

[0024] Optionally, the external parameter data is corrected according to the type of driving path when the unmanned driving device drives at the location corresponding to the positioning result, specifically including:

[0025] If it is determined that the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result is a straight path type, the translation matrix in the external parameter data is corrected according to the difference between the converted first positioning result and the second positioning result.

[0026] Optionally, the external parameter data is corrected according to the type of driving path when the unmanned driving device drives at the location corresponding to the positioning result, specifically including:

[0027] If it is determined that the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result is a turning path type, the rotation matrix in the external parameter data is corrected according to the difference between the converted first positioning result and the second positioning result.

[0028] Optionally, the external parameter data is corrected according to the type of driving path when the unmanned driving device drives at the location corresponding to the positioning result, specifically including:

[0029] If it is determined that the difference is less than a second preset threshold, adjusting the external parameter data according to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result and the difference to obtain adjusted external parameter data;

[0030] The external parameter data is corrected online according to the adjusted external parameter data.

[0031] Optionally, the external parameter data is corrected according to the type of driving path when the unmanned driving device drives at the location corresponding to the positioning result, specifically including:

[0032] If it is determined that the difference is greater than the second preset threshold, a warning message is sent to the terminal device corresponding to the unmanned driving device to obtain calibration data for recalibrating the external parameters between the first sensor and the second sensor in the unmanned driving device, and the external parameter data is corrected through the recalibrated calibration data.

[0033] Optionally, the first sensor is a radar, and the second sensor is an inertial measurement unit IMU.

[0034] This specification provides a monitoring device for calibrating external parameters of a sensor, including:

[0035] An acquisition module is used to acquire a positioning result of the unmanned driving device, wherein the positioning result includes a first positioning result obtained by positioning through a first sensor and a second positioning result obtained by positioning through a second sensor;

[0036] a conversion module, configured to convert the first positioning result into a positioning coordinate system corresponding to the second sensor based on pre-calibrated external parameter data between the first sensor and the second sensor, so as to obtain a converted first positioning result;

[0037] A determination module, used to determine the difference between the converted first positioning result and the second positioning result, and determine whether the external reference data needs to be corrected according to the difference;

[0038] The correction module is used to correct the external parameter data according to the driving path type of the unmanned driving device when it drives at the location corresponding to the positioning result.

[0039] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned monitoring method for sensor external parameter calibration is implemented.

[0040] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned monitoring method for sensor external parameter calibration when executing the program.

[0041] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0042] It can be seen from the above method that the positioning result of the unmanned driving device can be obtained, and the positioning result includes a first positioning result obtained by positioning through the first sensor and a second positioning result obtained by positioning through the second sensor. Then, based on the external parameter data between the first sensor and the second sensor that are pre-calibrated, the first positioning result is converted to the positioning coordinate system corresponding to the second sensor to obtain the converted first positioning result, and the difference between the converted first positioning result and the second positioning result is determined. Based on the difference, it is determined whether the external parameter data needs to be corrected. If so, the external parameter data is corrected according to the type of driving path when the unmanned driving device is driving at the location corresponding to the above positioning result.

[0043] It can be seen from the above content that although the external parameter data is usually calibrated offline in advance, by determining the difference between the converted first positioning result and the second positioning result in this method, it is possible to determine whether there is a certain error in the external parameter data, and under the premise of a certain error, the external parameter data can be corrected. Moreover, when correcting the external parameter data, the type of driving path traveled by the unmanned equipment can be taken into account to a certain extent to perform accurate error correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0045] Figure 1 A flow chart of a monitoring method for calibrating external parameters of a sensor in this specification;

[0046] Figure 2 A schematic diagram of a process for error monitoring of sensor external parameters provided in this specification;

[0047] Figure 3 A schematic diagram of a monitoring device for calibrating external parameters of a sensor in this specification;

[0048] Figure 4 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0050] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0051] Figure 1 The following is a flow chart of a monitoring method for calibrating external parameters of a sensor in this specification, which specifically includes the following steps:

[0052] S101: Acquire a positioning result of an unmanned driving device, wherein the positioning result includes a first positioning result obtained by positioning through a first sensor and a second positioning result obtained by positioning through a second sensor.

[0053] S102: Based on pre-calibrated external parameter data between the first sensor and the second sensor, convert the first positioning result into a positioning coordinate system corresponding to the second sensor to obtain a converted first positioning result.

[0054] In practical applications, unmanned driving equipment can locate itself by means of multiple sensor fusion methods. This method requires pre-calibration of external parameters between sensors. Since unmanned driving equipment may experience bumps during driving, the external parameter data calibrated in advance between sensors may have large errors, resulting in inaccurate positioning results during sensor fusion positioning. Therefore, a monitoring method for sensor external parameter calibration in this specification is used to monitor errors in the pre-calibrated external parameter data between sensors.

[0055] Based on this, the server can obtain the positioning result of the unmanned driving device, which may include the first positioning result obtained by positioning through the first sensor and the second positioning result obtained by positioning through the second sensor. Then, based on the pre-calibrated external parameter data between the first sensor and the second sensor, the first positioning result can be converted to the positioning coordinate system corresponding to the second sensor to obtain the converted second positioning result.

[0056] That is, the first positioning result may refer to the position and posture obtained by positioning only by the first sensor, or by combining other sensors except the second sensor and the first sensor for positioning, that is, the first positioning result does not use the second sensor. The second positioning result may refer to the position and posture obtained by positioning only by the second sensor, or by combining other sensors except the first sensor and the second sensor for positioning, that is, the second positioning result does not use the first sensor.

[0057] By using the external parameter data between the first sensor and the second sensor obtained in advance through calibration, the first positioning result can be converted to the positioning coordinate system corresponding to the second sensor to obtain the converted first positioning result (of course, the second positioning result can also be converted to the positioning coordinate system corresponding to the first sensor, which is not limited here). The external parameter data mentioned here can be used to represent the conversion relationship between the positioning coordinate system corresponding to the first sensor and the positioning coordinate system corresponding to the second sensor. The external parameter data may include a rotation matrix and a translation matrix.

[0058] The first sensor mentioned here may refer to a radar, and the second sensor may refer to an inertial measurement unit (IMU). Fusion positioning may be performed through global matching of the radar and trajectory recursion of the IMU. Of course, the first sensor may also refer to a camera, and the second sensor may be an IMU. The first sensor and the second sensor may be fused and positioned, and the first sensor and the second sensor are not specifically limited here.

[0059] S103: Determine a difference between the converted first positioning result and the second positioning result, and determine whether the external reference data needs to be corrected based on the difference.

[0060] After converting the first positioning result to the positioning coordinate system of the second sensor to obtain the converted first positioning result, the difference between the converted first positioning result and the second positioning result can be determined, and based on the difference, it can be determined whether the above external parameter data needs to be corrected.

[0061] That is to say, here the external parameter data is first considered to be correct, and then the converted first positioning result is obtained through the external parameter data, and the difference between the converted first positioning result and the second positioning result is determined, so that the error of the external parameter data is reflected through the difference. If the difference is larger, the error in the external parameter data may be larger.

[0062] Therefore, it can be determined whether the difference between the converted first positioning result and the above-mentioned second positioning result exceeds the first preset threshold value. If it exceeds the first preset threshold value, it can be determined that the above-mentioned external parameter data needs to be corrected. If it does not exceed the first preset threshold value, it can be determined that the above-mentioned external parameter data does not need to be corrected. Among them, the first preset threshold value can be preset manually, and the first preset threshold value can include preset threshold values ​​for various parameters in the above-mentioned external parameter data (such as, respectively preset threshold values ​​for the rotation matrix and the translation matrix). If any parameter in the difference between the converted first positioning result and the above-mentioned second positioning result exceeds the threshold value corresponding to the parameter in the first preset threshold value (such as, a parameter in the translation matrix can correspond to a threshold value in the first preset threshold value), it can be determined that the above-mentioned external parameter data needs to be corrected.

[0063] Of course, in addition to judging by whether the difference exceeds the first preset threshold, judgment can also be made in other ways. Specifically, the positioning result of the unmanned driving device and the positioning results of other unmanned driving devices can be determined, and for each of the unmanned driving device and the other unmanned driving devices, the external parameter data between the first sensor and the second sensor of the unmanned driving device can be determined as the external parameter data corresponding to the unmanned driving device.

[0064] Based on the external parameter data corresponding to the unmanned driving device, the first positioning result of the unmanned driving device can be converted to the positioning coordinate system corresponding to the second sensor to obtain the converted first positioning result corresponding to the unmanned driving device. Then, the difference between the converted first positioning result corresponding to the unmanned driving device and the second positioning result corresponding to the unmanned driving device is determined as the difference corresponding to the unmanned driving device. According to the differences corresponding to each unmanned driving device, each unmanned driving device is sorted to obtain a sorting result. Then, it is determined that the external parameter data of the unmanned driving device that is located before the set ranking in the sorting result needs to be corrected.

[0065] That is to say, among many unmanned driving devices, the difference between the converted first positioning result and the second positioning result is determined. Those unmanned driving devices with larger differences may be unmanned driving devices in which there are errors in the calibration data between the first sensor and the second sensor. Therefore, it can be determined that these unmanned driving devices need to be corrected.

[0066] It should be noted that the large difference between the first positioning result after conversion and the second positioning result mentioned above is not necessarily due to the error in the calibration data between the first sensor and the second sensor, but may also be due to the presence of noise in the positioning result obtained by the sensor when performing positioning. Therefore, the first positioning result of the unmanned driving equipment at each moment can be converted into the positioning result in the positioning coordinate system corresponding to the second sensor, and the difference between the converted first positioning result and the second positioning result during this period can be determined. Of course, the acquisition frequencies of different sensors are inconsistent. Therefore, each moment mentioned here may refer to the moment when the positioning result of the first sensor and the positioning result of the second sensor can be obtained at the same time.

[0067] Since there is a large difference between the first positioning result and the second positioning result after conversion at the same time, there may be noise in the positioning observation of a certain sensor. However, the noise observed by the sensor may not exist for a long time, or the error in the positioning result caused by the noise observed by the sensor will not always exist and be fixed.

[0068] Therefore, for each moment, the difference between the first positioning result and the second positioning result after conversion at that moment can be determined as the difference corresponding to that moment. Afterwards, several consecutive moments in which the change in the difference within the set time length does not exceed the set change amount can be screened out, and the difference between the first positioning result and the second positioning result after conversion at the screened consecutive moments can be used to determine whether it is necessary to correct the external reference data (that is, ignore the positioning results at moments with large difference fluctuations in a very short period of time, so as to avoid the influence of noise points observed by the sensor on the results of external reference data correction).

[0069] That is to say, if there is a sudden change in the difference between the first positioning result and the second positioning result after conversion at each moment within the set time period, or the difference value fluctuates, it may be that there are noise points in the positioning result observation performed by the sensor itself. Even if there are large differences at certain moments within the set time period, it is not considered that there is a problem with the external parameter data between the sensors.

[0070] It should also be noted that in order to more accurately detect whether there is a problem with the external parameter data between sensors, a vibration sensor can also be provided in the unmanned driving device. The unmanned driving device can collect the vibration value at each moment through the vibration sensor. If the vibration value of the unmanned driving device exceeds the set vibration value within a specified time period (such as 5 seconds), and after the specified time period, the difference between the converted first positioning result and the second positioning result is large (such as exceeding the first set threshold, or ranking higher among the unmanned driving devices), it can be determined that the external parameter data needs to be corrected.

[0071] S104: If yes, the external parameter data is corrected according to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result.

[0072] After determining the difference between the converted first positioning result and the second positioning result, if it is determined that the external parameter data needs to be corrected, the external parameter data can be corrected according to the type of driving path when the unmanned driving device is driving at the location corresponding to the positioning result.

[0073] Among them, the driving path type can include a turning path type and a straight path type. In practical applications, when the unmanned driving device is driving straight, the difference between the converted first positioning result and the second positioning result will more reflect the error of the translation matrix in the external parameter data, and when the unmanned driving device is converting, the difference between the converted first positioning result and the second positioning result will more reflect the error of the rotation matrix in the external parameter data. Therefore, in order to more accurately and efficiently monitor and correct the errors of the external parameter data of the unmanned driving device, the external parameter data of the unmanned driving device can be corrected according to the driving path type of the unmanned driving device.

[0074] Among them, if it is determined that the driving path type of the unmanned driving device when driving at the location corresponding to the above positioning result is a straight path type, the translation matrix in the external reference data can be corrected according to the difference between the converted first positioning result and the second positioning result. If it is determined that the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result is a turning path type, the rotation matrix in the external reference data can be corrected according to the difference between the converted first positioning result and the second positioning result.

[0075] It should also be noted that after determining that the external parameter data between the first sensor and the second sensor of the unmanned driving device needs to be corrected, there may be multiple ways to correct the external parameter data. For example, if the server determines that the difference between the converted first positioning result and the second positioning result is less than the second preset threshold, the external parameter data may be adjusted according to the type of driving path when the unmanned driving device is driving at the location corresponding to the above positioning result and the difference, to obtain the adjusted external parameter data, and the external parameter data may be corrected online according to the adjusted external parameter data.

[0076] That is to say, if it is determined that the external parameter data needs to be corrected, and the difference between the converted first positioning result and the second positioning result is relatively small, then the external parameter data of the unmanned driving device can be adjusted directly based on the difference, so that the unmanned driving device can use the adjusted external parameter data to perform positioning fusion between the first sensor and the second sensor.

[0077] If it is determined that the difference is greater than the above-mentioned second preset threshold, a warning message is sent to the terminal device corresponding to the unmanned driving device to obtain calibration data for recalibrating the external parameters between the first sensor and the second sensor in the unmanned driving device, and the external parameter data is corrected through the recalibrated calibration data.

[0078] That is to say, if the difference is large, the external parameter data can be recalibrated offline. Therefore, when the difference is large, a warning message can be sent to the terminal device corresponding to the unmanned driving device (such as a developer related to the unmanned driving device, a terminal device held by a person who specializes in calibrating the external parameter data of the unmanned driving device, or a terminal device provided in the unmanned driving device, and the calibration personnel or developers can view the warning message through the terminal device) to prompt that there may be errors in the external parameter data calibrated between the first sensor and the second sensor of the unmanned driving device, so as to perform offline recalibration, obtain the recalibrated calibration data, and correct the external parameter data.

[0079] Among them, the warning message can carry the driving path type of the unmanned driving device when it is driving at the location corresponding to the above positioning result, so as to remind relevant personnel that the error of the external parameter data of the unmanned driving device may appear more in the rotation matrix or the translation matrix.

[0080] The following is a complete example to fully introduce a monitoring method for sensor external parameter calibration provided in this manual. Figure 2 shown.

[0081] Figure 2 A schematic diagram of a process for error monitoring of sensor external parameters provided in this manual.

[0082] from Figure 2 It can be seen that the first positioning result obtained by the first sensor can be converted to the positioning coordinate system of the second sensor, thereby obtaining the converted first positioning result, and the converted first positioning result is compared with the second positioning result to obtain the difference between the two. Since both the first positioning result and the second positioning result can be postures, the difference between the two can be represented by calculating the transformation matrix between the postures, that is, the difference between the two can be a difference matrix (including a rotation matrix and a translation matrix), which can be calculated by the following formula:

[0083] innovation=a×tab×b.inverse()

[0084] innovation is the determined difference, that is, the difference matrix, a is the first positioning result, tab is the transformation matrix from the positioning coordinate system of the first sensor to the positioning coordinate system of the second sensor determined by the external parameter data between the first sensor and the second sensor, and b.inverse() is the inverse matrix corresponding to the second positioning result.

[0085] By calculating the difference between the two, the server can determine the unmanned driving equipment whose external parameter data needs to be corrected, and make online corrections to such unmanned driving equipment, or issue an early warning so that offline personnel can recalibrate the unmanned driving equipment (of course, the executor can also be the unmanned driving equipment, that is, the unmanned driving equipment can determine by itself whether its own external parameter data needs to be corrected).

[0086] It should be noted that the first preset threshold and the second preset threshold mentioned above can be set in advance, and the first preset threshold and the second preset threshold can include thresholds preset for various parameters in the rotation matrix and the translation matrix respectively, that is, the setting method of the second preset threshold can be similar to the first preset threshold, wherein, since the second preset threshold is set for the case where the difference between the first positioning result and the second positioning result is large and manual recalibration is required, the set second preset threshold can be greater than the first preset threshold.

[0087] It can be seen from the above method that although the external parameter data is usually calibrated offline in advance, by determining the difference between the converted first positioning result and the second positioning result in this method, it is possible to determine whether there is a certain error in the external parameter data, and correct the external parameter data under the premise of a certain error. Moreover, when correcting the external parameter data, the type of driving path traveled by the unmanned equipment can be taken into account to a certain extent, so as to perform efficient and accurate error correction.

[0088] The unmanned driving equipment mentioned above may refer to unmanned vehicles, drones, automatic delivery equipment and other equipment that can realize automatic driving. Based on this, the monitoring method for sensor external parameter calibration provided in this specification can be used to monitor and correct the errors of external parameter data between sensors in unmanned driving equipment. The unmanned driving equipment can be specifically applied to the field of delivery through unmanned equipment, such as the business scenarios of using unmanned driving equipment for express delivery, logistics, take-out and other delivery.

[0089] The above is a monitoring method for sensor external parameter calibration provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding monitoring device for sensor external parameter calibration, such as Figure 3 shown.

[0090] Figure 3 A schematic diagram of a monitoring device for calibrating external parameters of a sensor provided in this specification specifically includes:

[0091] The acquisition module 301 is used to acquire a positioning result of the unmanned driving device, wherein the positioning result includes a first positioning result obtained by positioning through a first sensor and a second positioning result obtained by positioning through a second sensor;

[0092] A conversion module 302, configured to convert the first positioning result into a positioning coordinate system corresponding to the second sensor based on pre-calibrated external parameter data between the first sensor and the second sensor, to obtain a converted first positioning result;

[0093] A determination module 303 is used to determine the difference between the converted first positioning result and the second positioning result, and determine whether the external reference data needs to be corrected according to the difference;

[0094] The correction module 304 is used to correct the external parameter data according to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result.

[0095] Optionally, the determination module 303 is specifically used to, if it is determined that the difference between the converted first positioning result and the second positioning result exceeds a first preset threshold, it is necessary to correct the external parameter data; if it is determined that the difference between the converted first positioning result and the second positioning result does not exceed the first preset threshold, it is not necessary to correct the external parameter data.

[0096] Optionally, the acquisition module 301 is specifically used to obtain the positioning result of the unmanned driving device and the positioning results of other unmanned driving devices; based on the pre-calibrated external parameter data between the first sensor and the second sensor, the first positioning result is converted to the positioning coordinate system corresponding to the second sensor, and before obtaining the converted first positioning result, the conversion module 302 is also used to, for each of the unmanned driving device and the other unmanned driving devices, determine the external parameter data between the first sensor and the second sensor of the unmanned driving device as the external parameter data corresponding to the unmanned driving device; the conversion module 302 is specifically used to, based on the external parameter data corresponding to the unmanned driving device, convert the first positioning result of the unmanned driving device to the positioning coordinate system corresponding to the second sensor, and obtain the converted first positioning result corresponding to the unmanned driving device; the determination module 303 is specifically used to determine the difference between the converted first positioning result corresponding to the unmanned driving device and the second positioning result corresponding to the unmanned driving device, as the difference corresponding to the unmanned driving device; according to the differences corresponding to each unmanned driving device, the unmanned driving devices are sorted to obtain a sorting result; and it is determined that the external parameter data of the unmanned driving device located before the set ranking in the sorting result needs to be corrected.

[0097] Optionally, the correction module 304 is specifically used to correct the translation matrix in the external parameter data according to the difference between the converted first positioning result and the second positioning result if it is determined that the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result is a straight path type.

[0098] Optionally, the correction module 304 is specifically used to correct the rotation matrix in the external parameter data according to the difference between the converted first positioning result and the second positioning result if it is determined that the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result is a turning path type.

[0099] Optionally, the correction module 304 is specifically used to, if it is determined that the difference is less than a second preset threshold, adjust the external parameter data according to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result and the difference, to obtain adjusted external parameter data; and perform online correction on the external parameter data based on the adjusted external parameter data.

[0100] Optionally, the correction module 304 is specifically used to send a warning message to the terminal device corresponding to the unmanned driving device if it is determined that the difference is greater than the second preset threshold, so as to obtain calibration data for recalibrating the external parameters between the first sensor and the second sensor in the unmanned driving device, and correct the external parameter data through the recalibrated calibration data.

[0101] Optionally, the first sensor is a radar, and the second sensor is an inertial measurement unit IMU.

[0102] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A monitoring method for calibrating external parameters of sensors is provided.

[0103] This manual also provides Figure 4 The schematic structure diagram of the electronic device shown in FIG. Figure 4 As mentioned above, at the hardware level, the electronic device and the unmanned driving device include a processor, an internal bus, a network interface, a memory and a non-volatile memory, and of course may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1The monitoring method for sensor external parameter calibration. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0104] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0105] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0106] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0107] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0108] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0113] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0114] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0116] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0118] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0119] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A monitoring method for sensor external parameter calibration, characterized in that: include: Acquire a positioning result of the unmanned driving device, wherein the positioning result includes a first positioning result obtained by positioning through a first sensor and a second positioning result obtained by positioning through a second sensor; Based on the pre-calibrated external parameter data between the first sensor and the second sensor, the first positioning result is converted into a positioning coordinate system corresponding to the second sensor to obtain a converted first positioning result; Determining a difference between the converted first positioning result and the second positioning result, and determining whether the external reference data needs to be corrected according to the difference; If so, the external reference data is corrected according to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result; The obtaining of the positioning result of the unmanned driving device specifically includes: Obtaining the positioning result of the unmanned driving device and the positioning results of other unmanned driving devices; Based on the pre-calibrated external parameter data between the first sensor and the second sensor, before converting the first positioning result to a positioning coordinate system corresponding to the second sensor, and obtaining the converted first positioning result, the method further includes: For each of the unmanned driving device and the other unmanned driving devices, determining external reference data between a first sensor and a second sensor of the unmanned driving device as the external reference data corresponding to the unmanned driving device; Based on the pre-calibrated external parameter data between the first sensor and the second sensor, converting the first positioning result into a positioning coordinate system corresponding to the second sensor to obtain the converted first positioning result specifically includes: Based on the external reference data corresponding to the unmanned driving device, convert the first positioning result of the unmanned driving device into the positioning coordinate system corresponding to the second sensor to obtain the converted first positioning result corresponding to the unmanned driving device; Determining a difference between the converted first positioning result and the second positioning result, and determining whether the external reference data needs to be corrected according to the difference, specifically includes: Determine a difference between a converted first positioning result corresponding to the unmanned driving device and a second positioning result corresponding to the unmanned driving device as a difference corresponding to the unmanned driving device; According to the differences corresponding to the unmanned driving devices, the unmanned driving devices are sorted to obtain a sorting result; It is determined that the external parameter data of the unmanned driving equipment that is ahead of the set ranking in the sorting results needs to be corrected.

2. The method according to claim 1, characterized in that According to the difference, determining whether the external reference data needs to be corrected includes: If it is determined that the difference between the converted first positioning result and the second positioning result exceeds a first preset threshold, the external parameter data needs to be corrected. If it is determined that the difference between the converted first positioning result and the second positioning result does not exceed the first preset threshold, the external parameter data does not need to be corrected.

3. The method according to claim 1, characterized in that According to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result, the external parameter data is corrected, specifically including: If it is determined that the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result is a straight path type, the translation matrix in the external parameter data is corrected according to the difference between the converted first positioning result and the second positioning result.

4. The method according to claim 1, characterized in that According to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result, the external parameter data is corrected, specifically including: If it is determined that the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result is a turning path type, the rotation matrix in the external parameter data is corrected according to the difference between the converted first positioning result and the second positioning result.

5. The method according to claim 1, characterized in that According to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result, the external parameter data is corrected, specifically including: If it is determined that the difference is less than a second preset threshold, adjusting the external parameter data according to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result and the difference to obtain adjusted external parameter data; The external parameter data is corrected online according to the adjusted external parameter data.

6. The method according to claim 5, characterized in that According to the driving path type of the unmanned driving device when driving at the location corresponding to the positioning result, the external parameter data is corrected, specifically including: If it is determined that the difference is greater than the second preset threshold, a warning message is sent to the terminal device corresponding to the unmanned driving device to obtain calibration data for recalibrating the external parameters between the first sensor and the second sensor in the unmanned driving device, and the external parameter data is corrected through the recalibrated calibration data, wherein the warning message carries the driving path type of the unmanned driving device when it is driving at the location corresponding to the positioning result.

7. The method according to any one of claims 1 to 6, characterized in that: The first sensor is a radar, and the second sensor is an inertial measurement unit (IMU).

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.