Online calibration of sensors
By updating radar sensor calibration online through computing units and machine learning models, the problem of difficult sensor calibration on vehicles is solved, and the system reliability and safety are improved.
Patent Information
- Application Number
- CN202410645954.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-25
- Filing Date
- 2024-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
Existing radar sensors are difficult to calibrate on vehicles, especially during use, and require a dedicated laboratory or workshop environment.
The computing unit determines the ground condition value from the sensor data, uses a machine learning model to update the calibration, and combines vehicle operating parameters and target reflection data to achieve online calibration updates.
It improves the reliability of sensor calibration and the safety of the vehicle, while reducing computational complexity and dependence on the laboratory environment.
Smart Images

Figure CN119024310A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a computer-implemented method, apparatus, in-vehicle device and computer program for updating the calibration of one or more sensors. Background Art
[0002] Today, vehicles (cars, ships, robots, etc.) often operate using vehicle assistance systems (e.g., such as Advanced Driver Assistance Systems (ADAS)) to provide a certain degree of driving assistance to the driver. In addition, the vehicle assistance system is also able to control the vehicle semi-autonomously or even fully autonomously. This requires the vehicle assistance system to have sufficient knowledge about its surroundings, in particular about obstacles such as other vehicles, pedestrians, etc., to avoid collisions and allow the vehicle to pass safely through its surroundings. Lidar or radar sensors are usually used to obtain the required data to determine objects, measure their speed, or classify the surroundings in the vehicle environment.
[0003] While LiDAR sensors have been the go-to sensor for automotive applications for many years, the use of radar sensors has seen a growth in popularity as radar or radar sensors are relatively inexpensive, have long range and are less weather dependent than other sensors. However, given the recent increase in popularity, optimizing radar sensors and their applications has seen an increase in interest.
[0004] One aspect of radar sensors that requires further development is the calibration of the radar sensors. Typically, radar sensors are calibrated in the factory before being sold. If the radar sensor is mounted on a vehicle, for example, one manufacturing step of the vehicle is to calibrate the radar sensor in the production line. However, it is difficult to recalibrate the sensor when the vehicle is sold and used by the customer. Furthermore, if parts of the vehicle are replaced or damaged, recalibration of the sensor may be necessary.
[0005] Against this background, it is an object of the present invention to provide a method for updating the calibration of one or more sensors, preferably online, without the need for a dedicated laboratory or workshop. Summary of the invention
[0006] Various aspects of the disclosure are set out in the accompanying independent and dependent claims. Combinations of features from the dependent claims may be combined with features of the independent claims as appropriate and not just as explicitly set out in the claims.
[0007] In one aspect, the present invention relates to a computer-implemented method for updating the calibration of one or more sensors in a vehicle. Sensor data may be obtained from the one or more sensors. In addition, the method may include the step of determining a ground truth value based on at least a portion of the sensor data and based on one or more vehicle operating parameters. The method may also include the step of determining calibration update data based on the ground truth value. The method may also include the step of updating the calibration using the calibration update data.
[0008] Using the above method, it can be determined that the current calibration of one or more sensors needs to be adjusted (by using ground truth values obtained using at least a portion of the sensor data and vehicle operating parameters), and the current calibration of the one or more sensors is updated accordingly. This improves the reliability of the system. In particular, by including such a determination unit in the vehicle, extensive recalibration in a laboratory environment / workshop environment, which is cumbersome for the owner of the vehicle, can be avoided.
[0009] In a further aspect, the method further includes determining one or more targets in the surroundings of the vehicle based on the sensor data, wherein the one or more targets are determined by considering sensor data having a high signal-to-noise ratio (SNR).
[0010] In a further aspect, the method further comprises: determining one or more target parameters of one or more targets in the surroundings of the vehicle based on the sensor data. In addition, the method may comprise the step of determining the ground truth value based on one or more target parameters of the targets in the surroundings of the vehicle.
[0011] By determining one or more targets in the surroundings of the vehicle, and in particular by determining ground truth values based on the determined targets, the computational complexity of updating the calibration of one or more sensors may be reduced, since only data associated with the targets are considered.
[0012] In a further aspect, the vehicle operating parameters include one or more of a location of the one or more sensors in the vehicle, a speed of the vehicle, and a yaw rate of the vehicle.
[0013] In another aspect, the calibration update data is determined based on sensor data reflected from the one or more targets in a surrounding of the vehicle.
[0014] In a further aspect, the reflected sensor data corresponds to a steering vector associated with the one or more targets and the one or more sensors.
[0015] On the other hand, the target parameters include one or more of the distance of the target relative to the vehicle, the rate of change of the distance of the target, based on the vehicle operating parameters and the sensor data, preferably based on a corresponding vector of complex values, and more preferably based on the target parameters, the complex values coming from information associated with different virtual antenna channels included in the sensor data.
[0016] In a further aspect, the target parameters include one or more of a position of the target relative to the vehicle, a speed of the target, and a yaw rate of the target based on vehicle information.
[0017] In another aspect, the step of updating the calibration further comprises updating the calibration by applying an algorithm to only a portion of the calibrated data using the calibration update data.
[0018] In a further aspect, the algorithm is applied by a trained machine learning model.
[0019] By using a trained machine learning model to update the calibration, the reliability of the resulting updated calibration is improved and the computational complexity is reduced, as this avoids the use of typical computer-intensive matrix calibration calculations.
[0020] On the other hand, the machine learning model is trained by a computing unit included in the vehicle.
[0021] Using the above, calibration updates may be customized for a particular vehicle including one or more sensors.
[0022] In a further aspect, the machine learning model is trained by a remote computing unit.
[0023] Utilizing the above, data obtained from additional data sources may be included to enhance the resulting trained machine learning model to improve reliability.
[0024] In yet another aspect, the ground truth values and / or the calibration update data are received from a remote server.
[0025] Through the above, since calculations for obtaining ground truth values and / or calibration update data can be performed remotely, the required computing power included in the vehicle can be reduced.
[0026] In another aspect, the method further includes determining whether the updated calibration is associated with a reference value; and if the updated calibration is not associated with the reference value, repeatedly updating the calibration using the sensor data.
[0027] By comparing the updated calibration to a predetermined reference value, the reliability of the updated calibration of the one or more sensors may be determined.
[0028] On the other hand, the method also includes: determining operating instructions for the vehicle based on an updated calibration affecting the function of the vehicle assistance system, wherein preferably the function includes: triggering a warning if it is determined that the calibration is not associated with the baseline value; and updating the warning after updating the calibration.
[0029] By the above, the safety of the vehicle may be improved, as the driver of the vehicle may be notified when one or more radar sensors are operating unreliably.
[0030] Another aspect of the invention relates to a device comprising means configured to perform the method described above.
[0031] Another aspect of the invention relates to a computer program comprising instructions which, when executed by a computing system, cause the computing system to perform the method described above.
[0032] Yet another aspect of the present invention relates to a carrier comprising a device configured to perform the above method. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Various aspects of the invention are described in more detail below with reference to the accompanying drawings, without the invention being limited to the embodiments of these drawings.
[0034] Figure 1a A typical setup for updating the calibration is illustrated.
[0035] Figure 1b A method for updating calibration according to the present invention is illustrated.
[0036] Figure 2 A flow chart for updating sensor calibration according to an embodiment is shown.
[0037] Figure 3 A vehicle is shown that performs a method for updating sensor calibration.
[0038] Figure 4 A system for remotely updating sensor calibration according to aspects of the invention is shown.
[0039] Figure 5 A system comprising units configured to perform a method for updating the calibration of a sensor is shown.
[0040] Figure 6 A carrier according to aspects of the invention is shown. DETAILED DESCRIPTION
[0041] Hereinafter, various aspects of the present invention are described in more detail with reference to the accompanying drawings.
[0042] exist Figure 1a A typical setup for updating the calibration of one or more sensors is shown in .
[0043] One or more sensors, such as one or more camera sensors 372, one or more lidar sensors 376, and / or one or more radar sensors 374, are mounted on the vehicle 100. For reliability reasons, the one or more sensors 370 are typically calibrated and / or aligned when used in the vehicle. Calibration of sensors typically refers to calibration of individual sensors (regarding the phase / amplitude of the signal), where calibration typically involves calibration of one or more sensors in the vehicle. In the following, calibration is used to refer to any calibration / alignment process used to ensure that a sensor works in a reliable manner.
[0044] Typically, one or more sensors 370 are mounted to the vehicle 100 during the manufacturing process of the vehicle 100. For example, one manufacturing step of the vehicle 100 may be to install / insert one or more sensors 370 on the vehicle. When used in the vehicle 100, the one or more sensors 370 may already have an initial calibration. Preferably, the initial calibration is a calibration matrix including information for transforming the complex signal vector to compensate for the amplitude and / or phase offset of the one or more sensors 370. In some examples, the one or more sensors 370 are mounted on the vehicle 100 before receiving the initial calibration. The initial calibration of the one or more sensors 370 is primarily performed in a laboratory environment over the entire FOV (field of view). For example, after the vehicle 100 is manufactured, the calibration may be performed in a separate laboratory for performing the calibration of the one or more sensors 370, such as Figure 1aAs illustrated. In another example, calibration of one or more sensors 370 is performed during the manufacture of the carrier 100 on the production line. Preferably, one or more sensors 370 receive initial calibration after the individual components on which the one or more sensors 370 are installed are fully manufactured, because changing the individual components may cause undesirable changes in the calibration of one or more sensors 370. In another example, one or more sensors 370 are installed after the carrier 100 has been manufactured. For example, the owner of the carrier 100 may install one or more sensors 370 after purchasing the carrier 100. In such an example, the one or more sensors 370 installed after the purchase of the carrier 100 may already have an initial calibration or be calibrated in a laboratory (e.g., in a workshop) to receive an initial calibration. Similarly, in a further example, the carrier 100 may have been issued to the workshop due to damage to the carrier 100. In such an example, it is also beneficial to (re)calibrate one or more sensors installed in the carrier 100.
[0045] exist Figure 1b An exemplary method for calibrating one or more sensors is illustrated in . Preferably, one or more sensors are calibrated after receiving an initial calibration. In a first step, the computing unit 300 may obtain sensor data from the one or more sensors 370 .
[0046] The sensor data obtained from one or more sensors 370 may include imaging data obtained from one or more cameras or one or more lidar sensors, and / or radar data obtained from one or more radar sensors. In one embodiment, one or more sensors 370 may include a plurality of radar sensors forming a radar antenna array, preferably a MIMO radar antenna array. The radar antenna array includes a plurality of radar sensors positioned close to each other to obtain improved spatial resolution, Doppler resolution, and dynamic range. Each radar sensor in one or more radar sensors in the radar antenna array transmits an orthogonal signal, which is received by a corresponding antenna, which extracts the obtained waveform using a corresponding filter. This allows the number of theoretical radar antennas to be increased without increasing the number of actual hardware radar antennas. In the art, this is also referred to as a virtual antenna array (X transmitting radar antennas and Y receiving radar antennas form an X*Y virtual radar antenna). Using the information obtained by each antenna in the X*Y virtual antenna, the specific resolution can be improved. The sensor data includes information about one or more objects 110, 120 around the vehicle 100. The sensor data may include distance information associated with one or more objects 110, 120, distance rate of change information associated with one or more objects 110, 120, preferably relative to the vehicle 100. The one or more objects 110, 120 may be any object that may be present around the vehicle 100. Preferably, the one or more objects 110, 120 are one or more other vehicles 110 around the vehicle 100 or one or more specific objects 120 around the vehicle 100. For example, the specific object 120 may be a sign or important object adjacent to the road, such as a tree or landmark that the vehicle 100 passes by while driving. The one or more important objects may also be calibration objects, such as fiducial markers designed to facilitate / allow calibration of the one or more sensors 370.
[0047] When the sensor data is radar data obtained from one or more radar sensors 374 , the sensor data includes one or more of angle, velocity, and / or Doppler information of one or more objects 110 , 120 surrounding the vehicle 100 .
[0048] Based on the sensor data, the computing unit determines a ground truth value. The ground truth value may indicate a reference for calibrating one or more sensors. The ground truth value may also indicate the quality / reliability of the calibration of the one or more sensors 370. The ground truth value may be determined based on at least a portion of the sensor data and one or more vehicle operating parameters. The one or more vehicle operating parameters include information about vehicle speed, vehicle position, and vehicle yaw rate.
[0049] Additionally or alternatively, based on the sensor data, one or more targets around the vehicle can be determined, wherein the one or more targets are determined by considering sensor data with a high signal-to-noise ratio SNR. The one or more targets can be one or more of the one or more objects 110, 120 around the vehicle 100. After the one or more targets around the vehicle 100 have been determined, one or more target parameters of the one or more targets can be determined based on the sensor data and the vehicle operating parameters. In some embodiments, the target parameters include distance information associated with the one or more targets relative to the vehicle 100, and distance change rate information associated with the one or more targets. Preferably, the target parameters are determined based on the vehicle operating parameters and the corresponding vector of complex values, which are from the information associated with different virtual antenna channels included in the sensor data. Additionally or alternatively, the target parameters include one or more of the position of the target relative to the vehicle, the speed of the target, and the yaw rate of the target. Based on the one or more target parameters, the sensor data, and the one or more vehicle operating parameters, the ground truth value can be determined.
[0050] Based on the ground truth values, the computing unit 300 may determine that the calibration of one or more sensors 370 appears to be unreliable. This may be the case where the calibration has changed significantly compared to the initial calibration of the one or more sensors 370. Therefore, the calibration needs to be adjusted. In some examples, the calibration is similar to the initial calibration of the one or more sensors 370. In the case where the calibration is similar to the initial calibration, the computing unit may determine that the calibration does not need to be updated.
[0051] After determining the ground truth value, the computing unit 300 may determine calibration update data based on the ground truth value. Preferably, the computing unit 300 also determines the calibration update data based on sensor data reflected from one or more targets in the vehicle's surroundings, wherein preferably, the reflected sensor data corresponds to a steering vector, or corresponds to a matrix of multiple steering vectors associated with one or more targets and one or more sensors.
[0052] Based on the calibration update data, the computing unit 300 may update the calibration of one or more sensors 370. The calibration may be updated by applying an algorithm that facilitates such an operation. Preferably, the calibration is updated by applying the algorithm to only a portion of the calibration data using the calibration update data. The algorithm is preferably based on a numerical optimization algorithm.
[0053] exist Figure 2 Various aspects of the methods described in the above examples are described in more detail in . Figure 2 Each step described may be additionally or alternatively included in the above examples.
[0054] In step S210, the computing unit 300 starts a calibration process. The calibration process can be started whenever a vehicle operation is started. For example, when the user starts the vehicle to perform a driving operation, the computing unit 300 can start the calibration process. In some examples, the computing unit 300 can start the calibration process based on determining that calibration may be necessary. The determination can be based on a certain time interval (e.g., every 6 months, preferably every month, more preferably every week, and most preferably every day), determining that the difference between the initial configuration and the calibration is higher than a predetermined threshold, determining that an accident has occurred, determining that a car part has been replaced, determining that an external signal indicates that calibration has started, and / or determining that a specific climate condition change has occurred. If it is determined that calibration may be necessary, a notification can be issued to the user of the vehicle, preferably as a voice / visual notification using an advanced driver assistance system (ADAS).
[0055] In step S211 (not shown), the computing unit 300 receives one or more vehicle operating parameters. The one or more vehicle operating parameters may include information about the arrangement of the one or more sensors 370 (e.g., the spacing and corresponding angles between the individual sensors 370 in the one or more sensors 370), information about the vehicle speed, information about the vehicle yaw rate, and information about other functional parameters associated with the vehicle 100 and / or the one or more sensors 370. In addition to the one or more vehicle operating parameters, the computing unit 300 may also obtain information associated with the first calibration of the one or more sensors 370 and the initial calibration of the one or more sensors 370 and / or any previous calibration of the one or more sensors 370. For example, the one or more vehicle operating parameters may also include an initial calibration matrix, a 0° calibration matrix, and / or a sector calibration matrix associated with the one or more sensors 370. The one or more vehicle operating parameters and the initial calibration of the one or more sensors 370 are considered as part of the input parameters for updating the calibration of the one or more sensors 370.
[0056] The computing unit 300 then obtains sensor data from the one or more sensors having calibration.
[0057] In step S220, the computing unit 300 determines one or more targets around the vehicle by filtering the sensor data. Preferably, the target is determined by considering sensor data with a high signal-to-noise ratio (SNR), which is very suitable for identifying a single target. In order to facilitate the determination of a single target, the computing unit 300 can obtain descriptive information, such as the size of a potential target among the one or more targets. Through the descriptive information, the single target determination can be confirmed. In addition, based on the radar installation position and one or more vehicle operating parameters, target parameters such as the speed, position and yaw rate of the target can be determined.
[0058] In addition, during the determination of one or more targets around the vehicle, the computing unit 300 can determine based on the sensor data that an expected portion of the sensor data is lost. Based on this determination, the computing unit 300 can request further sensor data and / or initiate an update to one or more sensors.
[0059] exist Figure 3 A typical environment for target filtering is illustrated in Figure 3 In the example shown, the vehicle 100 is traveling on a road, where two other vehicles 110 and a landmark 120 are present around the vehicle. Based on the sensor data, the computing unit 300 may determine object information associated with the objects 110, 120, including one or more of current position information, current speed information, and / or current yaw rate information of one or more objects 110, 120 around the vehicle 100. Based on the object information, the computing unit 300 may determine which of the one or more objects 110, 120 is most suitable for determining the ground truth. In some examples, the computing unit 300 determines that a moving object is more suitable for determining the ground truth than a static object. Figure 3 In the illustrated scenario, the computing unit 300 may determine that a vehicle 110 driving in the same lane as the vehicle 100 is a good candidate for target filtering. That is, the computing unit 300 may determine that a moving object is more suitable for determining ground truth when compared to a static object, and that a vehicle 110 driving in the same lane may be expected to be present for a relatively long period of time (particularly when compared to a vehicle 110 driving in an opposite direction from the vehicle 100), and to have a relatively low change in relative speed relative to the vehicle 100. This allows for a more reliable estimation of expected object information, and therefore a more reliable determination of ground truth, when compared to a vehicle 110 driving in an opposite direction, because the vehicle 110 will only be available for such estimation / determination for a short period of time. In particular, since the vehicle 110 driving in the same direction as the vehicle 100 will be available for a plurality of sensor data measurements that may be provided to the computing unit 300.
[0060] In step S221 (not shown), the computing unit 300 may determine whether the single target is sufficiently determined. If this is not the case, the computing unit 300 may repeat the target filtering step. That is, the computing unit 300 may consider another object of the one or more objects 110, 120, or start again by obtaining new sensor data. Otherwise, if the computing unit 300 determines that the target has been sufficiently determined, the computing unit 300 may continue to update the calibration.
[0061] Back to Figure 2In step S230, based on the target, vehicle information and sensor data, a steering vector of one or more sensors 370 is determined. Preferably, by considering the target parameters, one or more vehicle operating parameters and sensor data, a steering vector (or a steering vector matrix) of one or more sensors 370 associated with one or more targets (e.g., objects 110, 120 around the vehicle 100) can be determined.
[0062] Using the information obtained in the previous steps, such as Figure 1b As described, the computation unit 300 determines a ground truth value.
[0063] Subsequently, in step S240, the computing unit 300 updates the calibration using the calibration update data. Preferably, the computing unit updates a calibration matrix associated with the one or more sensors 370. The calibration matrix comprises information related to the amplitude and phase offsets in the sensor data when compared with the expected results associated with the initial calibration. The calibration update data is determined based on the ground truth values and the steering vector data obtained in step S230, wherein preferably only the portion of the calibration matrix associated with the calibration is updated.
[0064] Preferably, an algorithm is used to update the calibration matrix. The algorithm may be based on numerical optimization of a linear algebraic update of the calibration matrix. Preferably, by taking into account information included in the steering vector associated with one or more sensors 370 and ground truth values.
[0065] In some embodiments, the algorithm involves a machine learning engine. Preferably, the machine learning model is trained using input parameters and sensor data. The machine learning algorithm can be trained using the computing unit 300 or a remote computing unit in communication with the computing unit.
[0066] exist Figure 4 , an example is shown in which the computing unit 300 receives data (e.g., a trained machine learning model) from a remote computing unit using the remote server 200. The server 200 may also receive data sent from the vehicle 100. The server 200 may send data from / to the vehicle 100 using the API 204. The data may include sensor data, an initial calibration matrix, an updated calibration matrix obtained after updating the calibration matrix, the orientation of the vehicle, the position of the vehicle, the speed of the vehicle, and / or a trained machine learning model.
[0067] In one example, the vehicle 100 may send sensor data that is used at the remote computing device 300 to train a machine learning model that may then be sent to the vehicle 100 for updating the calibration. Preferably, the updating step S140 is configured to perform and determine when to update the calibration.
[0068] In another example, the computing device 300 may send data associated with the calibration and / or initial calibration of one or more sensors 370 to a remote computing device so that the remote computing device can train / adjust the machine learning model so that it is specifically trained for updating the calibration, thereby optimizing the updated calibration. Preferably, multiple computing devices 300a, 300b, and 300c associated with multiple corresponding vehicles 100a, 100b, 100c send / receive data to / from the remote server 200. This allows the amount of data at the remote server 200 to be increased so that the remote computing device can optimally train the machine learning model. In addition, a trained machine learning model (or other data) can be sent from one server to multiple vehicles 100a, 100b, 100c. The data transmitted between the server 200 and the vehicle 100 can be transmitted at regular intervals via wireless communication, or whenever the vehicle is at a service station / workshop.
[0069] In another example, the above-described data transmission from multiple vehicles 100a, 100b, 100c can be used to improve the calibration of one or more sensors 370 of the vehicle 100a. For example, sensor data from multiple vehicles 100a, 100b, 100c associated with the vehicle 100 can be used to improve calibration. That is, by improving the determination of target data and / or vehicle information, sensor data from multiple vehicles 100a, 100b, 100c associated with the vehicle 100 can be used to improve a single target determination. For example, the position, orientation, and velocity of a target and / or vehicle 100 can be determined based on sensor data from multiple vehicles 100a, 100b, 100c associated with the vehicle 100.
[0070] Although described above with respect to communication via a remote server, sensor data from multiple vehicles 100a, 100b, 100c and sensor data from vehicle 100 (as well as other data from vehicle 100) may also be communicated directly between vehicle 100 and multiple vehicles 100a, 100b, 100c.
[0071] Those skilled in the art will appreciate that the above examples and embodiments described with respect to vehicle 100 may also be applied to vehicles 100a, 100b, and 100c.
[0072] Back to Figure 2, after updating the calibration matrix using the algorithm, it is determined whether the method steps up to this point provide sufficient information / data so that the updated calibration can be expected to be successful. Otherwise, the process can be repeated starting from step S210, or the process ends, wherein if the process ends at this point, a notification can be issued to the user of the vehicle 100. In this case, the notification may include a warning that one or more sensors 370 require further calibration.
[0073] In step S250, a diagnostic step is performed to check whether the updated calibration matrix has improved the reliability of one or more sensors 370. This means that the ground truth value of the updated calibration indicates a good calibration. To this end, a reference value indicating the reliability of the calibration can be determined. For example, certain reference measurements can be performed, and a deviation threshold of the expected reference measurement from the actual reference measurement can be associated with the reference value. In some examples, the diagnostic step can be performed by comparing the calibrated calibration matrix with the updated calibration and / or initial calibration using cross-correlation. The reference value can be determined based on the result of determining the reference value using cross-correlation, so that the reference value indicates the reliability of the phase of the one or more sensors 370 about the sensor data. Additionally or alternatively, the Frobenius norm of the calibration matrix can be used to determine the quality (reliability) of the calibration relative to the determined ground truth value. Using the Frobenius norm, the reliability of the size of the sensor data of one or more sensors 370 can be determined. In addition, using the calibration matrix and the updated calibration matrix, comparing the angular error and correlation error at different angles for different ground truth targets can be used to determine the improvement obtained by updating the calibration matrix. That is, for each angular range of the total field of view of the one or more sensors 370 (ie, for a subset of the total viewing angle), target determination and ground truth determination are performed to update the calibration matrix. Based on this information, angular errors and correlation errors may be determined, as described above.
[0074] In step S260, the updated calibration is applied to one or more sensors 370. In addition, each of the corresponding information obtained during the update process can be stored on the vehicle 100 and / or sent to a remote server / computing device. In some examples, the information obtained in the aforementioned steps can be used to update / improve the trained machine learning model.
[0075] Preferably, the above method for updating the calibration of one or more sensors is performed on only a portion of the field of view of the one or more sensors 370. By performing the above method on only a portion of the field of view, the calibration can be updated more efficiently and the reliability of the update can be increased. In some examples, it may be beneficial to perform the above method multiple times to cover the entire field of view of the vehicle 100.
[0076] By applying the above method in an ADAS, the ADAS can perform an update of the calibration of one or more sensors in the vehicle, as described above. As a result, the vehicle assistance system can trigger one or more actions. Such an action can be any action performed by the ADAS, particularly taking into account the calibration of the sensor. That is, the ADAS can determine operating instructions for the vehicle based on the speed of the object determined to affect the function of the vehicle assistance system, wherein the function preferably includes: if it is determined that the calibration is not associated with a reference value, a warning is triggered, and after the calibration is updated using the sensor data, the warning is updated.
[0077] exist Figure 5An exemplary computing device 300 is illustrated in . The computing device 300 may include one or more long-range transceivers 310, which include one or more receivers 312 and one or more transmitters 314. The one or more transceivers are configured to send / receive data. Preferably, data is sent / received to / from the server 200. The computing device 300 may also include one or more short-range transceivers 320, which are configured to send / receive data at a short distance. One or more short-range transceivers 320 may include one or more receivers 322 and one or more transmitters 324. One or more short-range transceivers may also include a wired communication interface, which is configured to send / receive data via wired communication such as LAN, USB, etc. In addition, in order to estimate the current position of the vehicle 100, the computing unit 300 may also include one or more positioning sensors 330. One or more position sensors 330 may be configured to send / receive GPS, Galileo, GLONASS, BeiDou, and IRNSS data for determining the current position of the vehicle 100. Each of the one or more long-range transceivers, the one or more short-range transceivers, and the one or more positioning sensors includes antennas 319, 329, 339 for sending / receiving data. In addition, the computing device may include a data bus 340, which is configured to transmit data within the computing device. For example, the data bus 340 may be configured to transfer data from one or more positioning sensors 330 to a memory 350. The computing unit 300 also includes one or more processors 360, such as one or more CPUs 362 and / or one or more accelerators 364, which are configured to calculate instructions to perform any of the above methods. In addition, as described above, the computing unit 300 may also include one or more sensors 370, such as one or more cameras 372, one or more radars 374, and / or one or more laser radars 376. In some examples, one or more sensors 330 , 370 and / or one or more transceivers 310 , 320 are not part of computing unit 300 , but are in communication with computing unit 300 .
[0078] An exemplary vehicle 600 including an ADAS as described above and corresponding applications are described in Figure 6 . In this example, vehicle 600 is a car. However, in other examples, the vehicle may be any of a ship, a drone, a robot, or other vehicles that may benefit from considering the environment in their respective actions / operations.
[0079] In an example, the ADAS may be configured to perform the above method. For example, the ADAS may take into account the information obtained by the above method, such as by using a calibrated radar to control the vehicle. An exemplary application is a start-stop (Stop&Go) function for a vehicle, such as a robot in a manufacturing plant, to avoid collisions with other robots or people. In addition, in the example of a robot in a manufacturing plant, providing collision warnings to other vehicles or people around the robot may be critical for safety and collision avoidance. Another potential application described above is a pre-collision warning. This may be particularly critical in vehicles such as cars, such as in the case of a traffic accident, where a pre-collision warning may trigger safety-related applications such as airbags to avoid injuring the driver. In addition, another potential application is to help the driver of the car in a parking maneuver. To this end, the information obtained can be used for blind spot detection or as a general parking aid to facilitate parking. In addition, the information can also be used to assist the driver in changing lanes by triggering a warning or automatically performing a lane change.
[0080] The method according to the present invention can be implemented according to a computer program, which can be executed on any suitable data processing device, which includes a correspondingly configured device (e.g., a memory and one or more processors operably coupled to the memory). The computer program can be stored as a computer executable instruction on a non-transitory computer readable medium.
[0081] The embodiments of the present disclosure may be implemented in any of various forms. For example, in some embodiments, the present invention may be implemented as a computer-implemented method, a computer-readable storage medium, or a computer system.
[0082] In some embodiments, a non-transitory computer-readable storage medium may be configured to store program instructions and / or data, wherein the program instructions, when executed by a computer system, cause the computer system to perform a method, such as any one of the method embodiments described herein, or any combination of the method embodiments described herein, or any subset of any one of the method embodiments described herein, or any combination of such subsets.
[0083] In some embodiments, a computing device may be configured to include a processor (or a group of processors) and a storage medium, wherein the storage medium stores program instructions, wherein the processor is configured to read and execute program instructions from the storage medium, wherein the program instructions are executable to implement any of the various method embodiments described herein (or any combination of the method embodiments described herein, or any subset of any of the method embodiments described herein, or any combination of these subsets). The device may be implemented in any of various forms.
[0084] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even if only a single embodiment is described for a particular feature. The examples of features provided in the present disclosure are intended to be illustrative rather than restrictive, unless otherwise stated. The above description is intended to cover such alternatives, modifications, and equivalents that are obvious to those skilled in the art having the benefit of this disclosure.
[0085] The scope of the present disclosure includes any feature or combination of features disclosed herein (explicitly or implicitly), or any generalization thereof, whether or not it mitigates any or all of the problems addressed. In particular, with reference to the appended claims, features of dependent claims may be combined with features of the independent claims, and features of the respective independent claims may be combined in any appropriate manner, and not merely in the specific combinations listed in the appended claims.
[0086] Reference numerals list
[0087] 100 Vehicles
[0088] 100a Vehicle
[0089] 100b Vehicle
[0090] 100c Vehicle
[0091] 110, 120 objects
[0092] 200 Remote Server
[0093] 204API
[0094] 300 computing units
[0095] 300a computing unit
[0096] 300b computing unit
[0097] 300c computing unit
[0098] 310 Long Distance Transceiver
[0099] 312 Long Range Receiver
[0100] 314 Long Distance Transmitter
[0101] 319 Antenna
[0102] 320 short range transceiver
[0103] 322 Short Range Receiver
[0104] 324 Short Range Transmitter
[0105] 329 Antenna
[0106] 330 Positioning Sensor
[0107] 330 Antenna
[0108] 340 data bus
[0109] 350 Memory
[0110] 360 Processor
[0111] 362CPU
[0112] 364 Accelerator
[0113] 370 Sensors
[0114] 372 cameras
[0115] 374 Radar
[0116] 376 LiDAR
[0117] 600 Vehicles
Claims
1. A computer-implemented method for updating the calibration of one or more sensors in a vehicle, the method comprising the steps of: obtaining sensor data from the one or more sensors; determining a ground truth value based on at least a portion of the sensor data and based on one or more vehicle operating parameters; determining calibration update data based on the ground truth value; as well as The calibration is updated using the calibration update data.
2. The method according to claim 1, wherein: The step of determining the ground truth value further comprises: One or more targets in the surroundings of the vehicle are determined based on the sensor data, wherein the one or more targets are determined by considering the sensor data having a high signal-to-noise ratio (SNR).
3. The method according to claim 2, wherein: The step of determining the ground truth value further comprises: determining one or more target parameters of one or more targets in the vehicle's surroundings based on the sensor data; and The ground truth value is determined based on one or more target parameters of the target in the surroundings of the vehicle.
4. A method according to any one of the preceding claims, wherein: The vehicle operating parameters include one or more of: a location of the one or more sensors in the vehicle, a speed of the vehicle, and a yaw rate of the vehicle.
5. The method according to claim 2 or 3, wherein: The calibration update data is determined based on sensor data reflected from the one or more targets in the surroundings of the vehicle.
6. The method according to claim 5, wherein: The reflected sensor data corresponds to a steering vector associated with the one or more targets and the one or more sensors.
7. The method according to claim 3, wherein: The target parameters include one or more of the distance of the target relative to the vehicle, the rate of change of the distance of the target, based on the vehicle operating parameters and a corresponding vector of complex values, the complex values coming from information associated with one or more virtual antenna channels included in the sensor data.
8. A method according to any one of the preceding claims, wherein: The step of updating the calibration further comprises: The calibration is updated by applying an algorithm to only a portion of the calibrated data using calibration update data.
9. The method according to claim 8, wherein: The algorithm is applied by a trained machine learning model.
10. The method according to any one of the preceding claims, further comprising: The ground truth values and / or the calibration update data are received from a remote server.
11. The method according to any one of the preceding claims, further comprising: determining whether the updated calibration correlates with the baseline value; as well as If the updated first calibration does not correlate to the reference value, the calibration is repeatedly updated using the sensor data.
12. The method according to claim 11, further comprising: Determining operating instructions for the vehicle based on the updated calibration affecting functions of the vehicle assistance system, wherein preferably the functions include: If it is determined that the calibration does not correlate with the reference value, triggering a warning; and After updating the first calibration, the warning is updated.
13. A device comprising means configured to perform the method according to any one of claims 1 to 12.
14. A computer program comprising instructions which, when executed by a computing system, cause the computing system to perform the method according to any one of claims 1 to 12.
15. A carrier comprising the device according to claim 13.
