Cross-sensor vehicle sensor calibration based on object detection

Abstract

The method includes receiving a first output value from a first sensor of a vehicle, the first output value reflecting a position of an object external to the vehicle; receiving a second output value from a second sensor of the vehicle, the second output value reflecting the position of the object; determining a measurement difference regarding the position of the object based on the first and second output values; and performing an action related to at least one of the first or second sensors based on determining the measurement difference.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. Provisional Patent Application No. 63 / 369,688, filed July 28, 2022, entitled "CROSS-SENSOR VEHICLE SENSOR CALIBRATION BASED ON OBJECT DETECTIONS," the disclosure of which is incorporated herein by reference in its entirety.

[0002] This document relates to cross-sensor vehicle sensor calibration based on object detection. [Background technology]

[0003] Some vehicles manufactured today are equipped with one or more types of systems that can at least partially handle activities related to driving the vehicle. Some such assistance involves automatically monitoring the vehicle's surroundings and being able to take action regarding detected vehicles, pedestrians, or objects. Such automatic monitoring involves the use of sensors, and its performance depends on the accuracy of such sensors. In the past, vehicle calibration involved so-called end-of-line (EOL) calibration or traditional online calibration. For example, EOL calibration is performed in the factory using precisely marked equipment to establish a ground truth for sensor outputs. As another example, traditional online calibration is performed by the vehicle itself and attempts to independently calibrate each sensor based on its own sensor output. Summary of the Invention

[0004] In one aspect, a method includes receiving a first output value from a first sensor of a vehicle, the first output value reflecting a position of an object external to the vehicle; receiving a second output value from a second sensor of the vehicle, the second output value reflecting the position of the object; determining a measurement difference regarding the position of the object based on the first and second output values; and performing an action regarding at least one of the first or second sensors based on determining the measurement difference.

[0005] Implementations may include any or all of the following features: the first and second outputs include azimuth angle values ​​for the object relative to the vehicle, and the measurement difference includes an azimuth angle difference for the first and second sensors; at least one of the first and second sensors is a camera, a light detection and ranging device, or a radar; a measurement difference value is determined for each pair of first and second output values ​​such that a plurality of measurement difference values ​​is determined; determining the measurement difference includes determining a median or mean of the plurality of measurement difference values; the method further includes determining convergence of the measurement difference values, wherein the action is performed in response to determining the convergence of the measurement difference values; determining the convergence of the measurement difference values ​​includes determining a standard deviation of the measurement difference values, wherein the convergence of the measurement difference values ​​is determined in response to the standard deviation meeting a threshold. Determining the measurement difference includes performing a recursive process in which previously calculated statistics and current measurement difference values ​​are stored in a memory, and the previously calculated statistics are updated based on the current measurement difference value at each iteration of the recursive process. At least one calibration procedure has been previously performed for each of the first and second sensors of the vehicle, and the measurement difference is determined to detect calibration inaccuracies in the calibration procedures. A calibration procedure has not previously been performed for one of the first and second sensors of the vehicle, and the measurement difference is determined to obtain calibration parameters for one of the first and second sensors. Performing the action includes applying the obtained calibration parameters to one of the first and second sensors. A calibration status for at least one of the first and second sensors of the vehicle is unknown, and the measurement difference is determined to determine whether to (i) perform an online calibration of the vehicle or (ii) bring the vehicle in for repair. The action includes comparing the measurement difference to a threshold. The method further comprises performing a calibration of at least one of the first or second sensors in response to the measurement difference exceeding the threshold.The method further comprises correcting the measurement difference in a control algorithm for the vehicle in response to the measurement difference not exceeding the threshold. [Brief explanation of the drawings]

[0006] [Figure 1] 1 illustrates a schematic diagram of an example vehicle having sensors that generate outputs reflective of object positions;

[0007] [Figure 2] 10 shows an example of a graph of statistical values ​​relating to measurement differences in sensor outputs.

[0008] [Figure 3] 3 shows an example of a graph that may indicate convergence of statistics in the graph of FIG. 2.

[0009] [Figure 4] An example of a vehicle is shown.

[0010] [Figure 5] An example of the method is shown below.

[0011] [Figure 6] 1 illustrates an exemplary architecture of a computing device that may be used to implement aspects of the present disclosure.

[0012] Like reference symbols in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE INVENTION

[0013] This document describes example systems and techniques that may perform cross-sensor vehicle sensor calibration based on object detection. Calibration is performed across two or more sensors, rather than for each sensor independently. The present subject matter may provide an alternative approach for performing online calibration, whether or not combined with traditional online calibration. Approaches according to the present subject matter may be performed to determine whether vehicle calibration should be performed, and if so, what type of calibration should be performed. As another example, cross-sensor vehicle sensor calibration may be performed to determine calibration parameters for the sensors. Thus, the present subject matter can add an additional layer to traditional vehicle sensor calibration by cross-checking objects using more than one sensor. If automatic calibration for a sensor is not available, the present subject matter can be applied to calibrate the sensor by cross-checking one or more other sensors.

[0014] In prior approaches, EOL calibration is performed using a complex station precisely marked with sensor-detectable indicia. Due to inherent variations in manufacturing processes or device characteristics, the orientation of any individual sensor may vary between vehicles. Vehicles during production are therefore carefully positioned relative to the EOL station, and sensor outputs are read to establish ground truth for the calibration. Another prior approach involves calibration performed by a system local to the vehicle, sometimes referred to as online calibration. In traditional online calibration, a vehicle system calibrates one or more of the vehicle's sensors after the vehicle leaves the factory. This has been applied to some ADAS functions, which may not initially be operational in the vehicle until some time after customer delivery, during which traditional online calibration is completed. However, traditional online calibration is performed independently for each sensor using only the sensor's information alone, without the benefit of cross-vehicle measurements. Furthermore, conventional online calibration may have limitations, such as sensors that may have different calibration accuracies, sensors that do not detect lane markings on busy roads, or unfavorable weather conditions or vehicle surroundings, etc. These and other conditions may delay the completion of conventional online calibration.

[0015] The examples described herein refer to vehicles. A vehicle is a machine that transports passengers, cargo, or both. A vehicle may have one or more motors that use at least one type of fuel or other energy source (e.g., electricity). Examples of vehicles include, but are not limited to, cars, trucks, and buses. The number of wheels may vary between vehicle types, and one or more (e.g., all) of the wheels may be used to propel the vehicle, or the vehicle may be unpowered (e.g., when a trailer is attached to another vehicle). A vehicle may include a passenger compartment that accommodates one or more people. At least one vehicle occupant may be considered the driver; in this case, various tools, implements, or other devices may be provided to the driver. In the examples herein, any person carried by a vehicle may be referred to as the "driver" or "passenger" of the vehicle, regardless of whether that person is driving the vehicle, whether that person has access to the controls to drive the vehicle, or whether that person lacks the controls to drive the vehicle. The vehicles in this example are shown as being similar or identical to one another for illustrative purposes only.

[0016] Examples described herein refer to advanced driver-assistance systems (ADAS). In some implementations, an ADAS may perform driver assistance and / or autonomous driving. An ADAS may at least partially automate one or more dynamic driving tasks. An ADAS may operate based in part on the output of one or more sensors typically positioned on, under, or within the vehicle. An ADAS may plan one or more trajectories for a vehicle before and / or while controlling the vehicle's motion. The planned trajectories may define a path for the vehicle to travel. As such, propelling the vehicle according to the planned trajectories may correspond to controlling one or more aspects of the vehicle's operating behavior, such as, but not limited to, the vehicle's steering angle, gear (e.g., forward or reverse), speed, acceleration, and / or braking.

[0017] Although an autonomous vehicle is an example of an ADAS, not all ADAS are designed to provide fully autonomous vehicles. SAE International has defined multiple levels of driving automation, commonly referred to as Levels 0, 1, 2, 3, 4, and 5. For example, a Level 0 system or driving mode may not involve persistent vehicle control by the system. For example, a Level 1 system or driving mode may include adaptive cruise control, emergency brake assist, automatic emergency brake assist, lane keeping, and / or lane centering. For example, a Level 2 system or driving mode may include highway assist, autonomous obstacle avoidance, and / or autonomous parking. For example, a Level 3 or 4 system or driving mode may include incremental control of the vehicle by the driver assistance system. For example, a Level 5 system or driving mode may not require human intervention in the driver assistance system.

[0018] Examples described herein refer to sensors. A sensor is configured to detect one or more aspects of its environment and output a signal reflective of the detection. The detected aspect may be static or dynamic at the time of detection. By way of illustrative example only, a sensor may indicate one or more of the following: a distance between the sensor and an object, a speed of a vehicle carrying the sensor, a trajectory of the vehicle, or an acceleration of the vehicle. A sensor may generate an output without probing its surroundings with anything (e.g., passive sensing such as an image sensor capturing electromagnetic radiation), or the sensor may probe its surroundings (e.g., active sensing by sending out electromagnetic radiation and / or sound waves) and detect a response to the probing. Examples of sensors that may be used in one or more embodiments include, but are not limited to, optical sensors (e.g., cameras); light-based sensing systems (e.g., light ranging and detection (LiDAR) devices); radio-based sensors (e.g., radar); acoustic sensors (e.g., ultrasonic devices and / or microphones); inertial measurement units (e.g., gyroscopes and / or accelerometers); speed sensors (e.g., for the vehicle or its components); location sensors (e.g., for the vehicle or its components); orientation sensors (e.g., for the vehicle or its components); torque sensors; thermal sensors; temperature sensors (e.g., primary or secondary thermometers); pressure sensors (e.g., for the ambient air or vehicle components); humidity sensors (e.g., rain detectors); or occupancy sensors.

[0019] 1 schematically illustrates an example of a vehicle 100 having a sensor 102 that generates an output reflective of the position of an object. The vehicle 100 and / or sensor 102 may be used with one or more other examples described elsewhere herein. Here, the vehicle 100 is shown schematically as a box, and the vehicle 100 is shown from above while positioned on a surface 104. In this example, the vehicle 100 may be stationary or in motion on the surface 104. For example, the surface 104 may be a roadway, a floor, or any other terrain that supports the vehicle 100.

[0020] The sensors 102 may include two or more sensors. The sensors 102 may be of the same type as one another, or may be of two or more different types. Each of the sensors 102 may be attached or installed anywhere relative to the body of the vehicle 100. For example, the sensors 102 may be installed on the exterior or interior of the vehicle 100. Any of several types of sensors may be used for the sensors 102. In some implementations, at least one of the sensors 102 includes a camera. In some implementations, at least one of the sensors 102 includes a light detection and ranging (LiDAR) device. In some implementations, at least one of the sensors 102 may include a radar. Other sensors may be used in addition or instead.

[0021] The sensors 102 may detect one or more objects relative to the vehicle 100 and, in response, may generate at least one output value, where one of the sensors 102 generates an output 106A in response to detecting an object external to the vehicle 100. Similarly, another of the sensors 102 generates an output 106B in response to detecting the same object. Although the outputs 106A-106B reflect substantially the same size of the detected object, the outputs 106A-106B are not identical to one another because the sensors 102 in this example provide slightly different information regarding the object's location.

[0022] A corresponding detection of another object may occur relative to vehicle 100, where one of sensors 102 generates output 108A in response to the detection of another object external to vehicle 100, and another of sensors 102 generates output 108B in response to the detection of the same object. Thus, outputs 106A-106B correspond to the detection of one object by the respective sensor, and outputs 108A-108B correspond to the detection of another object by the respective sensor.

[0023] Each of the outputs 106A-106B and 108A-108B may be related to one or more aspects of the relationship of an object to the position of the vehicle 100. In some implementations, the outputs may reflect measurements relative to a reference. For example, the sensor 102 may determine an azimuth angle (e.g., horizontal angle) between the object and the virtual longitudinal axis L of the vehicle 100. As another example, the sensor 102 may determine an altitude (e.g., vertical angle) between the object and the virtual longitudinal axis L of the vehicle 100. Other measurements may be used, including, but not limited to, roll or pitch values. The selection of the type of measurement to be used may take into account the characteristics of the sensor 102. For example, a radar may not provide altitude information, so horizontal object detection (i.e., azimuth angle) may be selected. On the other hand, a LiDAR device may have altitude information, but another sensor may limit the available options. As another example, an image reader installed on the vehicle 100 may provide improved capabilities for object detection.

[0024] A measurement difference may be determined between at least two of outputs 106A-106B and outputs 108A-108B. The measurement difference reflects the difference between the positions of the same object detected by two or more of sensors 102. In some implementations, the measurements are in azimuth, in which case the measurement difference in terms of the object's position is an azimuth difference. For example, azimuth difference 110A is shown here with respect to outputs 106A-106B. Similarly, azimuth difference 110B is shown here with respect to outputs 108A-108B.

[0025] At least one of the obtained measurement differences may be compared to a threshold. If the measurement difference exceeds the threshold, action may be taken (or the action may be inhibited). In some implementations, the threshold may indicate the need to perform a calibration on the vehicle 100. If the threshold is met, this may indicate that the calibration of at least one of the sensors 102 is off. As such, the action taken may be to trigger one or more types of calibration. For example, an online calibration may be performed, or the operator of the vehicle 100 may be prompted to bring the vehicle 100 to a repair station for calibration.

[0026] Similarly, if the measurement difference does not exceed the threshold, another action may be taken (or the action may be suppressed). In some implementations, corrections for relatively small measurement differences may be performed. For example, the perception component of the ADAS may be designed to tolerate small aberrations in the outputs of various sensors. Based on the measurement difference, one or more parameters of the ADAS may be adjusted to correct for differences in the position of the same object by the sensors 102. As another example, the system may consider which of the sensors currently has a better calibration. This is done according to the type of sensor being used and its inherent characteristics. For example, a radar device may generally be considered to have higher accuracy than a LiDAR device. In such a situation, if a measurement difference is found between the radar and LiDAR device, the measurement difference can be applied to the LiDAR device to correct for it. Therefore, cross-sensor vehicle sensor calibration may be applied to correct for sensors with poor calibration accuracy based on sensors with good calibration accuracy.

[0027] Some examples illustrating the use of cross-sensor vehicle sensor calibration will now be described with reference to the above. Initially, in a particular situation, all sensors 102 of the vehicle 100 may have previously been calibrated. For example, an end-of-life (EOL) calibration may have been performed. Next, a cross-sensor vehicle sensor calibration may be performed to discover possible calibration inaccuracies in any of the sensors 102. For example, this may indicate that the EOL calibration was performed incorrectly, or that the vehicle 100 may have otherwise been subjected to a strong impact or other extreme condition that may have altered the installation of one or more of the sensors 102. That is, after at least one calibration procedure has previously been performed for each of the sensors 102, measurement differences may be determined and action may be taken to discover calibration inaccuracies in the calibration procedure. Again, if the detected measurement difference under such circumstances is relatively small (e.g., below a threshold), it may be sufficient to apply a correction in response (e.g., by a software algorithm). Otherwise, an online calibration or a visit to a repair station may be performed.

[0028] Second, some or all of the sensors 102 may not currently be calibrated. A cross-sensor vehicle sensor calibration may be performed to identify or determine at least one possible calibration parameter for any of the sensors 102. That is, a calibration procedure may not have been previously performed for one of the sensors 102, and a measurement difference may be determined to obtain a calibration parameter for one of the sensors 102. For example, if at least one of the sensors 102 is currently accurately calibrated, this one or more sensors may be used as a reference in obtaining at least a better calibration value for any of the other sensors. Actions performed based on determining the measurement difference may then include applying the determined calibration parameter to the sensor.

[0029] Third, the calibration status of the sensors 102 may be unknown. This may occur if the vehicle 100 system lacks information about whether one or more of the sensors 102 are calibrated, or if the system knows that none of the sensors 102 are calibrated. That is, the calibration status for at least one of the sensors 102 may be unknown, and a measurement difference may be determined to determine whether the vehicle 100 should perform an online calibration or whether the vehicle 100 should be brought in for repair. The use of cross-sensor vehicle sensor calibration may eliminate situations in which the vehicle 100 simply triggers a flag indicating that some sensors are off. Rather, the vehicle 100 may generate a list of at least one sensor that appears to need calibration in light of the check. This may then simplify the calibration work to be performed next at the repair station.

[0030] That is, in the present subject matter, a system of vehicle 100 obtains cross sensor outputs (e.g., outputs 106A-106B and outputs 108A-108B). A measurement difference regarding the position of an object may then be determined based on the outputs. Based on determining the measurement difference, an action may then be performed with respect to at least one of sensors 102.

[0031] 2 shows an example graph 200 of statistics related to measurement differences for sensor outputs. The illustrated statistics may represent one or more other examples described elsewhere herein. Graph 200 may represent any type of measurement. Here, the measurement is the median difference between the azimuth angles detected by two separate sensors for an object relative to the vehicle on which the sensors are installed. Thus, the azimuth angle difference is shown on the vertical axis of graph 200.

[0032] Each sensor may repeatedly generate its own output for the same object over a period of time. Therefore, multiple azimuth angle values ​​for an object may be generated by one sensor, and corresponding azimuth angle values ​​for the same object may be generated by another sensor. For each pair of these measurements from the two sensors, a measurement difference value may be determined. For example, the azimuth angle difference 110A or 110B in FIG. 1 may be determined for each pair of such measurements. A statistical measure may be determined for such determined measurement difference values, here the azimuth angle error. Any statistical measure that characterizes some aspect or characteristic of the entire data set may be used, including, but not limited to, the median or mean (e.g., the arithmetic mean). Graph 200 shows the median azimuth angle error over tens of thousands of sensor measurements. While there is some variation in the median during early measurements (i.e., toward the left end of the horizontal axis), the median begins to converge after around 10,000 measurements (in this example). The azimuth angle difference here converges toward a value of approximately −1.0 degrees. This value of the median azimuth angle difference does not change significantly as the number of measurements exceeds 70,000 data points, so convergence of the measurement difference values ​​may be determined, where an action is performed in response to determining convergence of the measurement difference values.

[0033] In some implementations, a recursive approach is applied when determining the measurement difference. For example, a recursive approach may be applied when a mean or standard deviation is calculated as a statistical value for the measurement difference. This may avoid using significant memory space or other computer resources to store large amounts of sensor data and the corresponding calculated difference values. For example, if the tens of thousands of data points shown in graph 200 were to be permanently stored, this may require significant system resources in the vehicle. Rather, a recursive process may be used in which only the current measurement difference value and previously calculated statistical values ​​are stored. N previous data points have already been processed, and the statistical value M N Let us assume that represents the mean value of these N data points. NWhen the next data point N+1 is acquired, the system updates the stored statistical value with the current measured difference value, resulting in an updated statistical value M representing the average value of these N+1 data points. N+1 Thus, only the previously calculated statistical values ​​and the current measurement difference value may be stored in memory, and the previously calculated statistical values ​​may be updated based on the current measurement difference value at each iteration of the recursive process.

[0034] 3 shows an example of a graph 300 that may indicate the convergence of statistics in the graph of FIG. 2. Graph 300 may represent one or more other examples described elsewhere herein. Graph 300 may represent any type of measurement, where the measurement is the standard deviation (std) of the difference in azimuth angles detected by two separate sensors for an object relative to the vehicle on which the sensors are installed. As such, the standard deviation of the azimuth angle difference is shown on the vertical axis of graph 300.

[0035] While there is some fluctuation in the standard deviation during initial measurements (i.e., toward the left end of the horizontal axis), after several thousand measurements (in this example), the standard deviation value begins to converge. Here, the standard deviation converges toward a value less than approximately 0.1 degrees. This value of the azimuth angle difference standard deviation does not change significantly once the number of measurements exceeds 70,000 data points. In some implementations, determining convergence of the measurement difference values ​​may include determining the standard deviation of the measurement difference values. For example, convergence of the measurement difference values ​​may be determined in response to the standard deviation meeting a threshold value (e.g., less than approximately 0.1 degrees). That is, when a recursive approach is used (e.g., only previously calculated statistics and the current measurement difference value may be stored in memory), convergence may not be immediately apparent from the stored measurement difference values, so the threshold value may be used as a flag to determine whether convergence has occurred. This flag may then indicate that the measurement difference values ​​have converged.

[0036] 4 illustrates an example of a vehicle 400. The vehicle 400 may be used with one or more other examples described elsewhere herein. The vehicle 400 includes an ADAS 402 and a vehicle controller 404. The ADAS 402 may be implemented using some or all of the components described with reference to FIG. 6 below. The ADAS 402 includes sensors 406 and a planning algorithm 408. Other aspects the vehicle 400 may include, including but not limited to other components of the vehicle 400 in which the ADAS 402 may be implemented, are omitted here for the sake of brevity.

[0037] Here, the sensor 406 is described as also including appropriate circuitry and / or executable programming for processing the sensor output and performing detection based on the processing. The sensor 406 may include a radar 410. In some implementations, the radar 410 may include any object detection system based at least in part on radio waves. For example, the radar 410 may be oriented in a forward direction relative to the vehicle and may be used to detect at least the distance to one or more other objects (e.g., another vehicle). The radar 410 may detect the surroundings of the vehicle 400 by sensing the presence of objects relative to the vehicle 400.

[0038] The sensors 406 may include an active optical sensor 412. In some implementations, the active optical sensor 412 may include any object detection system based at least in part on laser light. For example, the active optical sensor 412 may be oriented in any direction relative to the vehicle and may be used to detect at least the distance to one or more other objects. The active optical sensor 412 may detect the surroundings of the vehicle 400 by sensing the presence of objects relative to the vehicle 400. The active optical sensor 412 may be a scanning LiDAR or a non-scanning LiDAR (e.g., a flash LiDAR), to name just two examples.

[0039] Sensors 406 may include camera 414. In some implementations, camera 414 may include any image sensor whose signal is taken into account by vehicle 400. For example, camera 414 may be oriented in any direction relative to the vehicle and may be used to detect vehicles, lanes, lane markings, curbs, and / or road signs. Camera 414 may detect the surroundings of vehicle 400 by visually recording the situation in relation to vehicle 400.

[0040] The sensors 406 may include an ultrasonic sensor 416. In some implementations, the ultrasonic sensor 416 may include any transmitter, receiver, and / or transceiver used in detecting at least the proximity of an object based on ultrasonic waves. For example, the ultrasonic sensor 416 may be positioned on or near the exterior of the vehicle. The ultrasonic sensor 416 may detect the surroundings of the vehicle 400 by sensing the presence of objects in relation to the vehicle 400.

[0041] Any one of the sensors 406 alone, or two or more of the sensors 406 collectively, may detect the surroundings of the vehicle 400, regardless of whether the ADAS system 402 is controlling the movement of the vehicle 400. In some implementations, at least one of the sensors 406 may generate an output that is considered in providing an alert or other prompt to the driver and / or in controlling the movement of the vehicle 400. For example, the outputs of two or more sensors (e.g., the outputs of the radar 410, the active optical sensor 412, and the camera 414) may be combined. In some implementations, one or more other types of sensors may additionally or instead be included in the sensors 406.

[0042] The planning algorithm 408 may plan for the ADAS 402 to perform one or more actions, or no action, in response to monitoring the surroundings of the vehicle 400 and / or inputs by the driver. The output of one or more of the sensors 406 may be taken into account. In some implementations, the planning algorithm 408 may perform motion planning for the vehicle 400 and / or plan its trajectory.

[0043] Vehicle controls 404 may include steering controls 418. In some implementations, ADAS 402 and / or another driver of vehicle 400 controls the trajectory of vehicle 400 by adjusting the steering angle of at least one wheel by manipulating steering controls 418. Steering controls 418 may be configured to control the steering angle through a mechanical connection between steering controls 418 and adjustable wheels or may be part of a steer-by-wire system.

[0044] Vehicle controls 404 may include a gear control 420. In some implementations, the ADAS 402 and / or another operator of the vehicle 400 uses the gear control 420 to select among multiple operating modes of the vehicle (e.g., drive mode, neutral mode, or park mode). For example, the gear control 420 may be used to control an automatic transmission in the vehicle 400.

[0045] Vehicle control 404 may include a signal control 422. In some implementations, signal control 422 may control one or more signals that may be generated by vehicle 400. For example, signal control 422 may control the headlights, turn signals, and / or horn of vehicle 400.

[0046] Vehicle controls 404 may include brake controls 424. In some implementations, brake controls 424 may control one or more types of braking systems designed to slow the vehicle, stop the vehicle, and / or keep the vehicle stationary when stopped. For example, brake controls 424 may be actuated by ADAS 402. As another example, brake controls 424 may be actuated by the driver using a brake pedal.

[0047] Vehicle controls 404 may include a vehicle dynamics system 426. In some implementations, vehicle dynamics system 426 may control one or more functions of vehicle 400 in addition to, in the absence of, or instead of driver control. For example, when the vehicle is stopped on a hill, if the driver does not activate brake controls 424 (e.g., by pressing the brake pedal), vehicle dynamics system 426 may hold the vehicle stationary.

[0048] Vehicle controls 404 may include acceleration controls 428. In some implementations, acceleration controls 428 may control one or more types of traction motors of the vehicle. For example, acceleration controls 428 may control an electric motor and / or an internal combustion motor of vehicle 400.

[0049] Vehicle control 404 may further include one or more additional controls, collectively shown herein as control 430. Control 430 may provide vehicle control of one or more functions or components. In some implementations, control 430 may adjust one or more sensors of vehicle 400. For example, vehicle 400 may adjust sensor settings (e.g., frame rate and / or resolution) based on ambient data measured by the sensors of vehicle 400 and / or any other sensors.

[0050] Vehicle 400 may include a user interface 432. User interface 432 may include an audio interface 434 that may be used to generate alerts regarding sensor calibration. In some implementations, audio interface 434 may include one or more speakers positioned within the passenger compartment. For example, audio interface 434 may operate, at least in part, with an infotainment system within the vehicle.

[0051] User interface 432 may include a visual interface 436 that may be used to generate alerts regarding sensor calibration. In some implementations, visual interface 436 may include at least one display device in the passenger compartment of vehicle 400. For example, visual interface 436 may include a touchscreen device and / or an instrument cluster display.

[0052] 5 illustrates one example of a method 500. Method 500 may be used with one or more other examples described elsewhere herein. More or fewer operations than those shown may be performed. Unless otherwise indicated, two or more operations may be performed in a different order.

[0053] At operation 502, a first output value is received from a first sensor of the vehicle. The first output value reflects the position of an object external to the vehicle. For example, either output 106A or 108A in FIG. 1 may be received.

[0054] At operation 504, a second output value is received from a second sensor on the vehicle. The second output value reflects the position of the object. For example, either output 106B or 108B in FIG. 1 may be received.

[0055] In operation 506, a measurement difference regarding the position of the object based on the first and second output values ​​may be determined. For example, the azimuth angle difference 110A or 110B in Figure 1 may be determined. As another example, the data of graph 200 in Figure 2 may be determined.

[0056] At operation 508, an action may be performed with respect to at least one of the first or second sensors based on determining the measurement difference. For example, an online calibration may be performed. As another example, the ADAS may be adjusted to accommodate the measurement difference. As another example, the owner may be prompted to take the vehicle to a repair station for sensor calibration.

[0057] FIG. 6 illustrates an example architecture of a computing device 600 that may be used to implement aspects of the present disclosure, including any of the systems, devices, and / or techniques described herein, or any other systems, devices, and / or techniques that may be utilized in various possible embodiments.

[0058] The computing device illustrated in FIG. 6 may be used to execute the operating system, application programs, and / or software modules (including software engines) described herein.

[0059] In some embodiments, computing device 600 includes at least one processing device 602 (e.g., processor), such as a central processing unit (CPU). Various processing devices are available from various manufacturers, such as Intel or Advanced Micro Devices. In this example, computing device 600 also includes a system memory 604 and a system bus 606 that couples various system components, including the system memory 604, to the processing device 602. The system bus 606 is one of a number of types of bus structures that can be used, including, but not limited to, a memory bus or memory controller; a peripheral bus; and a local bus, using any of a variety of bus architectures.

[0060] Examples of computing devices that may be implemented using computing device 600 include a desktop computer, a laptop computer, a tablet computer, a mobile computing device (such as a smartphone, touchpad mobile digital device, or other mobile device), or other device configured to process digital instructions.

[0061] The system memory 604 includes a read-only memory 608 and a random access memory 610. A basic input / output system 612, containing the basic routines that act to transfer information within the computing device 600, such as during start-up, may be stored in the read-only memory 608.

[0062] In some embodiments, computing device 600 also includes a secondary storage device 614, such as a hard disk drive for storing digital data. The secondary storage device 614 is connected to system bus 606 by a secondary storage interface 616. The secondary storage device 614 and its associated computer-readable media provide non-volatile, non-transitory storage of computer-readable instructions (including application programs and program modules), data structures, and other data for computing device 600.

[0063] Although the exemplary environment described herein employs a hard disk drive as the secondary storage device, other types of computer-readable storage media are used in other embodiments. Examples of these other types of computer-readable storage media include a magnetic cassette, a flash memory card, a solid-state drive (SSD), a digital video disk, a Bernoulli cartridge, a compact disk read-only memory, a digital versatile disk read-only memory, a random access memory, or a read-only memory. Some embodiments include non-transitory media. For example, a computer program product may be tangibly embodied in a non-transitory storage medium. Additionally, such computer-readable storage media may include local storage or cloud-based storage.

[0064] A number of program modules may be stored in secondary storage device 614 and / or system memory 604, including an operating system 618, one or more application programs 620, other program modules 622 (such as the software engines described herein), and program data 624. Computing device 600 may utilize any suitable operating system.

[0065] In some embodiments, a user provides input to the computing device 600 through one or more input devices 626. Examples of input devices 626 include a keyboard 628, a mouse 630, a microphone 632 (e.g., for voice and / or other audio input), a touch sensor 634 (e.g., a touchpad or touch-sensitive display), and a gesture sensor 635 (e.g., for gesture input). In some implementations, the input devices 626 provide presence, proximity, and / or motion-based detection. Other embodiments include other input devices 626. The input devices may be connected to the processing device 602 through an input / output interface 636 that is coupled to the system bus 606. These input devices 626 may be connected by any number of input / output interfaces (e.g., a parallel port, a serial port, a game port, or a universal serial bus). In some possible embodiments, wireless communication between the input device 626 and the input / output interface 636 is also possible, including infrared, BLUETOOTH® wireless technology, 802.11a / b / g / n, cellular, ultra-wideband (UWB), ZigBee®, or other radio frequency communication systems, to name just a few.

[0066] In this exemplary embodiment, a display device 638, such as a monitor, liquid crystal display device, light emitting diode display device, projector, or touch-sensitive display device, is also connected to system bus 606 via an interface, such as a video adapter 640. In addition to the display device 638, computing device 600 may also include various other peripheral devices (not shown), such as speakers or a printer.

[0067] Computing device 600 may be connected to one or more networks through network interface 642. Network interface 642 may provide wired and / or wireless communication. In some implementations, network interface 642 may include one or more antennas for transmitting and / or receiving wireless signals. When used in a local area networking environment or a wide area networking environment (such as the Internet), network interface 642 may include an Ethernet interface. Other possible embodiments use other communication devices. For example, some embodiments of computing device 600 include a modem for communicating across networks.

[0068] Computing device 600 may include at least some form of computer-readable media. Computer-readable media includes any available media that can be accessed by computing device 600. By way of example, computer-readable media include computer-readable storage media and computer-readable communication media.

[0069] Computer-readable storage media include volatile and nonvolatile, removable and non-removable media implemented in any device configured to store information such as computer-readable instructions, data structures, program modules, or other data, including, but not limited to, random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory, or other memory technology, compact disk read-only memory, digital versatile disk, or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage device, or any other medium that can be used to store the desired information and that can be accessed by computing device 600.

[0070] Computer-readable communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, computer-readable communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency, infrared and other wireless media. Combinations of any of the above are also included within the scope of computer-readable media.

[0071] The computing device illustrated in FIG. 6 is also an example of a programmable electronic device that may include one or more such computing devices, and when multiple computing devices are included, such computing devices may be coupled together via a suitable data communications network to collectively perform various functions, methods, or operations disclosed herein.

[0072] In some implementations, computing device 600 may be characterized as an ADAS computer. For example, computing device 600 may include one or more components potentially used to process tasks arising in the field of artificial intelligence (AI). Computing device 600, in turn, includes sufficient processing power and the necessary support architecture for the demands of ADAS or AI in general. For example, processing device 602 may include a multi-core architecture. As another example, computing device 600 may include one or more coprocessors in addition to or as part of processing device 602. In some implementations, at least one hardware accelerator may be coupled to system bus 606. For example, a graphics processing unit may be used. In some implementations, computing device 600 may implement neural network-specific hardware to handle one or more ADAS tasks.

[0073] As used throughout this specification, the terms "substantially" and "about" are used to describe and take into account small variations, such as those due to processing variations. For example, they can refer to less than or equal to ±5%, such as less than or equal to ±2%, such as less than or equal to ±1%, such as less than or equal to ±0.5%, such as less than or equal to ±0.2%, such as less than or equal to ±0.1%, such as less than or equal to ±0.05%. Also, as used herein, indefinite articles such as "a" or "an" mean "at least one."

[0074] It should be understood that all combinations of the above concepts, and additional concepts discussed in more detail below, (provided that such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter disclosed herein. In particular, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the inventive subject matter disclosed herein.

[0075] Although several implementations have been described, it will nevertheless be understood that various modifications may be made without departing from the spirit and scope of the specification.

[0076] Additionally, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Additionally, other processes may be provided or processes may be eliminated from the described flows, and other components may be added to or removed from the described systems. Accordingly, other implementations are within the scope of the following claims.

[0077] While certain features of the described implementations have been shown and described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art. It should therefore be understood that the appended claims are intended to cover all such modifications and variations that fall within the scope of these implementations. They have been presented by way of example only, and not limitation, and it should be understood that various changes in form and detail may be made. Except for mutually exclusive combinations, any portion of the apparatus and / or methods described herein may be combined in any combination. The implementations described herein may include various combinations and / or subcombinations of the functions, components, and / or features of the different implementations described.

Claims

1. receiving a first output value from a first sensor on the vehicle, the first output value reflecting a position of an object external to the vehicle; receiving a second output value from a second sensor on the vehicle, the second output value reflecting the position of the object; determining a measurement difference related to the position of the object based on the first output value and the second output value; and performing an action with respect to at least one of the first sensor or the second sensor based on determining the measurement difference. A method for providing the above.

2. 2. The method of claim 1, wherein the first output value and the second output value comprise azimuth angle values ​​for the object relative to the vehicle, and the measurement difference comprises an azimuth angle difference for the first sensor and the second sensor.

3. The method of claim 1 , wherein at least one of the first sensor and the second sensor is a camera, a light detection and ranging device, or a radar.

4. The method of claim 1 , wherein a measurement difference value is determined for each pair of first and second output values ​​such that a plurality of measurement difference values ​​are determined.

5. The method of claim 4 , wherein determining the measurement difference comprises determining a median or mean of the plurality of measurement difference values.

6. The method of claim 4 , further comprising determining convergence of the measurement difference value, wherein the action is performed in response to determining the convergence of the measurement difference value.

7. 7. The method of claim 6, wherein determining the convergence of the measurement difference values ​​comprises determining a standard deviation of the measurement difference values, and wherein the convergence of the measurement difference values ​​is determined in response to the standard deviation satisfying a threshold.

8. 5. The method of claim 4, wherein determining the measurement difference comprises performing a recursive process in which previously calculated statistical values ​​and a current measurement difference value are stored in a memory, and the previously calculated statistical value is updated based on the current measurement difference value at each iteration of the recursive process.

9. 2. The method of claim 1, wherein at least one calibration procedure has previously been performed for each of the first and second sensors of the vehicle, and the measurement difference is determined to discover calibration inaccuracies in the calibration procedures.

10. 2. The method of claim 1, wherein a calibration procedure has not previously been performed on one of the first sensor and the second sensor of the vehicle, and the measurement difference is determined to obtain a calibration parameter for one of the first sensor and the second sensor.

11. The method of claim 10 , wherein performing the action comprises applying the obtained calibration parameters to one of the first sensor and the second sensor.

12. 2. The method of claim 1, wherein a calibration status for at least one of the first sensor and the second sensor of the vehicle is unknown, and the measurement difference is determined to determine whether to (i) perform an online calibration of the vehicle, or (ii) bring the vehicle in for repair.

13. The method of claim 1 , wherein the action comprises comparing the measurement difference to a threshold value.

14. The method of claim 13 , further comprising performing a calibration of at least one of the first sensor or the second sensor in response to the measurement difference exceeding the threshold.

15. The method of claim 13 , further comprising the step of correcting the measurement difference in a control algorithm for the vehicle in response to the measurement difference not exceeding the threshold.

Images