Vehicle monitoring and sensor calibration trigger system

Abstract

A vehicle computing system may monitor a vehicle using one or more sensors based on a set of sensor calibration triggers. Based on the monitoring of the vehicle, the computing system may detect a sensor calibration trigger of the set of sensor calibration triggers. In response to detecting the sensor calibration trigger, the computing system may output a recalibration alert to the user, where the recalibration alert notifies the user of recalibrating the sensor system of the vehicle.

Description

BACKGROUND

[0001] Sensor systems of autonomous and semi-autonomous vehicles are used to provide alerts (e.g., lane departure alerts, collision warnings, etc.), make emergency interventions, driver-assisted actions (e.g., automatic parallel parking, lane following and centering control, etc.), or for enabling automated navigation and control of the vehicle through a road network. These actions require the sensors of the vehicle to be properly calibrated. SUMMARY

[0002] A computing system for a vehicle can monitor the vehicle using one or more sensors based on a set of sensor calibration triggers. Based on the monitoring of the vehicle, the computing system can detect one of the set of sensor calibration triggers and output a recalibration alert to a user (e.g., a passenger or driver of the vehicle). The recalibration alert can inform the user to recalibrate the sensor system of the vehicle. As provided herein, each of the set of sensor calibration triggers can correspond to a severity level indicating whether one or more sensors of the sensor system need a full recalibration (e.g., extrinsic or intrinsic calibration), adjustment, repositioning, and / or need replacement. In certain examples, the calibration alert can be provided on a display screen of the vehicle (e.g., on the dashboard or infotainment system). In variant embodiments, the sensor calibration alert can be transmitted to a computing device of the user.

[0003] In certain implementations, the computing system monitors the vehicle using one or more inertial measurement units (IMUs). Accordingly, the computing system can monitor whether the vehicle has suffered impacts, vibrations, and other forces of different severities that can cause a change in orientation of the sensors, damage to the sensor mounts, or damage to the sensors themselves. Such impacts, vibrations, and other forces can correspond to situations in which the vehicle traverses over bumps or potholes on the road, the vehicle experiences a collision event, a severe acceleration or braking event, lateral forces exceed one or more thresholds, etc.

[0004] In various examples, the sensor system of the vehicle can include any combination of one or more LIDAR sensors, one or more image sensors, one or more radar sensors, or one or more ultrasonic sensors. Full autonomous or semi-autonomous vehicle operations can require such sensors. For example, an on-board computing system of the vehicle can execute a set of perception, motion prediction, motion planning, and / or motion execution models based on sensor data from the sensor system to navigate autonomously or semi-autonomously throughout a road network. These sensor-based models can rely on the calibration of the sensor system (e.g., relative to the center of the vehicle) to perform. In certain examples, the computing system can further monitor the vehicle based on sensor data from the sensor system of the vehicle.

[0005] According to examples described herein, the set of triggers can correspond to a set of scenarios that can include a windshield break, a windshield replacement, an impact or vibration experienced by the vehicle (e.g., exceeding one or more force or jolt thresholds), an over-the-air (OTA) update of software, a hardware upgrade, a hardware downgrade, a component adjustment or modification, or a component replacement. The hardware downgrade or upgrade can correspond to computing hardware (e.g., increasing computing power) or sensors (e.g., less expensive LIDAR or image sensors). The component can correspond to a vehicle part (e.g., a part including a sensor), such as a rearview mirror, a sideview mirror, a bumper, a wheel and / or tire, a brake, etc.

[0006] In various implementations, the computing system can detect different sensor calibration triggers of different severities that can require intrinsic recalibration and / or extrinsic recalibration of the sensor. As provided herein, intrinsic calibration can correspond to adjustments to internal characteristics of the sensor, such as focal length, bias, distortion, image center, laser alignment, mirror position, photodiode or light detector position, etc. Extrinsic calibration can correspond to the position and orientation of the sensor relative to other sensors, a vehicle center, and / or a surrounding environment. In certain examples, the computing system can determine that a sensor requires adjustments to intrinsic parameters that can be automatically recalibrated. For example, the computing system can detect a sensor calibration trigger corresponding to intrinsic recalibration of the sensor. In response to detecting the calibration trigger, the computing system can transmit a calibration command to the sensor to cause the sensor to perform intrinsic recalibration. BRIEF DESCRIPTION OF DRAWINGS

[0007] The disclosure herein is illustrated in the figures of the accompanying drawings, which are by way of illustration and not limitation, and in which like references indicate similar elements, and wherein:

[0008] Figure 1 is a block diagram depicting an example computing system for implementing vehicle monitoring and sensor recalibration alerts according to examples described herein;

[0009] Figure 2 is a block diagram illustrating an example vehicle computing system including a dedicated module for implementing vehicle monitoring and sensor recalibration alerts according to examples described herein;

[0010] Figure 3A and Figure 3B illustrates example recalibration triggers according to examples described herein; and

[0011] Figure 4 and Figure 5 is a flowchart describing an example method of implementing vehicle monitoring and sensor recalibration alerts according to examples described herein. DETAILED DESCRIPTION

[0012] Electrical and electronic (E / E) components of a vehicle can be included in safety systems that rely on sensing an external and / or internal vehicle environment to establish situational awareness. The intended functionality and implementation of E / E components for a particular safety system can lead to dangerous behavior, despite these E / E components being relatively immune to the faults mentioned in the ISO 26262 series of standards. Certain examples of such potentially dangerous behavior include, for example, the safety system failing to correctly perceive the environment, a lack of robustness in the functionality or algorithms of the safety system (e.g., with respect to sensor input variations), heuristics for sensor fusion, diversity of environmental conditions, or unexpected behavior of the safety system due to decision algorithms and / or different human expectations.

[0013] “Safety of the intended functionality” (SOTIF) corresponds to the absence of unreasonable risk due to dangerous behavior resulting from insufficient functionality. This includes both a lack of specification of the intended functionality of a safety system at the vehicle level, as well as a lack of specification or performance in the implementation of E / E components in the safety system. Accordingly, SOTIF mitigation measures correspond to implementing a particular safety system when the combination of detected vehicle hazards (e.g., a faulty component, a slipping tire, etc.) and the current driving scenario (e.g., rainy conditions, heavy traffic, high speed, etc.) generally creates a risk of injury or loss of life and the driver’s controllability of the vehicle or safety system is unlikely to prevent such injury or loss of life.

[0014] Mitigation measures to eliminate or reduce risk are implemented at various stages of a vehicle safety system, including the specification and design phase, the verification and validation phase, and the operational phase of the system. In particular, (i) modifications to vehicle functionality and / or sensor performance requirements in the development phase driven by identified system deficiencies or by dangerous scenarios, (ii) loop testing of the safety system in selected SOTIF-related scenarios (e.g., simulating potential trigger conditions), and (iii) long-term vehicle testing, track testing, simulation testing, and in-field monitoring of real-world SOTIF events are examples of such mitigation measures to eliminate or reduce risk when addressing SOTIF for a particular safety system. As provided herein, each potential dangerous event is evaluated according to the severity of possible injury in the context of how long the vehicle is exposed to the possibility of the dangerous event and the relative likelihood that a typical driver would control the vehicle to prevent injury.

[0015] Automotive Safety Integrity Level (ASIL) is a risk classification scheme defined by ISO 26262 (Road vehicles - Functional safety) and is typically established for E / E components of a vehicle by conducting a risk analysis of potential hazards involving a corresponding level of severity (i.e., severity of injury that can be expected to result from the hazard; classified as between S0 (no injury) and S3 (injury that is life threatening)), exposure (i.e., relative expected frequency of operating conditions in which injury can occur; classified as between E0 (extremely improbable) and E4 (high probability of high injury under most operating conditions)) and controllability of the vehicle operating scenario (i.e., relative likelihood that the driver can take action to prevent injury; classified as between C0 (totally controllable) and C3 (difficult to control or uncontrollable)). Thus, the safety goal for any potential hazard event includes a set of ASIL requirements.

[0016] Hazard identified as Quality Management (QM) does not dictate any safety requirements. By way of illustration, these QM hazards can be any combination of a low probability of exposure to the hazard, a low severity level of potential injury resulting from the hazard, and a high controllability level of the driver in avoiding the hazard and / or preventing injury. Other hazard events are classified as ASIL A, ASIL B, ASIL C, or ASIL D depending on the various severity levels, exposure, and controllability corresponding to the potential hazard. ASIL D events correspond to the highest integrity requirements (ASIL requirements) for a safety system or E / E component of a safety system, and ASIL A includes the lowest integrity requirements. For example, airbags, anti-lock brakes, and power steering systems of a vehicle will typically have an ASIL D level where the risk associated with failure of these components (e.g., possible severity of injury and lack of vehicle controllability to prevent these injuries) is relatively high.

[0017] As provided herein, ASILs can refer to both risk-based requirements and risk-dependent requirements, where various combinations of severity, exposure, and controllability are quantified to form an expression of risk (e.g., a vehicle’s airbag system can have a relatively low exposure level classification, but have high severity and controllability values). As provided above, the quantification of severity, exposure, and controllability for a given hazard traditionally uses the basic levels of severity (e.g., S0 to S3), exposure (e.g., E0 to E4), and controllability (e.g., C0 to C3) in the ISO 26262 series, which are then utilized to classify the ASIL requirements for components of a particular safety system. As provided herein, certain safety systems can have variable mitigation measures, which can range from alerts (e.g., visual, audible, or haptic alerts), minor interventions (e.g., braking or steering assistance) to major interventions and / or evasive maneuvers (e.g., taking over control of one or more control mechanisms such as steering, acceleration, or braking systems).

[0018] As further provided herein, fully autonomous and / or semi-autonomous vehicle operation requires precisely calibrated sensor systems on the vehicle in order to accurately measure the perceived environment. The sensor systems can include any combination of LIDAR sensors, image sensors (e.g., single cameras, binocular cameras, fisheye lens cameras, etc.), radar sensors, ultrasonic sensors, etc. Each of the sensors is internally and externally calibrated relative to the vehicle and to each other to provide an accurate representation of the environment around the vehicle. When a recalibration event occurs, such as the vehicle experiencing a significant impact, certain sensors can become misaligned, fail, or experience internal malfunctions. Such misalignments and failures can cause the vehicle to perform poorly or fail when performing autonomous and / or semi-autonomous tasks, such as SOTIF mitigation measures.

[0019] According to examples described herein, an on-board computing system of a vehicle can dynamically monitor whether the vehicle has been subjected to impacts, vibrations, collisions, and other forces of varying severity to determine whether one or more of the sensors need internal and / or external calibration, adjustment, and / or realignment. In various examples, each of the sensors can be mounted to the vehicle using a sensor mounting assembly that can be rated to withstand various levels of impacts, jolts, and vibrations that the vehicle can experience. In further examples, the sensors themselves can be rated to withstand various levels of impacts and vibrations.

[0020] In certain implementations, a computing system can use a learning-based approach to perform one or more functions described herein, such as processing the vehicle’s surroundings and other sensor data by executing an artificial neural network (e.g., a recurrent neural network, a convolutional neural network, etc.) or one or more machine learning models. Such learning-based approaches can further correspond to a computing system that stores or includes one or more machine learning models. In one implementation, the machine learning model can include an unsupervised learning model. In one implementation, the machine learning model can include a neural network (e.g., a deep neural network) or other type of machine learning model, including nonlinear models and / or linear models. The neural network can include a feedforward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. Some example machine learning models can utilize an attention mechanism, such as self-attention. For example, some example machine learning models can include a multi-head self-attention model (e.g., a transformer model).

[0021] As provided herein, a “network” or “one or more networks” can include any type of network or combination of networks that allow for communication between devices. In one implementation, the network can include one or more of a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, a point-to-point communication link, or some combination thereof, and can include any number of wired or wireless links. For example, communication over the network can be implemented via a network interface using any type of protocol, protection scheme, encoding, format, packaging, etc.

[0022] One or more examples described herein provide methods, techniques, and actions for execution by a computing device, programmatically or as a computer-implemented method. As used herein, programmatically means by use of code or computer-executable instructions. The instructions can be stored in one or more memory resources of the computing device. The steps of the programmatically executed methods, techniques, and actions can or can not be automatic.

[0023] One or more examples described herein can be implemented using programmed modules, engines, or components. Programmed modules, engines, or components can include programs, subroutines, portions of programs, or software or hardware components capable of performing one or more of the described tasks or functions. As used herein, a module or component can exist on the hardware of a device independently of other modules or components. Alternatively, a module or component can be a shared element or process of other modules, programs, or machines.

[0024] Some examples described herein can generally require the use of computing devices, including processing resources and memory resources. For example, one or more examples described herein can be implemented, in whole or in part, on computing devices such as servers and / or personal computers using network equipment (e.g., routers). Memory resources, processing resources, and network resources can all be used in connection with the establishment, use, or performance of any of the examples described herein, including the performance of any method or the implementation of any system.

[0025] Furthermore, one or more examples described herein can be implemented by use of instructions that are executable by one or more processors. These instructions can be carried by a non-transitory computer-readable medium. Machines shown or described with respect to the following figures provide examples of processing resources and computer-readable media for carrying out instructions implementing examples disclosed herein, and the instructions can be carried on the computer-readable medium and / or executed. Specifically, many of the machines shown with respect to examples of the present disclosure include processors and various forms of memory for storing data and instructions. Examples of non-transitory computer-readable media include permanent memory storage devices, such as hard drives on personal computers or servers. Other examples of computer storage media include portable storage units, such as flash drives or magnetic memory. Computers, terminal devices, network-enabled devices are all examples of machines and devices that utilize processors, memory, and instructions stored on computer-readable media. Additionally, examples can be implemented in the form of computer programs or computer-usable carrier media capable of carrying such programs.

[0026] Example computing system

[0027] Figure 1 is a block diagram depicting an example computing system for implementing a vehicle safety system of a vehicle in accordance with examples described herein. In one embodiment, the computing system 100 can include a control circuit 110, which can include one or more processors (e.g., microprocessors), one or more processing cores, programmable logic circuits (PLCs) or programmable logic arrays (PLAs) / programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or any other control circuit. In some implementations, the control circuit 110 and / or the computing system 100 can be embedded or otherwise disposed in a vehicle (e.g., a vehicle 200), such as a vehicle described herein. A vehicle control unit (also referred to as a vehicle controller) that is part of or can form a vehicle control unit in a vehicle (e.g., an automobile or truck). For example, the vehicle controller can be or can include an infotainment system controller (e.g., an infotainment head unit), a telematics control unit (TCU), an electronic control unit (ECU), a central powertrain controller (CPC), a central exterior and interior controller (CEIC), a zone controller, or any other controller (the term “or” is used interchangeably with “and / or” herein).

[0028] In variant embodiments, the control circuit 110 and / or the computing system 100 can be included on one or more servers (e.g., a backend server). In one embodiment, the control circuit 110 can be programmed by one or more computer-readable or computer-executable instructions stored on a non-transitory computer-readable medium 120. The non-transitory computer-readable medium 120 can be a memory device (also referred to as a data storage device) that can include electronic, magnetic, optical, electromagnetic, semiconductor, or any suitable combination of these. The non-transitory computer-readable medium 120 can form, for example, a computer floppy diskette, a hard disk drive (HDD), a solid-state drive (SDD), or solid-state integrated memory, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), static random access memory (SRAM), dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), and / or a memory stick. In some cases, the non-transitory computer-readable medium 120 can store computer-executable or computer-readable instructions, such as instructions for performing the methods described below in connection with Figure 4 and Figure 5 the instructions of the methods described below.

[0029] In various embodiments, the terms “computer-readable instructions” and “computer-executable instructions” are used to describe software instructions or computer code that is configured to perform various tasks and operations. In various embodiments, if the computer-readable or computer-executable instructions form a module, the term “module” refers broadly to a collection of software instructions or code that are configured to cause the control circuit 110 to perform one or more functional tasks. The modules and computer-readable / executable instructions can be described as performing various operations or tasks when the control circuit 110 or other hardware components are operating to execute the modules or computer-readable instructions.

[0030] In further embodiments, computing system 100 can include a communication interface 140 that enables communication over one or more networks 150 to transmit and receive data. Communication interface 140 can include any circuitry, components, software, etc. for communicating via one or more networks 150 (e.g., local area networks, wide area networks, the Internet, secure networks, cellular networks, mesh networks, and / or point-to-point communication links). In some implementations, communication interface 140 can include one or more of, for example, a communication controller, a receiver, a transceiver, a transmitter, a port, a conductor, software and / or hardware for communicating data / information.

[0031] As one example embodiment, computing system 100 can reside on an in-vehicle vehicle computing system and can receive sensor data from one or more sensors of the vehicle. In certain examples, these sensors can include an inertial measurement unit (IMU) that includes a combination of one or more magnetometers, accelerometers, and / or gyroscopes and can measure a particular force, angular rate, and / or orientation of the vehicle at any given time. In further examples, computing system 100 can receive sensor data from a sensor system of the vehicle that can include any combination of LIDAR sensors, image sensors, radar sensors, ultrasonic sensors, etc.

[0032] In various examples, computing system 100 can monitor the vehicle using sensor data from one or more of the sensors. Based on the monitoring of the vehicle, computing system 100 can determine whether a sensor recalibration trigger has occurred. As provided herein, a sensor recalibration trigger can correspond to any jolt or jounce experienced by the vehicle that can cause one or more sensors to require an intrinsic and / or extrinsic recalibration. In further aspects, a sensor recalibration trigger can correspond to a vehicle component breakage, failure, or replacement (e.g., a windshield, a side mirror, a rearview mirror, etc.) and / or can correspond to a software update or hardware update on the vehicle. In response to detecting a sensor recalibration trigger, computing system 100 can output an alert to a user (e.g., an owner or passenger of the vehicle).

[0033] In certain examples, the recalibration alert can provide a user with general notification to recalibrate the sensor system of the vehicle. In variant embodiments, the recalibration alert can identify the sensors that require recalibration and / or can provide a user or technician with contextual information regarding the required recalibration.

[0034] System description

[0035] Figure 2is a block diagram illustrating an example vehicle computing system including specialized modules for implementing vehicle safety systems, in accordance with examples described herein. In various implementations, a sensor system 205 of a vehicle can include external monitoring sensors, such as radar sensors (e.g., for automated or emergency braking assistance), LIDAR sensors, and / or cameras (e.g., for environmental perception and automated vehicle control), and / or ultrasonic sensors (e.g., for proximity determination). In further implementations, the sensor system 205 can include internal monitoring sensors, such as a global positioning or navigation system sensor, a wheel rotation sensor, a brake force sensor, an accelerometer, an accelerator pedal position sensor, an atmospheric pressure sensor, a tire pressure sensor, a fuel temperature sensor, a fuel pressure sensor, an exhaust gas temperature sensor, a safety system sensor (e.g., indicating performance or any failure of any particular vehicle safety system), a coolant and / or water temperature sensor, a mass air flow sensor, a manifold pressure sensor, a turbo boost sensor, an intake air temperature sensor, a throttle position sensor, an oxygen sensor, a NOx sensor, etc., in any combination.

[0036] In certain implementations, a vehicle can include an autonomous vehicle that navigates a road network based on perception, occupancy grid detection, object detection and classification, motion planning, and / or vehicle control operations. In variant embodiments, a vehicle can include a semi-autonomous vehicle that utilizes sensor data from the sensor system 205 to perform automated tasks, such as emergency braking, collision avoidance, automated parking, lane following, and various other operations (e.g., by an advanced driver assistance system of the vehicle). In various examples, the vehicle computing system 200 can include a vehicle monitoring module 220 that can process sensor data from the sensor system 205 to determine whether one or more sensors need recalibration or replacement. As provided herein, recalibration can include intrinsic recalibration or extrinsic recalibration. When the vehicle monitoring module 220 detects that a particular sensor of the sensor system 205 needs recalibration, the vehicle monitoring module 220 can transmit a recalibration command to the particular sensor, to cause the sensor to adjust a set of internal settings, for example, to address the recalibration accordingly, when possible.

[0037] In certain embodiments, the computing system 200 can include a database 230 that stores any number of preconfigured recalibration triggers 232 that the vehicle monitoring module 220 can reference to determine whether a particular recalibration trigger has occurred. The recalibration triggers 232 can include specified scenarios, such as detecting a windshield breakage, a sensor disconnection, a component failure, a particular sensor or the vehicle itself experiencing an impact or jolt, etc. Various scenarios are contemplated and can be individually set according to parameters of a particular sensor type (e.g., a gravity limit for a LIDAR sensor), individual sensors (e.g., a fisheye camera mounted to a rearview mirror), or sensor mounts (e.g., a gravity limit for a particular LIDAR sensor mount). Such scenarios can include detected force thresholds on the vehicle, such as a gravity threshold or an impact threshold experienced by the vehicle, a component replacement, an over-the-air software update, a hardware upgrade and / or downgrade, etc. As provided herein, any number of recalibration triggers 232 can be stored in the database 230 and dynamically referenced by the vehicle monitoring module 220.

[0038] In various examples, the computing system 200 can include or otherwise monitor IMU data from one or more IMUs 210 of the vehicle. As provided herein, the IMU data can be indicative of forces (e.g., gravity and / or impacts) experienced by the vehicle. In some examples, individual sensors or sensor mounts can include one or more IMUs 210 that can provide IMU data to the vehicle monitoring module 220. In such examples, the vehicle monitoring module 220 can determine whether a particular recalibration trigger 232 has occurred for an individual sensor or sensor mount based on the IMU data received from the specified co-located IMU 210. In variant embodiments, the vehicle can have a set of one or more IMUs 210 mounted in predetermined locations on the vehicle that can generally provide IMU data to the vehicle monitoring module 220.

[0039] Based on monitoring IMU data from one or more IMUs 210 and / or sensor data from the sensor system 205, the vehicle monitoring module 220 can detect a sensor calibration trigger corresponding to one or more of the recalibration triggers 232 in the database 230. In response to detecting a sensor calibration trigger, the vehicle monitoring module 220 can determine one or more sensors in the sensor system 205 that need recalibration and the nature of the recalibration. For example, the detected sensor calibration trigger can correspond to exceeding a gravity threshold for one or more sensors or sensor mounts. As provided herein, recalibration actions can be associated with detected sensor calibration triggers according to the severity of the trigger. For example, when a gravity threshold (e.g., 20g jolt) has been exceeded on the vehicle, the recalibration trigger 232 can indicate that a full system recalibration needs to be performed on the vehicle, which can include a system check (e.g., all mounts, sensor settings, and current alignment) and all actions needed to perform a recalibration of the sensor system 205 as a whole.

[0040] As provided herein, a full system recalibration can correspond to checking the internal settings, damage, and internal and / or external alignment of each sensor of the sensor system 205. If any settings are out of specification, any damage is detected, or any misalignment is detected, a technician can address the discrepancies to recalibrate the sensor system 205, which can include external adjustments, internal setting adjustments, sensor replacement, etc. Upon detecting a recalibration trigger 232, the vehicle monitoring module 220 can transmit one or more recalibration actions corresponding to the matching recalibration trigger to the alert module 240 of the computing system 200. In certain examples where a sensor calibration trigger can be directly resolved (e.g., via an intrinsic recalibration of a particular sensor), the vehicle monitoring module 220 can transmit a recalibration command to the affected sensor to cause or cause the affected sensor to perform an automatic calibration accordingly.

[0041] Based on the recalibration actions corresponding to the matching recalibration trigger, the alert module 240 generates a recalibration alert for transmission over one or more networks 280 via the communication interface 250 of the computing system or for display on a display device 260 of the computing system 200 or the vehicle. In certain examples, the alert module 240 can transmit the recalibration alert to a computing device of a user to notify the user that one or more sensors need recalibration. In such examples, the user can be located within the vehicle or can be remote from the vehicle. In either case, the alert module 240 can specify which particular sensors need recalibration (e.g., by name, identifier, code, etc.) or can generally notify the user to bring the vehicle to a service location for recalibration.

[0042] It is contemplated that each sensor of the sensor system 205 must be calibrated relative to the other sensors and relative to the vehicle itself (e.g., the center of the vehicle) in order to properly function in an autonomous or semi-autonomous function. Accordingly, the examples described herein can facilitate a quick recalibration of the sensor system 205 of a vehicle to ensure that such safety features operate correctly. It is further contemplated that the alert module 240 can further designate which vehicle safety systems and features will not function properly until the sensor system is recalibrated. In such examples, these safety systems and features can be listed or described on the display device 260 until the sensor system 205 is recalibrated or reset by a technician.

[0043] Example recalibration trigger

[0044] Figure 3A and Figure 3B Examples of example recalibration triggers in accordance with the examples described herein are illustrated, such as with respect to the recalibration trigger 232 shown and described with respect to Figure 2 As provided herein, examples relative to Figure 3A and Figure 3B are shown and described for illustrative purposes and are not intended to limit any embodiment of the present disclosure. Further, while Figure 3A and Figure 3B Examples describe two types of recalibration triggers, but many recalibration triggers are contemplated that can involve various levels of severity of forces or jolts experienced on a vehicle, component damage and replacement, or updates to software and / or hardware on a vehicle. Accordingly, Figure 3A and Figure 3B Examples of example recalibration triggers shown are provided for illustrative purposes only and do not constitute an exhaustive list of recalibration triggers for a vehicle.

[0045] Referring to Figure 3A , an example list of recalibration triggers 300 is provided for impact events having different levels of severity as measured in g-forces. In various implementations, the impact even recalibration triggers 300 can correspond to any acceleration or jolt event experienced by a vehicle as measured by one or more IMUs. In one aspect, various sensors of a vehicle can include an acceleration or impact rating and / or the mounting mechanism of a sensor can include an acceleration or impact rating. In further aspects, the impact event recalibration triggers 300 can be configured for a single sensor (e.g., a LIDAR sensor), a single sensor type (e.g., image sensors of a vehicle), or the entire sensor system 205 of a vehicle.

[0046] In Figure 3AIn the illustrated example, the shock event recalibration triggers 300 are categorized based on the severity of the shock event. As shown in the severity column 305, various severity levels of a shock event are associated with a recalibration action to be taken, as shown in the recalibration column 310. For a particular sensor, sensor type, or entire sensor system, a shock experienced with a severity less than 2g can be associated with no action needed to be taken. For example, the sensor, sensor type, and / or sensor system 205 can have internal and external mount ratings that can easily withstand a force of 2g. For forces from 2g to 5g, a system check can be needed to ensure that the shock experienced did not cause damage. As such, a notification can be provided to a user of the vehicle to have a technician inspect the vehicle to perform the system check.

[0047] In Figure 3A In the illustrated example, a shock experienced from 5g to 10g can require sensor adjustment of one or more of the sensors of the sensor system 205, which can correspond to one or more individual sensors needing adjustment to intrinsic or extrinsic parameters. The adjustment can correspond to sensor settings or realignment of the sensor relative to the sensor mount and / or one or more other sensors in the sensor system 205. As further shown in Figure 3A As further shown in the example, a shock experienced between 10g and 20g can involve a full system recalibration, which can correspond to a recalibration of the entire sensor system 205 relative to the vehicle. As further shown, a shock experienced above 20g can correspond to replacement of one or more sensors as well as a full system recalibration.

[0048] Figure 3B Another example of a set of recalibration triggers involving replacement of components caused by accidents, collisions, component failure, and / or general breakage of components is illustrated. The component recalibration triggers 350 can include a list of components as shown in column 355, and recalibration actions to be taken as shown in column 360. As shown in column 355, individual components for a vehicle can include side mirrors, rearview mirrors, front and rear bumpers, and windshields, each of which can include one or more sensors and be routinely replaced due to accidents or breakage. As such, replacement of these components can require certain types of sensor recalibration actions, as listed in column 360.

[0049] In Figure 3B In the illustrated example, replacement of a side mirror can require a full system recalibration due to, for example, multiple sensor types being housed in the side mirror housing (e.g., radar sensors, proximity sensors, and image sensors). In further examples, replacement of a rearview mirror can only require a sensor check of the sensors housed in the rearview mirror to ensure alignment (e.g., fisheye image sensors). InFigure 3B In another example shown, replacement of the front and / or rear bumper can require sensor adjustment (e.g., since only radar sensors are housed in the bumper). The component recalibration trigger 350 can also include a windshield breakage or replacement recalibration trigger, where full system recalibration can be required (e.g., since multiple sensor types require sensor observation through the windshield).

[0050] Numerous recalibration triggers are contemplated, which can be caused by various occurrences experienced by the vehicle, including the impact event recalibration trigger 300, the component recalibration trigger 350 as shown, Figure 3A and Figure 3B software and hardware update recalibration triggers (e.g., sensor upgrade, sensor downgrade, over-the-air software update, etc.). Each recalibration trigger can be associated with one or more recalibration actions, which can include sensor checks, alignment checks, intrinsic or extrinsic adjustments, partial recalibration of the sensor system 205 (e.g., image sensors are recalibrated relative to each other), or full sensor system recalibration. Thus, Figure 3A and Figure 3B The recalibration triggers shown are for illustrative purposes only and are not intended to be limiting in any way.

[0051] Methodology

[0052] Figure 4 and Figure 5 are flow diagrams that describe example methods of implementing vehicle monitoring and sensor recalibration alerts in accordance with the examples described herein. In Figure 4 and Figure 5 the following description, reference is made to the accompanying drawings that form a part hereof, and in which are shown, by way of illustration, various features for carrying out the examples of the present disclosure. It is to be understood that other specific arrangements can be utilized and that the examples described herein are not limited by the illustrated Figure 1 to FIG. 3. Moreover, the processes described in connection with the examples described herein can be embodied about an example computing system 200, as described with Figure 4 and Figure 5 reference to FIG. 1. Furthermore, certain steps in the processes described in connection with the flow diagrams of FIGS. 1-3 can be performed in an order different from the order in which the steps are described, and not all steps can be required, or even desirable. Figure 2 Figure 4 Figure 5 with reference to FIG. 1. Furthermore, certain steps in the processes described in connection with the flow diagrams of FIGS. 1-3 can be performed in an order different from the order in which the steps are described, and not all steps can be required, or even desirable.

[0053] with reference to FIG. 1. Furthermore, certain steps in the processes described in connection with the flow diagrams of FIGS. 1-3 can be performed in an order different from the order in which the steps are described, and not all steps can be required, or even desirable. Figure 4 ​​At block 400, the vehicle computing system 200 can monitor the vehicle using one or more sensors based on a set of recalibration triggers. As provided herein, the one or more sensors can include one or more IMUs of the vehicle, sensors from the sensor system 205 of the vehicle (e.g., LIDAR sensors, image sensors, and / or radar sensors), or sensors that detect that a component has been replaced. At block 405, the computing system 200 can further detect a sensor recalibration trigger from the set of recalibration triggers based on the monitoring of the vehicle. As described throughout this disclosure, the sensor recalibration trigger can correspond to an impact or jolt experienced by the vehicle that exceeds one or more thresholds (e.g., as determined from IMU data), or can correspond to a misalignment event due to physical contact with a sensor or sensor mount (e.g., based on a collision, pothole, accident, or intentional modification of the vehicle).

[0054] At block 410, the computing system 200 can output a recalibration alert to a user of the vehicle in response to detecting the recalibration trigger. As provided herein, the recalibration alert can provide an indication to the user that a sensor check, adjustment, and / or recalibration is needed. In further examples, the recalibration alert can indicate which particular sensors need a recalibration check, adjustment, or whether a full system recalibration of the sensor system is needed. As described herein, the computing system 200 can output the recalibration alert to a display screen of the vehicle, and / or can transmit the recalibration alert to a computing device of the user (e.g., via one or more networks 280, such as Bluetooth, Wi-Fi, or cellular networks).

[0055] Figure 5 is another example method of implementing vehicle monitoring and sensor recalibration alerts according to various examples. Reference is made to Figure 5 At block 500, the computing system 200 of the vehicle can monitor the vehicle based on sensor data from one or more sensors. As described herein, at block 502, the sensors can include one or more IMUs, and / or at block 504, the sensors can include one or more sensors from the sensor system 205 of the vehicle. By monitoring the vehicle, the computing system 200 can dynamically determine whether a sensor recalibration trigger has occurred through experienced forces or jolts, component misalignment, damage, replacement, etc.

[0056] At decision block 505, the computing system 200 can dynamically determine whether a recalibration trigger has been satisfied. As provided herein, the recalibration trigger can be satisfied if a predetermined condition has occurred on the vehicle, such as a force detected on the vehicle, component damage, misalignment or replacement, and / or sensor failure or malfunction. If the recalibration trigger has not been satisfied, the computing system 200 continues to dynamically monitor the vehicle. If one or more recalibration triggers have been satisfied, at block 515, the computing system 200 can determine one or more recalibration actions for one or more sensors of the sensor system 205 based on the recalibration trigger.

[0057] At block 516, the recalibration action can correspond to a sensor check, in which a user or technician determines whether any potentially affected sensors need recalibration. At block 517, the recalibration action can include a sensor adjustment, in which a user or technician can realign or adjust settings (e.g., intrinsic or extrinsic settings) of one or more sensors. At block 518, the recalibration action can correspond to a system recalibration, in which an affected sensor requires a full system recalibration relative to the vehicle and / or other sensors of the sensor system 205.

[0058] In certain scenarios, a potentially affected sensor can be automatically checked, adjusted, or recalibrated. For example, an image sensor can experience a jolt that has affected intrinsic parameters of the image sensor. The computing system 200 or the image sensor itself can determine that the image sensor can be intrinsically recalibrated via a recalibration command. As another example, an image sensor or LIDAR sensor can be mounted to the vehicle via a joint operated by an actuator that allows for a certain tolerance level of realignment. If the misalignment caused by the recalibration trigger is within that tolerance level, the computing system 200 or the sensor can automatically realign and / or recalibrate the affected sensor accordingly. Thus, in certain embodiments, at decision block 520, the computing system 200 can determine whether a sensor check, adjustment, and / or recalibration can be automatically performed.

[0059] In one embodiment, if a sensor check, adjustment, and / or recalibration can be automatically performed, at block 525, the computing system 200 can transmit a recalibration command to the sensor to facilitate automatic recalibration. In a variant embodiment, the sensor can automatically perform the recalibration action according to the recalibration command. If the recalibration action cannot be automatically performed, the computing system 200 can output a recalibration alert to a user, as described herein.

[0060] It can be appreciated that the vehicle computing system 200 can include components of an advanced driver assistance system (ADAS) of a vehicle, or can be included as components of an autonomous vehicle control system that automatically operates a vehicle along a travel route. It can be further appreciated that alignment and calibration of the sensor system 205 of a vehicle can be necessary for proper operation of safety systems of the vehicle. As provided herein, safety systems of a vehicle can include any safety system such as a brake assist system, an active lane-keeping assist system, a forward collision warning system, an automatic emergency braking system, a pedestrian detection and / or avoidance system, an adaptive cruise control system, a blind spot warning system, a rear cross-traffic alert system, a lane departure warning system, a lane-keeping assist system, an active headrest system, a backup camera system, a parking assist system, an airbag system, a seat jump lock system, a traction control system, an electronic stability control system, a collision intervention system, etc.

[0061] It can be appreciated that examples described herein can extend to individual elements and concepts described herein (independently of other concepts, ideas or systems), and examples can extend to combinations of elements described anywhere in the present disclosure. Although examples are described in detail herein with reference to the accompanying drawings, it is to be understood that the concepts are not limited to those precise examples. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. The scope of the concepts is defined by the following claims and their equivalents. Furthermore, it is intended that particular features described individually or as part of an example can be combined with other individually described features or parts of other examples, even if the other features or examples do not mention that particular feature.

Claims

1. A computing system for a vehicle, the computing system comprising: one or more processors; memory storing instructions that, when executed by the one or more processors, cause the computing system to: monitor the vehicle using one or more sensors based on a set of sensor calibration triggers; detect, based on monitoring the vehicle, a sensor calibration trigger of the set of sensor calibration triggers; and in response to detecting the sensor calibration trigger, output a recalibration alert to a user, the recalibration alert informing the user to recalibrate a sensor system of the vehicle.

2. The computing system of claim 1, wherein each sensor calibration trigger of the set of sensor calibration triggers corresponds to a severity level.

3. The computing system of claim 1, wherein the one or more sensors used to monitor the vehicle comprise one or more inertial measurement units (IMUs).

4. The computing system of claim 1, wherein the sensor system comprises any combination of one or more LIDAR sensors, one or more image sensors, one or more radar sensors, or one or more ultrasonic sensors.

5. The computing system of claim 4, wherein the one or more sensors used to monitor the vehicle are included in a sensor suite.

6. The computing system of claim 1, wherein the set of triggers corresponds to a set of scenarios, the set of scenarios comprising at least one of: a windshield break, a windshield replacement, an impact or vibration experienced by the vehicle, an over-the-air update to software, a hardware upgrade, a hardware downgrade, a component adjustment, or a component replacement.

7. The computing system of claim 1, wherein the vehicle comprises an autonomous vehicle or a semi-autonomous vehicle.

8. The computing system of claim 1, wherein the executed instructions further cause the computing system to: detect, based on monitoring the vehicle, a second sensor calibration trigger of the set of sensor calibration triggers, the second sensor calibration trigger corresponding to an intrinsic recalibration of a sensor of the sensor system; and in response to detecting the second sensor calibration trigger, transmit a calibration command to the sensor to cause the sensor to perform the intrinsic recalibration.

9. The computing system of claim 1, wherein the executed instructions cause the computing system to output the recalibration alert to a display screen of the vehicle.

10. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a computing system of a vehicle, cause the computing system to: monitor the vehicle using one or more sensors based on a set of sensor calibration triggers; detect, based on monitoring the vehicle, a sensor calibration trigger of the set of sensor calibration triggers; and in response to detecting the sensor calibration trigger, output a recalibration alert to a user, the recalibration alert informing the user to recalibrate a sensor system of the vehicle. ​ ​ 11. The non-transitory computer-readable medium of claim 10, wherein each sensor calibration trigger of the set of sensor calibration triggers corresponds to one severity level.

12. The non-transitory computer-readable medium of claim 10, wherein the one or more sensors used to monitor the vehicle comprise one or more inertial measurement units (IMUs).

13. The non-transitory computer-readable medium of claim 10, wherein the sensor system comprises any combination of one or more LIDAR sensors, one or more image sensors, one or more radar sensors, or one or more ultrasonic sensors.

14. The non-transitory computer-readable medium of claim 13, wherein the one or more sensors used to monitor the vehicle are included in a sensor suite.

15. The non-transitory computer-readable medium of claim 10, wherein the set of triggers corresponds to a set of scenarios, the set of scenarios comprising at least one of: a windshield breakage, a windshield replacement, an impact or vibration experienced by the vehicle, an over-the-air update of software, a hardware upgrade, a hardware downgrade, a component adjustment, or a component replacement.

16. The non-transitory computer-readable medium of claim 10, wherein the vehicle comprises an autonomous vehicle or a semi-autonomous vehicle.

17. The non-transitory computer-readable medium of claim 10, wherein the executed instructions further cause the computing system to: based on monitoring the vehicle, detect a second sensor calibration trigger of the set of sensor calibration triggers, the second sensor calibration trigger corresponding to an intrinsic recalibration of a sensor of the sensor system; and in response to detecting the second calibration trigger, transmit a calibration command to the sensor to cause the sensor to perform the intrinsic recalibration.

18. The non-transitory computer-readable medium of claim 10, wherein the executed instructions cause the computing system to output the calibration alert to a display screen of the vehicle.

19. A computer-implemented method for monitoring a vehicle for sensor recalibration, the method performed by one or more processors of the vehicle and comprising: based on a set of sensor calibration triggers, monitoring the vehicle using one or more sensors; based on monitoring the vehicle, detecting a sensor calibration trigger of the set of sensor calibration triggers; and in response to detecting the sensor calibration trigger, outputting a recalibration alert to a user, the recalibration alert informing the user to recalibrate a sensor system of the vehicle.

20. The method of claim 19, wherein each sensor calibration trigger of the set of sensor calibration triggers corresponds to one severity level.

Images