Method and system for vehicle speed estimation - Patents.com

Abstract

To provide an apparatus and a method for predicting a crash using a vehicle speed.SOLUTION: The method includes: receiving from a mobile device disposed within a vehicle a set of sensor measurements; converting the set of sensor measurements into a frequency domain; filtering with a bandpass filter the set of sensor measurements to eliminate high frequency sensor measurements from the set of sensor measurements; defining a set of contiguous windows on the basis of remaining sensor measurements in the set of sensor measurements, a set of features being generated by resampling the sensor measurements of the contiguous window at one or more predefined frequencies; generating a speed prediction by executing a trained neural network using the set of features; generating a vehicle crash prediction using the speed prediction; and transmitting to a remote device the vehicle crash prediction.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 888,182, filed August 16, 2019, and U.S. Patent Application No. 16 / 994,232, filed August 14, 2020, which are hereby incorporated by reference in their entireties for all purposes. [Background technology]

[0002]

[0002] Unless otherwise indicated in this specification, the material described in this section is not prior art to the claims of this application and is not admitted to be prior art by inclusion in this section.

[0003]

[0003] Mobile devices typically include a Global Positioning System (GPS) receiver and an Inertial Measurement Unit (IMU) with sensors such as a 3-axis accelerometer, a 3-axis gyroscope, and a 3-axis magnetometer to measure angular velocity, force, and magnetic field strength. A combination of GPS speed data and IMU measurements can be used to detect vehicle crashes with reasonable accuracy. However, relying on GPS data is undesirable because the GPS receiver consumes a lot of power on the mobile device, reducing its operating time. Summary of the Invention

[0004]

[0004] Embodiments of the present invention relate generally to estimating vehicle speed, and more particularly, to predicting crash events using vehicle speed predictions. [Means for solving the problem]

[0005] According to various aspects, a method for predicting a crash event using a vehicle speed prediction is provided. Some methods include receiving a set of sensor measurements from an accelerometer from a mobile device located in a vehicle, converting the set of sensor measurements to a frequency domain, filtering the set of sensor measurements with a bandpass filter to remove high frequency sensor measurements from the set of sensor measurements, defining a set of consecutive windows based on the remaining sensor measurements in the set of sensor measurements, where each consecutive window of the set of consecutive windows represents a consecutive portion of the remaining sensor measurements in the set of sensor measurements, generating a set of features by resampling the sensor measurements of the consecutive windows at one or more predefined frequencies for each consecutive window of the set of consecutive windows, generating a speed prediction by running a neural network trained with the set of features, generating a vehicle crash prediction using the speed prediction, and transmitting the vehicle crash prediction to a remote device.

[0006]

[0006] Another method includes receiving a set of sensor measurements from an accelerometer from a mobile device located in the vehicle; generating a set of successive windows using the set of sensor measurements, where each successive window in the set of successive windows represents a subset of the set of sensor measurements; generating a speed prediction for each successive window in the set of successive windows by running a neural network trained with the set of successive windows; receiving an indication that the vehicle has been involved in a crash event; authenticating the crash event using the speed prediction; determining that a crash event did not occur based on the authentication of the crash event; and deleting the crash event from memory in response to determining that a crash event did not occur.

[0007]

[0007] Another aspect of the present disclosure includes a system comprising one or more processors and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the first method and / or the second method described above.

[0008]

[0008] Another aspect of the present disclosure includes a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the first method and / or the second method described above.

[0009]

[0009] Various embodiments provide many advantages over conventional techniques. For example, various embodiments provide methods and systems that may be used to avoid excessive power consumption by a GPS receiver operating for vehicle crash detection in a mobile device. In some embodiments, the velocity estimates for crash detection are sensor measurements obtained from an inertial measurement unit, which consumes significantly less power from the mobile device than a GPS receiver. These and other embodiments, along with many of their advantages and features, are described in detail in conjunction with the following text and accompanying drawings.

[0010]

[0010] Aspects and features of various embodiments will become more apparent by describing each example with reference to the accompanying drawings. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a simplified block diagram illustrating an example of a system for collecting driving data in accordance with some aspects of the present disclosure. [Diagram 2] FIG. 2 is a simplified block diagram illustrating another example system for collecting driving data in accordance with some aspects of the present disclosure. [Diagram 3] 1 is a plot illustrating an example of vehicle speed prediction by a classifier using features extracted from an IMU signal sampled at 9 Hz, in accordance with various aspects of the present disclosure. [Figure 4]11 is a plot illustrating an example of vehicle speed prediction by a classifier using features extracted from an IMU signal sampled at 30 Hz, according to an embodiment of the present disclosure. [Diagram 5] 11 is a plot illustrating an example of vehicle speed prediction by a classifier using features extracted from an IMU signal sampled at 30 Hz, according to an embodiment of the present disclosure. [Figure 6] 4 is a flow diagram illustrating an example of a process for estimating vehicle speed according to an aspect of the present disclosure. [Figure 7] 11 is a plot illustrating an example of vehicle speed prediction by a classifier using features extracted from an accelerometer signal sampled at 30 Hz, in accordance with an embodiment of the present disclosure. [Figure 8] 11 is a plot illustrating an example of vehicle speed prediction by a classifier using features extracted from an accelerometer signal sampled at 30 Hz, in accordance with an embodiment of the present disclosure. [Figure 9] 1 is a cluster plot illustrating distance estimated by a classifier compared to distance measured by GPS, according to an embodiment of the present disclosure. [Figure 10] FIG. 2 is a block diagram illustrating an example of a collision prediction module according to aspects of the present disclosure. [Figure 11A] 1 is a plot illustrating an example of a collision prediction result according to an aspect of the present disclosure. [Figure 11B] 1 is a plot illustrating an example of a collision prediction result according to an aspect of the present disclosure. [Figure 11C] 1 is a plot illustrating an example of a collision prediction result according to an aspect of the present disclosure. [Figure 11D] 1 is a plot illustrating an example of a collision prediction result according to an aspect of the present disclosure. [Figure 12] 12 is an example representation of a graph data structure 1200 generated by the activity detection engine data, according to some embodiments. [Figure 13] FIG. 13 is an example block diagram of a system 1300 for predicting vehicle speed from mobile device sensor measurements, according to some embodiments. [Figure 14]1 is an example set of graphs illustrating division of processed sensor measurements into successive windows according to some embodiments. [Figure 15] 1 is an example representation of layers of a neural network that may generate a prediction of a vehicle's speed, according to some embodiments. [Figure 16] FIG. 1 illustrates an example block diagram of a system for predicting a velocity of a vehicle corresponding to a time step using sensor measurements of a mobile device, according to some embodiments. [Figure 17] 1 is a flow diagram of an example process for filtering a velocity prediction according to an aspect of the present disclosure. [Figure 18] 13 is a graph illustrating an example of filtering velocity predictions using "Sum of Sliding Window Gravity Angle Differences" in accordance with aspects of the present disclosure. [Figure 19] 1 is an exemplary graph illustrating the use of speed predictions to derive features for a machine learning model to generate a crash prediction, according to some embodiments. [Figure 20] FIG. 2 is a block diagram of a process for predicting a vehicle crash according to an aspect of the present disclosure. [Figure 21] 1 is a flow diagram of an example process for generating a crash prediction using a predicted speed of a vehicle, according to some embodiments. [Figure 22] FIG. 1 is a block diagram of a process for validating vehicle crash prediction using speed prediction, according to an aspect of the present disclosure. [Diagram 23] 1 is an example graph of predicted velocity over time, illustrating a correct-choice crash prediction, according to aspects of the present disclosure. [Figure 24] 1 is an example graph of predicted speed over time, illustrating false positive crash predictions, according to aspects of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012]

[0033] Although specific embodiments are described, these embodiments are presented by way of example only and are not intended to limit the scope of protection. The devices, methods, and systems described herein may be embodied in various other forms. Furthermore, various omissions, substitutions, and changes can be made in the form of the exemplary methods and systems described herein without departing from the scope of protection.

[0013]

[0034] FIG. 1 is a system diagram illustrating a system 100 for measuring acceleration of a device and detecting physical interactions, according to some embodiments. The system 100 includes a mobile device 104 with multiple processing, sensor, and communication resource components. The mobile device 104 may include a sensor data block 108, a data processing block 144, a data transmission block 164, and possibly a notification block 160. The sensor data block 108 includes data collected from sensors available to the mobile device 104, which may include external devices connected via Bluetooth, USB cable, etc. The data processing block 144 may include storage 156 that may include data from sensors in the sensor data block 108 that have been processed by the processor 122. Processing may include, but is not limited to, parsing, characterizing, manipulating, smoothing, subsampling, filtering, reformatting, etc. Examples of mobile devices include, but are not limited to, smartphones, tablets, laptops, application specific integrated circuits (ASICs), etc.

[0014]

[0035] The data transmission block 164 may process communications (e.g., transmitted and received communications) such as processed sensor data that is sent to an external computing device (e.g., a server 180), which may also store and / or process data obtained from the sensor data block 108. The server 180 may include its own processor 184 and storage 188.

[0015]

[0036] The notification block 160 may report the results of the analysis of the sensor data performed by the data processing block 144 to the user of the mobile device 104 via a display (not shown). For example, the notification block 160 may display or otherwise present a warning communication to the user of the mobile device 104 upon determining that the user may be an inattentive driver. In some examples, the determination of the physical interaction may be a process performed by the processor 148 of the mobile device 104. In other examples, the determination of the physical interaction may be a process performed by the processor 184, as further described herein with respect to FIG. 2.

[0016]

[0037] Some embodiments are described using examples in which driving data is collected using an electronic device 104, but these examples are not limited to a particular electronic device. For example, the electronic device may include various devices contained within or connected to the electronic device 104. Examples of electronic devices include, but are not limited to, devices having one or more position determining systems, such as a global positioning system (GPS) receiver 112, an accelerometer 116, a magnetometer 120, a gyroscope 124, a microphone 128, an external (sensor) device 132, a compass 136, a barometer 140, and communication capabilities. Exemplary electronic devices include smart watches, fitness monitors, Bluetooth headsets, tablets, laptop computers, smartphones, music players, motion analysis devices, and the like.

[0017]

[0038] One or more sensors of the mobile device 104 (e.g., sensors in the sensor data block 108) may operate to collect measurements to provide indications regarding physical interactions with the mobile device 104. In some examples, measurements may be collected when the electronic device is moving at a particular speed or when the device is located on a known road (e.g., a highway) as may be carried by a driver operating a vehicle. The sensors used to collect data may be components of the mobile device 104 and may use power resources available to the components of the mobile device 104 (e.g., battery power of the mobile device and / or data sources external to the mobile device 104).

[0018]

[0039] In some examples, mobile device settings can be used to enable various features described herein. For example, Apple iOS, Android OS, and / or wearable device operating systems with certain settings enabled can enable certain features of the embodiments. In some examples, enabling location services allows collection of location information (e.g., collected by a global positioning system (GPS) receiver 112) from the mobile device, and enabling background app refresh allows some embodiments to run in the background to collect and analyze driving data even when the application is not running. In some cases, location information can be determined by other sensors of the mobile device, such as by tracking the motion of the mobile device (e.g., using an accelerometer), by receiving location information from an external source radio triangulation (e.g., using a cellular, Bluetooth, or Wi-Fi radio), by the IP address of the mobile device, or by other means. In some implementations, physical processing can be performed in the background, so alerts are provided or displayed using notification block 160 while the app is running in the background.

[0019]

[0040] 2 is a simplified block diagram illustrating an example of another system 200 for collecting driving data according to some aspects of the present invention. The system 200 may comprise an electronic device 204 that may be incorporated within the mobile device 104 (e.g., as dedicated hardware or software) or may be a separate device (or execute on a separate device) that communicates with the mobile device 104. For example, as a separate device, the electronic device 204 may be a mobile device (e.g., the mobile device 104 of FIG. 1, a similar type of mobile device, a different type of mobile device, etc.), a computing device such as a server, a desktop or laptop computer, a dedicated processing device (e.g., one or more application specific integrated circuits, field programmable gate arrays, etc.), a distributed processing system (e.g., a cloud environment, etc.), a combination thereof (e.g., as a distributed process), etc. In some embodiments, the electronic device 204 may provide functionality using components including, but not limited to, a vector analyzer 208, a vector determiner 212, an external information receiver 216, a classifier 220 (e.g., a machine learning model), a data collection frequency engine 224, a driver detection engine 228, and an activity detection engine 232. Each component may include one or more processors (not shown) and a memory (not shown). Instructions stored in the memory of the component may be executed by the component's one or more processors to provide the functionality of the component. Alternatively, the one or more processors (not shown) of the electronic device 204 may execute instructions stored in a central memory of the electronic device 204 to provide the functionality of the component. The electronic device 204 may also include data storage 236. In some cases, one or more of the components 208-232 operating on the electronic device 204 may be stored in the memory 152 or storage 156 of the mobile device 104 and / or executed by the processor 148 of the mobile device 104.

[0020]

[0041] One or more of the sensors of the mobile device 104 (e.g., sensors of the sensor data block 108) are used to measure characteristics of the environment in which the mobile device is located. For example, one or more sensors are used to collect characteristics of the vehicle while the mobile device is located in the vehicle and while driving. In that case, the one or more sensors may operate while the mobile device is located in proximity to the driver during a time interval corresponding to when the driver is driving the vehicle. As used herein, the terms "driving" and "trip" refer to the movement of the vehicle over a time interval. Measurements obtained from the one or more sensors may be analyzed to determine the acceleration vector of the vehicle and various characteristics of the trip. In some cases, external data (e.g., weather, traffic, vehicle information, driver information) may be obtained and associated with the collected driving data.

[0021]

[0042] In some embodiments, a display of a mobile device (e.g., mobile device 104) may show a representation of driving data collected by one or more sensors or generated by any of components 208-232. For example, the representation of the driving data may be generated by converting the collected sensor data (e.g., driving data collected using sensor data block 108) into various results, including, but not limited to, an estimate of the activity of a user of mobile device 104 (e.g., stopping, walking, running, driving), an estimate of the occurrence of various driving events during the drive for which the data is being collected, a metric describing the driver's driving behavior during the drive, a metric describing the driver's driving behavior overall for all drives, a metric describing the driver's behavior associated with the occurrence of a particular event, and / or a combination of the converted driving data with geographic data.

[0022]

[0043] In some cases, the collected driving data can be analyzed to assign scores to the drive, drives, drivers, and / or driving actions based on various criteria. A scoring engine (not shown) can aggregate data collected by one or more sensors and apply one or more rules to generate a score for an embodiment. Detailed disclosure regarding scoring can be found in U.S. patent application Ser. No. 15 / 615,579, entitled "SYSTEMS AND METHODS FOR SCORING DRIVING TRIPS," filed Jun. 6, 2017 (the "'579 Application"), which is incorporated by reference herein in its entirety.

[0023]

[0044] The sensor data (e.g., sensor data collected using the sensor data block 108) may be used to analyze the motion of the mobile device to detect the occurrence of a driving event. The sensor data may be aggregated by the electronic device 204 and analyzed once a predetermined amount of sensor data is received. For example, the electronic device 204 may begin analyzing the sensor data once the electronic device 204 aggregates 50 megabytes of sensor data. In another example, the electronic device 204 may begin analyzing the sensor data once the electronic device 204 receives sensor data collected over a predetermined interval (e.g., 30 minutes of sensor data, 1 hour of sensor data, etc.). In yet another example, the electronic device 204 aggregates sensor data associated with a drive and analyzes the sensor data once all sensor data associated with the movement is received. Alternatively, the mobile device 104 may include one or more of the components 208-232 to provide analysis of the sensor data in real time (e.g., as one or more sensors take measurements).

[0024]

[0045] The GPS receiver may provide time-stamped position and speed data that may be used by various applications running on the mobile device. The time-stamped data may be used to accurately determine the vehicle's position and speed. The GPS receiver may detect collisions and determine the distance traveled by the vehicle. For example, the GPS receiver may detect a collision by detecting a sudden change in speed or position. However, because mobile devices operate with limited resources due to power and processing constraints and due to the high power consumption of GPS receiver operation, the electronic device 204 may use one or more other sensors of the mobile device 104 to detect the vehicle's position and / or speed.

[0025]

[0046] For example, a mobile device located in a vehicle experiences mechanical vibrations associated with vehicle activity. These vibrations can be measured using a subset of sensors in the sensor data block 108 of the mobile device 104 called the Inertial Measurement Unit (IMU). The mechanical vibration measurements can be made at various amplitudes and frequencies and can be used to identify vehicle activity or, in some cases, user activity. For example, some or all of the accelerometer, gyroscope, and magnetometer measurements can distinguish between a user's walking pattern and a vehicle's driving pattern (e.g., a vehicle speed of about 5 m / s).

[0026]

[0047] The IMU may comprise any of an accelerometer 116, a gyroscope 124, and a magnetometer 120. The IMU and the sensors contained therein may be separate units from the GPS receiver. The accelerometer 116 may be a three-axis accelerometer operable to measure longitudinal and lateral acceleration as well as acceleration due to gravity. The gyroscope 124 and magnetometer 120 may also be three-axis devices, measuring angular rotation and magnetic orientation in three dimensions, respectively. The IMU may combine the three-dimensional accelerometer data with the three-dimensional gyroscope data to determine the motion of the mobile device in six degrees of freedom (6DOF) (e.g., translation and rotation).

[0027]

[0048] In some cases, data obtained from the IMU may be filtered and used as input to train a classifier, such as classifier 220, to predict vehicle speed. Examples of such classifiers include, but are not limited to, an xgboost classifier. The classifier may be trained using features extracted from training data of a large number of driving trips. The extracted training features may include statistical features of the driving data, such as the median, variance, and maximum of the IMU signals (e.g., accelerometer, gyroscope, and magnetometer signals). In some cases, the orientation of the mobile device with respect to gravity may be determined and input to the classifier for training. Other statistical features may be used without departing from the scope of the present invention.

[0028]

[0049] During driving with the mobile device located within the vehicle, the IMU of the mobile device can be used to obtain motion measurements from any of the accelerometer, gyroscope, and magnetometer, which motion measurements can generate inputs for a classifier to predict vehicle speed. In some cases, the acceleration measurements used to predict vehicle speed can be acceleration measurements of a user. The acceleration measurements of a user can be acceleration measurements with the gravitational component of acceleration removed. In some cases, the acceleration measurements used to predict vehicle speed can be raw acceleration measurements. The raw acceleration measurements can be acceleration measurements that include a gravitational component.

[0029]

[0050] The motion measurement signals from the IMU sensors may be sampled at a designated sampling rate to obtain digital signals. In some cases, a sampling rate of 9 Hz may be used for the motion measurement signals. In other cases, a sampling rate of 30 Hz may be used for the motion measurement signals. Other sampling rates may be used, such as 50 Hz or another sampling rate. A higher sampling rate may improve speed estimation at the expense of increased resource consumption (e.g., processing resources and / or power resources). The electronic device 204 and / or mobile device 104 may vary the IMU sensor sampling in real time to optimize the amount of data collected and resource consumption (e.g., for accuracy of data analysis).

[0030]

[0051] For example, if the mobile device is connected to a reliable power source (e.g., a vehicle power source, etc.), the motion measurement signal may be sampled at the highest frequency (e.g., 50 Hz or a predetermined highest frequency). If the mobile device is not connected to a power source, the motion measurement signal may be sampled at a lower frequency (e.g., 30 Hz sampling or a predetermined intermediate frequency). If the mobile device's power source is below a threshold value (e.g., 25% of the maximum), the sampling of the motion measurement signal may be reduced to a lower frequency (e.g., 9 Hz or a predetermined low frequency) to conserve the remaining power of the mobile device. In some cases, the sampling rate of the motion measurement signal may be changed to improve the speed estimation. For example, an accuracy metric may be used to indicate the likelihood that a given speed estimation is valid. If the accuracy metric does not exceed the threshold, the sampling rate of the motion measurement signal may be temporarily or permanently increased until the accuracy metric exceeds the threshold. The mobile device may change the sampling rate in real time based on the operating conditions (e.g., resource consumption) or metrics of the mobile device.

[0031]

[0052] The filtered IMU signal can distinguish between driving patterns, stopping patterns, and user walking patterns. A bandpass filter (e.g., implemented in hardware or software), such as an infinite impulse response (IIR) filter, can be used to filter the IMU signal to isolate frequencies indicative of various vehicle activities and remove signal strength values ​​above a specified threshold. Portions of the signal having strength values ​​above a specified threshold may be excluded from further bandpass filtering. The digital bandpass filter can be designed to isolate the amount of vibration (i.e., frequency) occurring within a particular frequency range of interest. For example, the amount of vibration can be isolated into frequency ranges, such as 0.2 Hz to 1.1 Hz, 1.1 Hz to 2.0 Hz, etc., depending on the sampling frequency, by bandpass filtering the signal.

[0032]

[0053] Changes in low frequency bands, for example up to about 1 Hz, may contain information about a vehicle stopping, while changes in high frequency bands may correspond to a vehicle moving at high speed. The sources of vibration detected by the IMU sensor are complex interactions between engine vibrations due to speed changes, vibrations due to the vehicle interacting with the road surface at various speeds, etc. Machine learning models (e.g., classifiers) can learn these more complex interactions, which may be a combination of high and low frequencies corresponding to each vehicle's operation.

[0033]

[0054] In some cases, high intensity IMU sensor signals can adversely affect vehicle speed prediction. In those instances, filtering can be used to remove high intensity signals. For example, signals greater than about 10 m / s 2 Accelerometer signal strength values ​​that exceed the threshold value or another threshold value and subsequent portions of the signal may be filtered out. Portions of the IMU signal below the signal strength values ​​that exceed the threshold value may be bandpass filtered using an IIR filter.

[0034]

[0055] The IIR filtering process can use forward-backward filtering, where a portion of the IMU signal is filtered normally (i.e., forward filtered), and the forward filtered signal is "reversed" in time and filtered again with an IIR filter (i.e., backward filtered) to generate a squared magnitude response. The IIR filter can better isolate the signal of interest and minimize or eliminate nonlinear phase distortion of the signal. The IIR filter is applied recursively, such that the result of the last step of the filter algorithm is applied to the next step. IIR filtering methods can be more computationally efficient than filtering methods that require the calculation of all intermediate numbers that lead to a result (e.g., Fourier transforms). IIR filters are also preferred because they can isolate frequency ranges of interest with large signal amplitude attenuation outside the range of interest. In some implementations, finite impulse response (FIR) filters can be used to bandpass filter the IMU signal rather than IIR filters.

[0035]

[0056] The number of frequency bands used for bandpass filtering may be determined by the desired granularity and sampling frequency of the sensor data. For example, for data acquired using 9 Hz sampling, 14 passbands may be used in equally spaced 0.3 Hz frequency bands from 0.2 Hz to the Nyquist sampling frequency of 4.5 Hz, while for data acquired using 30 Hz sampling, 28 passbands may be used from 0.2 Hz to 15 Hz. If the IMU signals are sampled at a higher sampling frequency, more granular frequency bands may be used. The selection of the number and width of frequency bands may be determined based on the desired signal quality and granularity of information for each band. For example, too many frequency bands may result in reduced signal quality due to narrow bandwidth, while too few frequency bands may result in a loss of granularity of the captured information.

[0036]

[0057] Features, e.g., statistical features, may be extracted from some or all of the filtered signals. The features used as input to the classifier 220 may be summary statistics (e.g., median, variance, and maximum) over various signals covering different time spans. Features may be extracted from time windows of different lengths. In some implementations, each of the statistical features may be extracted from the IMU signals over a 5 second time window, a 10 second time window, and a 20 second time window. The center of each window may be placed at the time point under consideration. For each of the windows, summary statistics such as mean, median, variance, maximum, and minimum after various bandpass processing of the IMU sensor signals (e.g., accelerometer, gyroscope) included in these windows may be calculated.

[0037]

[0058] Windows of different lengths provide multiple levels of stability to the feature values, with longer window times producing more stable feature values. Other window lengths or different numbers of windows can be used without departing from the scope of the invention. For example, in some implementations, a single window may be used. For a 0.2 Hz to 1.1 Hz bandpass filtered accelerometer signal, nine features, e.g., median, variance, and maximum, may be extracted for each feature over a 5 second time window, a 10 second time window, and a 20 second time window. The feature extraction produces a single value list (e.g., feature vector) for each time point considered.

[0038]

[0059] The extracted features (e.g., feature vector) may be input to a classifier. The machine learning model (e.g., classifier) ​​may then make a speed prediction based on the feature vector input. The vehicle speed prediction by the classifier may be quantized, for example, in increments of 5 m / s or another increment. In some implementations, the orientation of the mobile device with respect to gravity may be determined and input to the classifier.

[0039]

[0060] The activity detection engine 232 identifies an activity corresponding to a sensor measurement received from one or more sensors of the sensor data block 108. For example, the activity detection engine 232 identifies when the mobile device 104 is stationary, carried by a user walking, carried by a user running, carried by a user biking, in a vehicle driving, in a vehicle moving at high speed, etc. In some cases, the activity detection engine 232 outputs a probability of an activity. In those examples, the activity detection engine 232 may output two or more probabilities, such as a 45% probability that the mobile device is walking, a 33% probability that the mobile device is driving, and a 22% probability that it is engaged in some other activity. The probabilities may be expressed as integers or real numbers, percentages, grades (such as low, medium, or high), or any manner that represents the probability of a given activity.

[0040]

[0061] The activity detection engine 232 may use the activity to detect driving from the sensor data. For example, the activity detection engine 232 may analyze data received from the mobile device 104 and identify a first time where the activity indicates a high probability that the mobile device 104 is in a vehicle while driving. The activity detection engine 232 may identify a second time after the first time where a high probability of another activity (e.g., stopping, walking) occurs. The activity detection engine 232 then defines driving as occurring from the first time to the second time. Other components of the electronic device 204 may then further analyze the sensor data received between the first time and the second time to identify driver behavior, driver score, crash detection, speed estimation, etc. In some cases, the activity detection engine 232 or any of the operations described in connection with the activity detection engine 232 may be performed by an operating system of the mobile device to manage data collection by the sensor data block 108.

[0041]

[0062] In some cases, the activity detection engine 232 may operate on the mobile device 104 to control the collection of measurements from the sensor data block 108. The mobile device 104 may execute a data collection application that controls the operation (e.g., sampling rate, etc.) of one or more sensors of the mobile device 104 and collects measurements from the one or more sensors. The data collection application may include one or more of the components 208-232. Because the mobile device operates with limited resources, the data collection application may be paused or terminated by a user of the mobile device due to the data collection application being inactive, such as when the mobile device is at rest. The activity detection engine 232 may operate in a background process to detect whether driving is occurring. If driving is occurring, the activity detection engine 232 may start the data collection application and begin collecting sensor data related to driving.

[0042]

[0063] In some cases, the activity detection engine 232 may generate a geofence around the mobile device 104 that, when crossed, causes the activity detection engine 232 to execute a data collection application or return the data collection application from a paused state to an active state. When the mobile device 104 crosses the geofence, the activity detection engine 232 may start the data collection application. For example, the geofence may surround a user's vehicle or residence such that crossing the geofence may be due to the user starting to drive. The geofence may be generated after a period of inactivity, such as when the mobile device has been stationary for a predetermined time interval. The geofence may be generated at a predetermined distance from the mobile device such that when the mobile device crosses the geofence, it may be due to the start of driving rather than other activities such as walking. The activity detection engine 232 may use other mechanisms to determine whether to activate the data collection application, including, but not limited to, detection of a visit (e.g., the mobile device being at a particular location), notifications, time intervals, one or more sensor readings exceeding a threshold, etc.

[0043]

[0064] Because the data collection application of the mobile device 104 cannot collect sensor measurements until it is running (or returns to an actively running state), some sensor measurements may be missed. For example, the data collection application may miss sensor measurements associated with a portion of a drive that occurred before the geofence was crossed. As a result, the data collection application may not collect sensor measurements for the entire drive, thereby missing valuable information about the drive, driver behavior, potential vehicle collisions, etc. In some cases, the mobile device 104 may not detect that a geofence has been crossed at all, thereby causing the data collection application to not be activated at all during the drive. In those instances, the mobile device 104 may miss a drive such that the data collection application does not collect sensor measurements associated with the missed drive. The data collection application may obtain some sensor measurements collected in the missed drive from other processes running on the mobile device 104.

[0044]

[0065] For example, the operating system of the mobile device 104 may collect and cache several sensor measurements over a sliding window, such as the previous time interval of a predetermined length. The sliding window may include the preceding 24 hours, 48 ​​hours, 72 hours, 96 hours, or any predetermined time interval. An application of the mobile device 104 may request and obtain sensor measurements from the operating system up to the length of the sliding window.

[0045]

[0066] The operating system begins collecting and caching sensor readings upon request by an application, such as a data collection application, and retains the cached sensor readings up to the length of the sliding window. At that point, the operating system discards the oldest sensor reading each time a new sensor reading is added. For example, the operating system may cache sensor readings up to the past 72 hours (e.g., 72 hours from a current time, such as now), at which point the oldest sensor readings (e.g., older than 72 hours) may be discarded so that sensor readings collected in the immediately preceding 72 hours are retained in the cache. In some cases, the operating system may only allow an application to request collection and caching of sensor readings for a specific time interval (e.g., which may be less than or equal to the length of the sliding window). If the specific time interval is shorter than the sliding window, the data collection application may not be able to request that the operating system cache the sensor readings for the entire sliding window. Instead, the data collection application generates a set of requests, with each subsequent request being sent at the end of the specific interval of the previous request. This allows the data collection application to request caching of sensor readings by the operating system for the entire sliding window.

[0046]

[0067] In the following example, the length of the sliding window may be 72 hours and the predetermined time interval may be 12 hours. When the data collection application runs (or returns from pause), it may generate an initial request for the operating system to collect and cache the next 12 hours of sensor readings. In response, the operating system may begin collecting and caching the sensor readings. The data collection application may then perform the intended operation that caused it to run (or return from pause) or terminate (or return to a paused state). When the 12-hour time interval ends, the data collection application may generate a second request to the operating system to run (or return from pause) and collect and cache another 12 hours of sensor data. In some cases, the data collection application may run before the 12-hour time interval ends. In that case, the application may generate a second request to the operating system to collect and cache sensor data for a 12-hour time interval beginning at the time of the second request (rather than the end of the previous 12-hour time interval). The data collection application may also access the sensor readings from the previous 12 hours (since the first request). This process may be repeated every 12 hours such that the operating system caches up to 72 hours of sensor measurements.

[0047]

[0068] The data collection application may access the cached sensor readings for the entire 72 hours (e.g., combining the requests for each 12 hours), even though the data collection application may be limited to sending requests for data collection and caching at a shorter time interval (e.g., 12 hours in this case). If the data collection application sends a first request for 12 hours of sensor readings (at time zero), when the data collection application runs (or returns from pause) 12 hours later, the operating system will collect and cache 12 hours of sensor readings that the data collection application may access. If the data collection application sends a second request to the operating system (at time 12) for caching an additional 12 hours of sensor readings, the operating system will continue to collect and cache the next 12 hours of sensor readings. When the data collection application runs 12 hours later (e.g., now at time 24), the data collection application now has access to 24 hours of sensor data, even though the data collection application only requests that the operating system collect and cache the next 12 hours of sensor readings.

[0048]

[0069] The data collection application may continue to request collection and caching of sensor readings even when the cache contains sensor readings over a 72 hour time interval (e.g., a complete sliding window). Because the cache can only store sensor readings over a sliding window (e.g., the past 72 hours), the oldest sensor readings (e.g., sensor readings older than 72 hours) may be discarded as new sensor readings are stored in the cache. Sensor readings may be continually discarded as new sensor readings are continually cached over the next requested time interval (e.g., 12 hours). Continued requests by the data collection application may cause the operating system to permanently cache the sensor readings for the past 72 hours.

[0049]

[0070] An application of the mobile device 104 comprising components 208-232 can request data collection by the operating system when the mobile device application (e.g., a data collection application) is paused or not running. The operating system can collect sensor readings over a predefined time interval. For example, the application can request sensor readings from the operating system up to 12 hours after the application is paused or terminated. When the application is run again, the application can request access to the sensor readings collected by the operating system while the application was paused or terminated.

[0050]

[0071] As previously described, the activity detection engine 232 can take sensor measurements collected by the operating system (or another application) of the mobile device and generate a probability of an activity associated with the mobile device. Alternatively, this can be performed by the operating system itself. For example, the operating system can output a probability that the mobile device 104 is stopped, walking, running, driving, cruising, etc. The activity detection engine 232 can use the activity data from the operating system to determine a time interval during which driving is likely to have occurred while the data collection application was paused or terminated (e.g., not running). The activity detection engine 232 can then request the sensor data collected by the operating system over the time interval. The sensor data collected by the operating system can be added to the sensor data collected by the data collection application.

[0051]

[0072] For example, the activity detection engine 232 can detect that the mobile device 104 has crossed a geofence and begin executing a data collection application to begin collecting sensor readings, such as from an IMU sensor. The data collection application then requests sensor data from the operating system for the time interval before the mobile device crossed the geofence. This allows the mobile device 104 to capture sensor readings for the entire duration of the drive, even if the application was run and began collecting sensor readings a few minutes before the drive.

[0052]

[0073] In another example, when a data collection application is executed, the data collection application requests sensor data from the operating system of the mobile device 104 for a time interval prior to execution of the data collection application. The data collection application identifies a first time interval from the activity data during which the operating system determines that driving likely occurred. The data collection application then requests sensor data collected by the operating system of the mobile device 104 for the first time interval. In some cases, there may be a delay between when driving begins and when the operating system detects the occurrence of a driving activity. Similarly, there may be a delay between when driving ends and when the operating system detects the end of driving. To ensure that sensor data for the entire trip is collected, the data collection application may request sensor data for a second (longer) time interval that begins before the first time interval (e.g., 1 minute, 5 minutes, 10 minutes, etc. before the first time interval begins) and ends after the first time interval (e.g., 1 minute, 5 minutes, 10 minutes, etc. after the first time interval ends).

[0053]

[0074] FIG. 3 is a plot 300 illustrating an example of vehicle speed prediction by a classifier using features extracted from IMU signals sampled at 9 Hz, according to an embodiment of the present disclosure. FIG. 3 illustrates accelerometer data 310 using user acceleration (i.e., with the gravity component of acceleration removed), and GPS vehicle speed data 320 of the movement, and a predicted vehicle speed 330 generated by the classifier. The IMU signals, specifically the accelerometer and gyroscope (not shown) signals, were sampled at a sampling rate of 9 Hz and forward and backward bandpass filtered with an IIR filter using 14 equally spaced frequency bands from 0.2 Hz to 4.5 Hz. Statistical features (e.g., median, variance, and maximum) were extracted from the filtered signals with three window sizes (20 s 340, 10 s 350, 5 s 360) and input to the classifier. As can be seen from FIG. 3, the predicted vehicle speed 330 generated by the classifier approximates the GPS vehicle speed data 320.

[0054]

[0075] 4 and 5 are plots 400, 500 illustrating examples of vehicle speed prediction by a classifier using features extracted from an IMU signal sampled at 30 Hz, according to aspects of the disclosure. FIG. 4 similarly illustrates accelerometer data 410 using user acceleration (i.e., with the gravity component of acceleration removed), gyroscope data 420, and GPS vehicle speed data of motion 430, as well as the predicted vehicle speed 440 generated by the classifier. FIG. 5 similarly illustrates accelerometer data 510 using user acceleration (i.e., with the gravity component of acceleration removed), gyroscope data 520, and GPS vehicle speed data of motion 530, as well as the predicted vehicle speed 540 generated by the classifier.

[0055]

[0076] The IMU signals, specifically the accelerometer and gyroscope signals, were sampled at a sampling rate of 30 Hz and forward and backward bandpass filtered with an IIR filter using 28 equally spaced frequency bands from 0.2 Hz to 14.8 Hz. Statistical features (e.g., median, variance, and maximum) were extracted from the filtered signals with three window sizes (20 s 440, 10 s 450, 5 s 460) and input to the classifier. Figures 4 and 5 show that a closer prediction of the vehicle speed is obtained when the IMU data is sampled at a higher sampling rate. Furthermore, as can be seen in Figure 4, the accelerometer signal 410 has a higher average speed of approximately 10 m / s 2 Thus, filtering of the IMU sensor signal may be performed on portions of the IMU signal that have signal intensity values ​​that exceed the threshold value within, for example, 2-5 seconds or another time range, but do not include signal intensity values ​​that exceed the signal threshold value or any subsequent value.

[0056]

[0077] Similarly, Figure 5 shows that the velocity of the 2 4 and 5, an accelerometer signal 510 is shown having an intensity value 550 that exceeds a threshold value. Thus, in Figures 4 and 5, only the portion of the IMU signal below the signal intensity value that exceeds the threshold value may be filtered using an IIR filter.

[0057]

[0078] 6 is a flow diagram illustrating an example of a method 600 for estimating vehicle speed according to an aspect of the disclosure. At block 610, it is determined that the mobile device is in the vehicle. For example, the accelerometer signal may be analyzed to determine the first accelerometer signal pattern consistent with vehicle motion after the last signal pattern consistent with a walking user. Detailed disclosure regarding determining that a user has entered a vehicle with a mobile device can be found in the '093 specification, which is incorporated herein.

[0058]

[0079] At block 620, sensors of the mobile device may operate to obtain measurements of motion. For example, the mobile device may include an IMU having an accelerometer, gyroscope, and magnetometer, among others. One or more IMU sensors (e.g., accelerometer, gyroscope, and magnetometer) may operate during driving to obtain measurements of motion while the GPS receiver is not active.

[0059]

[0080] A motion measurement signal may be received at block 630. A processor of the mobile device may receive signals corresponding to the motion measurements from one or more of an accelerometer, a gyroscope, and a magnetometer of the IMU.

[0060]

[0081] At block 640, it may be determined whether a value of one or more of the received signals exceeds a threshold value. In response to a determination (640-Y) that one or more of the received signals exceeds the threshold value, at block 650, it may be determined to exclude that portion of the received signal from subsequent filtering. The mobile device processor may determine to exclude from subsequent filtering the portion of the received signal within a specified time period around the portion of the received signal that exceeds the threshold value. The processor may cause the portion of the received signal up to the specified time to be filtered such that the portion of the received signal that exceeds the threshold value is excluded from subsequent filtering. For example, the processor may determine to exclude the portion of the received signal 0.5 seconds or another time period before or after the time the one or more of the received signals exceeds the threshold value. The time period may be determined based at least in part on the sampling frequency, with shorter time periods being applied for higher sampling frequencies.

[0061]

[0082] In some implementations, only the portion of the signal or signals preceding the time period may then be filtered. In some implementations, both the portion of the signal or signals preceding the time period and the portion of the signal or signals following the time period may then be filtered, effectively filtering out portions of the received signal that exceed the threshold value. The process may continue to block 660.

[0062]

[0083] In response to a determination (640-N) that one or more of the received signals do not exceed the threshold value, at block 660, the signals may be filtered.

[0063]

[0084] At block 660, the signal may be bandpass filtered with an IIR filter using a forward-backward filtering process. Forward-backward IIR filtering filters the IMU signal normally (i.e., forward filtering), then the forward filtered signal is "reversed" in time and filtered again with an IIR filter (i.e., backward filtering) to generate a squared magnitude response. The IIR filter may better isolate the signal of interest and minimize or eliminate nonlinear phase distortion of the signal. IIR filters are applied recursively, meaning that the result of the last step of the filter algorithm is applied to the next step. IIR filters are also preferred because they can isolate frequency ranges of interest with large signal amplitude attenuation outside the range of interest. Digital bandpass filters may be designed to isolate the amount of vibration occurring within a particular frequency range of interest. In some implementations, finite impulse response (FIR) filters may be used to bandpass filter the IMU signal rather than IIR filters.

[0064]

[0085] At block 670, features may be extracted from the filtered signal. Features, e.g., statistical features, may be extracted from some or all of the filtered signal. Features may be extracted from time windows of different lengths. In some implementations, each of the statistical features may be extracted from the IMU signal over a 5 second time window, a 10 second time window, and a 20 second time window. Windows of different lengths provide multiple levels of stability to the feature values, with longer window times producing more stable feature values. Other window lengths or different numbers of windows may be used without departing from the scope of the invention. For a 0.2 Hz to 1.1 Hz bandpass filtered accelerometer signal, nine features, e.g., median, variance, and maximum, may be extracted for each feature over a 5 second time window, a 10 second time window, and a 20 second time window.

[0065]

[0086] At block 680, a classifier may be executed to predict the vehicle speed. The features extracted from the filtered IMU signal may be input to a classifier, such as an xgboost classifier. The extracted features may be, for example, median, variance, and maximum. Using the features extracted from the filtered IMU signal, the classifier may predict the speed of the vehicle. The vehicle speed prediction by the classifier may be quantized, for example, in increments of 5 m / s or another increment. In some implementations, the orientation of the mobile device relative to gravity may also be determined and input to the classifier. Detailed disclosures regarding determining the orientation of a mobile device relative to gravity may be found in the '099, '157, and '446 patents listed and incorporated above.

[0066]

[0087] In some implementations, the method of FIG. 6 may be used with only the accelerometer signal. For example, the processor may receive signals from various IMU sensors (e.g., accelerometer, gyroscope, and magnetometer), but only the accelerometer signal may be further processed to predict vehicle speed. The motion measurement signal from the accelerometer may be sampled at a specified sampling rate to obtain a digital signal. A digital bandpass filter, for example an infinite impulse response (IIR) filter, may be used to filter the accelerometer signal to isolate frequencies indicative of various vehicle activities and remove signal strength values ​​above a specified threshold. Portions of the accelerometer signal having strength values ​​above a specified threshold may be excluded from further bandpass filtering. The digital bandpass filter may be designed to isolate the amount of vibration (i.e., frequencies) occurring within a particular frequency range of interest.

[0067]

[0088] The IIR filtering process may use forward-backward filtering, where a portion of the accelerometer signal is filtered normally (i.e., forward filtered), and the forward filtered signal is "reversed" in time and filtered again with an IIR filter (i.e., backward filtered) to produce a squared magnitude response. Features, e.g., statistical features, may be extracted from the filtered accelerometer signal and provided as input to a classifier for predicting vehicle speed.

[0068]

[0089] In some implementations, the acceleration measurements used to predict the vehicle speed may be the acceleration measurements of the user. The acceleration measurements of the user are defined as acceleration measurements with the gravitational component of the acceleration removed. In some implementations, the acceleration measurements used to predict the vehicle speed may be raw acceleration measurements. The raw acceleration measurements are defined as acceleration measurements that include a gravitational component.

[0069]

[0090] 7 and 8 are plots 700, 800 illustrating examples of vehicle speed prediction by a classifier using features extracted from an accelerometer signal sampled at 30 Hz, according to aspects of the disclosure. Fig. 7 shows accelerometer data 710 using user acceleration (i.e., with the gravity component of acceleration removed), and GPS vehicle speed data 730 of the movement, and the predicted vehicle speed 740 generated by the classifier. Similarly, Fig. 8 shows accelerometer data 810 using user acceleration (i.e., with the gravity component of acceleration removed), and GPS vehicle speed data 830 of the movement, and the predicted vehicle speed 840 generated by the classifier.

[0070]

[0091] The accelerometer signal was sampled at a sampling rate of 30 Hz and forward and backward bandpass filtered with an IIR filter. Statistical features (e.g., median, variance, and maximum) were extracted from the filtered signal and input to a classifier. As can be seen in FIG. 7, the accelerometer signal 710 exhibited a velocity of approximately 10 m / s 2 8 has an intensity value 750 that exceeds the threshold value of about 10 m / s. Thus, filtering of the accelerometer signal may be performed on portions of the accelerometer signal that have intensity values ​​of the signal that exceed the threshold value, for up to, for example, a fraction of a second, but do not include intensity values ​​of the signal that exceed the threshold value or any subsequent values ​​of the signal. 2 7 and 8, an accelerometer signal 810 is shown having an intensity value 850 that exceeds a threshold value of 100. Thus, in Figures 7 and 8, only the portion of the accelerometer signal that is less than the intensity value of the signal that exceeds the threshold value may be filtered using an IIR filter.

[0071]

[0092] It should be appreciated that the particular operations illustrated in FIG. 6 provide a particular method for estimating vehicle speed according to an embodiment of the present invention. Other sequences of operations may be performed according to alternative embodiments. For example, alternative embodiments may perform the operations outlined above in a different order. Additionally, each operation illustrated in FIG. 6 may include multiple sub-operations that may be performed in various orders depending on the individual operation. Additionally, additional operations may be added or removed depending on the particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0072]

[0093] The method of FIG. 6 may be embodied on a non-transitory computer-readable medium, known to those skilled in the art, having stored therein a program including computer-executable instructions that cause a processor, computer, or other programmable device to perform the operations of the method.

[0073]

[0094] In some implementations, a mobile device may be aligned to a vehicle to predict longitudinal acceleration by a method similar to that of FIG. 6. Detailed disclosures regarding aligning a mobile device to a vehicle may be found in the incorporated '099 patent. In some implementations, other features, such as the median, variance, and maximum of velocity change derived through integration of accelerometer data, and the median, variance, and maximum of the derivative of acceleration (also known as jerk), may be extracted from the bandpass filtered data and included as inputs to a classifier to predict vehicle speed and other vehicle motion characteristics. In some implementations, distance may be estimated by integrating the vehicle speed predicted by the classifier. FIG. 9 is a cluster plot 900 illustrating distance estimated by a classifier 910 compared to distance measured by GPS 920, according to aspects of the present disclosure.

[0074]

[0095] According to some aspects of the present disclosure, vehicle mileage may be calculated from IMU sensor data collected in the background. Accelerometer data may be collected in the background while the vehicle is moving. The accelerometer data is downsampled to a frequency lower than the sampling frequency, for example 9 Hz or another frequency, and the magnitude of the downsampled signal is calculated (e.g., by a processor of the mobile device).

[0075]

[0096] The downsampled signal magnitudes may be segmented into fixed size time windows and input into a machine learning model (e.g., a classifier) ​​to predict the distance traveled in each segment. The predicted distances traveled in each segment may be combined to determine the predicted distance traveled for that movement.

[0076]

[0097] In some implementations, the predicted distance traveled (i.e., mileage) may be accumulated for all vehicle movements over a specified period of time, such as a year or another period, to determine the total mileage traveled during that period.

[0077]

[0098] Vehicle speed estimation based on IMU signals and acceleration-only signals has applications in areas such as driving data collection, traffic analysis, and crash prediction, among other areas.

[0078]

[0099] In some embodiments, a vehicle crash may be predicted based solely on signals generated by the IMU. Crash detection based solely on IMU signals has an advantage over GPS-based crash detection, at least in that the IMU does not consume as much power on the mobile device as a GPS receiver. For example, IMU data may be collected in the background whenever the mobile device is in operation. In addition, the sampling frequency of the IMU sensor may be controlled based on the detected sensor activity. For example, a low sampling frequency used during steady state conditions may be increased when a sudden change in acceleration or change in direction is sensed. On the other hand, obtaining GPS data requires a GPS application to be launched and running on the mobile device, consuming additional power.

[0079]

[0100] According to aspects of the present disclosure, information typically obtained from GPS data may be inferred from IMU data. For example, IMU data may be used to determine whether a vehicle has been stationary for a particular period of time long enough to qualify as an event where the GPS measurements identify the vehicle as being stationary for that time.

[0080]

[0101] 10 is a block diagram illustrating an example of a collision prediction module 1000 according to an aspect of the disclosure. With reference to FIG. 10, the collision prediction module 1000 may include an event time filter 1010 and an IMU collision model 1020. Both the event time filter 1010 and the IMU collision model 1020 may be machine learning models. The event time filter 1010 may be trained using IMU sensor data to identify event times that may have a higher likelihood (e.g., as determined from historical data) of being indicative of a collision than adjacent times.

[0081]

[0102] Data from all times recorded by the IMU sensor may be received by the collision prediction module 1000 as event times 1005. The event times 1005 may include IMU sensor data acquired during a specified time window. The event time filter 1010 may identify candidate event times. For example, the event time filter 1010 may filter candidate event times based on a certain threshold magnitude (e.g., 10 m / s 2 Candidate event times may be identified as time windows that include accelerometer data that exceeds a threshold (or another magnitude). These candidate event times may have a higher likelihood of being indicative of a crash than adjacent times.

[0082]

[0103] The event time filter 1010 can further filter the candidate event times by evaluating the IMU sensor data within a time window around the candidate event time to determine whether various criteria are met. For example, using IMU data, the event time filter 1010 can provide an indication equivalent to a GPS indication of a vehicle speed exceeding a specified speed (e.g., 5 m / s or other speed) for a specified time (e.g., 2 minutes or other time). As another example, the event time filter 1010 can provide an indication equivalent to a GPS indication that the vehicle traveled a specified distance (e.g., 250 meters or other distance) in a specified time (e.g., 3 minutes or other time). Other indications equivalent to a GPS indication can be provided without departing from the scope of the present disclosure. Candidate event times and associated data that meet the specified criteria can be filtered out by the event time filter 1010. By removing candidate event times that are unlikely to be crash indicators, the results of the IMU crash model can be improved (e.g., reduced false positives).

[0083]

[0104] The event time filter 1010 can output filtered event times 1015 that meet specified criteria as input to the IMU crash model 1020. Using the filtered event times 1015, the IMU crash model 1020 can generate a prediction 1025 of whether or not the vehicle has been involved in a crash.

[0084]

[0105] 11A-11D are plots illustrating example crash prediction results according to embodiments of the present disclosure. In FIG. 11A-11D, the horizontal axis shows false positive predictions per 12,000 miles driven, and the vertical axis shows the rate of crashes that can be detected as a function of false positive predictions. In the United States, 12,000 miles is the average annual mileage, and thus the plots in FIG. 11A-11D can also be viewed as showing the rate of crashes that can be detected as a function of false positive predictions per year (e.g., the number of false predictions made in a driver-year).

[0085]

[0106] In FIGS. 11A-11D, plots 1110A-1110D represent crash prediction results generated by the IMU crash model 1020 without filtering the event times by the event time filter 1010. In other words, all event times generated by the IMU are used as input to the IMU crash model 1020. Plots 1120A-1120D represent crash prediction results generated by the IMU crash model 1020 with the event time filter 1010 having precise knowledge of which candidate event times should be filtered out. The filtered event times fed to the IMU crash model 1020 are based on GPS data known to be indicative of event times associated with crash events. Thus, plots 1120A-1120D show predictions by the IMU crash model 1020 based only on event times known to be associated with crashes, resulting in the best possible crash prediction.

[0086]

[0107] Plots 1130A-1130D represent crash prediction results generated by the IMU crash model 1020 with the event time filter 1010 filtering the event times recorded by the IMU sensors and providing the filtered event times to the IMU crash model 1020. In Figures 11A and 11B, the candidate event time thresholds for plots 1130A and 1130B are 10 m / s 2 In Figures 11C and 11D, the candidate event time threshold for plots 1130C and 1130D was set at 5 m / s 2 Set with a larger acceleration spike.

[0087]

[0108] 11A-11D show that the effect of including the event time filter 1010 to filter the event time input to the IMU crash model 1020 is to increase the crash prediction accuracy of the IMU crash model 1020. As shown by plots 1130A-1130D, by including the event time filter 1010, the crash prediction accuracy of the IMU crash model 1020 (e.g., plots 1110A-1110D) only approaches the best possible crash prediction accuracy (i.e., plots 1120A-1120D). Note that the expanded x-axis scales of FIGS. 11B and 11D show that when the event time filter 1010 is used to filter the event time (i.e., plots 1130A-1130D), the prediction of false positives per 12,000 miles driven is much less than 1, approaching the best possible crash prediction (i.e., plots 1120A-1120D).

[0088]

[0109] FIG. 12 is an exemplary representation of a graph data structure 1200 generated by an activity detection engine data according to some embodiments. A data collection application running on a mobile device collects measurements from one or more sensors (e.g., sensors from the sensor data block 108 of FIG. 1, etc.). During execution, the data collection application may consume resources of the mobile device (e.g., processing, network bandwidth, power, etc.). In some cases, the data collection application may only be intended to capture sensor data corresponding to a certain activity type (e.g., driving, etc.). Capturing sensor data for other activities (e.g., when the mobile device is stopped) may waste resources of the mobile device. To reduce the resource consumption of the mobile device applications, the operating system of the mobile device may capture and cache sensor data from some or all of the mobile device's sensors over a predetermined time interval (e.g., 12 hours). When an application such as a data collection application runs (or returns from a paused state), the application may request sensor data collected by the operating system over the preceding predetermined time interval (e.g., up to the past 12 hours) during which the collection application was not running.

[0089]

[0110] The operating system of the mobile device may also generate probabilities of concurrent activities of the mobile device over a predefined time interval. The operating system may generate probabilities that the mobile device is stopped, carried by a user walking, carried by a user running, carried by a user biking, carried by a user driving, carried by a user traveling at a high speed, etc. The activity detection engine may access the activity data generated by the operating system over a predefined time interval and use the activity data to identify potentially missed drives (e.g., drives for which the data collection application did not obtain sensor data). For example, if the data collection application is paused or not running and fails to detect that the mobile device crossed a geofence (if a geofence is generated), the data collection application may not collect or analyze sensor data from the drive. Instead, the activity detection engine may identify missed drives from the activity data generated by the operating system and collect the portion of the sensor data collected by the operating system while the data collection application was paused or not running.

[0090]

[0111] In some cases, the data collection application may request that the operating system collect sensor data. The activity detection engine may indicate the time interval for which sensor measurements are requested (e.g., up to a maximum allowed by the operating system, such as 12 hours). The operating system may then cache the sensor data for the time interval during which the data collection application is not running (or is paused). When the data collection application runs (or returns from pause), the activity detection engine accesses the activity data collected by the operating system (e.g., via an application programming interface exposed by the operating system). The data collection application may also generate new requests to the operating system to collect sensor data at subsequent time intervals, such that sensor data is always collected by either the data collection application (at run time) or the operating system (when the data collection application is not running).

[0091]

[0112] The activity detection engine can generate a graph data structure using activity data received from the operating system. As previously described, the activity data includes, but is not limited to, the probability that the mobile device is stopped, walking (e.g., the mobile device is carried by a user who is walking), driving low (e.g., a low probability that the mobile device is carried by a user who is driving), driving (e.g., a medium probability that the mobile device is carried by a user who is driving), driving high (e.g., a high probability that the mobile device is carried by a user who is driving), cycling (e.g., the mobile device is carried by a user who is cycling), etc. Any activity can also be represented by the graph data structure, such as those enumerated by a user (e.g., via user input), those detected by another application on the mobile device, or any other activity that can be detected by analysis of sensor measurements. The activity detection engine can receive an identification of any amount of activity over a preceding time interval. The activity detection engine can obtain an indication of activity at regular intervals (e.g., by polling the operating system), such as every 30 minutes, every hour, etc. Alternatively, the activity detection engine may receive an indication of activity whenever the operating system detects new activity (eg, a push notification from the operating system).

[0092]

[0113] In some cases, the activity detection engine can receive a probability (e.g., confidence that an activity is occurring) for each of multiple activities. In these examples, the activity detection engine can represent the most probable activities in the graph data structure. Alternatively, the activity detection engine can represent only activities with probabilities above a threshold. For example, the activity detection engine can represent only activities with probabilities above 40% in the graph data structure. If there are no activities above the 40% threshold, the graph data structure may not include the activity at that time (e.g., the activity may be represented as a null value). Alternatively, the data collection engine can represent all activities in the graph data structure with a corresponding probability for each activity.

[0093]

[0114] The graph data structure 1200 includes a time interval preceding a 12-hour wake event (e.g., the time a data collection application is executed or re-awakened from pause). The wake event may be any predefined event, such as, but not limited to, a mobile device crossing a geofence (or any particular boundary), a visit notification, a notification from a notification service (e.g., known as a "push notification"), a scheduled time, detection of sensor data indicative of movement, etc. A visit may be a notification from the operating system of the mobile device indicating that the mobile device is located at a location long enough for the operating system to determine that the mobile device is "visiting" the location. A visit may correspond to any location where the mobile device is located for longer than a threshold time interval. For example, the location may correspond to an institution (e.g., a business, a public institution, a residence, etc.). The activity detection engine receives hourly activity data for the preceding 12-hour time interval, and the highest probability activity (if multiple activities are received) is presented in the graph data structure. For example, at minus 12 hours (e.g., 12 hours prior to execution of the data collection application), activity 1204 corresponds to stopped, at minus 11 hours activity 1208 corresponds to walking, at minus 3 hours activity 1212 corresponds to driving, and at minus 9 hours activity 1216 corresponds to driving high.

[0094]

[0115] The activity detection engine can use the activity data of the graph data structure 1200 to identify missed driving. For example, the activity detection engine identifies a first time 1220, which is the earliest time that a driving high activity was detected. The activity detection engine identifies activity 1216 as the activity corresponding to the first time 1220. In some cases, the activity detection engine identifies the earliest time of any driving activity (e.g., medium or high). In the graph data structure 1200, the first time 1220 is detected minus 9 hours (e.g., 9 hours before the execution of the data collection application). The activity detection engine then identifies a second time 1224, which is the earliest time after the first time that a different activity was detected. For example, the activity detection engine identifies activity 1228 as a next activity that is not a driving activity. The different activity can be any activity other than a driving activity, such as walking or stopping, in the example described by the graph data structure 1200. In some cases, the activity detection engine identifies a second time period that corresponds to an activity other than driving high (e.g., driving, walking, cycling, or stopped). For example, the second time period may correspond to a driving activity even if the probability of the driving activity occurring is medium, low, or zero. The activity detection engine then identifies missed driving that occurred over a time interval from minus 9 hours to minus 6 hours, inclusive.

[0095]

[0116] The activity detection engine may then identify whether another missed trip occurred by identifying a third time, which is the earliest time after the second time that driving activity was detected, and a fourth time, which is the earliest time after the third time that non-driving activity was detected. This process may continue until all missed driving is identified. Although the process of identifying missed trips begins with an analysis from the earliest time (e.g., minus 12 hours) to the reactivation event, the process may operate by analyzing the activity data from the reactivation event toward the earliest time.

[0096]

[0117] The data collection application may receive sensor data from the operating system for the time interval during which the missed driving occurred (e.g., from minus 9 hours to minus 6 hours as shown in graph data structure 1200). This prevents the data collection application from having to analyze sensor data that corresponds to a time interval that may not correspond to a driving.

[0097]

[0118] 13 is an example block diagram of a system 1300 for predicting a vehicle's speed from sensor measurements of a mobile device, according to some embodiments. The system 1300 generates one or more predictions of the vehicle's speed from sensor measurements received by a mobile device disposed within the vehicle. In some cases, the process described in FIG. 13 is performed in real-time by the mobile device as the mobile device receives sensor measurements from sensors of the mobile device. In other examples, the process may be performed partially by the mobile device and partially by one or more other devices (e.g., a server, a mobile device, a computing system, etc.), or completely by one or more other devices.

[0098]

[0119] The system 1300 receives raw sensor measurements 1304 from one or more sensors (e.g., accelerometer, magnetometer, etc.) of the mobile device. The raw sensor measurements may include sensor measurements in one or more axes (e.g., x-axis, y-axis, and / or z-axis). The system 1300 also receives activity data 1308 including a prediction of an activity occurring at approximately the same time that the raw sensor measurements 1304 are received. For example, an operating system (or activity detection engine) may determine a probability that the mobile device is located in a vehicle currently moving. The raw sensor measurements 1304 and the activity data 1308 are input to a processing pipeline 1312. The processing pipeline 1312 includes four stages: frequency filtering 1316, normalization of filtered measurements 1320, definition of successive windows 1324, and resampling 1328. The output of the processing pipeline 1312 may be input to a neural network (or other machine learning model) that generates a prediction of the vehicle's speed.

[0099]

[0120] Frequency filtering 1316 includes filtering out portions of data values ​​that occur at specific frequencies. For example, the raw sensor measurements may be converted to the frequency domain using a transform (e.g., a Fourier transform, etc.). Once in the frequency domain, the raw sensor measurements may be filtered to remove noise or data values ​​that are less accurate in predicting the vehicle speed. Frequency filtering 1316 filters out frequencies that are higher than a threshold frequency. In some cases, the threshold may be a frequency between 9 and 50 Hz, for example. The frequency threshold may be selected by a process of system 1300, a user, a communication received from a remote device, etc. Processing pipeline 1312 then passes the remaining raw sensor measurements to filtered measurement normalization 1320, where the remaining raw sensor measurements are normalized to a predetermined scale.

[0100]

[0121] The processing pipeline 1312 then defines successive windows 1324 each including a portion of the normalized sensor measurement. The successive windows include a portion of the normalized sensor measurement (in the time domain) received during a time interval. The length of the time interval may be dynamically determined based on the normalized sensor measurement. For example, a median delta may be defined as the median difference in time between two successive values ​​of the normalized sensor measurement. Successive windows may be defined each time the time interval between the two successive values ​​is greater than a threshold based on the median delta. In some cases, the threshold may be a multiple of a standard deviation from the median delta, such as 1 standard deviation, 2 standard deviations, 3 standard deviations, etc. In other examples, the threshold may be selected by a user or from an input received from another device. Any number of successive windows may be defined for the normalized sensor value, including 1 (e.g., when the time difference between each successive value of the normalized sensor measurement is less than or equal to the threshold 1412) or any number greater than 1.

[0101]

[0122] Alternatively, defining successive windows 1324 may generate uniform windows. For example, each successive window may be a window of a predetermined time interval length, such as, but not limited to, 5 seconds, 10 seconds, 30 seconds, 1 minute, etc. The length of the predetermined time interval may be, for example, predetermined (e.g., before run time), dynamically selected (any time after run time), selected based on user input, selected based on a value received from a remote computing device.

[0102]

[0123] The final stage of the processing pipeline 1312 includes resampling 1328 where successive windows are resampled to a predetermined frequency. The predetermined frequency may be a frequency selected by the processing pipeline 1312, the system 1300, a mobile device, a user, a remote device, etc. In some cases, the predetermined frequency may be a frequency between 9 and 50 Hz.

[0103]

[0124] The output from the processing pipeline 1312 may be processed sensor measurements corresponding to each successive window. The processed sensor measurements corresponding to the successive windows are passed to a neural network 1332 that generates a predicted speed 1336 for each time unit (e.g., defined based on the median delta, by user input, or as a fraction of the total time interval of the successive windows) included in the successive windows. For example, if the successive windows include 10 seconds of processed sensor measurements and the time unit is 1 second, the successive windows include 10 time units and the neural network 1332 may output 10 predicted speed values. If the successive windows are not of uniform length (but based on a time difference between two values ​​that is greater than a multiple of a standard deviation from the time delta), the neural network 1332 may generate a different amount of predictions for each successive window.

[0104]

[0125] FIG. 14 is an exemplary set of graphs illustrating division of processed sensor measurements into successive windows, according to some embodiments. Graph 1404 includes a plot of normalized acceleration values ​​with the x-axis representing time. Graph 1408 includes a plot of the median difference between two successive values ​​of the normalized acceleration values ​​versus time. As a result, each set of successive values ​​plotted in graph 1404 has a corresponding value plotted in graph 1408. Graph 1404 includes a first successive window that may be defined before the start of the plot (e.g., at some point before time 7800). The first successive window extends until the time difference between successive values ​​of the normalized acceleration values ​​exceeds a threshold value 1412. For example, at 1416, there is a time difference between successive values ​​significantly greater than the threshold value 1412 at time 8050, signifying the end of the first successive window and the start of the second successive window 1420. At 1424, there is another time gap between consecutive values ​​significantly greater than the threshold 1412 at time 8150, signifying the end of the second consecutive window 1420 and the beginning of a third consecutive window. In some cases, the graphs 1404 and 1408 may be rendered or displayed. In other examples, the graphs 1404 and 1408 may not be rendered or displayed. In still other examples, the graphs 1404 and 1408 may be represented as metadata, data, data structures, and the like. Additionally, while the graphs 1404 and 1408 illustrate segmenting x-axis normalized acceleration values ​​into consecutive windows, other inputs may be used in addition to or instead of the x-axis normalized acceleration values, such as, but not limited to, y-axis normalized acceleration values, z-axis normalized acceleration values, normalized acceleration magnitude, activity data, and the like.

[0105]

[0126] FIG. 15 is an exemplary representation of layers of a neural network that may generate a prediction of a vehicle's speed, according to some embodiments. The neural network includes an input layer 1504 (e.g., a layer where features are input to the neural network), an output layer 1508, and one or more hidden layers disposed between the input layer and the output layer. Each layer includes one or more neurons that are respectively connected to one or more neurons of the previous layer (except the input layer) and one or more neurons of the subsequent layer (except the output layer). Each neuron of each hidden layer provides a nonlinear transformation of the features received from the previous layer. Weights can be assigned to the connections between neurons to dictate the transformation applied to the features.

[0106]

[0127] Neural network 1500 may be a convolutional neural network that includes one or more convolutional layers in addition to or instead of hidden layers. Each neuron in a convolutional layer includes a kernel defined by hyperparameters convolved over features received from a previous layer. In some cases, a convolutional layer is a dilated convolution where the kernel is applied over a larger region (of input features from a previous layer) than its length (e.g., the layer's distance from the input layer). Each subsequent layer may include further dilations. For example, the first convolutional layer may include a dilation of 1 where the kernel of neuron 1520 is applied over a region of 1 (e.g., one neuron from a neuron adjacent to neuron 1520). The kernel of neuron 1520 may be applied to neurons 1512 and 1516. The second convolutional layer may include a dilation of 2 where the kernel of neuron 1524 is applied over two regions from adjacent neurons 1520 of the previous layer. The kernel of neuron 1524 may be applied to neuron 1520 to neuron 1528.

[0107]

[0128] Further in this example, the next layer includes a dilation of 4, where the kernel of neuron 1532 is applied to a region of 4 adjacent neurons 1524 through 1536 of the previous layer. The kernel of neuron 1532 is applied to neurons 1524 through 1536. The final layer depicted has a dilation of 8. The kernel of neuron 1540 is applied to neurons 1532 through 1544 (e.g., 8 neurons from neuron 1532). Neural network 1500 can include any number of neurons (e.g., 64, 128, 256, etc.), each configured to apply a kernel. Neural network 1500 can include any number of dilations. For example, in addition to the dilations 1, 2, 4, and 8 shown, the neural network can include 16, 32, 64, etc., where each subsequent dilation is twice the dilation of the previous layer.

[0108]

[0129] Input features 1512 and 1516 correspond to sensor measurements received over time (e.g., from right to left is time (T) T=0 to time T=∞). For example, input feature 1512 may correspond to an accelerometer measurement collected at time T=0, and input feature 1516 may correspond to an accelerometer measurement collected at the next subsequent time interval T=1 (where the time between T=0 and T=1 may be any length of time, such as 1 second, 1 millisecond, etc.). Dilated convolution allows the neural network to include a larger receptive field of input features, allowing it to selectively use sensor measurements over longer time intervals. For example, a neural network such as neural network 1500 with 128 kernels and dilations of 1, 2, 4, 8, 16, and 32 includes a receptive field of 96. This means that there are 96 inputs to the neural network (e.g., 96 separate sensor measurements or sensor measurements collected over 10.6 seconds). This increases the efficiency of the neural network by improving both the speed and accuracy of predictions (e.g., speed predictions, collision predictions, etc.) generated by the neural network.

[0109]

[0130] FIG. 16 is an example block diagram of a system for predicting a vehicle's velocity corresponding to a time step using sensor measurements of a mobile device, according to some embodiments. A set of sensor measurements 1604 may be received as an input. For example, a mobile device, such as mobile device 104, collects a set of sensor measurements from a sensor of the mobile device while the mobile device is located in a vehicle. The set of sensor measurements includes one or more sensor measurements collected over a time interval (e.g., T0 to Tn), with each sensor measurement associated with a time T at which the measurement was collected. The set of sensor measurements may include one or more acceleration measurements of an x-axis, one or more acceleration measurements of a y-axis, one or more acceleration measurements of a z-axis, one or more acceleration magnitude measurements, an identification of a current activity from activity data for each time period for which there are acceleration measurements, a combination thereof, and the like. In some cases, in addition to acceleration measurements, the sensor measurements may include gyroscope measurements, magnetometer measurements, GPS measurements, a combination thereof, and the like.

[0110]

[0131] The pre-processing pipeline 1312 may receive the sensor measurements 1604 and generate a set of consecutive chunks 1608 of processed sensor measurements. Each consecutive chunk 1608 may correspond to processed sensor measurements corresponding to consecutive windows (e.g., as described in connection with FIGS. 13 and 14). For example, consecutive chunk 1 1608-1 may include processed sensor measurements of sensor measurements collected over a first consecutive window. These processed sensor measurements may include one or more values ​​of each input sensor type (e.g., x-axis acceleration measurement, y-axis acceleration measurement, z-axis acceleration measurement, acceleration magnitude, and activity data). Any number of consecutive chunks may be generated by the pre-processing pipeline 1312. Because each successive chunk includes processed sensor measurements of sensor measurements collected during successive windows, and because successive windows may be of varying lengths, the amount of sensor measurements in each successive chunk 1608-1 to 1608-n may vary (e.g., from a single value of each sensor type to a specific amount of values ​​of each sensor type).

[0111]

[0132] The neural network receives successive chunks as input and generates a predicted speed 1612-1 through 1612-n for the vehicle for each sensor measurement. For example, successive chunk 1 1608-1 includes sensor measurements for times T0 through T5, so the neural network 1332 may generate a predicted speed 1612-1 for the vehicle for each of T0 through T5. The neural network 1332 may then receive the next successive chunk 1608-2 (not shown) and generate a prediction 1612-2 for each time unit in that successive chunk. This process may continue generating predictions 1612-n until there are no more successive chunks for the neural network.

[0112]

[0133] FIG. 17 is a flow diagram of an example process for filtering velocity predictions according to aspects of the disclosure. The process includes generating a set of gravity vectors from raw accelerometer measurements, where each gravity vector in the set of gravity vectors corresponds to a time interval (block 1704). Each gravity vector in the set of gravity vectors may be one-dimensional (e.g., incorporating raw accelerometer measurements from a single axis, such as the x-axis, the y-axis, or the z-axis), two-dimensional (e.g., incorporating raw accelerometer measurements from two axes), or three-dimensional (e.g., incorporating raw accelerometer measurements from the x-axis, the y-axis, and the z-axis). Each gravity vector may be associated with a time at which the corresponding raw acceleration measurement was collected. The gravity vectors may be generated at predetermined periodic intervals (e.g., referred to herein as time steps), which may be based on the sampling rate of the accelerometer or determined by a user. For example, the gravity vectors may be generated every 0.5 seconds, every second, etc.

[0113]

[0134] In some cases, each gravity vector may be derived from the median of a set of raw acceleration signals. This may prevent signal noise from the accelerometer from propagating into the derived gravity vector. For example, a rolling window may be defined with a fixed length of time interval. The median of the accelerometer measurements collected over the time interval may be used to derive the gravity vector. For example, a sliding window may include 22 seconds (e.g., T=0 to 22) of raw accelerometer measurements, and a first gravity vector may be derived from the median of the accelerometer measurements collected over the 22 second sliding window. The sliding window may then be incremented such that the sliding window includes the most recent raw accelerometer measurements and the oldest raw accelerometer measurements are discarded (e.g., T=1 to 23, and the window no longer includes the accelerometer measurements collected at T=0, but includes the accelerometer measurements collected at T=23). A second gravity vector may be derived from the raw accelerometer measurements that fall within the sliding window. The sliding window may be incremented in real time, when new accelerometer measurements are received, at predetermined time intervals (eg, every 0.5 seconds, every second, etc.), etc.

[0114]

[0135] The process continues by calculating an angle between the current gravity vector and the previous gravity vector (e.g., calculating a "gravity angle difference") (block 1708). The process starts at a first gravity vector for which there is a previous gravity vector to calculate an angle between, and continues for each successive gravity vector in the set of gravity vectors until there are no more gravity vectors. In some cases, the previous gravity vector may be the gravity vector immediately before the current gravity vector. In this example, the process may start with the second gravity vector (as the current gravity vector) since there is a previous gravity vector to calculate an angle from. In other examples, the previous gravity vector may be a gravity vector at a predetermined distance from the current gravity vector, such as the gravity vector five before the current gravity vector, the gravity vector ten before the current gravity vector, etc.

[0115]

[0136] The process continues by determining whether there are more gravity vectors (e.g., there are gravity vectors derived after the current gravity vector in the set of gravity vectors) (block 1712). If there are more gravity vectors, the process continues by setting the current gravity vector to the next gravity vector in the set of gravity vectors (block 1716) and returning to block 1708. Blocks 1708-1716 form a loop through the set of gravity vectors. The current gravity vector (in block 1708) is initially set to the first (and oldest) gravity vector in the set of gravity vectors, and the calculation of the gravity angle difference between the current gravity vector and the previous gravity vector is repeated through the set of gravity vectors until there are no more gravity vectors in the set of gravity vectors. Once the loop is completed, a gravity angle difference value is available for each time step (e.g., the amount of gravity angle difference values ​​may be approximately equal to or less than the amount of gravity vectors in the set of gravity vectors).

[0116]

[0137] The process continues by calculating a running sum over the gravity angle difference within a sliding time window of a predetermined length (block 1720). For example, the sliding window may be 10 seconds, 15 seconds, 20 seconds, etc. In this process, a sum of the gravity angle difference values ​​that fall within the sliding window is calculated (e.g., from T=5 to T=20). Then, the sliding window is incremented by one time step, and another sum is calculated using the gravity angle values ​​that fall within the incremented sliding window (e.g., from T=6 to T=21). The running sum of the gravity angle difference within the sliding time window may be referred to as the "sliding window gravity angle difference sum." In some cases, the "sliding window gravity angle difference sum" may be calculated for every time step that there is a gravity angle difference value. In other examples, the "sliding window gravity angle difference sum" may be calculated for every complete sliding window. For example, for a sliding window of length x, the sliding window is not completed until T=x. In those examples, the "sliding window gravity angle difference sum" may be calculated for all values ​​that exist in T greater than or equal to x.

[0117]

[0138] The process then continues by identifying one or more speed predictions from the set of speed predictions that correspond to the same time interval in which the running sum exceeds the threshold (block 1724). The set of speed predictions includes the predicted speed of the vehicle for each time step in which there is a "sliding window sum of gravity angle differences." The process identifies each "sliding window sum of gravity angle differences" that exceeds a threshold value. In some cases, the threshold may be dynamic. For example, the threshold may be made more or less sensitive based on the detected event. If a vehicle crash is detected, for example, the threshold may be affected for the time interval immediately following the detected vehicle crash. The process then identifies one or more speed predictions that correspond to the same time step in which there is a "sliding window sum of gravity angle differences" that exceeds a threshold value.

[0118]

[0139] The process continues by filtering one or more speed predictions from the set of speed predictions (block 1728). In some cases, the acceleration measurements may contain spurious values, which may be in part the result of a user operating a mobile device rather than a vehicle, noise, etc. Using these acceleration values ​​to generate a speed prediction may produce inaccurate results. The "Sum of Sliding Window Gravity Angle Differences," which represents the variability of the acceleration measurements, may be used as a filter to filter out speed predictions that correspond to acceleration measurements with a high degree of variability (e.g., that may be due to non-vehicle forces or noise). If the "Sum of Sliding Window Gravity Angle Differences" exceeds a threshold value at a time step (e.g., if the variability of the acceleration measurements at that time step is high), the speed prediction corresponding to that time step may be filtered from the set of speed predictions.

[0119]

[0140] In some cases, if there are acceleration measurements to process, the process may continue indefinitely. For example, blocks 1704-1728 may be executed continuously while the mobile device is located in the vehicle during a move (e.g., while the vehicle is moving from a start location to a destination location). The process may derive speed predictions and filter those speed predictions in real time. In other examples, blocks 1704-1728 may be executed once the move is completed. For example, during the move, the mobile device may collect sensor measurements for the entire trip. Once the move is completed, the sensor measurements may be processed in a batch process to generate a set of speed predictions for the trip, and those speed predictions may be filtered.

[0120]

[0141] In yet other examples, the process of 1704-1728 may be performed as a semi-batch process in which blocks 1704-1728 may be executed each time a predetermined amount of sensor measurements are collected (e.g., based on a threshold amount of memory sufficient to generate a predetermined amount of gravity vectors, a time interval from when the vehicle begins moving to when the vehicle stops moving, etc.) and / or when movement is completed (e.g., if less than a threshold amount of sensor measurements have been collected but the movement has ended).

[0121]

[0142] FIG. 18 includes graphs illustrating an example of filtering speed predictions using a "sliding window sum of gravity angle differences," according to aspects of the disclosure. Graph 1804 is a plot of the "sliding window sum of gravity angle differences" over time. Graph 1808 is a plot of the vehicle's actual speed and the vehicle's predicted speed over the same time interval as graph 1804. Graphs 1804 and 1808 provide an illustration of an example in which speed predictions are filtered according to the process of FIG. 17. In graph 1804, when the "sliding window sum of gravity angle differences" exceeds a threshold 1812, as it does at about 210 seconds, the system begins filtering the corresponding speed prediction value 1816. Graphs 1804 and 1808 provide a graphical representation of the filtering of the speed prediction values. During the process of filtering the velocity predictions (e.g., the process of FIG. 17), the derived values ​​(e.g., predicted velocity, “sum of gravity angle differences for a sliding window”) may not be displayed graphically (as in FIG. 18), but may instead be represented as data, a data structure, etc.

[0122]

[0143] 19 is an exemplary graph illustrating the use of speed predictions to derive features for a machine learning model to generate a crash prediction, according to some embodiments. The machine learning model may generate a crash prediction for an in-service vehicle. In some cases, signal noise or other non-vehicle forces (e.g., a user operating a mobile device, etc.) may result in a false positive where a crash is detected but not occurring. Additional features characterizing the vehicle's environment after a crash occurs may be used to improve the accuracy of the predicted crash (e.g., eliminate false positives) and as input to the machine learning model for recalibrating or retraining the model.

[0123]

[0144] For example, after a collision, the driver of the vehicle must bring the vehicle to a complete stop in a safe location. The driver may then inspect the damage to the vehicle (e.g., to determine if the vehicle is safe to drive), assess injuries, and / or exchange information with other drivers. The speed prediction may be used to determine if / when the vehicle stops after a collision is detected. If the vehicle stops within a predefined time interval or distance after a collision is detected, the collision prediction was likely accurate. On the other hand, if the vehicle continues to drive after a collision is detected, the detected collision may be a false positive. The detection of a stopped vehicle (based on the speed prediction) and the distance the vehicle takes to stop after a detected collision may be additional features that may be used to determine the accuracy of the collision prediction (e.g., to filter out false positives of detection / non-detection) and to further improve the accuracy of the machine learning model in generating the collision prediction (e.g., by labeling the sensor data used to train the machine learning model).

[0124]

[0145] Distance from the location of a possible collision may be determined by calculating the average predicted speed over one or more windows after the time the collision was detected. The windows may be a fixed, predefined length, such as 30 seconds, 60 seconds, 120 seconds, 180 seconds, 300 seconds, or the average predicted speed (e.g., with a larger average predicted speed having a larger window) or any length window set by the user. Distance may be calculated by multiplying the average speed by the duration of each window. Distance may be calculated for each window using the predicted speed (e.g., as described in FIG. 13) and again using the filtered speed (e.g., as described in FIG. 17 and FIG. 18). For the five windows shown in FIG. 19 (e.g., 30 seconds, 60 seconds, 120 seconds, 180 seconds, 300 seconds), ten distance features are derived, one for each window using the unfiltered predicted speed and one for each window using the filtered predicted speed. Additional distance features may be generated as additional windows are selected.

[0125]

[0146] The distance features may be used to determine which predicted crashes actually occurred. For example, if the vehicle stops within a predicted distance after a crash is detected, it may be more likely that a crash occurred. On the other hand, if the vehicle does not stop within the predicted distance, it may be more likely that the predicted crash was a false positive. In that case, the detected crash may be discarded and data collection by the mobile device may continue until the drive is terminated.

[0126]

[0147] Alternatively or additionally, the distance features may be used as further inputs to a machine learning model (e.g., either the same machine learning model that generated the original collision prediction or a different machine learning model) configured to generate a collision prediction. By generating a new collision prediction using the distance features as further inputs, a more accurate collision prediction may be generated than the initial collision prediction.

[0127]

[0148] In some cases, the further information may be used to provide labels for future training. For example, if the driver does not stop after a crash is predicted, the sensor data input to the machine learning model may be labeled "not a crash." Similarly, if the vehicle stops after a predicted crash, the sensor data input to the machine learning model may be labeled "crash." The labels assigned to the sensor data may be used by the machine learning model to refine how it generates future predictions, and may improve the accuracy of the machine learning model in generating future crash predictions.

[0128]

[0149] FIG. 20 is a block diagram of a process for predicting a vehicle crash according to an aspect of the disclosure. At block 2004, a crash event associated with the vehicle is detected. A mobile device disposed in the vehicle can collect sensor measurements using sensors of the mobile device. These sensor measurements can be processed to generate a crash prediction and passed as input to a crash prediction machine learning model. The crash prediction machine learning model can be trained to generate a crash prediction using sensor measurements (e.g., raw sensor measurements or processed sensor measurements such as those described by the pre-processing pipeline 1312 of FIG. 13). The crash prediction machine learning model can be trained using supervised learning, unsupervised learning, or semi-supervised learning. The crash prediction machine learning model can generate a crash prediction continuously (e.g., at regular intervals while the vehicle is in operation). In other examples, a crash prediction can be generated upon detecting an anomalous sensor measurement (e.g., a sensor measurement above a threshold).

[0129]

[0150] At block 2008, it is determined whether the vehicle has stopped. For example, the mobile device may use sensor measurements to predict the vehicle's speed (in real time, continuously, at regular intervals, upon detection of an event such as a crash event, etc.). After a crash is detected, the mobile device may determine whether the vehicle's speed is greater than zero after a predetermined time interval or distance from the crash event.

[0130]

[0151] If the vehicle is still moving, the process moves to block 2012, where it is determined that no collision has occurred (e.g., the vehicle collision was a false positive), and the process of FIG. 20 returns to block 2004 to wait until another collision is detected or until the vehicle is no longer moving (e.g., the current movement ends).

[0131]

[0152] If the vehicle has stopped, the process moves to block 2016, where features may be generated from the sensor measurements. The features may include features used by the machine learning model to determine the original crash event in block 2004. The features may also include sensor measurements collected or derived after the crash event. For example, the features may include one or more distance features corresponding to the distance traveled by the automobile after the crash occurred and before stopping, as described in connection with FIG. 19. The one or more distance features may be derived by multiplying the average predicted speed over each of one or more time windows (e.g., 30 seconds, 60 seconds, 120 seconds, 180 seconds, and / or 300 seconds). Alternatively or additionally, the one or more distance features may be derived by multiplying the average filtered predicted speed (e.g., as described in FIG. 17) over each of one or more time windows (e.g., 30 seconds, 60 seconds, 120 seconds, 180 seconds, and / or 300 seconds).

[0132]

[0153] The distance feature may provide further information after a crash event that may provide an indication as to the severity of the crash and whether the vehicle's motion after the crash was the result of the crash event or other driving activity (e.g., reaching a destination, a stop sign or stoplight, traffic, etc.). For example, if the vehicle is broken down due to a crash, it should stop almost immediately after the crash. This may provide an indication that the crash event was severe enough to break down the vehicle (e.g., not just a "minor crash"). If the vehicle continues to drive for a short period of time before stopping, this may indicate that the crash event was not severe because the vehicle was able to continue driving. In another example, the distance feature (along with other information) may provide an indication that the vehicle stop was due to a stoplight or other routine driving maneuver, thereby providing an indication that the crash event in 2004 was a false positive.

[0133]

[0154] In block 2020, a collision prediction machine learning model can be run using the generated distance features to generate a new collision prediction. In some cases, this collision prediction machine learning model can be the same machine learning model that was run in block 2004. In other examples, the collision prediction machine learning model can be a second (different) machine learning model. In those examples, the second machine learning model can be trained using the sensor measurements along with the distance features. The second machine learning model can be trained using supervised learning, unsupervised learning, or semi-supervised learning.

[0134]

[0155] The collision prediction machine learning model may generate new collision predictions based on the additional information of the generated distance features. The collision prediction machine learning model may generate a probability that a collision occurred at a particular time. The probability may be represented by any predefined means (e.g., a number from 0 to 1, 1 to 100, a grade such as A to F, etc.).

[0135]

[0156] At block 2024, it is determined whether the new crash prediction is greater than a threshold. In some cases, the threshold may be dynamically selected (at run-time). Sensor measurements and / or the original crash prediction may be used to set the threshold. The threshold may vary for each crash event. For example, if the original crash event indicates that a crash likely occurred (although the car continued driving) (e.g., 75-100%), the threshold may be set to a low value. As a result, the lower, new crash detection probability meets the threshold, indicating that a crash occurred. This allows the system to determine that a crash occurred, depending in part on the original crash detection result. If the original crash prediction had a low probability (e.g., 50-75%), the threshold may be set higher to be more sensitive to possible false positives. In other examples, the threshold may be pre-determined. In those examples, the threshold may be set by a machine learning model, such as a crash prediction machine learning model, a user, a command from a remote device, etc.

[0136]

[0157] In some cases, multiple thresholds may be used. For example, if the collision prediction is greater than the first threshold, the process may proceed to block 2028. If the collision prediction is between the first and second thresholds, the process may continue by first asking the user to confirm whether a collision has occurred. If the user confirms the collision, the process proceeds to block 2028. If the user does not confirm the collision, the process proceeds to block 2012. In some cases, a collision may be inferred if the user does not respond within a predetermined time interval. For example, the collision may have caused injury or damage to the mobile device, which the user was not able to confirm. As a result, the process may proactively call for emergency services, etc. If the collision prediction is lower than the second threshold, the process may proceed to block 2012. The first threshold may be set higher than the second threshold. For example, the first threshold may be set to 85% and the second threshold may be set to 65%. The first and second thresholds may be set to any predetermined (or dynamically selected) values, provided that one threshold is higher than the other.

[0137]

[0158] If, at block 2024, it is determined that the new crash prediction is greater than the threshold, the process proceeds to block 2028 where it is indicated that a crash has occurred. In some cases, determining that a crash has occurred may result in one or more actions being performed, such as, but not limited to, automatically sending a crash indication to a user of the mobile device (e.g., initiating an insurance claim process, determining the safety of people involved in the crash, collecting additional information about the crash, providing information to the user on what to do after a crash, providing safety information, providing first aid information, etc.), automatically sending the crash indication to emergency services, automatically sending the crash indication to an insurance representative associated with the vehicle or driver, modifying the driver's driving profile to include the crash indication and possibly sensor measurements related to the crash, etc.

[0138]

[0159] If, at block 2024, it is determined that the new crash prediction is equal to or less than the threshold, the process proceeds to block 2012 where it is indicated that a crash did not occur. In some cases, the crash prediction (e.g., the crash prediction at 2004 and / or 2020) may be discarded from the system. In other examples, both the crash prediction and the sensor measurement that triggered the crash prediction may be discarded or otherwise marked to prevent the sensor measurement from being used in other data analysis. The data may include anomalous sensor measurements that may also be unreliable in other analysis operations (e.g., determining driver behavior, such as distracted driving) because the sensor measurement resulted in at least one false positive crash prediction. In yet other examples, the crash prediction may be stored and further used to train and / or improve the crash prediction machine learning model(s).

[0139]

[0160] FIG. 21 is a flow diagram of an example process for generating a crash prediction using a predicted speed of a vehicle, according to some embodiments. The process begins with receiving a set of sensor measurements from a mobile device located in the vehicle (block 2104). The set of sensor measurements may include measurements received or derived from one or more sensors of the mobile device, such as, but not limited to, measurements from an accelerometer, activity data (e.g., as described in FIG. 12), measurements from a magnetometer, and / or measurements from one or more other sensors of the mobile device. The sensor measurements may be received over a predefined time interval or event. For example, the set of sensor measurements may be received in real time or at periodic intervals, such as, but not limited to, every 0.5 seconds, every 1 second, every 5 seconds, or any predefined interval.

[0140]

[0161] Alternatively, the set of sensor measurements may be received (in real time or at regular intervals) while in motion (e.g., as the vehicle is operated to move from a start location to a destination location). For example, activity data may be collected from a mobile device as described in connection with FIG. 12 above. The activity data may be used to determine when an activity, such as driving, begins and ends. When the mobile device detects that driving has begun, the mobile device may begin collecting a set of accelerometer measurements from the mobile device's accelerometer until the activity data (user or remote communication) indicates that the driving has ended.

[0141]

[0162] Alternatively, the set of sensor measurements may be received from a remote device, such as a remote accelerometer, a mobile device, a server, etc. The set of sensor measurements may include sensor measurements collected during driving. In some cases, the set of sensor measurements may be received after a drive is completed, upon detection of a crash event, upon detection of another event, upon expiration of a time interval, when a sensor signal exceeds a threshold, etc.

[0142]

[0163] The process continues by transforming the set of sensor measurements into the frequency domain (block 2108). The set of sensor measurements may be transformed using any means, such as a Fourier transform. In some cases, this may be performed by a machine learning model trained to process the accelerometer signals. The machine learning model may be configured to further perform one or more of blocks 2112-2120.

[0143]

[0164] The process continues by filtering the set of sensor measurements with a bandpass filter to remove high frequency sensor measurements from the set of sensor measurements (block 2112). The bandpass filter filters out portions of the sensor measurements occurring at specific frequencies, such as frequencies associated with signal noise sensor measurements that do not correspond to a vehicle (e.g., motion of the mobile device independent of motion of the vehicle). In some cases, frequencies higher than a first threshold frequency and lower than a second threshold frequency may be filtered. The first threshold may be, for example, about 50 Hz, and the second threshold may be, for example, about 9 Hz. The first and second thresholds may be selected by a process on the mobile device (e.g., a data collection application, an operating system, etc.), by a user, from a communication received from a remote device, etc.

[0144]

[0165] The process continues by defining one or more successive windows based on the remaining sensor measurements in the set of sensor measurements (block 2116). Defining one or more successive windows from the remaining (filtered) set of sensor measurements may be performed as described by block 1324 of FIG. 13. The successive windows may be defined relative to time intervals during which the accelerometer signals were received. Each accelerometer measurement may correspond to a successive window when the accelerometer signal was generated by the accelerometer based on the time intervals represented by the successive windows.

[0145]

[0166] One or more successive windows may be defined by identifying the time difference between each pair of successive accelerometer signals (e.g., when the signals are generated by an accelerometer). Successive windows may be defined between a first pair of successive sensor readings that is greater than a threshold (e.g., 3 standard deviations from the average time difference between successive accelerometer readings) and a next pair of successive sensor readings that is greater than the threshold. Because the time difference between pairs of successive accelerometer readings varies, the lengths of the successive windows may not be of equal length. Each successive window includes a time interval during which a subset of the remaining set of sensor readings was generated by the accelerometer.

[0146]

[0167] The process continues by generating a set of features for each successive window by resampling a subset of the remaining set of sensor measurements represented by the successive window to one or more predefined frequencies (block 2116). The subset of the remaining set of sensor measurements may be resampled as described in 1328 of FIG. 13, e.g., as a frequency between 9 and 50 Hz, determined by user input, by communication from a remote device, etc.

[0147]

[0168] The process continues by generating one or more speed predictions for each set of features by running a neural network trained with the set of features. The trained neural network may be neural network 1332 of Figure 13, neural network 1500 of Figure 15 (which may, for example, be the same as neural network 1332), or a different neural network. The neural network may be trained using supervised or unsupervised learning to generate speed predictions from the sensor measurements.

[0148]

[0169] In some cases, the speed predictions may be filtered to prevent signal noise not associated with the vehicle or reliance on accelerometer signals from producing inaccurate speed predictions. For example, one or more speed predictions may be filtered as described by the process of FIG.

[0149]

[0170] The process continues by generating a vehicle crash prediction using the one or more speed predictions (block 2128). Generating the vehicle crash prediction from the one or more speed predictions may be performed as described by some or all of the blocks in FIG. 20. For example, the vehicle crash prediction may be provided by a crash prediction machine learning model that indicates that a crash event has occurred. The vehicle crash prediction may be validated using the speed prediction to determine whether the vehicle has stopped after the crash event. If the vehicle has not stopped, the crash prediction may be defined as false (e.g., a crash event did not occur). If the vehicle has stopped, the crash prediction may be defined as true (e.g., a crash event has occurred).

[0150]

[0171] In another example, the vehicle crash prediction may be provided by a crash prediction machine learning model that indicates that a crash event has occurred. The vehicle crash prediction may be validated using a speed prediction to determine if the vehicle has stopped after the crash event. The speed prediction may be used to derive a distance feature that indicates if and / or when the vehicle has stopped after the initial crash detection. The distance feature may be provided to a crash prediction machine learning model (e.g., the neural network that provided the speed prediction, the same crash prediction machine learning model that provided the initial crash prediction, or a different crash prediction machine learning model), which may run using the distance feature and (possibly) sensor measurements to provide a final crash detection to generate a final crash prediction. The final crash prediction may be a probability that a crash has occurred (e.g., 0-1, 0-100, a grade from A-F, a Boolean such as true or false, etc.).

[0151]

[0172] The probability may be compared to one or more thresholds. If the probability is greater than one or more thresholds, it may be determined that a collision has occurred (e.g., the vehicle collision prediction may be defined as true). If the probability is less than a first threshold but greater than a second threshold (if utilized), the mobile device may trigger a notification to a user or another device (e.g., a server, an on-board computing device of the vehicle, another mobile device, etc.) to request additional information and / or confirmation of whether a collision has occurred. The collision event may be defined as true (e.g., a collision has occurred) or false (e.g., a collision has not occurred) based on user input. If the probability is less than one or more thresholds, it may be determined that a collision has not occurred (e.g., and the vehicle collision prediction may be defined as false).

[0152]

[0173] The process continues by transmitting the vehicle crash prediction (and / or the final crash prediction) to a remote device (block 2132).

[0153]

[0174] In some cases, if it is determined that a crash has occurred, one or more actions may be performed by the mobile device, such as, but not limited to, sending a notification to emergency services (e.g., push notification, email, robocall, text message, etc.), initiating an insurance claim process, sending a notification to an insurance representative, sending a notification to another mobile device (e.g., notifying emergency contacts of the driver or passengers, requesting additional information related to the crash from other occupants of the vehicle or occupants of other vehicles involved in the crash, etc.), determining the severity of the crash, combinations thereof, etc.

[0154]

[0175] If a collision is not predicted, the process returns to block 2104 where a new set of sensor measurements may be collected. The new set of sensor measurements may be associated with a different portion from the same movement or a subsequent movement.

[0155]

[0176] 22 is a block diagram of a process for validating vehicle crash predictions using speed predictions according to aspects of the disclosure. The process begins with receiving a set of sensor signals from a mobile device located in the vehicle (block 2204). The set of sensor measurements may include measurements received or derived from one or more sensors of the mobile device, such as, but not limited to, measurements from an accelerometer, activity data (e.g., as described in FIG. 12), measurements from a magnetometer, and / or measurements from one or more other sensors of the mobile device. The sensor measurements may be received over a predefined time interval or event. For example, the set of sensor measurements may be received in real time or at regular intervals.

[0156]

[0177] Alternatively, the set of sensor measurements may be received (in real time or at regular intervals) while in motion (e.g., as the vehicle is operated to move from a start location to a destination location). For example, activity data may be collected from a mobile device as described in connection with FIG. 12 above. The activity data may be used to determine when an activity, such as driving, begins and ends. When the mobile device detects that driving has begun, the mobile device may begin collecting a set of accelerometer measurements from the mobile device's accelerometer until the activity data (user or remote communication) indicates that the driving has ended.

[0157]

[0178] Alternatively, the set of sensor measurements may be received from a remote device, such as a remote accelerometer, a mobile device, a server, etc. The set of sensor measurements may include sensor measurements collected during driving. In some cases, the set of sensor measurements may be received after a drive is completed, upon detection of a crash event, upon detection of another event, upon expiration of a time interval, when a sensor signal exceeds a threshold, etc.

[0158]

[0179] The process continues by generating a set of successive windows using the set of sensor measurements, with each successive window in the set of successive windows representing a subset of the set of sensor measurements (block 2208). Defining one or more successive windows from the set of sensor measurements may be performed as described by block 1312 of FIG. 13. The successive windows may be defined relative to a time interval during which the accelerometer signals were received. Each successive window may represent a subset of the set of accelerometer measurements generated by the accelerometer during the time included in the successive window.

[0159]

[0180] One or more successive windows may be defined by identifying the time difference between each pair of successive accelerometer signals (e.g., if the signals are generated by an accelerometer). In some cases, the sensor measurements may be processed before the successive windows are defined, such as filtering (e.g., as described by 1316) and normalizing the remaining sensor measurements (e.g., as described by 1320). Successive windows may be defined between a first pair of successive sensor measurements that is greater than a threshold (e.g., one or more standard deviations from the average time difference between successive sensor measurements) and a next pair of successive sensor measurements that is greater than the threshold. The length of the successive windows may vary because the time difference between pairs of successive accelerometer signals varies. Each successive window includes a time interval during which a subset of the remaining set of sensor measurements was generated by or derived from a sensor of the mobile device.

[0160]

[0181] The process continues by generating a speed prediction by running the trained neural network with the set of successive windows for each successive window in the set of successive windows (block 2212). One or more features may be derived from each successive window. Each feature is an input to the trained neural network. The neural network may be, such as neural network 1332 of FIG. 13 or neural network 1500 of FIG. 15. The trained neural network may be run with a subset of the set of sensor measurements corresponding to each successive window to generate one or more speed predictions for the successive windows. Each speed prediction may correspond to a time step (e.g., a particular time in the successive windows). In some cases, the neural network may generate a set of speed predictions, one speed prediction for each time step of the successive windows.

[0161]

[0182] The process continues by receiving an indication that the vehicle has been involved in a crash event (block 2216). The indication of the crash event may be determined by the mobile device from the sensor measurements. For example, a crash prediction machine learning model may be executed to generate a crash prediction using the sensor measurements. The crash prediction machine learning model may be a neural network or another model. In some cases, the indication of the crash event may be received from a remote device. For example, the sensor measurements may be transmitted to the remote device. The remote device may execute the crash prediction machine learning model to generate a crash prediction. The remote device may then return the indication of the crash event to the mobile device.

[0162]

[0183] The process continues by validating the crash event using the speed prediction (block 2220). The crash prediction may be validated by one or more of the processes described in FIG. 20. In some examples, validating the crash event may include using the speed prediction to determine whether the vehicle continued driving after the crash event. The mobile device may use multiple time windows (e.g., 30 seconds, 60 seconds, 120 seconds, 180 seconds, 300 seconds, etc.) to determine whether and / or when the vehicle stopped in each time window. If the vehicle did not stop, it may be determined that the indication of the crash event was a false positive (e.g., a crash event did not occur).

[0163]

[0184] In another example, authenticating the crash event can include using the speed prediction to determine whether the vehicle continued driving after the crash event. If the vehicle is determined to have stopped after the crash event, a distance feature can be derived from the speed prediction and the time window. The distance feature can indicate whether and / or when the vehicle stopped after the initial crash detection, as well as the distance from the crash event. The distance feature can be provided to a crash prediction machine learning model (e.g., the neural network that provided the speed prediction, the same crash prediction machine learning model that provided the initial crash prediction, or a different crash prediction machine learning model), which can run using the distance feature and (optionally) the sensor measurements to generate an updated crash prediction. The updated crash prediction can be a probability that a crash occurred (e.g., 0-1, 0-100, a grade from A-F, a Boolean such as true or false, etc.).

[0164]

[0185] The probability may be compared to one or more thresholds. If the probability is greater than the one or more thresholds, a collision event may be determined to have occurred. If the probability is less than a first threshold but greater than a second threshold (if utilized), the mobile device may trigger a notification to a user or another device (e.g., a server, an on-board computing device in the vehicle, another mobile device, etc.) to request additional information and / or confirmation of whether a collision occurred. A collision event may be defined as true (e.g., a collision occurred) or false (e.g., a collision did not occur) based on user input. If the probability is less than one or more thresholds, a collision is determined not to have occurred.

[0165]

[0186] The process continues by determining that a collision event did not occur based on validation of the collision event (block 2228).

[0166]

[0187] The process continues by deleting the crash event from memory in response to determining that a crash event did not occur. In some cases, deleting the crash event from memory may include deleting a local memory of the mobile device. In other examples, deleting the crash event from memory may include deleting a memory of one or more remote devices (e.g., other mobile devices, servers) that may be configured to process sensor measurements of the mobile device, provide a speed prediction using the sensor measurements of the mobile device, provide a crash prediction using the sensor measurements of the mobile device, etc.

[0167]

[0188] The process of blocks 2204-2224 continues to receive sensor measurements and, upon receiving another indication of a possible collision event, may qualify the collision event.

[0168]

[0189] 23 is an example graph of predicted speed over time, representing a correct decision crash prediction, according to aspects of the disclosure. Graph 2300 shows predicted speed 2304 (gray) of the vehicle while driving and actual speed 2308 (black) of the vehicle. At 2312 (T=2840), the crash prediction indicates the occurrence of a possible crash event. After 2312, the predicted speed (and actual speed) of the vehicle decreases to near zero, indicating that the vehicle has stopped after the possible crash event. Because the vehicle has stopped after the crash event, the possible crash event is more likely to be a correct decision (e.g., a crash has occurred).

[0169]

[0190] 24 is an example graph of predicted speed over time, representing a false positive crash prediction, according to aspects of the disclosure. Graph 2400 shows a predicted speed 2404 (gray) of a vehicle while driving, and an actual speed 2408 (black) of the vehicle. At 2412 (T=6200), when the crash prediction indicates a possible crash event is occurring, the predicted speed of the vehicle is about 36 mph. After 2412, the predicted speed (and actual speed) of the vehicle does not change and remains at 36 mph. Because the vehicle did not stop after the crash event, the possible crash event is likely a false positive (e.g., a crash event did not occur). As a result, the crash prediction may be removed from the local memory.

[0170]

[0191] Specific details are set forth in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure the embodiments with unnecessary detail. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail so as not to obscure the embodiments.

[0171]

[0192] It should also be noted that the particular embodiments may be described as a process that is depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. Although a flowchart may describe operations as a sequential process, many of the operations may occur in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process terminates when its operations are completed, but may have additional steps not included in the diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to a calling function or to a main function.

[0172]

[0193] The implementation of the techniques, blocks, steps, and means described above may be done in various ways. For example, these techniques, blocks, steps, and means may be implemented in hardware, software, or a combination thereof. In a hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), mask programmable gate arrays (MPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described above, and / or combinations thereof.

[0173]

[0194] It should also be noted that the embodiments and / or examples may be described as a process that is depicted as a flowchart, flow diagram, swim lane diagram, data flow diagram, structure diagram, or block diagram. Although the diagrams may describe operations as a sequential process, many of the operations may occur in parallel or simultaneously. In addition, one or more operations may be performed in an order different from the order depicted. A process may end when the operation is completed or when the process returns to a previous step or block. A process may have additional steps or blocks not included in a diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or to the main function.

[0174]

[0195] Furthermore, the devices and / or systems described herein may be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages, and / or any combination thereof. When implemented in software, firmware, middleware, scripting languages, and / or microcode, the program code or code segments to perform the necessary tasks may be stored in a non-transitory computer-readable medium, such as a storage medium. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures, and / or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, and / or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0175]

[0196] For a firmware and / or software implementation, the methodologies may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. Any non-transitory computer-readable medium that tangibly embodies instructions may be used in implementing the methodologies described herein. For example, software code may be stored in a memory. The memory may be implemented within the processor or external to the processor. As used herein, the term "memory" refers to any type of volatile, non-volatile, or other storage medium and is not limited to any particular type or number of memories or the type of media on which the memory is stored.

[0176]

[0197] Additionally, as disclosed herein, the term "storage medium" may refer to one or more memories for storing data, including read-only memory (ROM), random access memory (RAM), magnetic RAM, cache memory, magnetic disk storage media, optical storage media, flash memory devices, and / or other machine-readable media for storing information. The term "computer-readable medium" includes, but is not limited to, portable or fixed storage devices, optical storage devices, and / or various other storage media capable of storing, containing or carrying instruction(s) and / or data.

[0177]

[0198] The examples and embodiments described herein are for illustrative purposes only. Various modifications or changes will become apparent to those skilled in the art upon reading this specification. These are to be included within the spirit and scope of this application and the scope of the appended claims.

Claims

1. receiving a set of sensor measurements from an accelerometer from a mobile device located in a vehicle; transforming the set of sensor measurements into the frequency domain; filtering the set of sensor measurements with a bandpass filter to remove high frequency sensor measurements from the set of sensor measurements; defining a set of successive windows based on remaining sensor measurements in the set of sensor measurements, each successive window in the set of successive windows representing a successive portion of the remaining sensor measurements in the set of sensor measurements; for each successive window of the set of successive windows, generating a set of features by resampling the sensor measurements of the successive windows at one or more predefined frequencies; generating an estimate of velocity by running a neural network trained with the set of features; detecting a vehicle crash using the velocity estimate; transmitting the detected vehicle crash to a remote device; A method comprising:

2. The method of claim 1, further comprising the step of detecting a vehicle collision using the estimated velocity. The method of claim 1 , comprising determining whether the vehicle has stopped after a crash event based on the estimate of the velocity.

3. The method of claim 1 , wherein at least two consecutive windows in the set of consecutive windows represent different amounts of the remaining sensor measurements in the set of sensor measurements.

4. 2. The method of claim 1, wherein the trained neural network is a convolutional neural network with one or more dilated convolutions.

5. said step of defining a set of contiguous windows further comprising: deriving, for each acceleration signal in the set of sensor measurements, a time difference between when each pair of successive sensor measurements was taken; defining a first successive window of the set of successive windows as beginning at a first time at which a first time difference between a first pair of successive sensor measurements exceeds a threshold and ending at a second time, after the second time, a second time difference between a second pair of successive sensor measurements exceeds the threshold, the second time occurring after the first time; The method of claim 1 , comprising:

6. The method of claim 5 , wherein the threshold value is equal to one or more standard deviations from the average time difference between each pair of successive sensor measurements.

7. The method of claim 1, further comprising the steps of: detecting the vehicle collision using the estimated velocity; running a crash prediction machine learning model using the set of sensor measurements to detect a first vehicle crash, the detected first vehicle crash indicating a crash event occurring at a first time; determining, at a time after detection of the vehicle crash, that the estimate of velocity is greater than zero; determining that the crash event did not occur based on a determination that the speed of the vehicle is greater than zero; defining the detection of the vehicle crash to indicate that the crash event did not occur; The method of claim 1 , comprising:

8. The method of claim 7, further comprising the step of detecting the vehicle collision using the estimated velocity. running a crash prediction machine learning model using the set of sensor measurements to detect a first vehicle crash, wherein detection of the first vehicle crash indicates a crash event occurring at a first time; determining, at a time after detection of the first vehicle crash, that the vehicle has stopped after the first time; using the velocity estimate to generate a set of distance features, each distance feature of the set of distance features corresponding to a position of the vehicle relative to the crash event; defining the detection of the vehicle collision as being output from a second execution of the collision prediction machine learning model using the set of sensor measurements and the set of distance features; The method of claim 1 , comprising:

9. receiving a set of sensor measurements from an accelerometer from a mobile device located in a vehicle; generating a set of successive windows using the set of sensor measurements, each successive window in the set of successive windows representing a subset of the set of sensor measurements; generating, for each successive window of the set of successive windows, an estimate of velocity by running a neural network trained with the set of successive windows; receiving an indication that the vehicle has been involved in a crash event; validating the crash event using the velocity estimate; determining that the crash event did not occur based on authentication of the crash event; in response to determining that the crash event did not occur, deleting the crash event from memory. A method comprising:

10. The method of claim 9 , wherein at least two consecutive windows in the set of consecutive windows represent different amounts of sensor measurements in the set of sensor measurements.

11. 10. The method of claim 9, wherein the trained neural network is a convolutional neural network with one or more dilated convolutions.

12. The method of claim 1, further comprising the step of authenticating the crash event using the velocity estimate, determining whether the vehicle has stopped after the crash event, and if it is determined that the vehicle has not stopped, it is determined that the crash event has not occurred; 10. The method of claim 9, comprising:

13. The method of claim 12, further comprising the step of authenticating the crash event using the velocity estimate, determining that the vehicle has stopped at a distance from the crash event; generating one or more distance features representative of the distance from the crash event; running a collision prediction machine learning model using the set of sensor measurements and the distance features to detect a vehicle collision; determining that the detection of the vehicle crash is below a threshold, where a determination that the detection of the vehicle crash is below the threshold indicates that the crash event did not occur; 10. The method of claim 9, comprising:

14. The method of claim 1 or 9, wherein the set of sensor measurements includes activity data.

15. one or more processors; An accelerometer; A non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 14. A mobile device comprising:

Images