Techniques for automatic data collection and report generation after a vehicle incident

Abstract

Techniques for automatic data collection and report generation after a vehicle incident include generating incident reports includes receiving a signal indicating that a vehicle has been involved in an incident; receiving vehicle data collected by the vehicle; receiving supplemental data from a mobile device; generating a report of the incident based on the vehicle data and the supplemental data; and transmitting the report to an insurance company.

Description

TECHNIQUES FOR AUTOMATIC DATA COLLECTION AND REPORT GENERATION AFTER A VEHICLE INCIDENTCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional application titled, “INCIDENT REPORT GENERATION,” filed November 20, 2023, and having Serial No. 63 / 601,106. The subject matter of this related application is hereby incorporated by reference.BACKGROUNDField of the Various Embodiments

[0002] Embodiments of the present invention relate generally to artificial intelligence, and more specifically, techniques for automatic data collection and report generation after a vehicle incident.Description of the Related Art

[0003] Insurance companies have to track incidents involving a vehicle to assess liability and risk, calculate damages, and determine premiums for drivers with frequent incidents that can lead to higher insurance rates. Incidents refer to any event involving a vehicle that could potentially lead to an insurance claim, such as accidents, damages, or loss related to a vehicle. Incidents can be classified as collisions, non-collision events, and minor incidents. A collision is an accident where two or more vehicles collide, or a vehicle hits an object (e.g„ a tree or a fence). A non-collision event includes incidents, such as theft, fire, vandalism, or weather- related damage (e.g., hail or flooding). A minor incident involves a small damage, such as scratches or dents.

[0004] In such incidents, insurance companies need to collect and process relevant data to determine the party at fault. Each insurance company also generates a custom report to identify what damages are being compensated for and / or which party is responsible for the damages. Conventional techniques collect the incident relevant data in different ways. For example, when a policy holder files a claim and reports the information related to an accident and fills the related forms. Another source of relevant data can be police reports describing an incident which contains date and time, location, traffic data, parties involved, description of the incident, officer’s observations, parties’ statements, traffic violations, etc. Yet other sources of relevant data can be information from an adjuster, a mechanic after repairing the vehicle, surveillance footage, medical reports, and / or reports by the insurance company adjusters after examining the vehicle.

[0005] One drawback of conventional techniques is that the collected incident data can be contaminated. The collected data can be incomplete or inaccurate due to several factors, such as human error and bias, missing information, fraudulent claims, and / or environmental factors. Another drawback of conventional techniques is that to generate a report, each incident is prepared manually which can be costly and time consuming.

[0006] As the foregoing indicates, what is needed in the art are more effective techniques for vehicle data collection and insurance report generation.SUMMARY

[0007] In various embodiments, a computer-implemented method for generating incident reports includes receiving a signal indicating that a vehicle has been involved in an incident; receiving vehicle data collected by the vehicle; receiving supplemental data from a mobile device; generating a report of the incident based on the vehicle data and the supplemental data; and transmitting the report to an insurance company.

[0008] Further embodiments provide, among other things, methods and systems for implementing one or more aspects of the disclosed techniques.

[0009] At least one technical advantage of the disclosed techniques relative to the prior art is that with the disclosed techniques the likelihood of contaminated data associated with a vehicle incident is greatly reduced. The disclosed techniques further reduce the likelihood of fraudulent claims or miss-interpretation of incident data. In addition, with the disclosed techniques, automatically collected data can be easily processed and automatically generate a report which reduces the time and effort needed to generate a report. The disclosed techniques also support the generation of reports based on incidents other than accidents. Furthermore, generating custom reports reduces costs by reducing manual labor and other resources needed to generate vehicle incident reports for different clients. These technical advantages provide one or more technological improvements over prior art approaches.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] So that the manner in which the above recited features of the various embodiments can be understood in detail, a more particular description of the inventive concepts, briefly summarized above, can be had by reference to various embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of the inventive concepts and are therefore not to beconsidered limiting of scope in any way, and that there are other equally effective embodiments.

[0011] Figure 1 illustrates a block diagram of a computer-based system configured to implement one or more aspects of the various embodiments;

[0012] Figure 2 is a block diagram illustrating a cloud computing server of Figure 1, according to various embodiments;

[0013] Figure 3 is a block diagram illustrating a vehicle computing device of Figure 1, according to various embodiments;

[0014] Figure 4 is an illustration of an exemplar autonomous vehicle, according to various embodiments;

[0015] Figure 5 is a block diagram illustrating a mobile device of Figure 1, according to various embodiments;

[0016] Figure 6 is a flow diagram of method steps for detecting an incident, according to various embodiments;

[0017] Figure 7 is a flow diagram of method steps for collecting user supplemental data 206, according to various embodiments; and

[0018] Figure 8 is a flow diagram of method steps for generating a report 250 after detecting an incident, according to various embodiments.DETAILED DESCRIPTION

[0019] In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one skilled in the art that the inventive concepts may be practiced without one or more of these specific details.System Overview

[0020] Figure 1 illustrates a block diagram of a computer-based system 100 configured to implement one or more aspects of the various embodiments. As shown, system 100 includes, without limitation, a cloud computing server 110, a vehicle computing device 140, and a mobile device 150 in communication over a network 130. Cloud computing server 110includes, without limitation, one or more processors 102 and a memory 112. Memory 112 includes, without limitation, a report generator 114. Vehicle computing device 140 includes, without limitation, one or more processors 142 and a memory 144. Memory 144 includes, without limitation, an incident module 146. Mobile device 150 includes, without limitation, one or more processors 152 and a memory 154. Memory 154 includes, without limitation, an incident application 156.

[0021] Processor(s) 102 can be any technically feasible form of processing device configured to process data and execute program code. For example, any of processor(s) 102 could be a central processing unit (CPU), a graphics processing unit (GPU), an applicationspecific integrated circuit (ASIC), a field-programmable gate array (FPGA), and so forth. Processor(s) 102 can include any combination of two or more processors of a same or different types. For example, processor(s) 102 could include a CPU working in cooperation with a one or more GPUs.

[0022] Memory 112 of cloud computing server 110 stores content, such as software applications and data, for use by processor(s) 102 and / or other processing units. Memory 112 can be any type of memory capable of storing data and software applications, such as a random-access memory (RAM), a read-only memory (ROM), an erasable programmable readonly memory (EPROM or Flash ROM), or any suitable combination of the foregoing. In some embodiments, a storage (not shown) can supplement or replace system memory 112. The storage can include any number and type of external memories that are accessible to processor 102. For example, and without limitation, the storage can include a Secure Digital Card, an external Flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, and / or any suitable combination of the foregoing.

[0023] As also shown, memory 112 includes a report generator 114. Report generator 114 receives input data and generates a custom report for an insurance company based on the input data. The input data includes vehicle data received from incident module 146 that has detected an incident for a corresponding vehicle. The input data further includes user supplemental data which is collected by incident application 156 after the incident has occurred. In some embodiments, the format and granularity of the collected input data are pre-defined by each insurance company. After receiving the input data, report generator 114 initially processes the input data to generate timeline records of the incident by aligning and stitching different parts of the input data into a recreation of the timeline of the incident. For example, report generator 114 can stitch videos and / or images of different cameras or imaging devices taken at differenttimes and / or angles to generate a timeline of the incident. Report generator 114 then uses one or more machine learning (ML) models to generate a custom report. The generated report can be in a format requested by an insurance company. The generated report includes details of the incident, the insurance policy information, a summary of the incident, such as location, role of parties, external conditions, alertness state of the driver, technical issues (e.g„ technical problems or failure in one or more vehicle systems), police reports, etc.

[0024] The one or more ML models can be trained using any suitable training technique, such as supervised, unsupervised, and / or reinforcement learning. In various embodiments, the one or more ML models can be implemented as any technically feasible natural language processing technique, including, but not limited to, a transformer, a multi modal neural network (e.g., visual language models (VLM)), a large language model (LLM), or an ensemble technique. The operations performed by report generator 114 are described in greater detail below in conjunction with Figure 2.

[0025] Cloud computing server 110 shown herein is for illustrative purposes only, and variations and modifications are possible without departing from the scope of the present disclosure. For example, the number of processors 102, the number of memories 112, and / or the number of applications included in the memory 112 can be modified as desired. Further, the connection topology between the various units in Figure 1 can be modified as desired. In some embodiments, any combination of processor(s) 102 and memory 112 can be included in and / or replaced with any type of virtual computing system, distributed computing system, and / or cloud computing environment, such as a public, private, or a hybrid cloud system.

[0026] Network 130 includes any technically feasible type of communications network that allows data to be exchanged between cloud computing server 110, vehicle computing device 140, and mobile device 150. For example, network 130 can include a wide area network (WAN), a local area network (LAN), a cellular network, a wireless (WiFi) network, and / or the Internet, among others.

[0027] Similar to processor(s) 102 of cloud computing server 110, processor(s) 142 can be any technically feasible form of processing device configured to process data and execute program code. For example, any of processor(s) 142 could be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field- programmable gate array (FPGA), and so forth. Processor(s) 142, can include any combinationof two or more different processors of a same or different types. For example, processor(s) 142 could include a CPU working in cooperation with a one or more GPUs.

[0028] Similar to memory 112 of cloud computing server 110, memory 144 of vehicle computing device 140 stores content, such as software applications and data, for use by the processor(s) 142. Memory 144 can be any type of memory capable of storing data and software applications, such as a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash ROM), or any suitable combination of the foregoing. In some embodiments, a storage (not shown) can supplement or replace memory 144. The storage can include any number and type of external memories that are accessible to processor(s) 142. For example, and without limitation, the storage can include a Secure Digital Card, an external Flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, and / or any suitable combination of the foregoing.

[0029] Incident module 146 detects an incident involving the vehicle. An incident can include an impact with other vehicles or objects, a vehicle rollover, driving into a ditch, and / or the like. Incident module 146 detects the incident by receiving and processing sensor data and data from other computing systems on the vehicle, such as data from an advanced drive assistance system (ADAS), a driver monitoring system (DMS), one or more electronic control units (ECUs), an event data recorder (EDR), a climate control unit (CCU), and / or the like. The sensor data can include data associated with the vehicle or data about an environment surrounding the vehicle, can include image data captured by cameras, imaging devices (e.g„ light detection and ranging (LiDAR), radar), ultrasound sensor data, EDR data, traffic conditions, and / or the like. When an incident is detected, incident module 146 stores the in- vehicle and surrounding sensor data leading up to the incident, alerts report generator 114, and transmits the stored data to report generator 114 for further processing. In addition, incident module 146 can notify the driver of the incident and request that the driver collect user supplemental data by sending an alert to incident application 156. The operations performed by incident module 146 are described in greater detail below in conjunction with Figure 3.

[0030] Similar to processor(s) 102 of cloud computing server 110, processor(s) 152 can be any technically feasible form of processing device configured to process data and execute program code. For example, any of processor(s) 152 could be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field- programmable gate array (FPGA), and so forth. Processor(s) 152 can include any combinationof two or more processors of a same or different types. For example, processor(s) 152 could include a CPU working in cooperation with a one or more GPUs.

[0031] Similar to memory 112 of cloud computing server 110, memory 154 of mobile device 150 stores content, such as software applications and data, for use by the processor(s) 152. Memory 154 can be any type of memory capable of storing data and software applications, such as a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash ROM), or any suitable combination of the foregoing. In some embodiments, a storage (not shown) can supplement or replace memory 154. The storage can include any number and type of external memories that are accessible to processor(s) 152. For example, and without limitation, the storage can include a Secure Digital Card, an external Flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, and / or any suitable combination of the foregoing.

[0032] Incident application 156 is a custom mobile device application that prompts the driver to capture user supplemental data after incident application 156 receives an incident signal from incident module 146 and / or report generator 114. Examples of user supplemental data include other drivers information, incident scene images, witness information, any injuries report, and / or alike. Incident application 156 captures and uploads the user supplemental data to report generator 114. The format and granularity of the user supplemental data, and mobile application instructions are pre-defined by each insurance company via templates provided by each insurance company. The operations performed by incident application 156 are described in greater detail below in conjunction with Figure 5.Exemplary Cloud Computing System for Report Generation

[0033] Figure 2 is a block diagram illustrating cloud computing server 110, according to various embodiments. As shown, cloud computing server 110 includes, without limitation, a memory 112, one or more processors 102, a storage 214, an input / output (I / O) devices interface 216 coupled to one or more input / output (VO) devices 222, a network interface 218, and an interconnect (bus) 220. Memory 112 includes, without limitation, vehicle data 202, a report generator 114, user supplement data 206, a report 250, insurance policy information 208, and police report(s) 210. Report generator 114 includes, without limitation, a reconstruction module 232 and trained model(s) 234. Vehicle data 202 includes, without limitation, sensor data 224, camera data 226, and controller data 228.

[0034] In some embodiments, cloud computing server 110 can be a desktop computer, a laptop computer, server machine, or any other type of computing device configured to receive input, process data, and is suitable for practicing one or more embodiments of the present disclosure. Cloud computing server 110 described herein is illustrative and that any other technically feasible configurations fall within the scope of the present disclosure. For example, multiple instances of report generator 114 could execute on a set of nodes in a data center, cluster, or cloud computing environment to implement the functionality of cloud computing server 110. In another example, report generator 114 could be implemented using any number of hardware and / or software components or layers.

[0035] I / O device interface 216 enables communication of I / O devices 222 with processor(s) 102. I / O device interface 216 generally includes the logic for interpreting addresses corresponding to I / O devices 222 that are generated by processor(s) 102. I / O device interface 216 can also be configured to implement handshaking between processor(s) 102 and I / O devices 222, and / or generate interrupts associated with I / O devices 222. I / O device interface 216 can be implemented as any technically feasible interface circuit or system.

[0036] In some embodiments, I / O devices 222 include devices capable of receiving input, such as a keyboard, a mouse, a touchpad, and / or a microphone, as well as devices capable of providing output, such as a display device and / or speaker. Additionally, I / O devices 222 can include devices capable of both receiving input and providing output, such as a touchscreen, a universal serial bus (USB) port, and so forth. I / O devices 222 can be configured to receive various types of input from an end-user (e.g., a designer) of cloud computing server 110, and to also provide various types of output to the end-user of cloud computing server 110, such as displayed digital images or digital videos or text.

[0037] Network interface 218 serves as the interface between the computer and the network 130. Network interface 218 facilitates the transmission and reception of data. Network interface 218 includes, without limitation, hardware, software, or a combination of hardware and software. In some embodiments, network interface 218 supports one or more communication protocols, such as Ethernet, Wi-Fi, Bluetooth, among others.

[0038] Interconnect 220 connects subsystems and devices within cloud computing server 110. For example, interconnect 220 includes an interface provided by cloud computing server 110 that allows memory 112 to communicate with processor(s) 102, storage 214, I / O devices interface 216, and network interface 218.

[0039] Storage 214 can include non-volatile storage for applications and data, and can include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-Ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. In some embodiments, report generator 114, vehicle data 202, user supplement data 206, insurance policy information 208, one or more police reports 210, and report 250 can be stored in storage 214 and loaded into memory 112.

[0040] As previously described with respect to Figure 1, memory 112 includes a report generator 114. Report generator 114 receives input data and generates a custom report for an insurance company based on the input data. The input data includes vehicle data 202, user supplemental data 206, incident signal 204, insurance policy information 208, and one or more police reports 210. The generated report includes details of the incident, insurance policy information 208, a summary of the incident, such as location, role of parties, external conditions, alertness state of the driver, technical issues, police report(s) 210, etc., as described in further detail below.

[0041] Vehicle data 202 is received from incident module 146 that has detected an incident for a corresponding vehicle. Vehicle data 202 includes sensor data 224, camera data 226, and controller data 228. Sensor data 224 is the information collected by sensors, such as and without limitation, global navigation satellite systems (“GNSS”) sensor(s) (e.g., Global Positioning System sensor(s)), RADAR sensor(s), ultrasonic sensor(s), LIDAR sensor(s), inertial measurement unit (IMU) sensor(s) (e.g., accelerometer(s), gyroscope(s), magnetic compass(es), magnetometer(s), etc.), microphone(s), speed sensor(s) (e.g., for measuring the speed of the vehicle), vibration sensor(s), steering sensor(s), brake sensor(s) (e.g., as part of the brake sensor system), and / or other sensor types. The vehicle sensors used to collect sensor data 224 are described in more detail in conjunction with Figure 4.

[0042] Camera data 226 can include, but is not limited to, the data from digital cameras that can be adapted for use with the components and / or systems of the vehicle. Camera data 226 can be captured from the cameras located in the interior of the vehicle or exterior of the vehicle, such as front view, rear view, side view, driver view cameras, and / or the like. The camera used to capture camera data 226 can be stereo camera(s), wide-view camera(s) (e.g., fisheye cameras), infrared camera(s), surround camera(s) (e.g., 360 degree cameras), long- range and / or mid-range camera(s) or any other type of cameras. The data from additional and / or alternative cameras located at different locations on the vehicle can be included.Camera data 226 can be captured with any image capture rate, such as 60 frames per second(fps), 120 fps, 440 fps, etc., depending on the embodiment. Camera data 226 can be from cameras capable of using rolling shutters, global shutters, another type of shutter, or a combination thereof. Camera data 226 can be captured with a Bayer sensors (RGGB) color filter array, a monochrome sensor color filter array, and / or another type of color filter array. The vehicle cameras used to collect camera data 226 are described in more details in conjunction with Figure 4.

[0043] Controller data 228 is data indicating signals and data generated by computing and control systems in the vehicle. The computing and control systems can include, without limitation, one or more ECUs, an ADAS, an EDR, a DMS, and / or the like. Controller data 228 can include, without limitation, signals to operate the vehicle brakes, to operate the steering system, to operate the propulsion system, and / or the like generated by the one or more ECUs. Controller data 228 can further include other data and values generated by the computing and control systems based on sensor data.

[0044] For example, the ADAS uses sensors, cameras, and radar to detect and respond to a vehicle’s surroundings. The ADAS provides real-time alerts and automated actions like braking, steering, or acceleration to help driver avoid collisions, stay in lane, or maintain safe distance. In such an example, controller data 228 is the real-time alerts and information from the automated actions generated by the ADAS. As another example, the EDR is designed to store data from sensors, cameras, radar, and / or the like. Yet another example, the DMS uses sensors, cameras, and algorithms to monitor the driver’s attentiveness and well-being in realtime. The DMS provides real-time alerts to the driver by flashing lights, warning sounds, etc., to avoid collisions when the driver is not attentive. As an example, the DMS can collect images and health data, perform eye tracking, cognitive load, and mental awareness evaluation of the driver or other vehicle occupants. The vehicle controllers used to collect controller data 228 are described in more detail in conjunction with Figures 3 and 4.

[0045] Incident signal 204 is a signal generated by incident module 146 after an incident is detected. Incident signal 204 can trigger report generator 114 to initiate processing of received data and generating report 250. In some embodiments, upon receiving incident signal 204, report generator 114 can forward incident signal 204 to incident application 156 to request incident application 156 to provide user supplemental data 206.

[0046] User supplemental data 206 can be any data related to the incident that is collected from a user (e.g„ the driver of the vehicle) by incident application 156. For example, usersupplemental data 206 can include, without limitation, information about the driver(s) of other vehicles, information about pedestrians, bicyclists, etc., involved in the incident, incident scene images, witness information, and / or injury reports. In some embodiments, the format and granularity of the collected data are pre-defined by each insurance company. User supplemental data 206 is described in more detail in conjunction with Figure 5.

[0047] Insurance policy information 208 outlines the rights and responsibilities of both the insurer and the insured, including what risks are covered, how claims can be made, and any exclusions. Insurance policy information 208 includes, without limitation, the details of an insurance contract, such as the policyholder's name, coverage type, policy number, premium amount, coverage limits, deductibles, and the terms and conditions. Report generator 114 uses insurance policy information 208 to generate report 250.

[0048] Police report(s) 210 includes, without limitation, incident details such as the date, time, and location of the incident, statements from drivers, passengers, and witnesses, descriptions of vehicle damage and injuries, and a diagram or narrative describing how the incident occurred collected by police officers or other officials present at the incident scene. Police report(s) 210 can also include police observations, such as citations issued, weather and road conditions, and preliminary determinations of fault. Report generator 114 uses police report(s) 210 to generate report 250.

[0049] Reconstruction module 232 processes the received data to generate a reconstruction of the incident that includes timeline records of the incident by combining, aligning, and / or stitching different parts of the input data into a recreation of the timeline of the incident. Reconstruction module 232 can use one or more of trained model(s) 234 to generate the reconstruction of the incident. For example, reconstruction module 232 can combining videos and / or images included in camera data 226 taken at different times and / or angles to generate a timeline of the incident n pictures or videos based on the time stamps associated with camera data 226. As another example, reconstruction module 232 can align and stitch videos and / or images from user supplemental data 206 taken by the driver after the incident or images provided by police report(s) 210. Reconstruction module 232 can reconstruct the incident scene with computer graphic techniques to visualize the occurrences or events leading up to the incident. Reconstruction module 232 can further use the computer graphics techniques to create perspective of the incident from different positions and / or angles, generate animations at various speeds, and / or the like.

[0050] Reconstruction module 232 can further annotate and / or add notes to the timeline with information about relevant sensor data 224 and / or controller data 228. For example, the timeline can be annotated with an indication when certain events occurred (e.g., hard braking, hard steering, a collision), information about the driver (e.g., the driver’s attention level, the driver’s cognitive load, the driver’s vital signs, whether the driver was using a mobile phone (hands-free or otherwise) prior or during the incident), and / or the like. As another example, the timeline can be annotated with information regarding traffic and / or environmental data prior to or during the incident, such as the volume of traffic, traffic speed, the status of traffic lights, weather conditions, and / or the like.

[0051] Reconstruction module 232 can process the data from different sources to identify relevant information of the incident. For example, after an incident that damaged the front bumper of the vehicle, a video from a front camera of the vehicle in camera data 226 can be identified that shows the front bumper impact. In such an example, reconstruction module 232 can also identify images from other sources, such as user supplemental data 206 that show the damage to the front bumper after the impact. Reconstruction module 232 can identify relevant data from different sources using any feasible technique, such as matching the timestamps or other meta-data of data from different sources or comparing the content of the data from the different sources. For example, reconstruction module 232 can compare image and / or video features, such as shapes and colors to identify the relevant data.

[0052] Report generator 114 receives the input data including vehicle data 202, incident signal 204, user supplemental data 206, insurance policy information 208, police report(s) 210 and uses this information to generate report 250. In addition, report generation 114 can further request and / or receive data from other vehicle-to-everything (V2X) systems in the vicinity of the incident, such as traffic cameras, images for traffic lights, and / or the like. In some embodiments, vehicle data 202 can include sensor fusion data which can be a combination of one or more elements from vehicle data 202. Report generator 114 uses reconstruction module 232 to generate a reconstruction of the incident. Report generator 114 then uses trained model(s) 234 to generate report 250 based on relevant data from vehicle data 202, user supplemental data 206, insurance policy information 208, and / or police report(s) 210. When generating report 250, report generator 114 can further abide by data privacy rights of the driver or other persons by selectively omitting personal and / or other information from report 250 to maintain compliance with data privacy regulations and laws. Report generator 114 then transmits report 250 to the insurance company of the driver. For example, report generator 114can identify the insurance company and / or where to transmit report 250 based on insurance policy information 208.

[0053] Report 250 includes details of the incident and provides a complete summary of the available information that might be useful for the insurance company to assess the incident and make coverage and / or other determinations. Report 250 can include the date and location of the incident, information to identify the vehicles and people involved, information to contact the people involved and / or their insurance companies, information to identify and / or contact witnesses or first responders, and / or the like. Report 250 can include a timeline of events associated with the incident, such as reconstruction generated by reconstruction module 232. Report 250 can include a narrative describing the timeline of the incident, a summary of the damage from the incident, observations or conclusions included in police reports(s) 210, and / or the like. Report 250 can include information about the weather or road conditions, such as where it was raining or snowing at the time of the accident, whether there were high winds, whether the exterior temperature indicates there might be icy roads, and / or the like. Report 250 can include the health information of the driver, such as was the driver running a fever, experiencing other health conditions, and / or the like. Report 250 can include information about the alertness level, cognitive load of the driver, where the driver’s attention was focused before and during the incident, and / or the like. Report 250 can further include relevant information from the insurance policy information 208. Report 250 can further include preliminary assessments about the cause of the incident, fault for the incident, and / or the like.

[0054] In addition, report 250 can be customized based on the requirements and / or preferences of the insurance company (e.g„ format, content, sections, granularity, etc.) for which report 250 is generated. For example, Report 250 can be generated in different formats depending upon the insurance company for which report 250 is being generated. Report generator 114 can further transmit report 250 to the insurance company via email, fax, uploading to a cloud interface, or using any other technically feasible approach.

[0055] Trained model(s) 234 assist report generator 114 to generate report 250. generate custom reports based on each insurance company requested report format. Each of trained model(s) 234 is trained to generate one or more portions of report 250. For example, different ones of trained model(s) 234 can be trained to for a specific portion of report 250, such as timeline generation, narrative drafting, cause assessment, fault assessment, damage assessment, and / or the like. In some embodiments, trained model(s) 234 are trained using samples of reports or portions of reports along with the underlying data used to generate thesample reports or portions of the reports. In some embodiments, different trained models 234 are trained and used for different insurance companies to generate report 250 to meet the requirements of the corresponding insurance company. Trained model(s) 234 can be any type of model including, but not limited to, one or more transformer models, multi modal neural networks, visual language models (VLMs), large language models (LLMs), ensemble models, and / or the like.Exemplary Vehicle Computing System

[0056] Figure 3 is a block diagram illustrating vehicle system 300, according to various embodiments. As shown, vehicle system 300 includes, without limitation, vehicle computing device 140 and other vehicle computing units. Vehicle computing device 140 includes, without limitation, a memory 144, one or more processors 142, a storage 304, I / O devices interface 306 coupled to one or more I / O devices 310, a network interface 308 coupled to other vehicle computing units 314, and an interconnect 302. Memory 144 includes, without limitation, vehicle data 202, an incident module 146, and one or more task specific models 312. Incident module 146 includes, without limitation, an incident detector module 324. Other vehicle computing units 314 include, without limitation, one or more ECUs 316, an ADAS 318, an EDR 320, and a DMS 322.

[0057] In some embodiments, vehicle computing device 140 can be included in a head unit, an ECU, and / or any other type of computing device configured to receive input, process data, and is suitable for practicing one or more embodiments of the present disclosure, vehicle computing device 140 described herein is illustrative and that any other technically feasible configurations fall within the scope of the present disclosure. In another example, incident module 146 could be implemented using any number of hardware and / or software components or layers.

[0058] VO device interface 306 enables communication of VO devices 310 with processor(s) 142. I / O device interface 306 generally includes the logic for interpreting addresses corresponding to I / O devices 310 that are generated by processor(s) 142. VO device interface 306 can also be configured to implement handshaking between processor(s) 142 and VO devices 310, and / or generate interrupts associated with VO devices 310. VO device interface 306 can be implemented as any technically feasible interface circuit or system.

[0059] In some embodiments, I / O devices 310 include devices capable of receiving input, such as a keyboard, a mouse, a touchpad, and / or a microphone, as well as devices capable of providing output, such as a display device and / or speaker. Additionally, I / O devices 310 can include devices capable of both receiving input and providing output, such as a touchscreen, a universal serial bus (USB) port, and so forth. I / O devices 310 can be configured to receive various types of input from an end-user (e.g., a designer) of vehicle computing device 140, and to also provide various types of output to the end-user of vehicle computing device 140, such as displayed digital images or digital videos or text.

[0060] Network interface 308 serves as the interface between the computer and the network 130. Network interface 308 facilitates the transmission and reception of data. Network interface 308 includes, without limitation, hardware, software, or a combination of hardware and software. In some embodiments, network interface 308 supports one or more communication protocols, such as Ethernet, Wi-Fi, Bluetooth, among others.

[0061] Network interface 308 is coupled with other vehicle computing units 314. Each of the other vehicle computing units 314 can be a desktop computer, a laptop computer, server machine, or any other type of computing device configured to receive input, process data, and is suitable for practicing one or more embodiments of the present disclosure.

[0062] One or more ECUs 316 are embedded systems that manage specific functions of the vehicle by processing the vehicle data and controlling actuators in response. The ECUs 316 can be dedicated to a particular system or function, such as an engine control unit that manages the operations of the engine, brake control module that controls a braking system, and / or the like. One or more ECUs 316 use sensors, cameras, user inputs, radar, and / or the like to process the vehicle data and control actuators. For example, a CCU is an ECU that controls the climate inside the vehicle.

[0063] ADAS 318 is a suite of components to improve safety and enhance the driving experience by assisting drivers in various aspects of vehicle control, navigation, and awareness. ADAS 318 uses sensors, cameras, and radar to detect nearby obstacles to respond to the vehicle’s surroundings and / or to provide other assistance to the driver in operating the vehicle. For example, ADAS 318 can improve vehicle operation, such as by providing adaptive cruise control, lane keeping assistance, real-time alerts and / or the like. ADAS 318 can also perform automatic vehicle actions, such as braking, steering, or acceleration. The observations and / or actions taken by ADAS 318 can be included in controller data 228.

[0064] EDR 320 is a device that stores specific information about the vehicle and actions of the driver around the time of an incident is detected. EDR 320 can collect data from sensors, camera, and radar. EDR 320 retrieves and analyzes the collected data to understand the vehicle’s response and whether the vehicle or the driver correctly interpreted the environment and executed proper controls. For example, EDR 320 can record data from sensors and / or the like in response to detecting vehicle issues, such as failure and engine codes, and / or based on high acceleration, steering, and / or the like. The data recorded by EDR 320, can be included in controller data 228.

[0065] DMS 322 uses sensors, cameras, and algorithms to monitor the driver’s attentiveness, health conditions, eye gaze direction, cognitive load, etc., in real-time. DMS 322 can provide real-time alerts to the driver by flashing lights, warning sounds, etc., to avoid collisions when the driver is not attentive. The data monitored and / observations made by DMS 322, can be included in controller data 228.

[0066] Interconnect 302 connects subsystems and devices within vehicle computing device 140. For example, interconnect 302 includes an interface provided by vehicle computing device 140 that allows memory 144 to communicate with processor(s) 142, storage 304, VO devices interface 306, and network interface 308.

[0067] In some embodiments, storage 304 includes non-volatile storage for applications and data, and can include fixed or removable disk drives, flash memory devices, and CD- ROM, DVD-ROM, Blu-Ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. In some embodiments, incident module 146, vehicle data 202, and one or more task specific models 312 can be stored in storage 304 and loaded into memory 144.

[0068] Task specific models 312 receive input data, such as vehicle data 202 and generate a task specific output by processing the input data. Task specific models 312 can be specialized algorithms or machine learning models designed to perform particular tasks. Each of the task specific models 312 can be defined to focus on a specific aspect of driving, perception, or decision making. An example of a task performed by task specific models 312 can be perception tasks, that process sensor, camera, or radar data to interpret the environment around the vehicle. Examples of perception tasks are object detection that identifies and locates objects such as vehicles, pedestrians, cyclists, object tracking that follows detected objects across frames to understand object’s movement and predict future position of the object, semantic segmentation that classifies each pixel in an image or video to identify roadfeatures, such as lanes, sidewalks, and depth estimation that calculates the distance of objects from the vehicle, either through stereo vison or other sensor fusion technology. Another example of a task specific model 312 helps with incident detection by continuously processing sensor, camera, or radar data and can use a pre-defined threshold, sensor fusion, and / or an anomaly score to determine whether an incident has occurred. Yet another example of a task specific model 312 is an anomaly detection model that is trained from driving data and can identify unusual events indicating various types of incidents.

[0069] Other examples of task specific models 312 are prediction models, such as path prediction that determine the future path of the vehicle, pedestrian, cyclists, planning models that calculates feasible and safe routes from the current position of the vehicle, control models that controls the physical movements of the vehicle, and driver monitoring models that determine whether the driver is paying attention and is prepared to take control of the vehicle.

[0070] Incident module 146 detects an incident involving the vehicle and performs a series of actions in response to detecting the incident. Incident module 146 receives vehicle data 202 and the outputs from task specific models 312 to detect an incident. Incident module 146 uses incident detector module 324 to detect an incident. After an incident is detected, incident module 146 alerts report generator 114 by transmitting incident signal 204. Upon detecting an incident, incident module 146 stores vehicle data 202 leading up to and through the incident in storage 304. Incident module 146 can transmit the incident stored data in storage 304 to report generator 114 through network 130. In addition, incident module 146 can notify the driver of the vehicle and request that the driver or other user collect user supplemental data 206 by sending an alert to incident application 156.

[0071] Incident detector module 324 detects an incident by processing vehicle data 202 and outputs of task specific models 312 in real-time. Incident detector module 324 then notifies incident module 146 about the detected incident. An incident can include an impact with other vehicles or objects, a vehicle rollover, driving into a ditch, hard braking, hard steering, and / or the like. Incident detector module 324 monitors sensor data 224, camera data 226, controller data 228, and outputs of the task specific models 312 to check whether any unusual value is received. Incident detector module 324 can perform a filtering process to the input data to reduce noise and improve false positives. Incident detector module 324 can use any suitable technique, such as a machine learning model, a classifier, a regression model and / or the like to detect an incident. For example, incident detector module 324 can use a predefined threshold for acceleration, deceleration, impact force, and other parameters that whenout of a defined range indicates an incident. As another example, incident detector module 324 can use one or more of the task specific models 312 to determine whether an incident has occurred and / or information about the incident.

[0072] Incident detector module 324 can further assess the severity of the incident based on the vehicle data 202, such as speed and impact force. Incident detector module 324 can also determine the type of the incident, such as collision, rollover, or hard braking using one or more of task specific models 312 for classification. When incident detector module 324 detects an incident, incident detector module 324 generates incident signal 204, which is transmitted to report generator 114 and / or incident application 156.Example Vehicle

[0073] Figure 4 is an illustration of an example vehicle 400, according to various embodiments. Vehicle 400 can be, without limitation, a passenger vehicle, such as a car, a truck, a bus, a first responder vehicle, a shuttle, an electric or motorized bicycle, a motorcycle, a fire truck, a police vehicle, an ambulance, a boat, a construction vehicle, an underwater craft, a robotic vehicle, a drone, an airplane, a vehicle coupled to a trailer (e.g„ a semi-tractor-trailer truck used for hauling cargo), and / or another type of vehicle (e.g., that is unmanned and / or that accommodates one or more passengers).

[0074] Vehicle 400 can include components such as a chassis, a vehicle body, wheels (e.g., 2, 4, 6, 8, 18, etc.), tires, axles, and other components of a vehicle. The vehicle 400 can include a propulsion system 450, such as an internal combustion engine, hybrid electric power plant, an all-electric engine, and / or another propulsion system type. The propulsion system 450 can be connected to a drive train of the vehicle 400, which can include a transmission, to enable the propulsion of the vehicle 400. The propulsion system 450 can be controlled in response to receiving signals from the throttle / accelerator 452.

[0075] A steering system 454, which can include a steering wheel, can be used to steer the vehicle 400 (e.g., along a desired path or route) when the propulsion system 450 is operating (e.g., when the vehicle is in motion). The steering system 454 can receive signals from a steering actuator 456. The steering wheel can be optional for full automation (Level 5) functionality.

[0076] The brake sensor system 446 can be used to operate the vehicle brakes in response to receiving signals from the brake actuators 448 and / or brake sensors.

[0077] ECUs 316, which can include one or more system on chips (SoCs) and / or GPU(s), can provide signals (e.g., representative of commands) to one or more components and / or systems of the vehicle 400. For example, ECUs 316 can send signals to operate the vehicle brakes via one or more brake actuators 448, to operate the steering system 454 via one or more steering actuators 456, to operate the propulsion system 450 via one or more throttle / accelerators 452. ECUs 316 can include one or more onboard (e.g„ integrated) computing devices (e.g., supercomputers) that process sensor signals, and output operation commands (e.g., signals representing commands) to enable autonomous driving and / or to assist a human driver in driving the vehicle 400. ECUs 316 can include a first ECU 316 for autonomous driving functions, a second ECU 316 for functional safety functions, a third ECU 316 for artificial intelligence functionality (e.g., computer vision), a fourth ECU 316 for infotainment functionality, a fifth ECU 316 for redundancy in emergency conditions, and / or other ECUs. In some examples, a single ECU 316 can handle two or more of the above functionalities, two or more ECUs 316 can handle a single functionality, and / or any combination thereof.

[0078] ECUs 316 can provide the signals for controlling one or more components and / or systems of the vehicle 400 in response to sensor data received from one or more sensors (e.g., sensor inputs). The sensor data can be received from, for example and without limitation, GNSS sensor(s) 458, RADAR sensor(s) 460, ultrasonic sensor(s) 462, LIDAR sensor(s) 464, inertial measurement unit (IMU) sensor(s) 466 (e.g., accelerometer(s), gyroscope(s), magnetic compass(es), magnetometer(s), etc.), microphone(s) 496, stereo camera(s) 468, infrared camera(s) 472, surround camera(s) 474 (e.g., 360 degree cameras), speed sensor(s) 444 (e.g., for measuring the speed of the vehicle 400), vibration sensor(s) 442, steering sensor(s) 440, brake sensor(s) (e.g., as part of the brake sensor system 446), and / or other sensor types.

[0079] One or more ECUs 316 receive inputs (e.g., represented by input data) from an instrument cluster 432 of the vehicle 400 and provide outputs (e.g., represented by output data, display data, etc.) via a human-machine interface (HMI) display 434, an audible annunciator, a loudspeaker, and / or via other components of the vehicle 400. The outputs can include information such as vehicle velocity, speed, time, map data, location data (e.g., the vehicle’s 400 location, such as on a map), direction, location of other vehicles, information about objects and status of objects as perceived by ECUs 316, etc. For example, the HMI display 434 can display information about the presence of one or more objects (e.g., a street sign, caution sign, traffic light changing, etc.), and / or information about driving maneuvers thevehicle has made, is making, or will make (e.g., changing lanes now, taking exit 34B in two miles, etc.).

[0080] The vehicle 400 further includes a network interface 424 which can use one or more wireless antenna(s) 426 and / or modem(s) to communicate over one or more networks. For example, the network interface 424 can be capable of communication over Long-Term Evolution (“LTE”), Wideband Code Division Multiple Access (“WCDMA”), Universal Mobile Telecommunications System (“UMTS”), Global System for Mobile communication (“GSM”), IMT-CDMA Multi-Carrier (“CDMA4000”), etc. The wireless antenna(s) 426 can also enable communication between objects in the environment (e.g., vehicles, mobile devices, etc.), using local area network(s), such as Bluetooth, Bluetooth Low Energy (“LE”), Z-Wave, ZigBee, etc., and / or low power wide-area network(s) (“LPWANs”), such as LoRaWAN, SigFox, etc.Example of User Supplemental Data Collection

[0081] Figure 5 is a block diagram illustrating mobile device 150, according to various embodiments. As shown, mobile device 150 includes, without limitation, a memory 154, one or more processors 152, a storage 522, one or more EO devices interface 524 coupled to one or more EO devices 528, a network interface 526, and an interconnect 520. Memory 154 includes, without limitation, an incident application 156, and a user supplemental data 206. Incident application 156 includes, without limitation, a notification module 504, a user interface module 506, a recording module 508, and a file transfer module 510. User supplemental data 206 includes, without limitation, other driver(s) information 512, one or more incident scene images 514, a witness information 516, and injury report(s) 518. In operation, incident application 156 receives incident signal 204 from incident module 146 or report generator 114 and prompts a user (e.g., the driver of the vehicle) to collect user supplemental data 206.

[0082] In some embodiments, mobile device 150 can be a laptop computer, a smart phone, a personal digital assistant (PDA), tablet computer, or any other type of computing device configured to receive input, process data, and is suitable for practicing one or more embodiments of the present disclosure, mobile device 150 described herein is illustrative and any other technically feasible configurations fall within the scope of the present disclosure. For example, multiple instances of incident application 156 could execute on a set of nodes in a data center, cluster, or cloud computing environment to implement the functionality ofmobile device 150. In another example, incident application 156 could be implemented using any number of hardware and / or software components or layers.

[0083] I / O device interface 524 enables communication of I / O devices 528 with processor(s) 152. I / O device interface 524 generally includes the logic for interpreting addresses corresponding to I / O devices 528 that are generated by processor(s) 152. I / O device interface 524 can also be configured to implement handshaking between processor(s) 152 and I / O devices 528, and / or generate interrupts associated with I / O devices 528. I / O device interface 524 can be implemented as any technically feasible interface circuit or system.

[0084] In some embodiments, I / O devices 528 include devices capable of receiving input, such as a keyboard, a mouse, a touchpad, a camera, and / or a microphone, as well as devices capable of providing output, such as a display device and / or speaker. Additionally, I / O devices 528 can include devices capable of both receiving input and providing output, such as a touchscreen, a universal serial bus (USB) port, and so forth. I / O devices 528 can be configured to receive various types of input from an end-user a designer) of mobile device 150, andto also provide various types of output to the end-user of mobile device 150, such as displayed digital images or digital videos or text.

[0085] Network interface 526 serves as the interface between the computer and the network 130. Network interface 526 facilitates the transmission and reception of data. Network interface 526 includes, without limitation, hardware, software, or a combination of hardware and software. In some embodiments, network interface 526 supports one or more communication protocols, such as Ethernet, Wi-Fi, Bluetooth, among others.

[0086] Interconnect 520 connects subsystems and devices within mobile device 150. For example, interconnect 520 includes an interface provided by mobile device 150 that allows memory 154 to communicate with processor(s) 152, storage 522, VO devices interface 524, and network interface 526.

[0087] In some embodiments, storage 522 includes volatile and non-volatile storage for applications and data, and can include portions of memory 154, fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-Ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. In some embodiments, incident application 156 and user supplemental data 206 can be stored in storage 522 and loaded into memory 154 when executed.

[0088] Incident application 156 is a custom mobile device application that prompts the driver or other user to capture user supplemental data 206 after incident application 156 receives an incident signal 204 from incident module 146 and / or report generator 114. Incident application 156 captures and uploads user supplemental data 206 to report generator 114. The format and granularity of user supplemental data 206, and incident application 156 instructions are pre-defined by each insurance company via templates provided by each insurance company. Incident application 156 can identify user supplemental data 206 not collected by the user and prompt the user for the missing information. For example, incident application 156 can identify missing user supplemental data 206 by comparing the data collected in storage 522 with a list of information defined by an insurance company of the driver. In some embodiments, incident application 156 can ask for consent from the insurance policy holder (e.g„ the driver) before capturing and / or uploading user supplemental data 206 to report generator 114.

[0089] Notification module 504 receives incident signal 204 from report generator 114 and / or incident module 146 and delivers notifications to the incident application 156. Notifications can be messages or alerts that inform users about important events, or actions that need to be taken within incident application 156. After receiving incident signal 204, notification module 504 alerts the user to collect user supplemental data 206. User notification module 504 can use any suitable technique to inform the user to engage with incident application 156, such as push notifications, in-app notifications, email or SMS notifications.

[0090] User interface module 506 receives a notification from notification module 504 and displays the notification to the user. User interface module 506 can prompt the user to interact with user interface module 506 to record and / or transfer user supplemental data 206. User interface module 506 displays the visual elements and interactive components of the incident application 156. For example, user interface module 506 prompts the user to take photos, record a video, and / or enter textual content for inclusion in user supplemental data 206. User interface module 506 can include buttons and links, forms and input fields, navigation menus and tabs and a responsive design and state management for user state persistence.

[0091] Recording module 508 enables users to record different types of data that are part of user supplemental data 206. Recording module 508 receives user supplemental data 206 after interacting with user interface module 506 and stores the user supplemental data 206 in storage 522. Recording module 508 can support different data types, such as audio, video,images, and / or textual data. Recording module 508 can provide recording settings and quality control. For example, audio quality, video resolution, and / or frame rate settings.

[0092] File transfer module 510 interacts with user interface module 506 and transfers user supplemental data 206 stored in storage 522 to report generator 114. File transfer module 510 enables a secure exchange of files between incident application 156 and outside devices, such as cloud computing server 110. File transfer module 510 can include features like upload and download of files, progress indicators, cancel or pause options and / or the like. File transfer module 510 can support different transfer protocols, such as HTTP / HTTPS, FTP / SFTP, and P2P protocols. Additionally and / or alternatively, file transfer module 510 can receive user supplemental data 206 directly from recording module 508 and transfer user supplemental data 206 to report generator with or without user supplemental data 206 being stored in storage 522.

[0093] User supplemental data 206 can be any data related to the incident that is collected by a user (e.g„ the driver of the vehicle involved in an incident). For example, user supplemental data 206 can include, without limitation, information about the driver(s) of other vehicles, information about pedestrians, bicyclists, etc., involved in the incident, incident scene images, witness information, and / or injury reports. In some embodiments, the format and granularity of the collected data are pre-defined by each insurance company. The user supplemental data 206 includes, without limitation, other driver(s) information 512, one or more incident scene images 514, a witness information 516, and injury report(s) 518.

[0094] Other driver(s) information 512 can include other driver’s name, contact information, driver’s license number, insurance information, license plate number, etc. Other driver(s) information 512 can also include information about pedestrians, bicyclists, etc., involved in the incident. For example, the other driver(s) information 512 can be collected by taking, when prompted, images of the relevant documents using a camera of mobile device 150, entered via voice-to-text, or via a text box.

[0095] Incident scene images 514 can include images of each vehicle involved in the incident, such as images of damage to the driver’s vehicle, damage to other vehicles, etc. Incident scene images 514 can further include images of the overall incident scene, the roadway, traffic lights, roadway signs, damage to surrounding property (e.g., street lights, signs, etc.), and or the like. Each image can be captured by the user via using the camera of mobile device 150 as prompted by user interface module 506 and / or recording module 508.

[0096] Witness information 516 can include description of the vehicles and people involved, chronology of events, environmental factors, name, contact information of the witness, a spoken statement, and / or the like.

[0097] Injury reports 518 can include documents that record any injuries sustained by individuals involved in the incident, such as from paramedic reports, hospital reports, etc. Injury reports 518 can include time and date, medical professionals involved, injured person(s) name, contact information, such as address, phone number, email address, role in the incident, such as driver, passenger, details of the injury, such as type of injury, severity of injury, symptoms, causes of injury, immediate effects, medical examination, initial treatment given and / or the like.

[0098] Figure 6 is a flow diagram of method steps for detecting an incident, according to various embodiments. Although the method steps are described in conjunction with the systems of Figures 1-5, persons skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present embodiments.

[0099] As shown, a method 600 begins at step 602, where incident module 146 receives vehicle data 202 and outputs of task specific models 312. Vehicle data 202 includes sensor data 224, camera data 226, and / or controller data 228. Sensor data 224 is the information collected by sensors. Camera data 226 can include, but not limited to, the data from digital cameras that can be adapted for use with the components and / or systems of the vehicle. Controller data 228 is data indicating signals and data generated by computing and control systems in the vehicle. The computing and control systems can include, without limitation, one or more ECUs, an ADAS, an EDR, a DMS, and / or the like. Task specific models 312 can be specialized algorithms or machine learning models designed to perform particular tasks.

[0100] At step 604, incident detector module 324 detects an incident by processing the vehicle data 202 and outputs of task specific models 312 in real-time. Incident detector module 324 then invokes incident module 146 with the detected incident for further operations. Incident detector module 324 can detect different types of incidents, such as collision, rollover, or hard braking and / or the like by monitoring the input data and checking whether any unusual value is received. Incident detector module 324 can use any suitable technique, such as one or more of task specific models 312, an anomaly detection model, a classifier, a regression model and / or the like to detect an incident. For example, incident detector module 324 can use apredefined range for acceleration, deceleration, impact force, and / or other parameters that when outside the predefined range, indicate an incident.

[0101] Incident detector module 324 can further assess the severity of the incident, based on the input data, such as speed and impact force of a collision, and / or the like. Incident detector module 324 can also determine the type of the incident, such as collision, rollover, or hard braking using any suitable machine learning model for classification.

[0102] At step 606, incident module 146 transfers vehicle data 202 to report generator 114. After an incident is detected by incident detector module 324, incident module 146 alerts report generator 114 by transmitting incident signal 204 to report generator 114. Upon detecting an incident, incident module 146 stores vehicle data 202, such as any of the sensor data 224, camera data 226, and / controller data 228 leading up to and through the incident as well as outputs of task specific models 312. in storage 304. Incident module 146 can transmit the data in storage 304 to report generator 114 through network 130. In addition, incident module 146 can transmit incident signal to report generator 114 and / or incident application 156, to indicate that an incident has occurred.

[0103] Figure 7 is a flow diagram of method steps for collecting user supplemental data 206, according to various embodiments. Although the method steps are described in conjunction with the systems of Figures 1-5, persons skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present embodiments.

[0104] As shown, a method 700 begins at step 702, where incident application 156 receives a request to capture user supplemental data 206. Incident application 156 can receive the request via incident signal 204 from incident module 146 and / or report generator 114. User supplemental data 206 can be any data related to the incident that is collected from a user (e.g., the driver of the vehicle) by incident application 156.

[0105] At step 704, incident application 156 notifies the driver to capture user supplemental data 206 via mobile device 150. After receiving incident signal 204, notification module 504 alerts the user to collect user supplemental data 206 through the user interface module 506. User interface module 506 prompts the user to take photos, record a video, record audio, and / or enter textual content for inclusion in user supplemental data 206. For example, incident application 156 and user interface module 506 can prompt the user for any of otherdriver(s) information 512, incident scene images 514, witness information 516, and injury reports 518. In some embodiments, incident application generates a list of user supplemental data 206 to be collected.

[0106] At step 706, incident application 156 collects user supplemental data 206. As incident application 156 collects user supplemental data 206 using recording module 508, incident application 156 stores the collected user supplemental data 206 in storage 522. Depending on the type of the data being collected, incident application 156 and user interface module 506 can display one or more prompts, text boxes, and / or the like for input of the information and / or work using a microphone or camera in mobile device to capture the user supplemental data 206. In some embodiments, incident application uses the list of user supplemental data 206 generated during step 704 to keep track of which user supplemental data 206 has been collected or not collected, and the status of whether the user supplemental data 206 on the list has been provided. This list can be displayed to the user as a checklist of what has been collected and what still needs to be collected.

[0107] At step 708, incident application 156 transmits user supplemental data 206 to report generator 114. Incident application 156 retrieves user supplemental data 206 from storage 214 and transmits user supplemental data 206 to report generator 114 through network 130. Additionally and / or alternatively, file transfer module 510 can receive user supplemental data 206 from recording module 508 and transmit the received user supplemental data 206 to report generator 114.

[0108] Figure 8 is a flow diagram of method steps for generating a report 250 after detecting an incident, according to various embodiments. Although the method steps are described in conjunction with the systems of Figures 1-5, persons skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present embodiments.

[0109] As shown, a method 800 begins at step 802, where report generator 114 receives vehicle data 202 from a vehicle. Vehicle data 202 includes sensor data 224, camera data 226, and / or controller data 228 collected by the vehicle by incident module 146. Report generator 114 can further receive incident signal 204 from incident module 146. Report generator receives vehicle data 202 and / or incident signal 204 from incident module 146 in response to incident module 146 detecting an incident involving the vehicle.

[0110] At step 804, report generator 114 receives user supplemental data 206. User supplemental data 206 can be any data related to the incident that is collected from a user (e.g., the driver of the vehicle) by incident application 156. For example, user supplemental data 206 can include, without limitation, other driver(s) information 512, incident scene images 514, witness information 516, and injury reports 518. In some cases, report generator 114 receives user supplemental data 206 automatically from incident application 156 (e.g., when incident module 146 sends incident signal 204 to incident application 156) or in response to report generator 114 sending incident signal 204 to incident application 156.[oni] At step 806, reconstruction module 232 generates a timeline reconstruction of the incident, processes media files included in vehicle data 202 and user supplemental data 206. Reconstruction module 232 generates the reconstruction timeline by aligning and stitching different parts of vehicle data 202 and user supplemental data 206. For example, reconstruction module 232 can stitch videos and / or images included in camera data 226 taken at different times and / or angles to generate a timeline of the incident in pictures or videos based on the time stamps associated with camera data 226. Reconstruction module 232 can further annotate and / or add notes to the timeline with information about relevant vehicle data 202 and / or user supplemental data 206. Reconstruction module 232 uses one or more of trained model(s) 234 to generate the timeline.

[0112] At step 808, report generator 114 generates a report 250. Report generator 114 uses vehicle data 202, user supplemental data 206, insurance policy information 208, police report(s) 210 as well as other collected data (e.g., V2X data) to generate report 250. Report generator 114 can further include the reconstructed timeline generated by reconstruction module 232 in report 250. Report generator 114 also uses one or more of trained model(s) 234 to generate one or more portions of the report. Report 250 that includes details of the incident, the insurance policy information, a summary of the incident, such as location, role of parties, external conditions, such as weather conditions, road conditions, and traffic data, alertness state of the driver, the driver’s health and vital signs, technical issues, police reports, etc. Report 250 can include a narrative describing the timeline of the incident, a summary of the damage from the incident, and / or one or more observations or conclusions. Report generator 114 can further customize report 250 depending upon the insurance company to which report 250 is being submitted.

[0113] At step 810, report generator 114 transmits report 250 to an insurance company. Report generator 114 can identify the insurance company of the driver and the correspondingcontact information. Report generator 114 can transmit report 250 to the insurance company via email, fax, uploading to a user interface, or any other suitable way. Report generator 114 can identify the insurance company and / or where to transmit report 250 based on insurance policy information 208.

[0114] In sum, the disclosed techniques provide for automatic data collection and custom report generation following an incident involving a vehicle. The disclosed techniques continuously receive in-vehicle and surrounding sensor data. The disclosed techniques then detect an incident an impact) by processing the collected. Upon detecting an incident,sensor data leading up to the incident are captured and uploaded to a cloud infrastructure hosting a report generator. A phone application is also used to prompt a driver of the vehicle to capture and upload other information to the report generator. For example, the driver is instructed to take driver license photos of the other party, get contact details of potential witnesses, take photos of the other party insurance documents, police reports, etc. After receiving the data from the vehicle and the information collected using the custom phone application, the report generator processes the acquired data to generate an incident report. The generated report includes details of the incident, the driver policy details, a summary of the incident, such as location, role of parties, external conditions, alertness state of the driver, technical issues, police records, etc.

[0115] At least one technical advantage of the disclosed techniques relative to the prior art is that with the disclosed techniques the likelihood of contaminated data associated with a vehicle incident is greatly reduced. The disclosed techniques further reduce the likelihood of fraudulent claims or miss-interpretation of incident data. In addition, with the disclosed techniques, automatically collected data can be easily processed and automatically generate a report which reduces the time and effort needed to generate a report. The disclosed techniques also support the generation of reports based on incidents other than accidents. Furthermore, generating custom reports reduces costs by reducing manual labor and other resources needed to generate vehicle incident reports for different clients. These technical advantages provide one or more technological improvements over prior art approaches.

[0116] 1. In some embodiments, a computer-implemented method for generating incident reports comprises receiving a signal indicating that a vehicle has been involved in an incident, receiving vehicle data collected by the vehicle, receiving supplemental data from a mobile device, generating a report of the incident based on the vehicle data and the supplemental data, and transmitting the report to an insurance company.

[0117] 2 The method of clause 1, wherein the signal is generated by the vehicle in response to the vehicle data.

[0118] 3 The method of clauses 1 or 2, wherein generating the report comprises generating a timeline of the incident based on at least the vehicle data.

[0119] 4. The method of any of clauses 1-3, wherein generating the timeline comprises combining a plurality of images from the vehicle data based on time stamps associated with the plurality of images.

[0120] 5. The method of any of clauses 1-4, wherein generating the timeline comprises annotating the timeline based on one or more of indications of when one or more events occurred or information about a driver of the vehicle at one or more times.

[0121] 6. The method of any of clauses 1-5, wherein the information about the driver includes one or more of an attention level of the driver, a cognitive load of the driver, one or more vital signs of the driver, or whether the driver was using a mobile phone.

[0122] 7. The method of any of clauses 1-6, wherein generating the report comprises generating one or more of a narrative describing a timeline of the incident or a summary of damage due to the incident.

[0123] 8. The method of any of clauses 1-7, wherein generating the report comprises generating one or more observations or conclusions.

[0124] 9. The method of any of clauses 1-8, wherein generating the report comprises using one or more trained models to generate one or more portions of the report.

[0125] 10. The method of any of clauses 1-9, wherein generating the report comprises customizing the report based on one or more requirements of the insurance company.

[0126] 11. The method of any of clauses 1-10, wherein the vehicle data comprises one or more of sensor data from one or more sensors in the vehicle, camera data from one or more imaging devices in the vehicle, or controller data from one or control units or other systems in the vehicle.

[0127] 12. The method of any of clauses 1-11, wherein the supplemental data comprises one or more of information about drivers or other vehicles, one or more images of a scene ofthe incident, information about one or more witnesses to the incident, or one or more injury reports.

[0128] 13. The method of any of clauses 1-12, wherein generating of the report is further based on one or more of one or more police reports, insurance policy information, or vehicle to everything (V2X) data.

[0129] 14. In some embodiments, one or more non-transitory computer-readable media include instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of receiving a signal indicating that a vehicle has been involved in an incident, receiving vehicle data collected by the vehicle, receiving supplemental data from a mobile device, generating a report of the incident based on the vehicle data and the supplemental data, and transmitting the report to an insurance company.

[0130] 15. The one or more non-transitory computer-readable media of clause 14, wherein generating the report comprises generating a timeline of the incident by combining a plurality of images from the vehicle data based on time stamps associated with the plurality of images, and annotating the timeline based on one or more of indications of when one or more events occurred or information about a driver of the vehicle at one or more times.

[0131] 16. The one or more non-transitory computer-readable media of clauses 14 or 15, wherein the information about the driver includes one or more of an attention level of the driver, a cognitive load of the driver, one or more vital signs of the driver, or whether the driver was using a mobile phone.

[0132] 17. The one or more non-transitory computer-readable media of any of clauses 14-16, wherein generating the report comprises generating one or more of a narrative describing a timeline of the incident, a summary of damage due to the incident, or one or more observations or conclusions.

[0133] 18. The one or more non-transitory computer-readable media of any of clauses 14-17, wherein the steps further comprising transmitting a request to the mobile device for the supplemental data.

[0134] 19. The one or more non-transitory computer-readable media of any of clauses 14-18, wherein the mobile device is a mobile device of a driver of the vehicle.

[0135] 20. In some embodiments, a system comprises one or more memories storing instructions, and one or more processors coupled to the one or more memories that, when executing the instructions, perform the steps of receiving a signal indicating that a vehicle has been involved in an incident, receiving vehicle data collected by the vehicle, receiving supplemental data from a mobile device, generating a report of the incident based on the vehicle data and the supplemental data, and transmitting the report to an insurance company.

[0136] Any and all combinations of any of the claim elements recited in any of the claims and / or any elements described in this application, in any fashion, fall within the contemplated scope of the present invention and protection.

[0137] The descriptions of the various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

[0138] Aspects of the present embodiments may be embodied as a system, method, or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “module,” a “system,” or a “computer.” In addition, any hardware and / or software technique, process, function, component, engine, module, or system described in the present disclosure may be implemented as a circuit or set of circuits. Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

[0139] Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portablecompact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0140] Aspects of the present disclosure are described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine. The instructions, when executed via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / acts specified in the flowchart and / or block diagram block or blocks. Such processors may be, without limitation, general purpose processors, special-purpose processors, applicationspecific processors, or field-programmable gate arrays.

[0141] The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0142] While the preceding is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method for generating incident reports, the method comprising: receiving a signal indicating that a vehicle has been involved in an incident; receiving vehicle data collected by the vehicle; receiving supplemental data from a mobile device; generating a report of the incident based on the vehicle data and the supplemental data; and transmitting the report to an insurance company.

2. The method of claim 1, wherein the signal is generated by the vehicle in response to the vehicle data.

3. The method of claim 1, wherein generating the report comprises generating a timeline of the incident based on at least the vehicle data.

4. The method of claim 3, wherein generating the timeline comprises combining a plurality of images from the vehicle data based on time stamps associated with the plurality of images.

5. The method of claim 3, wherein generating the timeline comprises annotating the timeline based on one or more of indications of when one or more events occurred or information about a driver of the vehicle at one or more times.

6. The method of claim 5, wherein the information about the driver includes one or more of an attention level of the driver, a cognitive load of the driver, one or more vital signs of the driver, or whether the driver was using a mobile phone.

7. The method of claim 1, wherein generating the report comprises generating one or more of a narrative describing a timeline of the incident or a summary of damage due to the incident.

8. The method of claim 1, wherein generating the report comprises generating one or more observations or conclusions.

9. The method of claim 1, wherein generating the report comprises using one or more trained models to generate one or more portions of the report.

10. The method of claim 1, wherein generating the report comprises customizing the report based on one or more requirements of the insurance company.

11. The method of claim 1, wherein the vehicle data comprises one or more of: sensor data from one or more sensors in the vehicle; camera data from one or more imaging devices in the vehicle; or controller data from one or control units or other systems in the vehicle.

12. The method of claim 1, wherein the supplemental data comprises one or more of: information about drivers or other vehicles; one or more images of a scene of the incident; information about one or more witnesses to the incident; or one or more injury reports.

13. The method of claim 1, wherein generating of the report is further based on one or more of: one or more police reports; insurance policy information; or vehicle to everything (V2X) data.

14. One or more non-transitory computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: receiving a signal indicating that a vehicle has been involved in an incident; receiving vehicle data collected by the vehicle; receiving supplemental data from a mobile device; generating a report of the incident based on the vehicle data and the supplemental data; and transmitting the report to an insurance company.

15. The one or more non-transitory computer-readable media of claim 14, wherein generating the report comprises: generating a timeline of the incident by combining a plurality of images from the vehicle data based on time stamps associated with the plurality of images; and annotating the timeline based on one or more of indications of when one or more events occurred or information about a driver of the vehicle at one or more times.

16. The one or more non-transitory computer-readable media of claim 15, wherein the information about the driver includes one or more of an attention level of the driver, a cognitive load of the driver, one or more vital signs of the driver, or whether the driver was using a mobile phone.

17. The one or more non-transitory computer-readable media of claim 14, wherein generating the report comprises generating one or more of a narrative describing a timeline of the incident, a summary of damage due to the incident, or one or more observations or conclusions.

18. The one or more non-transitory computer-readable media of claim 14, wherein the steps further comprising transmitting a request to the mobile device for the supplemental data.

19. The one or more non-transitory computer-readable media of claim 14, wherein the mobile device is a mobile device of a driver of the vehicle.

20. A system comprising: one or more memories storing instructions; and one or more processors coupled to the one or more memories that, when executing the instructions, perform the steps of: receiving a signal indicating that a vehicle has been involved in an incident; receiving vehicle data collected by the vehicle; receiving supplemental data from a mobile device; generating a report of the incident based on the vehicle data and the supplemental data; and transmitting the report to an insurance company.

Images