Vehicle video recording framework

Abstract

A system for processing a plurality of video recordings of a vehicle includes a camera system having a camera buffer. The system also includes a video storage device. The system also includes a vehicle communication system configured to communicate with a remote server. The system also includes a vehicle controller. The vehicle controller is programmed to capture one of the plurality of video recordings using the camera system. The one of the plurality of video recordings is stored to the camera buffer. The vehicle controller is further programmed to generate at least one event video file based on the one of the plurality of video recordings and save the at least one event video file to the video storage device. The vehicle controller is further programmed to upload the at least one event video file from the video storage device to the remote server using the vehicle communication system.

Description

Technical Field

[0001] This disclosure relates to systems and methods for enhancing occupant awareness of a vehicle, and more specifically to vehicle digital video recording systems and methods. Background Technology

[0002] To enhance occupant awareness and convenience, vehicles may be equipped with digital video recording systems configured to capture images and / or video of events occurring in the vehicle's surrounding environment. These systems may utilize at least one camera fixed inside / on the vehicle to monitor events and capture images / video. Digital video recording systems can categorize events based on multiple characteristics. However, they may not be able to account for other factors, such as multiple events occurring simultaneously. Furthermore, digital video recording systems may lack the ability to allow vehicle occupants to view the recordings captured by the system in a timely manner.

[0003] Therefore, while current digital video recording systems and methods have achieved their intended purpose, a new and improved system and method for digital video recording of vehicles is needed. Summary of the Invention

[0004] According to several aspects, the present invention provides a system for processing multiple video recordings of a vehicle. The system includes a camera system configured to capture multiple video recordings of the vehicle's surrounding environment. The camera system includes a camera buffer configured to store the multiple video recordings. The system also includes a video storage device. The system further includes a vehicle communication system configured to communicate with a remote server. The system also includes a vehicle controller electrically communicating with the camera system, the video storage device, and the vehicle communication system. The vehicle controller is programmed to capture one of the multiple video recordings using the camera system. One of the multiple video recordings is stored in the camera buffer. The vehicle controller is also programmed to generate at least one event video file based on one of the multiple video recordings. The vehicle controller is further programmed to save the at least one event video file to the video storage device. The vehicle controller is also programmed to upload the at least one event video file from the video storage device to the remote server using the vehicle communication system.

[0005] In another aspect of this disclosure, in order to capture one of a plurality of video recordings, the vehicle controller is also programmed to identify multiple events in one of the plurality of video recordings. In order to capture one of the plurality of video recordings, the vehicle controller is also programmed to determine the importance level of each of the multiple events in one of the plurality of video recordings.

[0006] In another aspect of this disclosure, in order to generate at least one event video file, the vehicle controller is also programmed to generate at least one event video file based on one of a plurality of video recordings. The at least one event video file includes metadata identifying one of a plurality of events from the plurality of video recordings that has the highest importance level.

[0007] In another aspect of this disclosure, one of the multiple video recordings includes multiple sets of images. To capture one of the multiple video recordings, the vehicle controller is also programmed to capture one of the multiple video recordings into a camera buffer. For each of the multiple sets of images in one of the multiple video recordings, the vehicle controller is also programmed to identify at least one event in that set of images. For each of the multiple sets of images in one of the multiple video recordings, the vehicle controller is also programmed to store metadata for that set of images in the camera buffer. The metadata includes an event field identifying at least one event, a camera field identifying the camera that captured the set of images, and a timestamp field including the time the set of images was captured.

[0008] In another aspect of this disclosure, at least one event video file comprises multiple event video files. To generate the multiple event video files, the vehicle controller is also programmed to determine multiple unique events contained in the event fields of the metadata of all multiple sets of images in one of the multiple video recordings. To generate the multiple event video files, the vehicle controller is also programmed to generate the multiple event video files based on multiple sets of images in one of the multiple video recordings. Each of the multiple event video files corresponds to one of the multiple unique events.

[0009] In another aspect of this disclosure, in order to generate one of a plurality of event video files corresponding to one of a plurality of unique events, the vehicle controller is also programmed to generate one of the plurality of event video files. One of the plurality of event video files is a concatenation of subsets of images from one of a plurality of video recordings. The event field of the metadata for each subset of the plurality of images contains at least one of the plurality of unique events. The camera field of the metadata for each subset of the plurality of images is identical.

[0010] In another aspect of this disclosure, in order to upload multiple event video files, the vehicle controller is also programmed to determine an event upload priority for each of the multiple unique events. In order to upload multiple event video files, the vehicle controller is also programmed to determine a file upload priority for each of the multiple event video files. In order to upload multiple event video files, the vehicle controller is also programmed to upload each of the multiple event video files based at least in part on the event upload priority and the file upload priority.

[0011] In another aspect of this disclosure, for uploading multiple event video files, the vehicle controller is also programmed to determine multiple event upload time limits. One of the multiple event upload time limits corresponds to each of the multiple unique events. For uploading multiple event video files, the vehicle controller is also programmed to determine a file upload time allocation for each of the multiple event video files, at least in part based on the multiple event upload time limits and file upload priority. For uploading multiple event video files, the vehicle controller is also programmed to upload each of the multiple event video files, at least in part based on the event upload priority, file upload priority, and file upload time allocation for each of the multiple event video files.

[0012] In another aspect of this disclosure, in order to upload one of a plurality of event video files, the vehicle controller is also programmed to establish a connection with a remote server using a vehicle communication system. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to use the connection with the remote server to upload a first set of images from one of the plurality of event video files. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to measure the connection speed with the remote server. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to determine the expected achievable data upload amount based at least in part on file upload time allocation and connection speed. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to compare the expected achievable data upload amount with the remaining data amount of one of the plurality of event video files. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to adjust the bitrate of a second set of images from one of the plurality of event video files in response to determining that the expected achievable data upload amount is less than the remaining data amount of one of the plurality of event video files. The bitrate adjustment is based at least in part on the expected achievable data upload amount and the remaining data amount of one of the plurality of event video files. In order to upload one of multiple event video files, the vehicle controller is also programmed to upload a second set of images from one of the multiple event video files.

[0013] In another aspect of this disclosure, in order to upload multiple event video files, the vehicle controller is also programmed to identify high-priority regions in each of the multiple event video files.

[0014] According to several aspects, the present invention provides a method for processing multiple video recordings of a vehicle. The method includes capturing one of the multiple video recordings using a camera system. One of the multiple video recordings is stored in a camera buffer. The method also includes generating at least one event video file based on one of the multiple video recordings. The method further includes saving the at least one event video file to a video storage device. The method also includes uploading the at least one event video file from the video storage device to a remote server using a vehicle communication system.

[0015] In another aspect of this disclosure, capturing one of a plurality of video recordings may further include identifying multiple events within one of the plurality of video recordings. Capturing one of a plurality of video recordings may also include determining the importance level of each of the multiple events within one of the plurality of video recordings.

[0016] In another aspect of this disclosure, generating at least one event video file may further include generating at least one event video file based on one of a plurality of video records. The at least one event video file includes metadata identifying one of a plurality of events from the plurality of video records that has the highest importance level.

[0017] In another aspect of this disclosure, capturing one of a plurality of video recordings may further include capturing one of the plurality of video recordings into a camera buffer. One of the plurality of video recordings includes multiple sets of images. For each of the multiple sets of images in one of the plurality of video recordings, the method further includes identifying at least one event in that set of images. For each of the multiple sets of images in one of the plurality of video recordings, the method further includes storing metadata of that set of images in the camera buffer. The metadata includes an event field identifying at least one event, a camera field identifying the camera that captured the set of images, and a timestamp field including the time the set of images was captured.

[0018] In another aspect of this disclosure, at least one event video file includes multiple event video files. Generating multiple event video files may further include identifying multiple unique events contained in the event fields of the metadata of all multiple sets of images in one of the multiple video records. Generating multiple event video files may also include generating multiple event video files based on multiple sets of images in one of the multiple video records. Each of the multiple event video files corresponds to one of the multiple unique events.

[0019] In another aspect of this disclosure, generating one of a plurality of event video files corresponding to one of a plurality of unique events may further include generating one of a plurality of event video files, wherein one of the event video files is a concatenation of a subset of multiple sets of images from one of a plurality of video recordings. The event field of the metadata for each subset of the multiple sets of images contains at least one of the plurality of unique events. The camera field of the metadata for each subset of the multiple sets of images is identical.

[0020] In another aspect of this disclosure, at least one event video file includes multiple event video files. Uploading multiple event video files also includes establishing a connection with a remote server using a vehicle communication system. Uploading multiple event video files also includes uploading a first set of images from one of the multiple event video files using the connection with the remote server. Uploading multiple event video files also includes measuring the connection speed with the remote server. Uploading multiple event video files also includes determining the expected achievable data upload volume based at least in part on file upload time allocation and connection speed. Uploading multiple event video files also includes comparing the expected achievable data upload volume with the remaining data volume of one of the multiple event video files. Uploading multiple event video files also includes adjusting the bitrate of a second set of images from one of the multiple event video files in response to determining that the expected achievable data upload volume is less than the remaining data volume of one of the multiple event video files. The bitrate adjustment is based at least in part on the expected achievable data upload volume and the remaining data volume of one of the multiple event video files. Uploading multiple event video files also includes uploading the second set of images from one of the multiple event video files.

[0021] According to several aspects, this disclosure provides a system for processing multiple video recordings of a vehicle. The system includes a camera system configured to capture video recordings of the environment surrounding the vehicle. The camera system includes a camera buffer configured to store the video recordings. The system also includes a video storage device. The system further includes a vehicle communication system configured to communicate with a remote server. The system also includes a vehicle controller electrically communicating with the camera system, the video storage device, and the vehicle communication system. The vehicle controller is programmed to capture video recordings using the camera system. One of the multiple video recordings is stored in the camera buffer. One of the multiple video recordings includes multiple sets of images. The vehicle controller is also programmed to identify at least one event in the set of images. The vehicle controller is also programmed to store metadata of the set of images in the camera buffer. The metadata includes an event field identifying at least one event, a camera field identifying the camera that captured the set of images, and a timestamp field including the time the set of images was captured. The vehicle controller is also programmed to determine multiple unique events contained in the event fields of the metadata of all the multiple sets of images in one of the multiple video recordings. The vehicle controller is also programmed to generate multiple event video files based on the multiple sets of images in one of the multiple video recordings. Each of the multiple event video files corresponds to one of multiple unique events. Each of the multiple event video files is a concatenation of a subset of images from one of multiple video recordings. The vehicle controller is also programmed to save the multiple event video files to a video storage device. The vehicle controller is also programmed to upload the multiple event video files from the video storage device to a remote server using the vehicle communication system.

[0022] In another aspect of this disclosure, for uploading multiple event video files, the vehicle controller is also programmed to determine an event upload priority for each of the multiple unique events. For uploading multiple event video files, the vehicle controller is also programmed to determine a file upload priority for each of the multiple event video files. For uploading multiple event video files, the vehicle controller is also programmed to determine multiple event upload time limits, wherein one of the multiple event upload time limits corresponds to each of the multiple unique events. For uploading multiple event video files, the vehicle controller is also programmed to determine a file upload time allocation for each of the multiple event video files, at least in part based on the multiple event upload time limits and the file upload priority. For uploading multiple event video files, the vehicle controller is also programmed to identify a high-priority region for each of the multiple event video files. For uploading multiple event video files, the vehicle controller is also programmed to upload the high-priority region for each of the multiple event video files. For uploading multiple event video files, the vehicle controller is also programmed to upload each of the multiple event video files, at least in part based on the event upload priority, file upload priority, and file upload time allocation for each of the multiple event video files.

[0023] In another aspect of this disclosure, in order to upload one of a plurality of event video files, the vehicle controller is also programmed to establish a connection with a remote server using a vehicle communication system. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to upload a first set of images from one of the plurality of event video files using the connection to the remote server. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to measure the connection speed with the remote server. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to determine the expected achievable data upload amount based at least in part on file upload time allocation and connection speed. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to compare the expected achievable data upload amount with the remaining data amount of one of the plurality of event video files. In order to upload one of the plurality of event video files, the vehicle controller is also programmed to adjust the bitrate of a second set of images from one of the plurality of event video files in response to determining that the expected achievable data upload amount is less than the remaining data amount of one of the plurality of event video files. The bitrate adjustment is based at least in part on the expected achievable data upload amount and the remaining data amount of one of the plurality of event video files. In order to upload one of multiple event video files, the vehicle controller is also programmed to upload a second set of images from one of the multiple event video files.

[0024] Further areas of application will become apparent from the description provided herein. It should be understood that these descriptions and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0025] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0026] Figure 1 This is a system diagram of a system for processing multiple video recordings of a vehicle, according to an exemplary embodiment;

[0027] Figure 2 This is a flowchart illustrating a method for processing multiple video recordings of a vehicle according to a first exemplary embodiment;

[0028] Figure 3 This is a flowchart illustrating a method for processing multiple video recordings of a vehicle according to a second exemplary embodiment; and

[0029] Figure 4 This is a method for uploading multiple event video files to a remote server system according to an exemplary embodiment. Detailed Implementation

[0030] The following description is merely exemplary in nature and is not intended to limit this disclosure, application, or use.

[0031] Reference Figure 1 A system for processing multiple video recordings from a vehicle is shown, generally indicated by reference numeral 10. System 10 is shown together with an exemplary vehicle 12. Although a passenger vehicle is shown, it should be understood that vehicle 12 can be any type of vehicle without departing from the scope of this disclosure. System 10 typically includes a vehicle controller 14, a camera system 16, a video storage device 18, and a vehicle communication system 20.

[0032] The vehicle controller 14 is used to implement methods 100 and 300 for processing multiple video recordings of a vehicle, as described below. The vehicle controller 14 includes at least one processor 22 and a non-transitory computer-readable storage device or medium 24. The processor 22 may be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among multiple processors associated with the vehicle controller 14, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally a device for executing instructions. The computer-readable storage device or medium 24 may include, for example, volatile and non-volatile storage devices in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operational variables when the processor 22 is powered off. The computer-readable storage device or medium 24 may be implemented using multiple storage devices, such as PROM (Programmable Read-Only Memory), EPROM (Electrically Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, or other electrical, magnetic, optical, or combined storage devices capable of storing data (some of which represent executable instructions used by the vehicle controller 14 to control various systems of the vehicle 12). The vehicle controller 14 may also consist of multiple controllers that are electrically in communication with each other. The vehicle controller 14 may interconnect with other systems and / or controllers of the vehicle 12, thereby allowing the vehicle controller 14 to access data such as the speed, acceleration, braking, and steering angle of the vehicle 12.

[0033] The vehicle controller 14 is in electrical communication with the camera system 16, the video storage device 18, and the vehicle communication system 20. In an exemplary embodiment, electrical communication is established using, for example, a CAN bus, a Wi-Fi network, a cellular data network, etc. It should be understood that various other wired and wireless technologies and communication protocols used for communicating with the vehicle controller 14 are within the scope of this disclosure.

[0034] Camera system 16 is used to capture images and / or record video of the environment surrounding vehicle 12. In an exemplary embodiment, camera system 16 includes a photographic and / or video camera positioned to observe the environment 24 surrounding vehicle 12. In a non-limiting example, camera system 16 includes a camera fixed inside vehicle 12 (e.g., in the roof of vehicle 12) having a field of view through the windshield of vehicle 12. In another non-limiting example, camera system 16 includes a camera fixed outside vehicle 12 (e.g., on the roof of vehicle 12) having a field of view of the environment surrounding vehicle 12.

[0035] In another exemplary embodiment, the camera system 16 is a surround-view camera system, including a plurality of cameras (also referred to as satellite cameras) arranged to provide views of the environment adjacent to all sides of the vehicle 12. In a non-limiting example, the camera system 16 includes a front camera (e.g., mounted in the front grille of the vehicle 12), a rear camera (e.g., mounted on the tailgate of the vehicle 12), and two side cameras (e.g., mounted below each of the two side mirrors of the vehicle 12). In another non-limiting example, the camera system 16 also includes an additional rear-view camera mounted near the center high-mounted brake light of the vehicle 12.

[0036] In an exemplary embodiment, the camera system 16 further includes a camera buffer 26. The camera buffer 26 is used to temporarily store video recordings captured by the camera system 16 during recording until the video recordings are processed and stored on the video storage device 18, as will be discussed in further detail below. In an exemplary embodiment, the camera buffer 26 is a volatile data storage device, such as random access memory (RAM), including dynamic RAM (DRAM) and / or static RAM (SRAM). As described above, the camera buffer 26 is in electrical communication with each camera of the camera system 16 and the vehicle controller 14.

[0037] It should be understood that camera systems with other cameras and / or other mounting locations are all within the scope of this disclosure. It should also be understood that cameras with various sensor types (including, for example, charge-coupled device (CCD) sensors, complementary metal-oxide-semiconductor (CMOS) sensors, and / or high dynamic range (HDR) sensors) are all within the scope of this disclosure. Furthermore, cameras with various lens types, including, for example, wide-angle lenses and / or narrow-angle lenses, are also within the scope of this disclosure.

[0038] Video storage device 18 is used to store video recordings on vehicle 12 for subsequent retrieval. In an exemplary embodiment, video storage device 18 is a non-volatile, persistent storage medium, such as a hard disk drive, tape drive, solid-state drive, etc. In a non-limiting example, video storage device 18 includes a solid-state drive (SSD) that utilizes NAND flash memory to store video recordings in digital data form. In another non-limiting example, video storage device 18 includes a hard disk drive (HDD) that utilizes a magnetizable material to store video recordings in digital data form. As described above, video storage device 18 is in electrical communication with vehicle controller 14.

[0039] Vehicle controller 14 uses vehicle communication system 20 to communicate with other systems outside vehicle 12. For example, vehicle communication system 20 includes the ability to communicate with vehicles (“V2V” communication), infrastructure (“V2I” communication), remote systems at remote call centers (e.g., General Motors’ ON-STAR), and / or personal devices. Generally, the term vehicle-to-everything communication (“V2X” communication) refers to communication between vehicle 12 and any remote system (e.g., vehicles, infrastructure, and / or remote systems). In some embodiments, vehicle communication system 20 is a wireless communication system configured to communicate using the IEEE 802.11 standard or via a wireless local area network (WLAN) using cellular data communication. However, additional or alternative communication methods such as dedicated short-range communication (DSRC) channels and / or mobile telecommunications protocols based on 3GPP standards are also considered within the scope of this disclosure. A DSRC channel refers to a one-way or two-way short- to medium-range wireless communication channel designed specifically for automotive use, along with a set of corresponding protocols and standards. 3GPP refers to a partnership among multiple standards organizations that develop mobile telecommunications protocols and standards. 3GPP standards are structured as “releases”. Therefore, communication methods based on 3GPP releases 14, 15, 16, and / or future 3GPP releases are considered within the scope of this disclosure. Thus, the vehicle communication system 20 may include one or more antennas and / or communication transceivers for receiving and / or transmitting signals, such as cooperative sensing messages (CSM). The vehicle communication system 20 is configured to wirelessly transmit information between vehicle 12 and another vehicle. Furthermore, the vehicle communication system 20 is configured to wirelessly transmit information between vehicle 12 and infrastructure or other vehicles.

[0040] Continue to refer to Figure 1 A remote server system is shown and is generally indicated by reference numeral 28. The remote server system 28 includes a server controller 30 that is in electrical communication with a server database 32 and a server communication system 34. In a non-limiting example, the remote server system 28 is located in a server cluster, data center, etc., and is connected to the Internet. The server controller 30 includes at least one server processor 36 and a server non-transitory computer-readable storage device or server medium 38. The description of the type and configuration given above for the vehicle controller 14 also applies to the server controller 30. The description given above for the vehicle communication system 20 also applies to the server communication system 34. The server communication system 34 is used to communicate with external systems (e.g., the vehicle controller 14) via the vehicle communication system 20.

[0041] Reference Figure 2The diagram shows a flowchart of a method 100 for processing multiple video recordings from a vehicle. Method 100 begins at box 102 and proceeds to boxes 104 and 106.

[0042] At block 104, the vehicle controller 14 uses the camera system 16 to permanently capture video recordings to the camera buffer 26. In an exemplary embodiment, when the camera buffer 26 is full, the video recordings stored on the camera buffer 26 are overwritten by new video recordings.

[0043] At box 106, vehicle controller 14 monitors the permanent video recording captured at box 104 and determines whether an activation event has occurred. In a non-limiting example, an activation event includes one of a detected event and an occupant-initiated event. Detected events include, for example, collisions and / or near-collisions between vehicle 12 and / or other vehicles, pedestrians or animals, theft and / or attempted theft of items from vehicle 12 and / or vehicle 12, vandalism and / or attempted vandalism of vehicle 12 and / or other vehicles, anomalous events such as natural disasters, road rage incidents involving vehicle 12 and / or other vehicles, traffic violations by vehicle 12 and / or other vehicles, etc. In an exemplary embodiment, other vehicle sensors, such as accelerometers, vision sensors (e.g., camera system 16), etc., are used to detect the detected event.

[0044] An occupant-initiated event is an event manually activated by an occupant of vehicle 12. In an exemplary embodiment, the occupant of vehicle 12 uses a human-machine interface device (HID), such as a button (e.g., an electromechanical button, a capacitive button, etc.), a touch screen, a knob, etc., to initiate an occupant-initiated event.

[0045] At box 106, if the activation event has not yet occurred, method 100 returns to box 104 and continues video recording. If the activation event has occurred, method 100 proceeds to box 108.

[0046] At box 108, vehicle controller 14 identifies multiple events in the video recording captured at box 104. These events include detected events or events initiated by the occupant, as described above. Following box 108, method 100 proceeds to box 110.

[0047] At block 110, vehicle controller 14 determines the importance level of each of the plurality of events identified at block 108. In an exemplary embodiment, the importance level is determined based on the nature of each of the plurality of events. In a non-limiting example, a collision involving vehicle 12 is determined to have a higher importance level than a collision involving only other vehicles and not vehicle 12. In another non-limiting example, a collision involving vehicle 12 is determined to have a higher importance level than a road rage event involving vehicle 12. In an exemplary embodiment, medium 24 of vehicle controller 14 includes a lookup table that includes a plurality of possible events and a corresponding predetermined importance level for each of the plurality of possible events. In another exemplary embodiment, the importance level of each of the plurality of possible events in the lookup table may be configured by the occupants of vehicle 12 using an HID (e.g., a touchscreen display). Thus, vehicle controller 14 uses the lookup table to determine the importance level of each of the plurality of events. After block 110, method 100 proceeds to block 112.

[0048] At block 112, vehicle controller 14 generates at least one event video file based on the video recording captured at block 104. In an exemplary embodiment, the event video file is a digital video file that includes views from camera system 16 from at least one event. In a non-limiting example, the event video file may include a sub-portion of the video recording captured at block 104, for example, thirty seconds before the occurrence of the first event identified at block 108 and sixty seconds after the occurrence of the last event identified at block 108. In an exemplary embodiment, the event video file includes metadata identifying one of the plurality of events identified at block 108. In a non-limiting example, as determined at block 110, the metadata identifies the event with the highest importance level among the plurality of events. For example, if the video recording includes a collision event and a road rage event, and the collision event has a higher importance level than the road rage event, the metadata of the event video file will identify the collision event. After block 112, method 100 proceeds to block 114.

[0049] At box 114, the event video file generated at box 112 is stored in video storage device 18, making camera buffer 26 available for further recording. In the exemplary embodiment, the event video file in video storage device 18 is categorized according to the metadata discussed above with reference to box 112. After box 114, method 100 proceeds to box 116.

[0050] At box 116, the vehicle communication system 20 uploads the event video file saved at box 114 from video storage device 18 to remote server system 28. In an exemplary embodiment, the vehicle communication system 20 establishes a connection with the server communication system 34 of the remote server system 28 and uploads the event video file to the remote server system 28. In a non-limiting example, the remote server system 28 saves the uploaded event video file in server database 32. In an exemplary embodiment, the event video files in server database 32 are categorized according to the metadata discussed above with reference to box 112. In an exemplary embodiment, the event video file is then retrieved from server system 28 for further action. For example, an occupant can then use a mobile device (e.g., smartphone, tablet, personal computer, etc.) to establish a connection with remote server system 28 and view the event video file.

[0051] It should be understood that there may be a delay (e.g., several seconds, minutes, hours, etc.) between the execution of blocks 114 and 116 of method 100. In other words, the event video file may not be uploaded to the remote server system 28 immediately, but may be stored in the video storage device 18 for a delay period before being uploaded in block 116.

[0052] Following block 116, method 100 enters a standby state at block 118. In an exemplary embodiment, controller 14 repeatedly exits standby state 118 and restarts method 100 at block 102. In a non-limiting example, controller 14 exits standby state 118 and restarts method 100 according to a timer (e.g., every 300 milliseconds).

[0053] refer to Figure 3 A flowchart of a method 300 for processing multiple video recordings of a vehicle is shown. Method 300 begins at block 302 and proceeds to block 304. At block 304, the vehicle controller 14 uses the camera system 16 to capture a group of pictures (GOPs) into the camera buffer 26. Within the scope of this disclosure, a GOP is a collection of consecutive pictures within an encoded video stream. Each encoded video stream includes multiple GOPs. Therefore, the video recordings captured by the camera system 16 include multiple GOPs. After block 304, method 300 proceeds to block 306.

[0054] At block 306, vehicle controller 14 identifies an event in the GOP captured at block 304. Within the scope of this disclosure, the event identified at block 306 includes at least one of detected events and occupant-initiated events. Detected events include, for example, collisions and / or near-collisions between vehicle 12 and / or other vehicles, pedestrians or animals, theft and / or attempted theft of items from vehicle 12 and / or vehicle 12, vandalism and / or attempted vandalism of vehicle 12 and / or other vehicles, anomalous events such as natural disasters, road rage incidents involving vehicle 12 and / or other vehicles, traffic violations by vehicle 12 and / or other vehicles, etc. In an exemplary embodiment, other vehicle sensors, such as accelerometers, vision sensors (e.g., camera system 16), etc., are used to detect the detected event.

[0055] An occupant-initiated event is an event manually activated by an occupant of vehicle 12. In an exemplary embodiment, the occupant of vehicle 12 uses a human-machine interface device (HID), such as a button (e.g., an electromechanical button, a capacitive button, etc.), a touch screen, a knob, etc., to initiate an occupant-initiated event.

[0056] If no event is detected at box 306, method 300 returns to box 304 to capture another GOP into camera buffer 26. If at least one event is detected at box 306, method 300 proceeds to box 308.

[0057] At box 308, vehicle controller 14 saves the metadata of the GOP captured at box 304 to camera buffer 26. In an exemplary embodiment, the metadata includes multiple fields. Each of the multiple fields contains information about the GOP. In a non-limiting example, the metadata includes an event field, a camera field, and a timestamp field. The event field includes the event identified at box 306. The camera field identifies which of the multiple cameras of camera system 16 captured the GOP. In a non-limiting example, each of the multiple cameras of camera system 16 is assigned a unique identifier (UID), and the UID of one of the multiple cameras that captured the GOP is stored in the camera field of the metadata for that GOP. The timestamp includes the time when the GOP was captured. After box 308, method 300 proceeds to box 310.

[0058] At box 310, if camera buffer 26 is not full, method 300 returns to box 304 to capture subsequent GOPs. Therefore, camera buffer 26 can contain multiple GOPs (also called video recordings), with metadata corresponding to each of the multiple GOPs. If camera buffer 26 is full, method 300 proceeds to box 312.

[0059] At box 312, the vehicle controller 14 determines multiple unique events included in the metadata of all multiple GOPs in the camera buffer 26. For example, if the multiple GOPs include two GOPs, the first GOP having metadata for "vehicle collision" in the event field, and the second GOP having metadata for "vehicle collision, road rage event" in the event field, then the multiple unique events are "vehicle collision, road rage event". After box 312, method 300 proceeds to box 314.

[0060] At block 314, vehicle controller 14 generates multiple event video files, each corresponding to a given one of a plurality of unique events identified at block 312. Each of the multiple event video files is stored in video storage device 18. In an exemplary embodiment, to generate one of the multiple event video files corresponding to a given event among the multiple events, vehicle controller 14 concatenates a subset of multiple Groups of Pictures (GOPs) in camera buffer 26. The event field of the metadata of each GOP in the subset of multiple GOPs contains the given one of the multiple events. Furthermore, the camera field of the metadata of each GOP in the subset of multiple GOPs is identical (i.e., each GOP in the subset of multiple GOPs was captured by the same camera). The subset of multiple GOPs is concatenated chronologically based on the timestamp field of the metadata of each of the multiple GOPs. After block 314, method 300 proceeds to block 316.

[0061] At box 316, each of the multiple event video files generated at box 314 will be uploaded from video storage device 18 to remote server system 28, as will be referred to below. Figure 4 This will be discussed in more detail. It should be understood that a delay (e.g., several seconds, minutes, hours, etc.) may occur between the execution of blocks 314 and 316 of method 300. In other words, the event video file is not necessarily uploaded to the remote server system 28 immediately, but may be stored in the video storage device 18 for a delay before being uploaded in block 316. After block 316, method 300 enters the standby state at block 318.

[0062] In an exemplary embodiment, controller 14 repeatedly exits standby state 318 and restarts method 300 at block 302. In a non-limiting example, controller 14 exits standby state 318 and restarts method 300 according to a timer (e.g., every 300 milliseconds).

[0063] Reference Figure 4The diagram illustrates an exemplary embodiment of block 316 described above. The exemplary embodiment of block 316 begins with blocks 316a and 316b. At block 316a, the vehicle controller 14 determines the file upload priority for each of a plurality of event video files generated at block 314. In the exemplary embodiment, the file upload priority is determined at least in part based on the visibility of each of the plurality of unique events in each of the plurality of event video files. For example, if a first event video file displays a complete view of the event (e.g., the front camera of camera system 16 displays a complete view of a collision with the front end of vehicle 12), the first event video file displaying the complete view is determined to have the highest file upload priority. In another example, if a second event video file displays a partial view of the event (e.g., a side camera of camera system 16 displays a partial view of a collision with the front end of vehicle 12), the second event video file displaying the partial view is determined to have a lower file upload priority relative to the first event video file. In yet another example, if a third event video file does not show a view of the event (e.g., the rear camera of camera system 16 does not show a view of the collision with the front of vehicle 12), then the third event video file that does not show a view is determined to have the lowest file upload priority relative to the first and second event video files. In another exemplary embodiment, each of the plurality of event video files corresponding to one of the plurality of unique events is determined to have the same file upload priority, regardless of which camera of camera system 16 is used to record each of the plurality of event video files. Following block 316a, the exemplary embodiment of block 316 proceeds to block 316c.

[0064] At block 316b, vehicle controller 14 determines the event upload priority of each of the plurality of unique events determined at block 312. Within the scope of this disclosure, the event upload priority is determined at least in part based on the importance and / or time sensitivity of each of the plurality of unique events. For example, if the first of the plurality of unique events is a frontal collision involving vehicle 12, then the first of the plurality of unique events is determined to have the highest event upload priority. In another example, if the second of the plurality of unique events is an attempt to steal vehicle 12, then the second of the plurality of unique events is determined to have a lower event upload priority relative to the first of the plurality of unique events. In yet another example, if the third of the plurality of unique events is a minor traffic violation, then the third of the plurality of unique events is determined to have the lowest event upload priority relative to the first and second of the plurality of unique events. In an exemplary embodiment, the medium 24 of vehicle controller 14 includes a lookup table that includes a plurality of possible events and a predetermined event upload priority corresponding to each of the plurality of possible events. Following block 316b, the exemplary embodiment of block 316 proceeds to block 316c.

[0065] At block 316c, vehicle controller 14 determines a plurality of event upload time limits, each of which corresponds to one of a plurality of unique events determined at block 312. Within the scope of this disclosure, the event upload time limit is a time length within which all event video files corresponding to one of the plurality of unique events must be uploaded to remote server system 28. For example, if the first of the plurality of events is a vehicle collision, then the event upload time limit corresponding to the first of the plurality of events is equal to the police response time (e.g., ten minutes), allowing the police to review the event video file of the vehicle collision upon arrival. In an exemplary embodiment, the event upload time limit for each of the plurality of unique events is determined at least in part based on the event upload priority of each of the plurality of unique events determined at block 316b. In another exemplary embodiment, the medium 24 of vehicle controller 14 includes a lookup table comprising a plurality of possible events and a predetermined event time constraint corresponding to each of the plurality of possible events. Following block 316c, the exemplary embodiment of block 316 proceeds to block 316d.

[0066] At block 316d, vehicle controller 14 determines a plurality of file upload time allocations, each of which corresponds to one of a plurality of event video files. Within the scope of this disclosure, a file upload time allocation is an allocation of time length for uploading event video files. In an exemplary embodiment, the plurality of file upload time allocations are determined at least in part based on a plurality of event upload time limits determined at block 316c, a file upload priority for each of the plurality of event video files determined at block 316a, and a file size for each of the plurality of event video files. In a non-limiting example, for an exemplary event having three corresponding event video files, the following formula is used:

[0067]

[0068]

[0069]

[0070] Variable A1 is the file upload time allocation for the first of the three corresponding event video files. Variable A2 is the file upload time allocation for the second of the three corresponding event video files. Variable A3 is the file upload time allocation for the third of the three corresponding event video files. Variable T is one of the multiple event upload time limits corresponding to the exemplary event, as determined at box 316c. Variable S1 is the file size of the first of the three corresponding event video files. Variable S2 is the file size of the second of the three corresponding event video files. Variable S3 is the file size of the third of the three corresponding event video files. Constant α is the first file upload time allocation constant for the second of the three corresponding event video files. Constant β is the second file upload time allocation constant for the third of the three corresponding event video files. Both constants α and β are between 0 and 1, where α is greater than β. The first of the three corresponding event video files is determined at box 316a to be the one with the highest file upload priority among the three corresponding event video files. The second of the three corresponding event video files is determined at box 316a to be the one with a lower upload priority relative to the first of the three corresponding event video files. The third of the three corresponding event video files is the one identified at box 316a as having the lowest file upload priority relative to the first and second of the three corresponding event video files. Following box 316d, the exemplary embodiment of box 316 proceeds to box 316e.

[0071] At block 316e, vehicle controller 14 identifies one of a plurality of event video files to be uploaded to remote server system 28. In the first exemplary embodiment, in order to identify one of the plurality of event video files to be uploaded, vehicle controller 14 first identifies a plurality of unuploaded event video files, including each of the plurality of event video files that have not yet been uploaded to remote server system 28. Then, vehicle controller 14 identifies a subset of the plurality of unuploaded event video files determined at block 316a that has the highest file upload priority. One of the plurality of event video files to be uploaded is identified as one of the subset of unuploaded event video files, which corresponds to one of a plurality of unique events determined at block 316b that has the highest event upload priority.

[0072] In the second exemplary embodiment, in order to identify one of a plurality of event video files to be uploaded, the vehicle controller 14 first identifies a plurality of unuploaded event video files. Then, the vehicle controller 14 identifies a subset of the plurality of unuploaded event video files, which corresponds to one of a plurality of unique events with the highest event upload priority determined at block 316b. One of the plurality of event video files to be uploaded is identified as one of the subset of the plurality of unuploaded event video files with the highest file upload priority determined at block 316a.

[0073] The first and second exemplary embodiments described above may further include identifying a high-priority region among the plurality of event video files to be uploaded. Within the scope of this disclosure, a high-priority region is a sub-portion of one of the plurality of event video files. In a non-limiting example, the high-priority region includes the event that is the subject of the event video file, as well as predetermined regions of the event video files before and after that event. For example, if the exemplary event occurs at the three-minute timestamp in a ten-minute exemplary event video file, the high-priority region is, for example, the region in the exemplary event video file with timestamps from two minutes and thirty seconds to four minutes. The high-priority region of each of the plurality of event video files is uploaded first to allow for the rapid uploading of useful sub-portions of each of the plurality of event video files. After uploading the high-priority region of each of the plurality of event video files, the exemplary embodiment of box 316 may be repeated to upload the full version of each of the plurality of event video files. After box 316e, the exemplary embodiment of box 316 proceeds to box 316f.

[0074] At block 316f, vehicle controller 14 uses vehicle communication system 20 to establish a connection between system 10 and remote server system 28, and uploads the GOP of the event video file identified at block 316e to remote server system 28. Following block 316f, the exemplary embodiment of block 316 proceeds to block 316g.

[0075] At block 316g, the vehicle controller 14 uses the vehicle communication system 20 to measure the connection speed between system 10 and remote server system 28. In an exemplary embodiment, the connection speed is an upload speed measured in megabits per second (e.g., 15 megabits per second). In another exemplary embodiment, the vehicle communication system 20 performs multiple connection speed measurements over time and calculates a moving average. Following block 316g, the exemplary embodiment of block 316 proceeds to block 316h.

[0076] In block 316h, the vehicle controller 14 determines the expected achievable data upload amount. Within the scope of this disclosure, the expected achievable data upload amount is the amount of data expected to be uploaded within the remaining time allocated for the upload time of the currently uploaded event video file (i.e., the event video file identified at block 316e). The expected achievable data upload amount is determined based on the connection speed measured at block 316g and the remaining time allocated for the upload time of the currently uploaded event video file. For example, if the upload time allocation for the currently uploaded event video file is sixty seconds, and twenty seconds have elapsed, then the remaining time allocated for the upload time of the currently uploaded event video file is forty seconds. If the connection speed is ten megabits per second, then the expected achievable data upload amount is four hundred megabits per second. Following block 316h, the exemplary embodiment of block 316 proceeds to block 316i.

[0077] At block 316i, the vehicle controller 14 compares the expected achievable data upload amount determined at block 316h with the remaining file size (i.e., the remaining data amount) of the event video file being uploaded (i.e., the event video file identified at block 316e). The remaining file size is the file size of the portion of the event video file that has not yet been uploaded. If the expected achievable data upload amount is less than the remaining file size, the exemplary embodiment of block 316 proceeds to block 316j. If the expected achievable data upload amount is greater than or equal to the remaining file size, the exemplary embodiment of block 316 proceeds to block 316k.

[0078] At block 316j, vehicle controller 14 adjusts the bitrate of subsequent GOPs of the event video file being uploaded (i.e., the event video file identified at block 316e). In a non-limiting example, reducing the bitrate of subsequent GOPs results in a reduction in the remaining file size of the event video file being uploaded. In an exemplary embodiment, the bitrate is reduced such that the remaining file size of the event video file being uploaded is less than or equal to the expected achievable data upload amount determined at block 316h. Following block 316j, the exemplary embodiment of block 316 proceeds to block 316k.

[0079] At block 316k, vehicle controller 14 determines whether the event video file being uploaded (i.e., the event video file identified at block 316e) has been fully uploaded. In other words, vehicle controller 14 determines whether each GOP of the event video file being uploaded has been uploaded to remote server system 28. If the event video file being uploaded has not been fully uploaded, the exemplary embodiment of block 316 returns to block 316f to upload the subsequent GOPs of the event video file being uploaded. If the event video file being uploaded has been fully uploaded, the exemplary embodiment of block 316 proceeds to block 316l.

[0080] At block 3161, vehicle controller 14 determines whether all of the multiple event video files generated at block 314 have been uploaded. If not all of the multiple event video files have been uploaded, the exemplary embodiment of block 316 returns to block 316e to identify one of the multiple event video files for uploading to remote server system 28. If all of the multiple event video files have been uploaded, the exemplary embodiment of block 316 ends and method 300 continues.

[0081] The system 10 and methods 100 and 300 disclosed herein offer several advantages. By storing metadata for each GOP used to identify events occurring within a GOP, multi-event recording and classification are possible while minimizing the size required for the camera buffer 26. Using the upload method discussed in reference box 316, event video files can be uploaded within a time limit even if network connection quality varies during the upload process.

[0082] The descriptions in this disclosure are merely exemplary in nature, and changes that do not depart from the spirit and scope of this disclosure are intended to fall within its scope. Such changes should not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A system for processing multiple video recordings from a vehicle, the system comprising: A camera system is configured to capture multiple video recordings of the environment surrounding a vehicle, wherein the camera system includes a camera buffer configured to store the multiple video recordings; Video storage devices; The vehicle communication system is configured to communicate with a remote server; and The vehicle controller is electrically connected to the camera system, the video storage device, and the vehicle communication system, wherein the vehicle controller is programmed to: The camera system is used to capture one of the plurality of video recordings, wherein one of the plurality of video recordings is stored in the camera buffer, wherein one of the plurality of video recordings comprises multiple sets of images, and wherein, in order to capture one of the plurality of video recordings, the vehicle controller is further programmed to: One of the plurality of video recordings is captured into the camera buffer, wherein, for each of multiple sets of images of one of the plurality of video recordings, the vehicle controller is further programmed to: Identify at least one event in the image group; and The metadata of the image group is stored in the camera buffer, wherein the metadata includes an event field that identifies the at least one event, a camera field that identifies the camera that captured the image group, and a timestamp field that includes the time when the image group was captured. Based on one of the plurality of video recordings, at least one event video file is generated, wherein the at least one event video file comprises a plurality of event video files, and wherein, in order to generate the plurality of event video files, the vehicle controller is further programmed to: Determine multiple unique events contained in the event fields of the metadata of all the multiple sets of images in one of the multiple video records; and Multiple event video files are generated based on the multiple sets of images from one of the multiple video recordings, wherein each of the multiple event video files corresponds to one of the multiple unique events, and wherein, in order to generate one of the multiple event video files corresponding to one of the multiple unique events, the vehicle controller is further programmed to: Generate one of the plurality of event video files, wherein the plurality of event video files is a concatenation of a subset of the plurality of images in one of the plurality of video records, wherein the event field of the metadata of each subset of the plurality of images contains at least one of the plurality of unique events, and wherein the camera field of the metadata of each subset of the plurality of images is the same; Save the at least one event video file to the video storage device; and Using the vehicle communication system, the at least one event video file is uploaded from the video storage device to the remote server.

2. The system according to claim 1, wherein, In order to capture one of the multiple video recordings, the vehicle controller is also programmed to: Identify multiple events in one of the plurality of video recordings; as well as Determine the importance level of each of the multiple events in one of the multiple video recordings.

3. The system according to claim 2, wherein, In order to generate the at least one event video file, the vehicle controller is also programmed to: The at least one event video file is generated based on one of the plurality of video records, wherein the at least one event video file includes metadata identifying one of a plurality of events in the plurality of video records that has the highest importance level.

4. The system according to claim 1, wherein, In order to upload the multiple event video files, the vehicle controller is also programmed to: Determine the event upload priority for each of the plurality of unique events; Determine the file upload priority for each of the plurality of event video files; as well as Each of the plurality of event video files is uploaded, at least in part, based on the event upload priority and the file upload priority.

5. The system according to claim 4, wherein, In order to upload the multiple event video files, the vehicle controller is also programmed to: Determine multiple event upload time limits, wherein one of the multiple event upload time limits corresponds to each of the multiple unique events; and The file upload time allocation for each of the multiple event video files is determined, at least in part, based on the multiple event upload time limits and the file upload priority of each of the multiple event video files; and Each of the plurality of event video files is uploaded based at least in part on the event upload priority, file upload priority, and file upload time allocation of each of the plurality of event video files.

6. The system according to claim 5, wherein, In order to upload one of the multiple event video files, the vehicle controller is also programmed to: Use the vehicle communication system to establish a connection with the remote server; Using the connection to the remote server, upload the first set of images from one of the plurality of event video files; Measure the connection speed to the remote server; The expected achievable data upload volume is determined at least in part based on the file upload time allocation and the connection speed; The expected achievable data upload volume is compared with the remaining data volume of one of the plurality of event video files; In response to determining that the expected achievable data upload volume is less than the remaining data volume of one of the plurality of event video files, the bitrate of a second set of images in one of the plurality of event video files is adjusted, wherein the bitrate is adjusted at least in part based on the expected achievable data upload volume and the remaining data volume of one of the plurality of event video files; as well as Upload the second set of images from one of the multiple event video files.

7. The system according to claim 4, wherein, In order to upload the multiple event video files, the vehicle controller is also programmed to: Identify the high-priority regions of each of the plurality of event video files; and Upload the high-priority region of each of the multiple event video files.

Images