Information provision system, information provision device, and information provision method

Abstract

To provide an information provision system, an information provision device, and an information provision method which can appropriately provide appropriate traffic information to a user who requires the traffic information.SOLUTION: An information provision device, when an event occurs, determines a vehicle at a transmission source of vehicle data including time data corresponding to time at which the event occurs and data showing the same travel direction as that of a lane in which the event occurs as a candidate from among vehicles loaded with imaging units which image videos capturing the event on the basis of event data including data on a place and time at which the event occurs, and the vehicle data including position data, time data, and data showing the travel direction, collects video data stored in an on-vehicle storage unit of the determined candidate vehicle, extracts specific image data capturing the event from the collected video data, and transmits to a user data regarding the event including the extracted specific image data.SELECTED DRAWING: Figure 1

Description

【Technical Field】 【0001】 The present invention relates to an information providing system, an information providing apparatus, and an information providing method that utilize video obtained by an imaging device such as a drive recorder mounted on a vehicle. 【Background Art】 【0002】 Events that impede traffic, such as disasters, accidents, and traffic jams occurring on the road where a vehicle is traveling, may be captured by a drive recorder mounted on the vehicle. Patent Document 1 discloses that a center related to traffic information provision identifies vehicles present in an area where a traffic jam has occurred based on their positions, instructs the vehicles to transmit the videos captured by them to the center, and provides the received videos outside the area. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Patent Application Laid-Open No. 2006-221537 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 The number of vehicles equipped with drive recorders is increasing. There are many vehicles that are extracted only under the condition of being present in an area where a traffic jam has occurred. Imaging images captured from many vehicles are not always appropriate videos that can enable understanding of the situation of an event. Also, it is required to consider personal information, such as the problem that the face of the driver of a vehicle related to an accident can be identified instead of simply providing the imaging image outside the area. 【0005】 In one aspect, an object of the present invention is to provide an information providing system, an information providing apparatus, and an information providing method that can appropriately provide appropriate traffic information to users who need the traffic information. 【Means for Solving the Problems】 【0006】 An information provision system in one embodiment of the present disclosure includes an in-vehicle device mounted in each vehicle and an information provision device that provides users with images of events occurring on and around the road while each vehicle is traveling, wherein the in-vehicle device includes an imaging unit that captures images of the surroundings and / or interior of the vehicle on which it is mounted, an in-vehicle storage unit that stores the captured image data in association with metadata relating to imaging conditions, including the date and time of capture, and a vehicle data transmission unit that performs a process of transmitting vehicle data, including position data indicating the position of the vehicle, time data indicating the time the vehicle was at the position, and data indicating the direction of travel of the vehicle, to the information provision device, and the information provision device includes an acquisition unit that acquires the vehicle data from the in-vehicle device and vehicle identification data of the transmitting vehicle and the acquired vehicle data. The system includes a storage unit that stores data in association with other data, an event data acquisition unit that acquires notification of the occurrence of the event along with event data including data on the location and time the event occurred, and data indicating the lane on the road where the event occurred, a determination unit that determines as a candidate vehicle the source vehicle of the vehicle data, which includes time data corresponding to the time the event occurred and data indicating the same direction of travel as the lane where the event occurred, among vehicles equipped with an imaging unit that captures video of the event based on the vehicle data and event data, an imaging unit that captures video of the event, an imaging unit that captures time data corresponding to the time the event occurred, and data indicating the same direction of travel as the lane where the event occurred, an imaging unit that collects video data stored in the in-vehicle storage unit of the determined candidate vehicle, an extraction unit that extracts specific image data capturing the event from the collected video data, and a transmission unit that transmits data related to the event, including the extracted specific image data, to the user. 【0007】 In one embodiment of the information provision system of this disclosure, the information provision device determines candidate vehicles that are highly likely to have captured images of the event that occurred, collects the images, extracts appropriate images, and provides them to vehicles equipped with an in-vehicle device including an imaging unit. Candidate vehicles are determined by whether or not the lane in which the event occurred and the direction of travel are the same, and whether or not the vehicle is the source of vehicle data that includes time data corresponding to the time the event occurred. 【0008】 An information providing device in one embodiment of the present disclosure is an information providing device that provides a user with video footage of events that occurred on and around a road while each vehicle is traveling, comprising: a vehicle data acquisition unit that acquires vehicle data including location data indicating the position of the vehicle, time data indicating the time the vehicle was at the position, and data indicating the direction of travel of the vehicle; a storage unit that stores the acquired vehicle data in association with vehicle identification data of the transmitting vehicle; and an event data unit that acquires notification of the occurrence of the event, along with event data including data indicating the location and time the event occurred, and data indicating the lane on the road where the event occurred. The system includes an acquisition unit, a determination unit that determines as a candidate vehicle the source of the vehicle data, which is equipped with an imaging unit that captures video of the event based on the vehicle data and event data, and which includes time data corresponding to the time the event occurred and data indicating the same direction of travel as the lane in which the event occurred, and video acquisition unit that collects video data captured by the imaging unit of the determined candidate vehicle, an extraction unit that extracts specific image data capturing the event from the collected video data, and a specific image data transmission unit that transmits data related to the event, including the extracted specific image data, to the user. 【0009】 In one embodiment of the information provision device of this disclosure, vehicle data, including vehicle location data and time data, is not limited to being transmitted directly from the in-vehicle device to the information provision device, but may also be acquired by the information provision device from data stored in another server device. 【0010】 The collection range of candidate vehicles may differ depending on whether the event is an accident or a traffic jam. The collection time may also differ depending on the type of event. Candidate vehicles may be selected based on their track record, or they may be selected based on the likelihood of capturing higher-resolution video. 【0011】 An information provision method in one embodiment of the present disclosure is an information provision method in which a computer provides a user with video footage of events that occurred on and around a road while each vehicle was traveling, wherein the computer acquires vehicle data including location data indicating the position of each vehicle, time data indicating the time each vehicle was at the position, and data indicating the direction of travel of each vehicle, stores the acquired vehicle data in a storage unit in association with vehicle identification data of the transmitting vehicle, acquires an event occurrence notification together with event data including data on the place and time the event occurred, and data indicating the lane on the road where the event occurred, and based on the vehicle data and event data, determines as a candidate vehicle a vehicle that is equipped with an imaging unit that captures video footage of the event, and which is the source vehicle of the vehicle data including time data corresponding to the time the event occurred and data indicating the same direction of travel as the lane where the event occurred, collects video data captured by the imaging unit of the determined candidate vehicle, extracts specific image data capturing the event from the collected video data, and executes a process to transmit data relating to the event, including the extracted specific image data, to the user. [Effects of the Invention] 【0012】 According to this disclosure, appropriate traffic information will be appropriately provided to the users who need it. [Brief explanation of the drawing] 【0013】 [Figure 1] This is an overview diagram of the information provision system. [Figure 2] This is a block diagram showing the configuration of the in-vehicle equipment included in the information provision system. [Figure 3] This is a functional block diagram of the in-vehicle equipment. [Figure 4] This is a block diagram showing the configuration of the information provision device included in the information provision system. [Figure 5] This is a functional block diagram of the information provision device. [Figure 6] This flowchart shows an example of the procedure for collecting information using an information provision device. [Figure 7] This flowchart shows an example of the procedure for providing information using an information provision device. [Figure 8] This flowchart shows an example of the procedure for providing information using an information provision device. [Figure 9] This is a flowchart showing an example of the filtering procedure. [Figure 10] This figure shows an example of a screen that notifies the occurrence of an event in the in-vehicle device 1. [Modes for carrying out the invention] 【0014】 Embodiments of the information provision system, information provision device, and information provision method disclosed herein will be described below with reference to the drawings. The same reference numerals are used for identical components shown in the drawings, and repeated descriptions are omitted. 【0015】 Figure 1 is an overview diagram of the information provision system 100. The information provision system 100 includes on-board devices 1 installed in some of the vehicles V1, V2, V3, V4, and V6, which are among multiple vehicles V (shown individually as V1 to V7 in the figure). Vehicle V5 and vehicle V7 involved in the accident are not subscribed to the information provision service provided by the information provision system 100 of this disclosure and are not equipped with the on-board devices 1. 【0016】 The information providing system 100 includes an information providing device 4 that acquires images from each in-vehicle device 1 and executes a process of providing information including the images for the in-vehicle device 1. In the information providing system 100, the plurality of in-vehicle devices 1 and the information providing device 4 can transmit and receive data via the network 2. The network 2 includes a so-called Internet, an intranet, a wireless LAN, a mobile communication network, and the like. The network 2 includes a carrier network, a network for ETC (registered trademark) (Electronic Toll Collection System), a network for VICS (registered trademark) (Vehicle Information and Communication System), a dedicated line of the service provider of the information providing system 100, the manufacturer of the in-vehicle device 1, and the dealer of the vehicle V. 【0017】 The information providing system 100 can acquire data from the map server 3. The map server 3 stores map data and outputs the map data of the requested range in response to a request. Each in-vehicle device 1 requests the map server 3 for the map data of a predetermined range including the position indicated by the position data of the vehicle V in, for example, a navigation system which is one function of the in-vehicle device 1. The in-vehicle device 1 can acquire and display the map data from the map server 3. 【0018】 The information providing system 100 can acquire data from the traffic center 5. The traffic center 5 is an organization that sequentially collects the situation in the transportation network from roadside equipment, cameras installed on roads, and in-vehicle devices 1, etc., detects the occurrence of events such as disasters, accidents, traffic jams, etc., and notifies each place as necessary. When the above-mentioned event is detected at the traffic center 5, the information providing device 4 can acquire event data including the position data and time data of the place where the event occurred, the identification data of the road, and the lane data from the traffic center 5. 【0019】 The information provision system 100 disclosed herein is used in a service that provides video footage of accidents and other incidents occurring in the transportation network to service subscribers. By installing (downloading) the program for this service on an in-vehicle device 1 installed in a vehicle owned by the user and configuring the service usage settings (user registration), the in-vehicle device 1 can receive information from the information provision device 4. 【0020】 In the information provision system 100 disclosed herein, when an accident occurs on a road, the traffic center 5 detects it, and the information provision device 4 extracts vehicles V1, V2, V3, and V6 located near the accident site from their location data. The information provision device 4 collects video (including still images) captured by each vehicle V1, V2, V3, and V6. Conventional traffic information provision systems also collected video from vehicles near the accident site, but they have not been able to select and present appropriate video from among them. In the information provision system 100 disclosed herein, the information provision device 4 extracts video appropriate for provision based on the location and direction of travel of each vehicle V1, V2, V3, and V6. The information provision device 4 may also extract video appropriate for provision based on the information provision history of each vehicle V1, V2, V3, and V6. The information provision device 4 then identifies a vehicle V4 that is scheduled to pass through the location where the accident occurred, and sends a notification of the accident, including the selected video footage, to the onboard device 1 of vehicle V4. 【0021】 Figure 2 is a block diagram showing the configuration of the in-vehicle device 1 included in the information provision system 100, and Figure 3 is a functional block diagram of the in-vehicle device 1. The in-vehicle device 1 is, for example, a microcomputer. The in-vehicle device 1 includes an in-vehicle control unit 10, an in-vehicle storage unit 11, an in-vehicle communication unit 12, a position detection unit 13, an imaging unit 14, and a UI (User Interface) unit 15. 【0022】 The in-vehicle control unit 10 is a processor such as one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), GPGPUs (General-purpose computing on graphics processing units), or TPUs (Tensor Processing Units). The in-vehicle control unit 10 uses a non-temporary storage medium such as RAM (Random Access Memory) (not shown) to store data generated during processing in the non-temporary storage medium, and performs its functions based on the in-vehicle program P1 stored in the in-vehicle storage unit 11. 【0023】 The in-vehicle storage unit 11 is a non-volatile storage medium such as a hard disk, SSD (Solid State Drive), or flash memory. The in-vehicle storage unit 11 stores the in-vehicle program P1 and setting data, etc., which are read by the in-vehicle control unit 10. The in-vehicle program P1 may be a copy of the in-vehicle program P9 stored on an external non-temporary storage medium 9, which is read by the in-vehicle control unit 10. The in-vehicle program P1 may also be one that is distributed by a remote server device, which the in-vehicle control unit 10 acquires via the in-vehicle communication unit 12 and stores in the in-vehicle storage unit 11. 【0024】 The in-vehicle storage unit 11 stores vehicle type data 117 of the vehicle on which the in-vehicle device 1 is installed. The vehicle type data 117 is stored as reference data for the installation height of the imaging unit 14. The in-vehicle storage unit 11 may store height data of the imaging unit 14 instead of vehicle type data 117. The in-vehicle storage unit 11 stores a history of position data 111 indicating the position of the vehicle V acquired by the in-vehicle control unit 10. The in-vehicle storage unit 11 stores map data 311 acquired by the in-vehicle control unit 10 from the map server 3. The in-vehicle storage unit 11 stores video data 141 acquired by the in-vehicle control unit 10 from the imaging unit 14, and recording data 114 including metadata 142 including the date and time the video was captured, camera information including camera resolution, vehicle type information, and other imaging conditions. The video format of the video data 141 is, for example, MP4. MP4 can store not only video but also audio, still images, and subtitle data. The video format is not limited to MP4. The in-vehicle storage unit 11 stores event data 511 acquired by the in-vehicle control unit 10 from the traffic center 5 and the information provision device 4. The in-vehicle storage unit 11 also stores specific image data 416 for displaying information provided by the information provision device 4. 【0025】 The in-vehicle communication unit 12 is a communication device that enables communication via the network 2. The in-vehicle communication unit 12 uses at least one of several communication modules or antennas that support ETC communication, wireless communication with carrier base station BS, Wi-Fi, Bluetooth (registered trademark), and VICS communication. The in-vehicle control unit 10 can send and receive data to and from the information provision device 4 and a server located outside the information provision system 100 via the in-vehicle communication unit 12. 【0026】 The position detection unit 13 includes a compass sensor for detecting the direction of travel of the vehicle V, a GPS sensor for detecting GPS signals from GPS (Global Positioning System) satellites, and the like. Based on the outputs from the compass sensor and the GPS sensor, the position detection unit 13 sequentially outputs to the on-board control unit 10 the position data 111 of the vehicle V, time data 113 indicating the date and time the vehicle was at the position indicated by the position data 111, and data indicating the direction of travel. The position data 111 is, for example, data expressed in a global coordinate system that includes at least longitude and latitude. The position data 111 may also be data expressed in a local coordinate system that can be converted to a global coordinate system, and may include altitude. The time data 113 is expressed in, for example, year, month, day, hour, and second. The format of the time data 113 is not limited as long as it can be expressed in a format that can be converted to year, month, day, hour, and second. The position data 111 or the time data 113 may be in a format in which the data size has been compressed by, for example, a predetermined compression method. 【0027】 The imaging unit 14 is a camera having a housing, a lens, and an image sensor. The image sensor may be, for example, a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide-Semiconductor). The housing of the imaging unit 14 is installed to capture images in the direction in front of the vehicle through the lens. The imaging unit 14 outputs video data consisting of multiple frame images in a predetermined video format. The video data is divided into, for example, predetermined time intervals. The predetermined time interval may be, for example, 1 minute, 5 minutes, 10 minutes, etc., and the length of the predetermined time interval may be set by the user (owner) of the vehicle V. 【0028】 The imaging unit 14 may be mounted as a separate device (a so-called drive recorder) from the in-vehicle device 1. In this case, the imaging unit (imaging device) 14 incorporates a processor and memory, and the processor stores the video data 141 obtained from the image sensor in memory along with metadata 142 including the date and time of acquisition, and successively overwrites the memory area where the older video data 141 is stored. 【0029】 The imaging unit 14 may include not only a camera that images the area in front of the vehicle, but also a camera that images the area behind the vehicle. The imaging unit 14 may include a camera that captures the interior of the vehicle, or it may include a panoramic camera that captures images in all directions, not just the front or rear. 【0030】 The UI unit 15 includes a display 151, an input unit 152, and an audio device 153. The display 151 is an organic EL or liquid crystal display that displays map images and images stored in the in-vehicle storage unit 11 through processing by the in-vehicle control unit 10. The display 151 displays a GUI (Graphical User Interface) through processing by the in-vehicle control unit 10. 【0031】 The input unit 152 is a touch panel built into the display 151. The input unit 152 may also include physical buttons located near the display 151. The in-vehicle control unit 10 receives user operations via the input unit 152 and can identify the input data based on the screen displayed on the display 151 and the operation content detected by the input unit 152. 【0032】 The audio device 153 includes a speaker and a microphone. The audio device 153 can output voice or music from the speaker and create audio data from voice received by the microphone, in accordance with sound data provided by the in-vehicle control unit 10. The audio device 153 may also have a built-in voice recognition unit that performs voice recognition processing and can convert the audio data received by the microphone into text data. The in-vehicle control unit 10 may accept user input through the audio device 153. 【0033】 In the in-vehicle device 1 configured in this way, the in-vehicle control unit 10 reads and executes the in-vehicle program P1 stored in the in-vehicle storage unit 11, thereby functioning as a location data acquisition unit 101, a video data storage unit 102, a map data acquisition unit 103, a lane discrimination unit 104, a vehicle data transmission unit 105, and a specific image data output unit 106. 【0034】 While the vehicle V is ON, the on-board control unit 10 acquires position data 111 from the position detection unit 13 and sequentially stores it in the on-board storage unit 11 in association with time data 113 indicating the time when the vehicle V was at that position (position data acquisition unit 101). The position data 111 is retained as history, but older data may be deleted (overwritten) in order. 【0035】 While the vehicle V is ON (it may also be operating when the vehicle is OFF), the in-vehicle control unit 10 stores the video data 141 output by the imaging unit 14, along with metadata 142 including the date and time of acquisition, etc., in the in-vehicle storage unit 11 (or the RAM built into the in-vehicle control unit 10) as needed (video data storage unit 102). The metadata 142 may include vehicle type data (or height data of the imaging unit 14). The in-vehicle control unit 10 successively overwrites the storage area where older video data 141 is stored with newly acquired video data 141. 【0036】 While the vehicle V is ON, the in-vehicle control unit 10 sends a request to the map server 3 via the in-vehicle communication unit 12 for map data 311 within a predetermined range including the location indicated by the location data 111, and acquires the map data 311 (map data acquisition unit 103). The in-vehicle control unit 10 outputs a map image based on the map data 311 acquired by the function of the map data acquisition unit 103 to the display 151. 【0037】 The in-vehicle control unit 10 has the function of detecting lanes and determining the driving lane (lane) of the vehicle V on which the in-vehicle device 1 is mounted by performing image processing on the frame images that constitute the video data output by the imaging unit 14 (lane determination unit 104). By functioning as the lane determination unit 104, the in-vehicle control unit 10 sequentially stores data identifying the lane being driven (which lane it is from the center line) in RAM. 【0038】 The on-board control unit 10 transmits vehicle data, including the position data 111 and corresponding time data 113 acquired by the position data acquisition unit 101, to the information providing device 4 via the on-board communication unit 12, either upon request or autonomously (vehicle data transmission unit 105). The on-board control unit 10 also transmits lane identification data, determined by the lane discrimination unit 104, along with the vehicle data. 【0039】 The in-vehicle control unit 10 outputs an event occurrence notification screen, which includes specific image data 416 obtained by processing by the information provision device 4 (described later), to the display 151 (specific image data output unit 106). 【0040】 Details of the processing performed by each function of the on-board control unit 10 of the on-board device 1 will be described later with reference to the flowchart. 【0041】 Figure 4 is a block diagram showing the configuration of the information providing device 4 included in the information providing system 100, and Figure 5 is a functional block diagram of the information providing device 4. The information providing device 4 uses a server computer. The information providing device 4 may consist of a single server computer, or it may be configured to be distributed among multiple computers according to processing content, function, or region. 【0042】 The information providing device 4 comprises a control unit 40, a storage unit 41, and a communication unit 42. The control unit 40 employs one or more processors such as a CPU, MPU, GPU, GPGPU, or TPU. The control unit 40 uses a non-temporary storage medium such as RAM (not shown) to store data generated during processing in the non-temporary storage medium, and performs its functions based on the information providing program P4 stored in the storage unit 41. 【0043】 The storage unit 41 is a non-volatile storage medium such as a hard disk, SSD, or flash memory. The storage unit 41 stores the information provision program P4 read by the control unit 40, data referenced by the control unit 40, data transmitted and received by the control unit 40, etc. The information provision program P4 may be a copy of the information provision program P8 stored on a non-temporary storage medium 8 outside the device, read by the control unit 40. The information provision program P4 may also be one that is distributed by a remote server device, acquired by the control unit 40 via the communication unit 42, and stored in the storage unit 41. 【0044】 The storage unit 41 stores user data 411 of service subscribers to the information provision service implemented by the information provision system 100 in advance. The user data 411 includes data that identifies the vehicle V (and its on-board device 1) in which the user is riding, which is associated with a user ID that identifies the user. The data that identifies the on-board device 1 may be, for example, a vehicle number or the MAC address of the on-board device 1. 【0045】 The storage unit 41 stores vehicle data 412, which includes location data 111 for each vehicle acquired by the function of the vehicle data acquisition unit 401. Vehicle identification data that identifies vehicle V is associated with the vehicle data 412. The storage unit 41 stores map data 311 acquired by the control unit 40 from the map server 3. When the control unit 40 receives notification of an event occurrence from the traffic center 5, the storage unit 41 stores event data 511, which includes the location of the event, the type of event, and the estimated date and time of the event. The storage unit 41 stores specific image data 416 of a specific image extracted from the video recording data 114 acquired by the control unit 40 from the in-vehicle device 1. 【0046】 The communication unit 42 is a communication device that enables communication via the network 2. The communication unit 42 includes a communication device for sending and receiving data with multiple in-vehicle devices 1, and a communication device for acquiring data from the map server 3 and the traffic center 5. The communication unit 42 may include multiple types of communication devices depending on the communication standard with the communication partner. 【0047】 In this configuration, the information providing device 4 functions as a vehicle data acquisition unit 401, an event data acquisition unit 402, a determination unit 403, a video collection unit 404, an extraction unit 405, and a specific image data transmission unit, as the control unit 40 reads and executes the information providing program P4 stored in the storage unit 41. 【0048】 While the information providing device 4 is running, the control unit 40 transmits a request for vehicle data 412, including position data indicating the location of each vehicle V and data indicating the direction of travel of the vehicle V, to the vehicle V equipped with the in-vehicle device 1 at predetermined intervals. If the navigation system is guiding the vehicle, the control unit 40 may also request data of the planned route to be included in the vehicle data 412. The control unit 40 may also request vehicle type data (or height data of the imaging unit 14) 117 to be included in the vehicle data 412. The control unit 40 acquires the vehicle data 412 transmitted in response to the request via the communication unit 42 (vehicle data acquisition unit 401) and stores the vehicle data 412 in the storage unit 41 in association with the vehicle identification data of the vehicle V. As the vehicle data acquisition unit 401, the control unit 40 may not only acquire vehicle data directly from the in-vehicle device 1, but may also acquire vehicle data transmitted from the in-vehicle device 1 to other storage devices (not shown) via the network 2 from those storage devices. 【0049】 While the information provision device 4 is running, the control unit 40 sequentially receives event occurrence notifications from the traffic center 5 and receives event data 511 indicating the content of the event transmitted along with the event occurrence notification (event data acquisition unit 402). The control unit 40 stores the received event data 511 in the storage unit 41. Event identification data that identifies the event is associated with the event data 511. 【0050】 The control unit 40 determines which vehicles may have captured video of the detected event based on the location and surrounding area where the event occurred, and which vehicles were traveling around the time the event occurred and immediately before and after (determination unit 403). From the determined candidates, the control unit 40 collects recorded video data 114 captured by the imaging unit 14 around the time the event occurred and immediately before and after (video collection unit 404). From the collected recorded video data 114, the control unit 40 extracts appropriate video that captures the event and extracts appropriate frame images from the frame images contained in that video (extraction unit 405). Using the functions of the extraction unit 405, the control unit 40 performs processing to exclude inappropriate video related to personal information, or to extract frame images that capture, for example, a line of cars in a traffic jam. The control unit 40 may also use a trained model to extract specific frame images sufficient to understand the event. The control unit 40 transmits the image data of the extracted specific images to the on-board device 1 of the vehicle V heading towards the location where the event occurred (specific image data transmission unit 406). 【0051】 Details of the processing performed by each function of the control unit 40 of the information provision device 4 will be described later with reference to the flowchart. 【0052】 Figure 6 is a flowchart illustrating an example of the information collection procedure performed by the information providing device 4. While the information providing device 4 is running, it continuously executes the following processes. In the flowcharts used in the following explanation, "S" indicates a step. 【0053】 The control unit 40 of the information providing device 4 requests vehicle data 412 from the vehicle V on which the in-vehicle device 1 is installed (S101), receives the vehicle data 412 and vehicle identification data transmitted from the vehicle V (S102), and stores them in the storage unit 41 (S103). The processing in S102 corresponds to the vehicle data acquisition unit 401. Alternatively, the request for vehicle data 412 in S101 may be omitted, and the vehicle data 412 may be autonomously transmitted from the in-vehicle device 1 of the vehicle V at a predetermined interval, and the information providing device 4 may receive and store it sequentially. 【0054】 Figures 7 and 8 are flowcharts illustrating an example of the information provision procedure by the information provision device 4. While the information provision device 4 is running, it performs the following processing independently of the processing procedure shown in Figure 6, each time it receives notification of an event occurrence from the traffic center 5. 【0055】 The control unit 40 determines whether or not it has received notification of an event from the traffic center 5 (S201). If it is determined that it has not received notification (S201: NO), the control unit 40 returns to S201 and continues the process of collecting the latest location of each vehicle V. 【0056】 If the control unit 40 determines in S201 that it has received an event occurrence notification (S201: YES), it stores the type of event, the date and time of the event occurrence (detection date and time), and the location of the event included in the event data 511 received along with the occurrence notification, in association with the event identification data (S202). S201 and S202 correspond to the event data acquisition unit 402. 【0057】 Event data 511 includes, if the event type is traffic congestion, location data of the location where the congestion is occurring (beginning and end), road identification data of the road where the congestion is occurring, and lane data identifying the lane (upbound / downbound, direction of travel) where the congestion is occurring. Event data 511 includes, if the event type is a traffic accident, location data of the location where the traffic accident occurred, and road identification data of the road where the traffic accident occurred. Event data 511 related to traffic accidents may also include lane data identifying the lane in which the accident vehicle is located. Event data 511 includes, if the event type is a disaster, location data of the location where the disaster occurred, and road identification data of roads near the location where the disaster occurred. 【0058】 Based on the received event data 511, the control unit 40 determines the collection range for recorded data 114 on a map, according to the type of event, based on the location of the event, the lane, etc. (S203). For example, if the event is a traffic jam, the control unit 40 determines the collection range to be the lane where the congestion is occurring, within a first predetermined distance (5m from the front, 5m from the back, etc.) from the location of the congestion on the road where the congestion is occurring. The collection range may also include the opposing lane. For example, if the event is a traffic accident, the control unit 40 determines the collection range to be both lanes within a second predetermined distance (e.g., 20m) from the location of the accident. For example, if the event is a disaster, the control unit 40 determines the collection range to be both lanes within a third predetermined distance (e.g., 10m) from the location of the disaster. The first predetermined distance, the second predetermined distance, and the third predetermined distance are included in the setting data stored in the storage unit 41. 【0059】 Based on the received event data 511, the control unit 40 determines the collection time period for the recorded video data 114 according to the type of event (S204). For example, if the event is a traffic jam, the control unit 40 determines the collection time period to be 30 minutes after the time the traffic jam was detected. For example, if the event is a traffic accident, the control unit 40 determines the collection time period to be 60 minutes after the time the traffic accident was detected. For example, if the event is a disaster, the control unit 40 determines the collection time period to be 90 minutes after the time the disaster was detected. 【0060】 The control unit 40 determines candidate vehicles V that may have captured the event that occurred with the imaging unit 14 (S205). In S205, the control unit 40 extracts vehicle data 412 that includes the determined collection range as location data 111 within the determined collection time period from the vehicle data 412 stored in the storage unit 41, and determines them as candidate vehicles to be collected. The processing in S203, S204, and S205 corresponds to the determination unit 403. 【0061】 The control unit 40 identifies the vehicle identification data (vehicle) of the extracted candidate vehicles to be collected (S206), and temporarily stores the identified vehicle identification data (vehicle) as the vehicle identification data of the candidate vehicles to be collected (S207). 【0062】 The control unit 40 extracts the vehicles to be notified (S208). In S208, the control unit 40 extracts vehicle data 412 that includes position data 111 located within a predetermined fourth distance from the determined collection range and that corresponds to the direction ahead of the determined collection range. The control unit 40 identifies the vehicle identification data of the vehicles to be notified extracted in S208 (S209), and temporarily stores the vehicle identification data identified in S209 as the identification data of the vehicles to be notified (S210). In S208, the control unit 40 may also extract vehicle data 412 that are included in the planned route within the determined collection range. 【0063】 The control unit 40 determines whether the number of vehicle identification data for candidate vehicles to be collected (= number of candidate vehicles) is greater than or equal to a predetermined number (S211). If it is determined that the number is greater than or equal to a predetermined number (S211: YES), the control unit 40 executes a process to narrow down the candidate vehicles to be collected (S212) and proceeds to the next step, S213. 【0064】 If the control unit 40 determines in S211 that the number of candidate vehicles to be collected is less than a predetermined number (S211: NO), it proceeds to the next step, S213. The narrowing down process in S212 is included in the processing of the decision unit 403. 【0065】 The control unit 40 transmits a request from the communication unit 42 to the candidate vehicle for collection of recorded video data 114 from the time of the accident until a predetermined time (S213). The control unit 40 receives and collects the recorded video data 114, including the video data 141 and metadata 142 transmitted in response to the request (S214). The collection process in S214 corresponds to the video collection unit 404. The collected recorded video data 114 is temporarily stored in association with the vehicle identification data of the sender (S215). 【0066】 The control unit 40 processes each frame image contained in the recorded video data 114 received in S214 with respect to the event in the event data 511 received in S201, and extracts one or more frame images in which the event is depicted (S216). The processing in S216 corresponds to the extraction unit 405. In S216, the control unit 40 may determine whether or not the event is depicted in the frame image and extract only the frame images in which it is determined that the event is depicted. The control unit 40 may determine whether or not the event is depicted using a pre-trained learning model. The learning model is trained to output, when one or more images are input, the probability that the event is depicted in the image and data indicating the range in which the event is depicted. The learning model is trained using image data annotated with the range in which the event is depicted as training data. 【0067】 The control unit 40 stores the extracted frame image as specific image data 416 in the storage unit 41, associating it with the event identification data of the target event data (S217). The control unit 40 stores the vehicle identification data of the source of the video data containing the extracted specific image data 416 as actual data in the storage unit 41 (S218). 【0068】 The control unit 40 creates screen data for an event occurrence notification screen that includes the extracted specific image data 416 (S219), and transmits the created screen data to the vehicle to be notified, which is stored in the vehicle identification data in S210 (S220), and then terminates the process. S220 corresponds to the specific image data transmission unit 406. 【0069】 The control unit 40 can then provide information to the in-vehicle device 1, which has subsequently requested information regarding map data 311 that includes the location where the event indicated by the stored event data occurred, in order to display specific image data 416 at the location where the event occurred on the map. The control unit 40 continues to provide this information until the event is resolved. After the event is resolved, the information providing device 4 may store the event data and specific image data in the storage unit 41 as history for a certain period of time. 【0070】 Figure 9 is a flowchart showing an example of the narrowing procedure. The processing procedure shown in Figure 9 corresponds to the details of the processing in S212. The control unit 40 assigns the following priorities to narrow down the vehicles from which video data is to be collected. The control unit 40 may perform the processing up to S216, and if no event is captured, request video data from the vehicle with the next priority and repeat the processing of S216. 【0071】 The control unit 40 assigns points to vehicles (vehicle identification data) that traveled within the collection range determined in S203 during the time period determined in S204, with higher points assigned to vehicles that traveled closer to the location where the event occurred (S301). These points are used to assign priority in S306, which will be described later. 【0072】 The control unit 40 adds priority points to vehicles that traveled within the collection range determined in S203 during the time period determined in S204, with the priority given to vehicles whose travel time is closer to the time the event occurred (S302). 【0073】 The control unit 40 adds points for priority to vehicles whose direction of travel matches the lane in which the event occurred, or to vehicles whose direction of travel is in the opposite direction, depending on the event (S303). In S303, for example, if the event is a traffic jam, the control unit 40 adds points to vehicles that traveled in the same direction lane, and if the event is a traffic accident, it adds points to vehicles that traveled in a lane closer to the center of the opposite lane. If the collection range is determined in S203 to include vehicles that traveled in the same direction lane as candidates, the processing in S303 may be omitted. 【0074】 The control unit 40 adds higher points to vehicles that transmit recording data 114 that includes higher resolution imaging conditions in the metadata 142 (S304). This increases the likelihood of extracting images that are more clearly captured. 【0075】 The control unit 40 adds higher points to vehicles that have a large number of consecutive vehicle data including position data 111 or time data 113 included in the event collection range (S305). In S305, the control unit 40 can prioritize narrowing down the range close to where the event occurred and vehicles that are likely to have captured more video data 141 at a time close to when the event occurred. 【0076】 The control unit 40 adds points to the vehicle identified by the vehicle identification data associated with the previously collected video data 114, according to the number of video data entries 114 (S306). 【0077】 The control unit 40 assigns priority to the vehicles in descending order of the points given in S301 to S306 (S307), narrows down the selection to a predetermined number of vehicles with higher priority (S308), and then terminates the selection process. Some of the processes from S301 to S306 may be omitted, or all of them may be executed. 【0078】 Figure 10 shows an example of a screen 500 that notifies the user of an event occurrence in the in-vehicle device 1. Screen 500 shows information provided by the information provision device 4 to the user, who is a passenger in vehicle V4, when the traffic center 5 detects that an accident has occurred involving vehicle V7 in Figure 1. Screen 500 includes images based on specific image data 416 extracted from video data captured by the in-vehicle devices 1 of vehicles V1, V2, V3, and V6 that were driving nearby at the time of the accident involving vehicle V7 in Figure 1. Screen 500 includes the content of the event that occurred (accident, traffic jam, etc.) and an icon 501 and a graphic 502 indicating the area where the event occurred, based on event data 511. 【0079】 As shown in Figure 10, by displaying a screen like the one shown on the display 151 installed in the vehicle V4, which is heading towards the location where the event occurred and whose route includes that location, the user, as a passenger, can visually understand that the event is occurring ahead in the direction of travel. Appropriate images are extracted as images capturing the event, and appropriate information that satisfies the user viewing the information can be presented. Personal information and images that do not relate to the event are not displayed, and appropriate information is provided. 【0080】 The device on which screen 500 is displayed is not limited to the in-vehicle device 1, but may also be a communication terminal device running an application program for searching traffic information, such as a smartphone, tablet, or personal computer. 【0081】 The embodiments disclosed above are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the claims, and all modifications within the meaning and scope equivalent to the claims are included. [Explanation of symbols] 【0082】 100 Information Provision System 1 On-vehicle device 10. On-board control unit 11 Onboard storage 14 Imaging Unit 15 UI section 151 displays 4 Information provision device 40 Control Unit 401 Vehicle Data Acquisition Unit 402 Event Data Acquisition Unit 403 Decision Section 404 Video Collection Department 405 Extraction part 406 Specific Image Data Transmission Unit 41 Storage section 42 Communications Department P1 In-vehicle program P4 Information Provision Program

Claims

[Claim 1] An information provision system including an in-vehicle device installed in each vehicle and an information provision device that provides users with video footage of events that occurred on the road and in the surrounding area while each vehicle was traveling, The in-vehicle device is An imaging unit is installed to capture images of the surroundings and / or interior of the vehicle, An in-vehicle storage unit stores captured video data in association with metadata related to the imaging conditions, including the date and time of capture. A vehicle data transmission unit performs a process of transmitting vehicle data to an information providing device, which includes position data indicating the position of the vehicle, time data indicating the time the vehicle was at the position, and data indicating the direction of travel of the vehicle. Equipped with, The aforementioned information providing device is An acquisition unit that acquires the aforementioned vehicle data from the in-vehicle device, A storage unit that stores the acquired vehicle data in association with the vehicle identification data of the transmitting vehicle, An event data acquisition unit acquires the notification of the occurrence of the aforementioned event along with event data including data on the location and time the event occurred, and data indicating the lane on the road where the event occurred. Based on the vehicle data and event data, a determination unit determines, as a candidate, the vehicle that is the source of the vehicle data, which includes time data corresponding to the time the event occurred and data indicating the same direction of travel as the lane in which the event occurred, among the vehicles equipped with an imaging unit that captures video of the event. A video collection unit collects video data stored in the onboard memory unit of the selected candidate vehicle, An extraction unit inputs each frame image constituting the collected video data into a trained model to estimate whether or not the event is depicted in the frame image and the extent to which the event is depicted, selects frame images sufficient to grasp the event based on the estimation results, and further extracts specific image data that captures the event by excluding inappropriate images related to personal information from the selected frame images. A transmission unit that transmits data relating to the aforementioned event, including the extracted specific image data, to the user. An information provision system equipped with the following features. [Claim 2] An information provision device that provides users with video footage of events that occurred on the road and in the surrounding area while each vehicle was traveling, A vehicle data acquisition unit acquires vehicle data including position data indicating the position of the vehicle, time data indicating the time the vehicle was at the position, and data indicating the direction of travel of the vehicle. A storage unit that stores the acquired vehicle data in association with the vehicle identification data of the transmitting vehicle, An event data acquisition unit acquires the notification of the occurrence of the aforementioned event along with event data including data on the location and time the event occurred, and data indicating the lane on the road where the event occurred. Based on the vehicle data and event data, a determination unit determines, as a candidate, a vehicle that is equipped with an imaging unit for capturing images of the event, and which is the source vehicle of the vehicle data, including time data corresponding to the time the event occurred and data indicating the same direction of travel as the lane in which the event occurred. The video collection unit collects video data captured by the imaging unit of the selected candidate vehicle, An extraction unit inputs each frame image constituting the collected video data into a trained model to estimate whether or not the event is depicted in the frame image and the extent to which the event is depicted, selects frame images sufficient to grasp the event based on the estimation results, and further extracts specific image data that captures the event by excluding inappropriate images related to personal information from the selected frame images. A specific image data transmission unit transmits data relating to the aforementioned event, including the extracted specific image data, to the user. An information-providing device equipped with the following features. [Claim 3] A method for providing information to a user, wherein a computer provides video footage of events that occurred on the road and in the vicinity of the road while each vehicle was traveling, The aforementioned computer, Vehicle data is acquired that includes position data indicating the location of each vehicle, time data indicating the time each vehicle was at the said location, and data indicating the direction of travel of each vehicle. The acquired vehicle data is stored in the storage unit in association with the vehicle identification data of the transmitting vehicle. The notification of the occurrence of the aforementioned event is obtained along with event data including data on the location and time the event occurred, and data indicating the lane on the road where the event occurred. Based on the vehicle data and event data, among the vehicles equipped with an imaging unit that captures images of the event, the vehicle that transmitted the vehicle data, which includes time data corresponding to the time the event occurred and data indicating the same direction of travel as the lane in which the event occurred, is selected as a candidate. The video data captured by the imaging unit of the selected candidate vehicle is collected. Each frame image constituting the collected video data is input into a trained model to estimate whether or not the event is depicted in the frame image and the extent to which the event is depicted. Based on the estimation results, frame images sufficient to grasp the aforementioned event are selected, and further inappropriate images containing personal information are excluded from the selected frame images to extract specific image data that captures the aforementioned event. The data relating to the aforementioned event, including the extracted specific image data, is sent to the user. A method for providing information to execute a process.

Images