Event video provision system, management device, event video provision method, and program

Abstract

To extract a video more appropriately presenting an event situation and provide the video to a user.SOLUTION: In an event video provision system including each on-vehicle unit 1 that each vehicle mounts thereon and a management device 4 for transmitting an event video relating to an event, which occurs on the road, to the on-vehicle unit 1 of a target vehicle to which the event video should be provided, the management device 4 determines a collection range from an event occurrence location and a date, extracts a specific vehicle existing in the collection range and the target vehicle moving toward the event occurrence location, receives meta information corresponding to video data from the specific vehicle, converts the meta information into an evaluation point in accordance with conversion information, receives specific video data corresponding to high-score meta information of a highest evaluation point from the specific vehicle, and provides the specific video data to the target vehicle.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for providing a user who needs a video of a disaster, accident, traffic jam, or the like (hereinafter simply referred to as an "event") that has occurred on a road on which a vehicle is traveling. [Background technology]

[0002] The following Patent Document 1 discloses a system that identifies vehicles present in an area where congestion has occurred, instructs the imaging equipment installed in the vehicle to transmit images captured by the imaging equipment to a center, and provides the images to other vehicles outside the congestion area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-221537 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, there may be multiple vehicles in the area where an event occurs that can transmit video captured within the area. As long as all of the captured video adequately conveys the event situation, there is no problem with, for example, randomly selecting one vehicle and having it transmit that video.

[0005] However, in reality, the majority of footage does not adequately convey the event situation. For example, a dashcam is one example of a vehicle-mounted imaging device, but most of the footage captured by a dashcam in a vehicle in a traffic jam area only shows a large image of the rear of the vehicle in front. In particular, if the vehicle in front is a large bus or truck, the footage only shows the rear of the vehicle.

[0006] To properly convey the state of traffic congestion, it is desirable to have images captured from a high position that show, for example, the accident vehicle that caused the traffic congestion and the line of vehicles lined up.

[0007] In view of the above, this specification discloses an event video providing system, a management device, a video providing method, and a program that extracts video that more appropriately conveys the event situation and provides it to users. [Means for solving the problem]

[0008] (1) The event video provision system disclosed in this specification includes an on-board device installed in each vehicle and a management device that transmits event video related to an event that occurred on the road on which the vehicle is traveling to the on-board device of the target vehicle to which the video should be provided.

[0009] Each in-vehicle device includes a location information acquisition unit, an in-vehicle communication unit, an imaging unit, a meta information generation unit, and a recording unit. The location information acquisition unit sequentially acquires location information including the location of the vehicle on which it is installed and the time the vehicle was at that location. The in-vehicle communication unit sequentially transmits vehicle location information including identification information for identifying the vehicle and location information to the management device. The imaging unit generates video data showing video captured around the vehicle. The meta information generation unit generates meta information including location information as information related to the video. The recording unit records the video data in association with the meta information.

[0010] The management device includes a management communication unit, an event information acquisition unit, a collection range determination unit, a vehicle extraction unit, a meta information request unit, a management storage unit, an evaluation unit, a specific video data request unit, and a provision unit. The management communication unit sequentially receives vehicle position information. The event information acquisition unit acquires event information including the location and date and time of an event occurrence. The collection range determination unit determines a predetermined area and time as a collection range for event video based on the event information. The vehicle extraction unit extracts, based on the vehicle position information, specific vehicles that were present within the collection range and target vehicles that were not present within the collection range but were traveling toward the occurrence location and were present within a predetermined range from the occurrence location. The meta information request unit requests meta information associated with video data recorded within the collection range from the in-vehicle device of the specific vehicle. The management storage unit stores conversion information that converts meta information into a score. The evaluation unit stores multiple pieces of meta information received from the in-vehicle device by the management communication unit and converts them into a score using the conversion information. The specific video data request unit requests the specific video data associated with the high-score meta information from the in-vehicle device that transmitted the high-score meta information with the highest score among the plurality of meta information. The providing unit provides the event information and the specific video data to the in-vehicle device of the target vehicle.

[0011] The on-board device of the target vehicle further includes a map information acquisition unit and a display unit. The map information acquisition unit acquires map information showing a map including at least roads. The display unit displays the position of the target vehicle on the road, the occurrence position, and the event video superimposed on the map based on the map information, location information, event information, and specific video data provided by the providing unit.

[0012] (2) Each of the in-vehicle devices described in (1) above may further include an object detection unit and a calculation unit. The object detection unit analyzes the video to detect a predetermined object included in the video. The calculation unit calculates a usefulness rate indicating the proportion of the video that does not include a detection area of ​​the object. The meta information generation unit generates meta information including the usefulness rate.

[0013] (3) The event information in (1) above may include event occurrence lane information indicating the lane in which the event occurred, and each in-vehicle device may further include a lane marking detection unit, a road information acquisition unit, and a lane determination unit. The lane marking detection unit analyzes the video to detect lane markings of the road included in the video. The road information acquisition unit acquires road information including the number of lanes on the road based on the position information. The lane determination unit determines the driving lane in which the vehicle is traveling based on the road information and the results of the lane marking detection. The meta information generation unit generates meta information including driving lane information indicating the driving lane.

[0014] (4) Each of the in-vehicle devices in (1) above may further include an in-vehicle storage unit that stores vehicle model information indicating the vehicle model of the vehicle. The meta information generating unit generates meta information including the vehicle model information.

[0015] (5) The management device disclosed in this specification includes a management communication unit, an event information acquisition unit, a collection range determination unit, a vehicle extraction unit, a meta information request unit, a management storage unit, an evaluation unit, a specific video data request unit, and a provision unit. The management communication unit sequentially receives vehicle position information, including identification information identifying each on-board device itself, the location of the vehicle in which each on-board device is installed, and the date and time of the location, from each on-board device that associates and records video data of video images of the vehicle's surroundings with meta information related to the video. The event information acquisition unit acquires event information, including the location and date and time of an event that occurred on a road on which the vehicle is traveling. The collection range determination unit determines a predetermined area and time as a collection range for event video related to the event based on the event information. The vehicle extraction unit extracts, based on the vehicle position information, specific vehicles that were present within the collection range and target vehicles that were not present within the collection range but were traveling toward the occurrence location and were present within a predetermined range from the occurrence location. The meta information request unit requests meta information of video captured within the collection range from the in-vehicle device of the specific vehicle. The management memory unit stores conversion information, which is information for converting the meta information into a score. The evaluation unit stores multiple pieces of meta information received from the in-vehicle device by the management communication unit and converts them into a score using the conversion information. The specific video data request unit requests specific video data associated with the high-score meta information from the in-vehicle device that sent the high-score meta information with the highest score among the multiple pieces of meta information. The provision unit provides the event information and specific video data to the in-vehicle device of the target vehicle.

[0016] (6) The present specification discloses an event video providing method used in a management device for providing video related to an event. The event video providing method sequentially receives, from each on-board device that records video data of video captured around the vehicle in association with meta information related to the video, identification information that identifies each on-board device itself, vehicle location information including the location of the vehicle in which each on-board device is installed, and the date and time of the location, obtains event information including the location and date and time of occurrence of an event that occurred on a road on which the vehicle is traveling, determines a predetermined area and time as a collection range for event video related to the event based on the event information, extracts, based on the vehicle location information, specific vehicles that were present within the collection range and target vehicles that were not present in the collection range but were traveling toward the location of the occurrence and were present within a predetermined range from the location of the occurrence, requests meta information of the video captured within the collection range from the on-board device of the specific vehicle, stores the multiple pieces of meta information received from the on-board device, converts the meta information into scores using conversion information, requests specific video data associated with the high-score meta information from the on-board device that sent the high-score meta information with the highest score among the multiple pieces of meta information, and provides the event information and specific video data to the on-board device of the target vehicle.

[0017] (7) The program disclosed in this specification is a program that causes a computer that provides event-related video to execute an event video providing process. The event video providing process includes the steps of: sequentially receiving, from each in-vehicle device that records video data of a video captured around the vehicle in association with meta information related to the video, vehicle location information including identification information that identifies each in-vehicle device itself, the location of the vehicle on which each in-vehicle device is installed, and the date and time of the location; acquiring event information including the location and date and time of an event that occurred on a road on which the vehicle is traveling; determining a predetermined area and time as a collection range for event video related to the event based on the event information; and identifying specific vehicles that were present in the collection range and the collection points based on the vehicle location information. The method includes the steps of: extracting target vehicles that are not within the collection area, traveling toward the location of occurrence, and within a predetermined range from the location of occurrence; requesting meta information of images captured within the collection area from the on-board device of the specific vehicle; storing multiple pieces of meta information received from the on-board device and converting the meta information into scores using conversion information; requesting specific video data associated with the high-score meta information from the on-board device that sent the high-score meta information with the highest score among the multiple pieces of meta information; and providing the event information and specific video data to the on-board device of the target vehicle. [Effects of the Invention]

[0018] By using the event video providing system according to one aspect of the present disclosure, it is possible to extract the most appropriate video to inform users of the event situation and provide it to them. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating an example of the configuration of an event video providing system 100. FIG. [Figure 2] 1 is a diagram illustrating an example of the configuration of an in-vehicle device 1. FIG. [Figure 3] FIG. 10 is a diagram for explaining an example of a method for calculating a useful rate. [Figure 4] 2 is a diagram showing an example of a recording file stored in the in-vehicle storage unit 25 of the in-vehicle device 1. FIG. [Figure 5] 10 is a diagram showing an example of an event video displayed by the display unit 241. FIG. [Figure 6] FIG. 2 is a diagram illustrating an example of the configuration of a management device 4. [Figure 7] FIG. 10 is a diagram showing an example of a list of meta-information that the management device 4 has requested and received from each specific vehicle within the collection range. [Figure 8] FIG. 10 is a diagram illustrating an example of conversion information. [Figure 9] FIG. 10 is a diagram illustrating an example of conversion information. [Figure 10] FIG. 10 is a diagram illustrating an example of conversion information. [Figure 11] FIG. 10 is a diagram illustrating an example of conversion information. [Figure 12] FIG. 10 is a diagram showing an example of meta information converted into a score. [Figure 13] FIG. 10 is a diagram showing an example of an operation flow of the management device 4. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the same components shown in the drawings are given the same reference numerals, and repeated explanations will be omitted.

[0021] [Configuration of event video providing system 100] 1, the event video providing system 100 includes in-vehicle devices 1 mounted in vehicles 11 to 14, and a management device 4. The event video providing system 100 is a network system that is connected for communication via a network 2. The network 2 is configured from the Internet, an intranet, a wireless LAN, a mobile communication network, or the like.

[0022] The map server 3 stores map information used by the in-vehicle device 1 and transmits map information of a specified area to the in-vehicle device 1 in response to a request from the in-vehicle device 1. The traffic center 5 is an organization that collects and understands events that occur on roads, such as disasters, accidents, and congestion. The traffic center 5 generates event information 257 including the details of the event, the location and date and time when the event occurred, and information about the lane in which the event occurred, and transmits it to the management device 4.

[0023] The in-vehicle device 1 has an imaging function for capturing an image of the surroundings of the vehicle and generating image data 255, and a display function for displaying the captured image superimposed on a map. The in-vehicle device 1 generates meta information 256, which is information about the captured image, and records it in association with the image data 255. The in-vehicle device 1 obtains map information for displaying the map from the map server 3.

[0024] Vehicle 10 is a vehicle that has been involved in an accident. Vehicle 15 is a vehicle that does not have an in-vehicle device 1. Vehicles 10 and 15 are not included in the event video providing system 100 because they belong to users who are not subscribed to the service described below.

[0025] The event video providing system 100 is a system used for a service that provides users with videos of events that occur on roads. When a user installs a program dedicated to this service in an in-vehicle device 1 mounted in the user's vehicle and configures the service, the in-vehicle device 1 communicates with a management device 4 and becomes included in the event video providing system 100.

[0026] 1 shows an example of event video provision performed by the event video provision system 100. Here, an outline of the operation of the event video provision system 100 will be briefly described.

[0027] Each in-vehicle device 1 generates vehicle position information 432 including the position of the vehicle and the date and time, and sequentially transmits it to the management device 4, and the management device 4 grasps the current position of each vehicle equipped with each in-vehicle device 1. When an accident involving vehicle 10 causes congestion, each in-vehicle device 1 of vehicles 11 to 13 caught in the congestion records video 255 captured during the congestion period. When an event such as an accident occurs, the management device 4 receives event information 257 from the traffic center 5 via the network 2.

[0028] The management device 4 determines a collection range for collecting video data 255 based on the event information 257, and extracts specific vehicles (vehicles 11 to 13) that were present within the collection range, and a target vehicle (vehicle 14) that is not included in the collection range, is traveling toward the location where the event occurred, and is present within a predetermined range from the location where the event occurred. The management device 4 receives meta information 256 associated with the video data 255 from the specific vehicle, and converts the meta information 256 into an evaluation score based on conversion information 431 stored in advance.

[0029] The management device 4 receives specific video data 258, which is video data associated with the high-score meta information having the highest evaluation score, from the specific vehicle (vehicle 11), and transmits the specific video data 258 to the target vehicle (vehicle 14). The target vehicle (vehicle 14) displays an event video based on the specific video data 258 received from the management device 4 on a display unit.

[0030] By using the event video providing system 100, it is possible to extract and provide the optimum event video for informing the user of the event situation. The configurations of the in-vehicle device 1 and the management device 4 included in the event video providing system 100 will be described below.

[0031] [Configuration of in-vehicle device 1] 2, the in-vehicle device 1 includes, as functional units, an imaging unit 20, an in-vehicle communication unit 21, a position detection unit 22, an in-vehicle control unit 23, a UI (User Interface) unit 24, and an in-vehicle storage unit 25. The in-vehicle device 1 includes, as hardware units, a camera (imaging unit 20), a communication module (in-vehicle communication unit 21), various sensors, a CPU (Central Processing Unit), a memory, a storage medium, a display (display unit 241), and a touch panel (input unit 242).

[0032] The imaging unit 20 is a camera having a housing, a lens, and an imaging element, such as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS).

[0033] The housing of the imaging unit 20 is installed so as to capture an image in front of the vehicle through a lens. The imaging unit 20 has a function of generating video data 255 made up of a plurality of frame images in a predetermined video format.

[0034] The video data 255 may be in a video format such as MP4. MP4 can store not only video but also audio, still images, and subtitle data. The imaging unit 20 may generate the video data 255 using a format other than MP4. The imaging unit 20 may be capable of capturing images not only in the front direction of the vehicle but also in the rear direction and including the interior or exterior of the vehicle within its imaging range.

[0035] The in-vehicle communication unit 21 has the function of communicating with the management device 4 via the network 2 and transmitting and receiving various data. The in-vehicle communication unit 21 transmits to the management device 4 identification information for identifying itself and vehicle position information 432 including position information of the vehicle in which the in-vehicle communication unit 21 is installed. When the in-vehicle communication unit 21 receives a request for collection range meta information 256 or specific video data 258 from the management device 4, the in-vehicle communication unit 21 transmits the requested meta information 256 or specific video data 258 to the management device 4. The in-vehicle communication unit 21 can also receive the latest map information from the map server 3 via the network 2.

[0036] The position detection unit 22 is composed of a direction sensor that detects the traveling direction of the vehicle, a vehicle speed sensor that detects the vehicle speed, a GPS (Global Positioning System) sensor that detects GPS signals from GPS satellites, etc. The position detection unit 22 sequentially outputs sensor information for identifying the current position and date and time detected by each sensor to the on-board control unit 23.

[0037] The in-vehicle control unit 23 includes a position information acquisition unit 231, an object detection unit 232, a calculation unit 233, a lane marking detection unit 234, a road information acquisition unit 235, a lane determination unit 236, a meta information generation unit 237, a recording unit 238, and a map information acquisition unit 239. Each functional unit included in the in-vehicle control unit 23 is realized by the CPU executing a program.

[0038] The location information acquisition unit 231 has a function of generating location information 252 including the location of the vehicle and the date and time when the vehicle was at that location, based on the sensor information sequentially output by the location detection unit 22. The location information 252 is expressed, for example, in a global coordinate system including at least longitude information and latitude information.

[0039] The location information 252 may be expressed in a local coordinate system that can be converted into a global coordinate system, or may include altitude information. The date and time may be expressed in any format as long as it is expressed in, for example, year, month, day, hour, and second and can be converted into year, month, day, hour, and second. For example, the data size may be compressed using a predetermined compression method.

[0040] The object detection unit 232 has a function of analyzing the video captured by the imaging unit 20 and detecting a predetermined object contained in the video. The image analysis process for detecting the predetermined object contained in the video uses a known image recognition technology. For example, the image recognition technology disclosed in Japanese Patent Application Laid-Open No. 2018-163425 may be applied, or other known image recognition technology may be applied.

[0041] 3 shows an example of detecting an object by analyzing an image captured by the imaging unit 20. In the figure, a detection area 32 indicating the area where a vehicle is detected as an object is shown within an image area 31 indicating the area of ​​the entire image.

[0042] The calculation unit 233 has a function of calculating a usefulness rate indicating the proportion of the image that does not include the detection area 32 of the object. The usefulness rate may be calculated, for example, as usefulness rate = (image area 31 - detection area 32) / image area 31 (%). The sizes of the image area 31 and the detection area 32 may be expressed, for example, by pixel values.

[0043] For example, if the image area 31 is 1200 pixels wide and 800 pixels high, with a size of 1200 x 800 = 960,000 pixels, and the detection area 32 detected within the image area 31 is 800 pixels wide and 400 pixels high, with a size of 800 x 400 = 320,000 pixels, the usefulness rate is (960,000 - 320,000) / 960,000 ≒ 66.7%.

[0044] The usefulness rate indicates the proportion of the image area 31 that does not include a target object such as a vehicle, and the usefulness rate increases when the proportion of the area of ​​the detection area 32 in the image area 31 is small. Images with a high usefulness rate are images that do not show large vehicles, and are therefore more likely to be images that provide a relatively clear picture of the surrounding situation.

[0045] The lane marking detection unit 234 has a function of analyzing the video captured by the imaging unit 20 and detecting lane markings on the road contained in the video. The image analysis process for detecting lane markings uses a known image recognition technique. For example, the image recognition technique disclosed in Japanese Patent Application Laid-Open No. 2008-197905 may be applied, or other known image recognition techniques may be applied. Japanese Patent Application Laid-Open No. 2008-197905 discloses detecting the type of lane marking (solid line, dashed line) and the number of lane markings.

[0046] The road information acquisition unit 235 has a function of acquiring road information including the number of lanes of the road on which the vehicle is traveling, based on the acquired vehicle position information 252. The road information may be, for example, road information included in map information. The road information may be link information used for road search. The link information includes node information connected to the link, link position information, the number of lanes, etc.

[0047] The lane determination unit 236 has a function of determining the driving lane in which the host vehicle is traveling based on the acquired road information and the results of the detection of the lane markings. The method of determining the driving lane may be, for example, the lane determination method disclosed in the above-mentioned Japanese Patent Application Laid-Open No. 2008-197905. For example, if the road on which the host vehicle is traveling is a three-lane road (lanes 1, 2, and 3 from left to right), and the lane marking detection unit 234 detects two dashed lane markings, it may determine that lane 2 is the driving lane of the host vehicle.

[0048] The meta information generation unit 237 has a function of generating meta information 256 including location information 252 as information related to the video. The meta information 256 includes the location information 252, vehicle model information 251 indicating the vehicle model, the usefulness rate calculated by the calculation unit 233, and driving lane information determined by the lane determination unit.

[0049] The imaging unit 20 generates the video data 255 by dividing it into predetermined time intervals. The user may be able to set the video data 255 to be divided into time intervals such as 1 minute, 5 minutes, or 10 minutes. The meta information generation unit 237 generates meta information 256 for each video data 255.

[0050] The recording unit 238 has a function of recording a recording file 254 in which each piece of video data 255 generated by the imaging unit 20 is associated with each piece of meta information 256 generated by the meta information generating unit 237, in the in-vehicle storage unit 25. As shown in Fig. 4, the recording file 254 associates a recording file ID that identifies the recording file 254 with the video data 255 and the meta information 256.

[0051] The map information acquisition unit 239 has a function of acquiring map information 253 of a map to be displayed on the display unit 241 from the map server 3. The map information 253 is, for example, map information 253 that represents a map of the area around the current position of the vehicle in which the in-vehicle device 1 is installed.

[0052] The UI unit 24 has a display unit 241 and an input unit 242 in order to present information to the user and receive instructions from the user. The UI unit 24 is, for example, a touch panel display.

[0053] The display unit 241 displays a map based on the acquired map information 253, and superimposes on the map the event occurrence position based on the event information 257 provided from the management device 4, the position of the vehicle acquired by the position information acquisition unit 231, and the event video based on the specific video data 258. Note that the technology itself for displaying the event occurrence position and the vehicle position on a map using icons or the like is well known, and therefore a detailed description thereof will be omitted.

[0054] The input unit 242 receives input of instructions indicated by GUI buttons displayed on the display unit 241 when the user touches the GUI buttons. The input unit 242 receives input of instructions such as zooming in and out of the displayed map, obtaining traffic congestion information, etc. The input 242 may be a physical button (hardware button) provided on the in-vehicle device 1.

[0055] 5 is an example of the display of an event video by display unit 241. Display unit 241 displays map 33 based on map information 253, and superimposes on the map an event icon 34 indicating the location of the event based on event information 257, a host vehicle icon 35 indicating the location of the host vehicle, and a pop-up window 36 showing the event video based on specific video data 258.

[0056] The in-vehicle storage unit 25 is a storage medium that stores programs and data necessary to realize each functional unit of the in-vehicle device 1. The hardware of the in-vehicle storage unit 25 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a dynamic random access memory (DRAM), a static random access memory (SRAM), a read-on memory (ROM), or a combination of these.

[0057] The in-vehicle storage unit 25 stores vehicle type information 251 , location information 252 , map information 253 , recording files 254 , video data 255 , meta information 256 , event information 257 , and specific video data 258 .

[0058] [Configuration of management device 4] 6, the management device 4 includes a management communication unit 41, a management control unit 42, and a management storage unit 43. The management device 4 is a computer, and realizes various functions by the CPU executing programs.

[0059] The management device 4 may be, for example, a server computer installed in the facility of the system provider of the event video provision system 100, or may be a server computer owned by a company that provides cloud services.

[0060] The system provider of the event video provision system 100 can prepare the management device 4 by borrowing the management device 4 from the business operator and installing the program described below. In this case, the system provider of the event video provision system 100 does not need to own the management device 4.

[0061] The management communication unit 41 has a function of communicating with each in-vehicle device 1 via the network 2 and transmitting and receiving various data. For example, the management communication unit 41 receives vehicle position information 432 from each in-vehicle device 1 and event information 257 from the traffic center 5.

[0062] The management control unit 42 includes an event information acquisition unit 421, a collection range determination unit 422, a vehicle extraction unit 423, a meta information request unit 424, an evaluation unit 425, a specific video data request unit 426, and a provision unit 427. Each functional unit included in the management control unit 42 is realized by the CPU executing a program.

[0063] The event information acquisition unit 421 has a function of acquiring the event information 257 from the traffic center 5. The acquired event information 257 is stored in the management storage unit 43.

[0064] The collection range determination unit 422 has a function of determining a predetermined area and time as a collection range for event videos based on the acquired event information 257. The collection range determination unit 422 may determine, for example, a distance range with a radius of 300 m centered on the event occurrence position indicated by the event information 257 and within 10 minutes from the time the event occurred as the collection range.

[0065] The vehicle extraction unit 423 has a function of extracting specific vehicles that were present within the determined collection range and target vehicles that were not present within the collection range, were traveling toward the event occurrence location, and were present within a predetermined range from the event occurrence location, based on the vehicle position information 432 received from and stored in each in-vehicle device 1. In FIG. 1, the specific vehicles are vehicles 11 to 13, and the target vehicle is vehicle 14. The specific vehicles and target vehicles can be identified by identification information included in the vehicle position information 432.

[0066] For example, if the collection range is a distance range with a radius of 300 m centered on the location where the event occurred, the specified range may be a distance range with a radius of 1 km centered on the location where the event occurred excluding the collection range, or the specified range may be up to a specified number of roads connected to the road where the event occurred.

[0067] The meta information request unit 424 has a function of requesting the in-vehicle device 1 of a specific vehicle to transmit the meta information 256 associated with the video data 255 recorded within the collection range. The management communication unit 41 transmits a request for the meta information 256 to the in-vehicle device 1 identified by the identification information included in the vehicle position information 432.

[0068] The evaluation unit 425 has a function of storing multiple pieces of meta information 256 received from the in-vehicle device 1 by the management communication unit 41 and converting them into scores using conversion information 431. Fig. 7 shows an example of meta information that the management device 4 requests and receives from a specific vehicle. The meta information is associated with an in-vehicle device ID and a recording file ID. Figs. 8 to 11 show examples of the conversion information 431.

[0069] 8 shows an example of conversion information 431 for converting the proximity between the location indicated by location information 252, which is part of meta information 256, and the location where an event occurred into a score. "Proximity" is the difference between location information 252 and the location where the event occurred. Conversion information 431 is set so that the closer the video is captured from a location to the location where the event occurred, the higher the score. Note that, as an example, conversion information 431 is table information, but it may also be a mathematical formula.

[0070] 9 identifies the vehicle height from the vehicle type information 251, which is part of the meta information 256, and converts the vehicle height so that the higher the vehicle height, the higher the score. This is because video captured from a vehicle with a high vehicle height is more likely to capture the surrounding conditions than video captured from a vehicle with a low vehicle height.

[0071] The conversion information 431 shown in Fig. 10 converts the usefulness rate, which is part of the meta information 256, into a score. The conversion information 431 shown in Fig. 11 converts into a score according to the matching status of lane information, which is part of the meta information 256. It is assumed that the event information 257 includes information about the lane in which the event occurred.

[0072] In the example of conversion shown in Figure 11, if the lane in which the event occurred matches the lane the vehicle was traveling in, the score is 5 points; if the lane is adjacent, the score is 3 points; otherwise, the score is 1 point; however, other score values ​​can also be set.

[0073] 12, the evaluation unit 425 converts each element of the meta information 256 into a score and calculates the total score for each recording file ID. In the example shown in Fig. 12, the recording file ID "F_05" is the highest-scoring meta information.

[0074] The specific video data request unit 426 has a function of identifying the in-vehicle device 1 that transmitted the high score meta information from the in-vehicle device ID, and requesting the specific video data 258 associated with the high score meta information from the in-vehicle device 1. Upon receiving the request, the in-vehicle device 1 transmits the specific video data 258 to the management device 4.

[0075] The providing unit 427 has a function of providing the event information 257 and the specific video data 258 to the in-vehicle device 1 of the target vehicle. The providing unit 427 causes the management communication unit 41 to transmit the event information 257 and the specific video data 258 to the in-vehicle device 1 of the target vehicle.

[0076] The management storage unit 43 is a storage medium that stores programs and data necessary to realize each functional unit included in the management device 4. The hardware of the management storage unit 43 is configured by, for example, a large-capacity HDD or SSD.

[0077] The management storage unit 43 stores the event information 257 , the specific video data 258 , the conversion information 431 , the vehicle position information 432 , and the specific vehicle meta information 433 .

[0078] [Operation of management device 4] Here, a method for providing an event video to a target vehicle by the management device 4 will be described using the operation flow shown in Fig. 13. The symbol "S" in the figure indicates a step. First, the management device 4 sequentially receives vehicle position information 432 from each in-vehicle device 1 (S1).

[0079] When the management device 4 acquires the event information 257 from the traffic center 5 (S2: Yes), the collection range determination unit 422 determines the collection range of the event video based on the event information 257 (S3). If the management device 4 does not acquire the event information 257 (S2: No), the process returns to S1.

[0080] Next, the vehicle extraction unit 423 extracts specific vehicles that were present within the determined collection range and target vehicles that were not present within the collection range, were traveling toward the location where the event occurred, and were present within a predetermined range from the location where the event occurred (S4).

[0081] The meta information request unit 424 requests the meta information 256 from the in-vehicle device 1 of the extracted specific vehicle (S5). In response to the request, the in-vehicle device 1 of the specific vehicle transmits the meta information 256 of the video captured within the collection range to the management device 1.

[0082] The evaluation unit 425 stores the multiple pieces of meta information 256 received from the in-vehicle device 1 and converts them into scores using the conversion information (S6).

[0083] The specific video data request unit 426 requests the specific video data 258 associated with the high score meta information from the vehicle-mounted device 1 that transmitted the high score meta information (S7).

[0084] The providing unit 427 provides the event information 257 and the specific video data 258 to the in-vehicle device 1 of the target vehicle (S8).

[0085] The in-vehicle device 1 of the target vehicle that receives the specific video data 258 from the management device 4 can present the user with a video that more appropriately conveys the event situation.

[0086] Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments and should be understood to include various modifications and alternative forms of the above embodiments. For example, the present invention includes embodiments in which the components are modified and embodied within the scope of the present invention, embodiments in which multiple components disclosed in the above embodiments are appropriately combined, and embodiments in which some components are deleted from all the components shown in the embodiments or some components are added to the components shown in the embodiments.

[0087] In the above embodiment, the specific video data 258 is a moving image, but a still image may also be provided. Since the data amount of a still image is smaller than that of a moving image, the communication load can be reduced.

[0088] Various types of image analysis are envisioned for the image analysis performed by the in-vehicle device 1. For example, the image analysis may be performed to detect cracks or damage on the road on which the vehicle is traveling, or to detect whether the road is flooded. The image analysis may be performed using a machine learning model.

[0089] The present invention may be an event video providing method performed by the management device 4, a program that causes a computer to execute a process for extracting event videos to be provided, or a storage medium that stores such a program.

[0090] The program can be stored and supplied to a computer using various types of non-transitory computer readable media. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CDR / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)).

[0091] The program may be provided to the computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media may provide the program to the computer via a wired communication path such as an electrical wire or optical fiber, or via a wireless communication path. [Explanation of symbols]

[0092] 1 In-vehicle device 2 Network 3 Map Server 4 Management device 10-15 cars 20 Imaging unit 21 In-vehicle communication unit 22 Position detection unit 23 On-board control unit 24 UI section 25 Onboard storage 31 Video Area 32 detection area 33 Map 34 Event Icons 35 Vehicle icon 36 Pop-up Windows 41 Management and Communications Department 42 Management and Control Unit 43 Management storage unit 231 Location information acquisition unit 232 Object detection unit 233 Calculation Unit 234 Lane line detection unit 235 Road information acquisition department 236 Lane Judgment Unit 237 Meta Information Generation Unit 238 Recording Department 239 Map Information Acquisition Unit 241 Display section 242 Input section 251 Vehicle Information 252 Location information 253 Map Information 254 recording files 255 video data 256 Meta Information 257 Event Information 258 Specific Video Data 421 Event Information Acquisition Department 422 Collection Range Determination Unit 423 Vehicle Extraction Unit 424 Meta Information Request 425 Evaluation Department 426 Specific video data request part 427 Provision Department 431 Conversion Unit 432 Vehicle location information 433 Specific vehicle meta information

Claims

1. An event video providing system including an in-vehicle device mounted on each vehicle and a management device that transmits event video relating to an event occurring on a road on which the vehicle is traveling to the in-vehicle device of a target vehicle to which the event video should be provided, Each of the in-vehicle devices is a location information acquisition unit that sequentially acquires location information including the location of the vehicle and the time when the vehicle was at the location; an in-vehicle communication unit that sequentially transmits vehicle position information including identification information for identifying itself and the position information to the management device; an imaging unit that generates video data representing an image of the surroundings of the vehicle; a meta information generating unit that generates meta information including the location information as information related to the video; a recording unit that records the video data and the meta information in association with each other, The management device a management communication unit that sequentially receives the vehicle position information; an event information acquisition unit that acquires event information including an occurrence location and an occurrence date and time of the event; a collection range determination unit that determines a predetermined area and time as a collection range of the event video based on the event information; a vehicle extraction unit that extracts, based on the vehicle position information, specific vehicles that existed within the collection range among the vehicles, and target vehicles that do not exist within the collection range, are traveling toward the occurrence position, and exist within a predetermined range from the occurrence position; a meta information request unit that requests the meta information associated with the video data recorded within the collection range from the in-vehicle device of the specific vehicle; a management storage unit that stores conversion information that converts the meta information into a score; the conversion information includes information for converting the location included in the meta information so that the closer the location included in the meta information is to the occurrence location included in the event information, the higher the score; an evaluation unit that stores the plurality of pieces of meta information received from the in-vehicle device by the management communication unit and converts the pieces of meta information into the score using the conversion information; a specific video data requesting unit that requests specific video data associated with the high-score meta information from the in-vehicle device that has transmitted the high-score meta information having the highest score among the plurality of pieces of meta information; a providing unit that provides the event information and the specific video data to the in-vehicle device of the target vehicle, The on-board device of the target vehicle further a map information acquisition unit that acquires map information indicating a map including at least the road; a display unit that displays the location of the vehicle, the occurrence location, and the event video on the map, based on the map information, the location information, the event information provided by the providing unit, and the specific video data, and An event video providing system characterized by:

2. Each of the in-vehicle devices further comprises: an object detection unit that analyzes the video and detects a predetermined object included in the video; a calculation unit that calculates a usefulness rate indicating a rate at which the detection area of ​​the object is not included in the video, the meta information generation unit generates the meta information including the usefulness rate; The conversion information further includes information for converting the information so that the higher the usefulness rate, the higher the score.

2. The event video providing system according to claim 1.

3. the event information includes event occurrence lane information indicating the lane in which the event occurred; Each of the in-vehicle devices further comprises: a lane marking detection unit that analyzes the image and detects lane markings of the road included in the image; a road information acquisition unit that acquires road information including the number of lanes of the road based on the location information; a lane determination unit that determines a driving lane in which the host vehicle is traveling based on the road information and the detection result of the lane markings, the meta information generation unit generates the meta information including driving lane information indicating the driving lane; The conversion information further includes information for converting the score to the highest score when the driving lane is the same as the lane in which the event occurred, and the score to the next highest score when the driving lane is an adjacent lane to the lane in which the event occurred.

2. The event video providing system according to claim 1.

4. Each of the in-vehicle devices further comprises: an in-vehicle storage unit that stores vehicle model information indicating the vehicle model of the vehicle; the meta information generation unit generates the meta information including the vehicle model information; The conversion information further includes information that associates the vehicle type information with a vehicle height, and converts the vehicle height to a higher score as the vehicle height increases.

2. The event video providing system according to claim 1.

5. a management communication unit that sequentially receives, from each in-vehicle device that records video data of an image of the vehicle's surroundings in association with meta information related to the video, identification information that identifies each in-vehicle device itself, vehicle position information including the position of the vehicle in which each in-vehicle device is installed, and the date and time of the position; an event information acquisition unit that acquires event information including an occurrence location and an occurrence date and time of an event that has occurred on a road on which the vehicle is traveling; a collection range determination unit that determines a predetermined area and time as a collection range of event video related to the event based on the event information; a vehicle extraction unit that extracts, based on the vehicle position information, specific vehicles that existed within the collection range from among the vehicles, and target vehicles that do not exist within the collection range, are traveling toward the occurrence position, and exist within a predetermined range from the occurrence position; a meta information request unit that requests meta information of the video captured within the collection range from the in-vehicle device of the specific vehicle; a management storage unit that stores conversion information that converts the meta information into a score; the conversion information includes information for converting the location included in the meta information so that the closer the location included in the meta information is to the occurrence location included in the event information, the higher the score; an evaluation unit that stores the plurality of pieces of meta information received from the in-vehicle device by the management communication unit and converts the pieces of meta information into the scores using the conversion information; a specific video data requesting unit that requests specific video data associated with the high-score meta information from the in-vehicle device that has transmitted the high-score meta information having the highest score among the plurality of pieces of meta information; a providing unit that provides the event information and the specific video data to the in-vehicle device of the target vehicle.

6. An event video providing method used in a management device that provides video related to an event, Sequentially receiving vehicle position information from each in-vehicle device that records video data of an image of the surroundings of the vehicle in association with meta information related to the video, the vehicle position information including identification information for identifying each in-vehicle device itself, the position of the vehicle in which each in-vehicle device is installed, and the date and time of the position; Acquire event information including the location and date and time of the event that has occurred on the road on which the vehicle is traveling; determining a predetermined area and time as a collection range for event video related to the event based on the event information; extracting, based on the vehicle position information, specific vehicles that existed within the collection range from among the vehicles, and target vehicles that did not exist within the collection range, were traveling toward the occurrence position, and existed within a predetermined range from the occurrence position; requesting meta information of the video captured within the collection range from the in-vehicle device of the specific vehicle; storing the plurality of pieces of meta information received from the in-vehicle device, and converting the meta information into a score using conversion information; the conversion information includes information for converting the location included in the meta information so that the closer the location included in the meta information is to the occurrence location included in the event information, the higher the score; requesting specific video data associated with the high-score meta information from the in-vehicle device that has transmitted the high-score meta information having the highest score among the plurality of pieces of meta information; providing the event information and the specific video data to the in-vehicle device of the target vehicle; How to provide event footage.

7. A program for causing a computer that provides video related to an event to execute an event video providing process, The event video providing process includes: receiving, from each in-vehicle device that records video data of an image of the surroundings of the vehicle in association with meta information relating to the video, identification information for identifying each in-vehicle device itself, vehicle position information including the position of the vehicle in which each in-vehicle device is installed, and the date and time of the position; acquiring event information including the location and date and time of the event that has occurred on a road on which the vehicle is traveling; determining a predetermined area and time as a collection range of event video related to the event based on the event information; extracting, based on the vehicle position information, specific vehicles that existed within the collection range from among the vehicles, and target vehicles that did not exist within the collection range, traveled toward the occurrence position, and existed within a predetermined range from the occurrence position; a step of requesting meta information of the video captured within the collection range from the in-vehicle device of the specific vehicle; storing the plurality of pieces of meta information received from the in-vehicle device, and converting the meta information into a score using conversion information; the conversion information includes information for converting the location included in the meta information so that the closer the location included in the meta information is to the occurrence location included in the event information, the higher the score; requesting specific video data associated with the high-score meta information from the in-vehicle device that has transmitted the high-score meta information having the highest score among the plurality of pieces of meta information; providing the event information and the specific video data to the in-vehicle device of the target vehicle; Programs including.

Images