Accident analysis device, accident analysis method, and program

Abstract

An accident analysis device and the like that improves the accuracy of analyzing accident situations is provided. [Solution] In a collision timing identification process, the accident analysis device identifies the collision timing between the host vehicle and the collision object based on the audio data (S15). The accident analysis device determines whether the host vehicle has rolled over based on the vertical acceleration (S16). If it determines that the host vehicle has rolled over, it filters the acceleration data after the rollover timing so that it is ignored in other processes (S18). The accident analysis device identifies the collision object based on the collision direction of the host vehicle before and after the collision timing, and the distance and direction of surrounding objects from the host vehicle (S20).

Description

[Technical Field]

[0001] The present invention relates to an accident analysis device, an accident analysis method, and a program. [Background technology]

[0002] An accident analysis device has been proposed that acquires vehicle data measured by sensors equipped in the accident vehicle and video data captured by a camera equipped in the accident vehicle, and analyzes the circumstances of the accident caused by the accident vehicle based on the acquired vehicle data and video data (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6679152 Summary of the Invention [Problem to be solved by the invention]

[0004] In the accident analysis device as described in Patent Document 1, it is desired to improve the accuracy of the analysis of the accident situation.

[0005] Therefore, an object of the present invention is to provide an accident analysis device and the like that improves the accuracy of analysis of accident situations. [Means for solving the problem]

[0006] In order to achieve the above object, the accident analysis device of the present invention comprises: an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and the acquired video data, The analysis unit beforeIf there is a timing when the volume of the audio data included in the video data is higher than a predetermined threshold, the timing is determined to be the timing of an accident. death, A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed. .

[0008] When there are a plurality of timings in which the volume of the audio data included in the video data is higher than the determination value, the analysis unit may determine the timing with the largest volume as the timing when the accident occurred.

[0009] The analysis unit may be configured to determine that, if there are multiple times when the volume of the audio data included in the video data is higher than the judgment value, the first time when the volume exceeds the judgment value is the time when the accident occurred.

[0010] The analysis unit may determine the severity of the accident based on audio data included in the video data.

[0011] Another accident analysis device of the present invention includes: an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and the acquired video data, The analysis unit If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs; before A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of an accident occurrence is determined based on the audio data from which the frequency corresponding to a human scream has been removed. do.

[0012] The accident analysis method of the present invention also includes: An accident analysis method using an accident analysis device, an acquisition step of acquiring vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis step of analyzing the circumstances of the accident caused by the vehicle based on the acquired vehicle data and the acquired video data, In the analyzing step, before If there is a timing when the volume of the audio data included in the video data is higher than a predetermined threshold, the timing is determined to be the timing of an accident. death, If there is no timing when the volume of the audio data included in the video data is higher than the determination value, the timing when the volume is at its maximum is determined to be the timing when the accident occurred. . Another self-analysis method of the present invention comprises: An accident analysis method using an accident analysis device, an acquisition step of acquiring vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis step of analyzing the circumstances of the accident caused by the vehicle based on the acquired vehicle data and the acquired video data, In the analyzing step, If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs; A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed.

[0013] The program of the present invention also includes: Computer, an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and video data; The analysis unit before If there is a timing when the volume of the audio data included in the video data is higher than a predetermined threshold, the timing is determined to be the timing of an accident. death, If there is no timing when the volume of the audio data included in the video data is higher than the determination value, the timing when the volume is at its maximum is determined to be the timing when the accident occurred. . Another program of the present invention is Computer, an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and video data; The analysis unit If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs; A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed. [Effects of the Invention]

[0014] According to the present invention, the accuracy of the analysis of the accident situation can be improved. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a diagram illustrating an example of an accident analysis system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram showing the state of the present embodiment from inside the vehicle facing forward. [Figure 3] FIG. 1 is a functional block diagram of a drive recorder according to an embodiment of the present invention. [Figure 4] 1 is a diagram illustrating an example of a hardware configuration of an accident analysis device according to an embodiment of the present invention. [Figure 5] 1 is a functional block diagram of an accident analysis device according to an embodiment of the present invention. [Figure 6] 10 is a flowchart illustrating an example of an accident situation analysis process. [Figure 7] 10 is a flowchart illustrating an example of a collision timing identification process. [Figure 8] 10 is a flowchart illustrating an example of a collision object identification process. [Figure 9] 10A and 10B are diagrams illustrating a method for determining whether a vehicle has rolled over. [Figure 10] FIG. 10 is a diagram illustrating an example of identifying a collision target. [Figure 11] FIG. 10 is a diagram illustrating an example of identifying a collision target. [Figure 12] FIG. 10 is a diagram illustrating an example of information indicating the status of an accident output by the accident analysis device. [Figure 13] 10 is a diagram for explaining a processing procedure when the accident analysis device detects surrounding objects and identifies their coordinates. FIG. [Figure 14] 4 is a diagram for explaining a processing procedure when the accident analysis device estimates the relative positional relationship between a vehicle and a surrounding object. FIG. [Figure 15] 4 is a diagram for explaining a processing procedure when the accident analysis device estimates the absolute positions of a vehicle and a surrounding object. FIG. [Figure 16] FIG. 10 is a diagram for explaining a process for estimating the absolute position of a vehicle. [Figure 17] FIG. 10 is a diagram illustrating an example of an image generated by the accident analysis device. [Figure 18] FIG. 10 is a diagram for explaining a collision direction. [Figure 19] FIG. 10 is a diagram illustrating a process for determining a contact object. [Figure 20] FIG. 10 is a diagram for explaining a contact portion. [Figure 21] 10 is a diagram for explaining a process for identifying the positional relationship between a pedestrian and a crosswalk / safety zone. FIG. [Figure 22] FIG. 2 is a diagram for explaining the traveling directions of a vehicle and other vehicles. [Figure 23] FIG. 2 is a diagram illustrating an example of an accident case DB. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings, in which the same or corresponding parts are designated by the same reference numerals.

[0017] Fig. 1 is a diagram showing an example of an accident analysis system according to this embodiment. In the accident analysis system 1 according to this embodiment, a drive recorder 1100 (see Fig. 2) is mounted on a vehicle 1000 of a policyholder, and the drive recorder 1100 is connected to a cloud storage server (not shown) via a network (including a public line network) by wireless communication, and data (video data and vehicle data) acquired by the drive recorder 1100 is stored in the cloud storage server. In addition, an accident analysis device 10 and a terminal 20 are connected to the network. The accident analysis system 1 includes at least the drive recorder 1100 and the accident analysis device 10.

[0018] In this embodiment, an example is shown in which the drive recorder 1100 connects to a cloud storage server (not shown) via a network (including a public line network, etc.) via wireless communication, but for example, a dedicated mirror-type terminal device with similar functions may be prepared and installed in place of the mirror 1300.

[0019] A data storage area for this customer service is secured in the cloud storage server. More specifically, a data storage area that can be accessed by the general center server 30 or the like is secured for each drive recorder 1100 installed in the vehicle 1000.

[0020] The accident analysis device 10 has a function of acquiring vehicle data measured by sensors provided on the vehicle 1000 involved in the accident (hereinafter, sometimes referred to as "own vehicle") or the drive recorder 1100, and video data captured by a camera provided on the drive recorder 1100 of the vehicle 1000, from a cloud storage server via a network, and analyzing the circumstances of the accident caused by the vehicle 1000 based on the acquired vehicle data and video data. Note that, in this embodiment, an example will be described in which the vehicle data and video data are acquired from a cloud storage server, but the vehicle data and video data may also be acquired directly from the vehicle.

[0021] The accident analysis device 10 also has a function of searching for an accident case corresponding to the situation of the accident caused by the vehicle 1000 by comparing the situation of the accident obtained by the analysis with an accident case database that associates the situations of accidents that have occurred in the past with information on past accident cases (hereinafter referred to as "accident cases"). The accident analysis device 10 also has a function of generating an image (reconstruction) showing the situation of the accident caused by the vehicle 1000 by mapping the position of the vehicle 1000 and the positions of other vehicles obtained by analyzing the situation of the accident on map data. The image showing the situation of the accident caused by the vehicle 1000 may be of any type, and may be, for example, a bird's-eye view or a video.

[0022] The accident analysis device 10 may be configured using one or more physical information processing devices, or may be configured using a virtual information processing device that operates on a hypervisor, or may be configured using a cloud server.

[0023] The terminal 20 is a terminal operated by, for example, an operator of an insurance company, and displays the accident situation analyzed by the accident analysis device 10, accident cases corresponding to the accident conditions, and images generated by the accident analysis device 10. The terminal 20 can be any information processing device equipped with a display, such as a personal computer (PC), a notebook PC, a tablet terminal, or a smartphone.

[0024] The drive recorder 1100 may be connected via a network to a server of a general center that provides road service and emergency services. In this case, for example, the server of the general center is configured with one or more computers and is connected to a terminal device (not shown) installed in the general center and operated by each operator of the general center. The operator of the general center may be able to communicate with the drive recorder 1100 installed in the vehicle 1000, road service providers, emergency services (fire departments, private emergency service providers), and the like, using, for example, a headset connected to the terminal device. The network may also be connected to a road service provider system, an emergency service system, and the like, which may also retrieve data from a specified data storage area in a cloud storage server in response to a request from the server of the general center.

[0025] FIG. 2 shows a view of the vehicle 1000 from inside the vehicle 1000 looking forward. The drive recorder 1100 includes a camera, which is attached to the front portion or windshield of the vehicle 1000 as shown in the figure so as to capture at least video in the direction of travel of the vehicle 1000. The camera may also be capable of capturing video from the side and rear of the vehicle 1000. As shown in FIG. 2, an automatic diagnostic system 1400 (e.g., an OBD2 (On-Board Diagnostic system-II)) is installed inside the vehicle 1000, and the automatic diagnostic system 1400 is connected to the drive recorder 1100. The automatic diagnostic system 1400 is connected to control units (e.g., ECUs (Electronic Control Units)) of each part of the vehicle 1000 (e.g., engine, accelerator, brake, blinker, etc.) via a CAN (Controller Area Network) or the like, and is able to acquire information about the status of each part (e.g., whether the accelerator or brake is operated, whether a malfunction is present, speed, engine speed, etc.).

[0026] 3 shows a functional block diagram of a drive recorder 1100 according to this embodiment. The drive recorder 1100 mounted on the vehicle 1000 includes a first communication unit 1110, a second communication unit 1120, a positioning unit 1130, a video recording unit 1140, a sound recording unit 1150, an acceleration measurement unit 1160, an automatic diagnosis data acquisition unit 1170, a control unit 1180, a sensor unit 1190, and a data storage unit 1200.

[0027] The control unit 1180 is composed of processors such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), and causes each component to execute processing related to this customer service. The control unit 1180 transmits measurement data (vehicle data and video data) from each component of the drive recorder 1100 to a cloud storage server via the second communication unit 1120. As a result, the vehicle data and video data of the vehicle 1000 acquired by the drive recorder 110 are stored in the cloud storage server.

[0028] The first communication unit 1110 is a communication unit for communicating with a general center or the like using, for example, VoIP (Voice over Internet Protocol). However, it may also be a communication function of a general mobile phone. The second communication unit 1120 has a function for transmitting data to a cloud storage server, for example, via wireless communication (public line network).

[0029] Furthermore, when instructed by the control unit 1180, the positioning unit 1130 acquires the absolute position (for example, latitude and longitude) of the vehicle 1000 using, for example, a GPS (Global Positioning System) and stores it in the data storage unit 1200.

[0030] The recording unit 1140 stores video data (video data) captured by a camera mounted on the drive recorder 1100 in the data storage unit 1200. The sound recording unit 1150 stores audio data (audio data) input from a microphone in the data storage unit 1200. The sound recording unit 1150 may be integrated with the recording unit 1140. Furthermore, the recording unit 1140 continuously records video when instructed to do so by the control unit 1180, for example, and stores the video data in the data storage unit 1200. The control unit 1180 is capable of extracting both video data from a certain period before a specific time (such as the timing of an accident) and video data from a certain period after the specific time. The same applies to the sound recording unit 1150. The recording unit 1140 is capable of capturing video images of both the exterior and interior of the vehicle 1000. In this embodiment, the video data is described as including audio data, but the video data may also refer to video data that does not include audio.

[0031] The acceleration measuring unit 1160 measures an acceleration value using, for example, an acceleration sensor and outputs the value to the control unit 1180. When instructed by the control unit 1180, the automatic diagnosis data acquiring unit 1170 acquires automatic diagnosis data from the automatic diagnosis system 1400 installed inside the vehicle 1000. The automatic diagnosis data includes data indicating the presence or absence of damage to each of the engine, battery, fuel system, etc. The automatic diagnosis data also includes vehicle data used to recreate the accident situation, such as control information for each part of the vehicle 1000 (accelerator, brake, steering wheel, turn signal), operation information (vehicle speed, throttle opening (accelerator opening), brake operation information, steering wheel operation information, turn signal operation status, etc.). The vehicle data includes at least position information and acceleration information of the vehicle 1000. In this embodiment, the vehicle data also includes vehicle speed data acquired from the vehicle. The vehicle data also includes information related to the driver, such as brake operation information and steering wheel operation information.

[0032] The sensor unit 1190 includes a driver's gaze sensor composed of a camera or the like, an exhalation sensor that detects the driver's breath, and the like. The gaze sensor is composed of a camera that captures an image of the driver and an image processing device, and is capable of detecting the driver's gaze (gaze direction), and outputs the detected information about the driver's gaze (gaze direction) to the control unit 1180. The breath sensor detects the alcohol concentration (ethanol concentration) in the driver's breath (or the air inside the vehicle), and outputs the detected alcohol concentration information to the control unit 1180. That is, in this embodiment, the vehicle data includes information about the driver, such as the driver's gaze (gaze direction) and information about the driver's breath (whether or not they have drunk alcohol). In this embodiment, the drive recorder 1100 is configured to have a sensor unit 1190 including a gaze sensor, a breath sensor, etc., but the drive recorder 1100 may also acquire vehicle data such as information regarding the driver's gaze (direction of gaze) and the driver's breath (whether or not the driver has been drinking) from sensors (gaze sensor, breath sensor, etc.) provided in the vehicle 1000 or installed in the vehicle 1000.

[0033] The sensor unit 1190 may include, for example, a vehicle speed sensor, a geomagnetic sensor, a throttle sensor, and / or a turn signal detection sensor. The vehicle data measured by the sensors provided in the vehicle 1000 may also include, for example, the speed of the vehicle 1000, the direction in which the vehicle 1000 is facing (for example, the angle when north is 0 degrees), the throttle opening (accelerator opening), and the operation status of the turn signals. At least some of these may be acquired by the automatic diagnosis data acquisition unit 1170 from the automatic diagnosis system 1400 (sensors provided in the vehicle 1000).

[0034] The data storage unit 1200 is composed of storage devices such as memory, HDD (Hard Disk Drive) and / or SSD (Solid State Drive), and stores in advance equipment data such as company name, organization name, vehicle registration number, driver identifier (ID), telephone number, etc., and also stores data acquired by each component in response to instructions from the control unit 1180.

[0035] 4 is a diagram illustrating an example of the hardware configuration of the accident analysis device 10. The accident analysis device 10 includes a processor 11 such as a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit), a storage device 12 such as a memory, an HDD (Hard Disk Drive) and / or an SSD (Solid State Drive), a communication IF (Interface) 13 for wired or wireless communication, an input device 14 for accepting input operations, and an output device 15 for outputting information. The input device 14 is, for example, a keyboard, a touch panel, a mouse, and / or a microphone. The output device 15 is, for example, a display and / or a speaker.

[0036] FIG. 5 is a functional block diagram of the accident analysis device 10 according to this embodiment. The accident analysis device 10 includes a storage unit 100, an acquisition unit 101, an analysis unit 102, a generation unit 103, a search unit 104, and an output unit 105. The storage unit 100 can be implemented using the storage device 12 included in the accident analysis device 10. The acquisition unit 101, the analysis unit 102, the generation unit 103, the search unit 104, and the output unit 105 can be implemented by the processor 11 of the accident analysis device 10 executing a program stored in the storage device 12. The program can be stored in a storage medium. The storage medium storing the program may be a non-transitory computer-readable medium. The non-transitory storage medium is not particularly limited and may be, for example, a storage medium such as a USB (Universal Serial Bus) memory or a CD-ROM.

[0037] The storage unit 100 stores an accident case database (DB) that associates accident situations with accident cases of the accidents, and a map data DB. The accident case DB may include information indicating the fault ratio. The accident case and fault ratio corresponding to the situation of the accident that occurred may be searchable using the information indicating the accident situation as a key. The map data DB includes various data such as road data, road width, direction of travel, road type, traffic signs (stop sign, no entry, etc.), speed limit, location of traffic lights, and the number of intersecting roads at intersections. The storage unit 100 may be realized by an external server that can communicate with the accident analysis device 10. Although not shown in the figure, the storage unit 100 also stores operator information about insurance company operators.

[0038] The acquisition unit 101 has a function of acquiring vehicle data and video data measured and captured by the vehicle 1000 (accident vehicle) from a cloud storage server.

[0039] The analysis unit 102 has a function of analyzing the circumstances of an accident caused by the vehicle 1000 based on the vehicle data and video data acquired by the acquisition unit 101.

[0040] The analysis unit 102 has a function of determining the timing of an accident based on the audio data included in the video data.

[0041] The analysis unit 102 also has the function of estimating the collision direction of the vehicle in an accident based on the acceleration data included in the vehicle data, and identifying, as a collision target, surrounding objects included in the video data within a predetermined time before and after the determined timing of the accident that are in the collision direction and whose distance to the vehicle 1000 is decreasing.

[0042] In addition, the analysis unit 102 determines whether the vehicle overturned at the time of the accident based on the vertical acceleration data included in the vehicle data, and if it determines that the vehicle overturned, it does not use the acceleration data after the overturn in analyzing the situation of the accident.

[0043] The analysis unit 102 also has a function of analyzing the driver's condition at the time of the accident as the accident situation, based on information about the driver included in the vehicle data.

[0044] Furthermore, the vehicle data of the vehicle 1000 may include information indicating the absolute position of the vehicle 1000 measured by at least the GPS of the positioning unit 1130, and the analysis unit 102 may estimate the absolute position of the other vehicle based on the information indicating the absolute position of the vehicle 1000 (position data acquired by the positioning unit 1130) and information indicating the relative positional relationship between the vehicle 1000 and the other vehicle, which is obtained by analyzing an image of the other vehicle captured in the image data. Furthermore, the analysis unit 102 may estimate the absolute positions of the vehicle 1000 and the other vehicle in time series.

[0045] For example, the analysis unit 102 estimates the time series of the absolute position and orientation of the vehicle 1000 from information indicating the absolute position of the vehicle 1000 measured by GPS and the movement of feature points in each frame included in the video data. That is, the analysis unit 102 integrates the information indicating the absolute position of the vehicle 1000 measured by GPS and information indicating the relative position obtained from the video data to estimate the time series of the absolute position and orientation of the vehicle 1000.

[0046] Furthermore, the analysis unit 102, for example, (1) estimates "depth of the target = depth of the target in the direction directly in front of the camera" using the formula "depth of the other vehicle (target) = focal length of the camera × real-world height of the assumed other vehicle (target) / height of the other vehicle (target) in the image." (2) Based on the image coordinates (X, Y) of the target center, converts "depth" into "distance = straight-line distance between the optical center of the camera and the target." Specifically, the distance may be calculated by extending a vector from the optical center of the camera in the direction of the target until it reaches the depth calculated in (1), and then calculating the length of that vector.

[0047] In addition, the analysis unit 102 may calculate the angle between the direction directly ahead of the camera equipped in the drive recorder 1100 and the direction from the optical center of the camera to another vehicle (target), and transfer the vector in the camera coordinate system from the camera to the other vehicle (target) to a world coordinate system (latitude and longitude) using GPS to analyze the circumstances of the accident.

[0048] In addition, the analysis unit 102 may estimate the absolute position of the vehicle 1000 by analyzing the video data, and may analyze the circumstances of the accident by considering the absolute position of the accident vehicle estimated by analyzing the video data and the absolute position of the vehicle 1000 measured by the positioning unit 1130 as the absolute position averaged based on a predetermined weight.

[0049] In addition, the analysis unit 102 may analyze the circumstances of the accident by comparing the image size of the part of the video data in which the other vehicle is captured with data indicating the correspondence between image size and distance, and estimating the distance between the vehicle 1000 and the other vehicle, which is one piece of information indicating the relative positional relationship between the vehicle 1000 and the other vehicle.

[0050] In addition, the analysis unit 102 may analyze the circumstances of the accident by comparing the difference between the coordinates of the point in the video data where the other vehicle is captured and the center coordinates of the video data with data indicating the correspondence between coordinates and angles, and estimating the angle difference between the direction of travel of the vehicle 1000 and the direction in which the other vehicle is located, which is one piece of information indicating the relative positional relationship between the vehicle 1000 and the other vehicle.

[0051] The accident circumstances analyzed by the analysis unit 102 may include at least the color of the traffic light when the vehicle 1000 passes through the intersection, the priority relationship when passing through the intersection, and whether the vehicle speed of the vehicle 1000 exceeds the speed limit.

[0052] The generation unit 103 has a function of generating an image (reconstruction) showing the situation of an accident caused by the vehicle 1000 by mapping the absolute position of the vehicle 1000 and the absolute position of the other vehicle onto map data. The image may include an overhead view or a video. For example, the generation unit 103 may generate a video in which images showing the situation of the accident are arranged in chronological order. Note that if there is no other vehicle involved in the accident, the generation unit 103 generates an image showing the situation of the accident caused by the vehicle 1000 by mapping at least the absolute position of the vehicle 1000 onto map data. In this case, surrounding objects of the collision target (people, utility poles, guardrails, etc.) may be mapped onto the map data.

[0053] The search unit 104 has a function of comparing the circumstances of the accident caused by the vehicle 1000 analyzed by the analysis unit 102 with an accident case DB (accident case data) that associates the circumstances of the accident with past accident cases, thereby searching for an accident case corresponding to the circumstances of the accident caused by the vehicle 1000. In addition, the search unit 104 may search for the fault ratio of the vehicle 1000 in the accident caused by the vehicle 1000 as an accident case corresponding to the circumstances of the accident caused by the vehicle 1000.

[0054] In addition, the accident case may include one or more correction ratios (correction information) for correcting the fault ratio. If a correction ratio corresponding to the circumstances of the accident caused by the accident vehicle is present among the one or more correction ratios, the search unit 104 may correct the fault ratio of the accident vehicle in accordance with the correction ratio.

[0055] The output unit 105 has the function of outputting to the terminal 20 information indicating the accident situation analyzed by the analysis unit 102, accident cases corresponding to the accident conditions searched for by the search unit 104, and images indicating the accident situation generated by the generation unit 103.

[0056] Note that the functions of the accident analysis device 10 may be realized in cooperation with the terminal 20, and therefore some functions may be provided on the terminal 20 side. Furthermore, the accident analysis device 10 and the terminal 20 may be integrated into a single information processing device.

[0057] FIG. 6 is a flowchart showing an example of an accident situation analysis process in which the accident analysis device 10 analyzes the situation of an accident. First, the accident analysis device 10 acquires vehicle data and video data of the vehicle 1000 involved in the accident at the time of the accident (a predetermined period before and after the accident) (S10, S11). Specifically, the accident analysis device 10 acquires the vehicle data and video data from a cloud storage server via a network. Alternatively, for example, the vehicle data and video data may be transmitted to the accident analysis device 10 by wireless signals using a communication function provided in the vehicle 1000 or the drive recorder 1100. Furthermore, the vehicle data and video data may be imported into the accident analysis device 10 by connecting a storage medium on which the vehicle data and video data are stored to the accident analysis device 10.

[0058] Next, the accident analysis device 10 performs image analysis on the video data at the time of the accident for each frame to identify one or more peripheral objects (other vehicles, people, signs, road structures, etc.) present around the vehicle 1000 that are captured in the image for each frame, and further identifies coordinates (coordinates in the image) indicating the position of the identified peripheral objects in the image (S12). Next, the accident analysis device 10 estimates the relative positional relationship between the vehicle 1000 and the peripheral objects present around the vehicle 1000 for each peripheral object based on the coordinates (depth of the peripheral object) of the identified peripheral objects (S13). Note that the terms "identified" and "estimated" are used here, and "identified" may be used when there is a higher probability, but it does not necessarily mean that it can be "identified" and there is no significant difference in meaning from "estimated." This is the same for this embodiment.

[0059] Next, the accident analysis device 10 estimates the absolute position of the vehicle 1000 using the position information of the vehicle 1000 acquired by the positioning unit 1130 and / or the video data at the time of the accident. Furthermore, the accident analysis device 10 estimates the absolute positions of the surrounding objects present around the vehicle 1000 based on the estimated absolute position of the vehicle 1000 and the relative positional relationship between the vehicle 1000 and the surrounding objects present around the vehicle 1000 estimated in the processing procedure of step S13 (S14).

[0060] Steps S10 to S14 will be described in detail later.

[0061] Next, the accident analysis device 10 executes a collision timing identification process for identifying (determining) the collision timing between the vehicle 1000 and the collision object (S15).

[0062] 7 is a flowchart showing an example of the collision timing identification process. In the collision timing identification process, the accident analysis device 10 first acquires audio data at the time of the accident (S101). For example, audio data at the time of the accident (a predetermined period before and after the accident) may be extracted from video data already acquired from a cloud storage server.

[0063] Next, the accident analysis device 10 sets a high-frequency exclusion filter on the acquired audio data to remove high-frequency (predetermined frequency) audio equivalent to screams from the audio data (S102). Therefore, the high-frequency exclusion filter may be any filter capable of removing audio with a frequency equivalent to a human scream (e.g., 4 kHz or higher). It is also possible to set a filter that removes audio with a predetermined frequency equivalent to braking noise before a collision or other noise. In this way, audio with a predetermined frequency is removed from the audio data, making it possible to prevent human screams and the like from being detected as the collision sound of an accident, and to suitably analyze and reproduce the accident situation.

[0064] Next, the accident analysis device 10 determines whether there is a timing when the volume of the sound data from which high frequencies have been removed is higher than an accident determination value (S103). The accident determination value may be determined in advance based on statistical data on the volume of sounds generated during accidents, etc.

[0065] If there is a timing when the volume is higher than the accident determination value (S103; Yes), that timing is identified as the collision timing, i.e., the timing when the accident occurred (S104). If there is no timing when the volume is higher than the accident determination value (S103; No), the timing when the volume is at its maximum is identified as the collision timing (S105). Note that if there are multiple timings when the volume is higher than the accident determination value, the timing when the volume is at its maximum may be identified as the collision timing, or the first timing when the volume exceeds the accident determination value may be identified as the collision timing. Thereafter, the collision timing identification process is terminated, and the process returns to FIG. 6.

[0066] In this embodiment, the timing of the collision (timing of the occurrence of the accident) is determined based on the audio data (volume), but the type and severity of the accident may also be determined based on the audio data (volume). For example, if the volume exceeds a threshold for determining a major accident, the accident may be determined to be a major accident. Also, if there is no time when the volume is higher than the threshold for determining an accident, the accident may be determined to be a minor accident.

[0067] In this way, the accident analysis device 10 of this embodiment identifies the collision timing based on the audio data, thereby improving the accuracy of identifying the collision timing. In particular, the collision timing is identified using audio data from which audio of a predetermined frequency has been excluded, thereby improving the accuracy of identifying the collision timing. Note that, although the collision timing is identified based on the volume in this embodiment, the collision timing may also be identified based on the quality, type, or frequency of the sound.

[0068] If it is difficult to determine the timing of the collision, for example, if the audio data cannot be acquired normally or if there is no timing when the volume is higher than the accident determination threshold, the timing of the collision may be determined based on acceleration data included in the vehicle data, or based on both the audio data and acceleration data.

[0069] After executing the collision timing identification process, the accident analysis device 10 determines whether or not the vehicle 1000 has rolled over based on the vertical acceleration data included in the vehicle data (S16).

[0070] 9A and 9B are diagrams illustrating a method for determining whether the vehicle 1000 has rolled over according to this embodiment. FIG. 9A shows time-series vertical acceleration data before and after an accident involving the vehicle 1000. FIG. 9A shows two peaks where the acceleration is positive (upward). The second peak (MAX) has a higher acceleration than the first peak (2nd).

[0071] FIG. 9(B) shows the vertical acceleration integral value of vehicle 1000 in a time series before and after an accident. In FIG. 9(B), the acceleration integral value increases from the second acceleration point in FIG. 9(A), and then suddenly increases off the scale at the maximum acceleration point. In this embodiment, if the acceleration integral value exceeds a predetermined rollover determination value, it is determined that vehicle 1000 has rolled over at that point. In FIG. 9(B), the acceleration integral value exceeds the rollover determination value at the maximum acceleration point, so it is determined that vehicle 1000 has rolled over at the maximum acceleration point.

[0072] In this way, the accuracy of the rollover determination can be improved because the rollover determination is based on the vertical acceleration (integral value) of the vehicle 1000. Note that the rollover determination may be based on the vertical acceleration value instead of the integral value.

[0073] If it is determined that the vehicle 1000 has overturned because the vertical acceleration exceeds the overturn determination value (S17; Yes), the acceleration data from the time of the overturn is filtered so that it is not used in other processing or analysis (S18).

[0074] This prevents the accident situation from being analyzed and reconstructed based on the acceleration data after the fall, thereby improving the accuracy of the reconstruction of the accident situation. For example, when creating a reconstruction of the accident situation by estimating the trajectory of the vehicle 1000 based on the acceleration information, it is possible to prevent the trajectory of the vehicle from being drawn based on the acceleration after the fall, resulting in an inappropriate reconstruction diagram or trajectory being drawn.

[0075] If it is determined that the vehicle 1000 has not turned over (S17; No), step S18 is skipped.

[0076] Next, the accident analysis device 10 estimates the direction in which the vehicle 1000 collided with another vehicle or an obstacle (e.g., a head-on collision) based on the acceleration data in the forward / backward and left / right directions at the collision timing identified in step S15 (S19). The acceleration data in the forward / backward and left / right directions is included in the vehicle data. Step S19 will be described in detail later.

[0077] Next, the accident analysis device 10 executes a collision object identification process for identifying a collision object from the detected surrounding objects (S20).

[0078] 8 is a flowchart showing an example of the collision target identification process. In the collision target identification process, the accident analysis device 10 first extracts surrounding objects detected within N seconds (for example, 2 seconds; N may be any number) before and after the collision timing identified in the process of step S15 (S201). The surrounding objects extracted here become collision target candidates.

[0079] Then, the accident analysis device 10 excludes from the extracted surrounding objects any surrounding objects that are not in the collision direction estimated in the process of step S19 (S202). Here, for example, it is determined that any surrounding object that is not on an extension line of a range of X degrees (for example, 120 degrees; X may be any value) from the collision direction estimated in the process of step S19 cannot be a collision target, and the accident analysis device 10 excludes the surrounding object from the collision target candidates.

[0080] Next, the accident analysis device 10 determines that surrounding objects whose distance from the vehicle (vehicle 1000) is increasing during the N seconds before and after the collision timing are not likely to be collision targets, and excludes the surrounding objects from the list of collision target candidates (S203).

[0081] Next, the accident analysis device 10 determines that surrounding objects that are at a distance of D or more (for example, 10 meters or more) from the vehicle (vehicle 1000) at the time of collision cannot be collision targets, and excludes the surrounding objects from collision target candidates (S204).

[0082] After narrowing down the collision object candidates in this way, the accident analysis device 10 determines whether there are multiple objects around the collision object candidates (S205).

[0083] If there are not multiple peripheral objects of the collision target candidate (S205; No), the remaining peripheral objects of the collision target candidate are identified as the collision target (S206). If there are multiple peripheral objects of the collision target candidate (S205; Yes), the peripheral object with the shortest distance from the vehicle among the remaining collision target candidates is identified as the collision target (S207). Thereafter, the collision target identification process ends, and the process returns to FIG. 6.

[0084] In this way, the accident analysis device 10 of this embodiment identifies the collision object based on the collision direction of the own vehicle and the distance and direction of surrounding objects from the own vehicle, thereby improving the accuracy of identifying the collision object.

[0085] It should be noted that the method is not limited to that of this embodiment, and any method may be adopted as long as it can suitably identify the object of collision.

[0086] 10 and 11 are diagrams showing examples of identifying a collision target. As shown in FIG. 10, when the collision direction of the host vehicle is approximately 7 o'clock (rear left direction), surrounding objects to the right front and right, which are not in the collision direction, are excluded from the candidates for collision target as they are not likely to collide. The direction of collision is the opposite direction of the detected acceleration (impact direction). Furthermore, even if a surrounding object is in the impact direction, a surrounding object that is increasing in distance from the host vehicle (the vehicle at the bottom right in the figure) is excluded from the candidates for collision target as it is not likely to collide. As a result, vehicles in the vicinity of the collision direction (rear left direction) are identified as the collision target.

[0087] As shown in Figure 11, when the collision direction of the host vehicle is approximately in the 3 o'clock direction (to the right), even if a surrounding object is in the impact direction, the surrounding object (vehicle in the upper right in the figure) that is getting closer to the host vehicle is excluded from the candidates for collision object because it is unlikely to collide. As a result, vehicles in the vicinity of the collision direction are identified as collision objects.

[0088] Next, the accident analysis device 10 generates an image showing the accident situation (a re-creation of the accident situation including an overhead view and a video) by mapping the calculated absolute positions of the vehicle 1000 and the surrounding objects around the vehicle 1000 onto map data (S21).

[0089] Next, the accident analysis device 10 uses at least the vehicle data of the vehicle 1000, the video data at the time of the accident, the absolute positions of the vehicle 1000 and the surrounding objects around the vehicle 1000 estimated in the process of step S14, the collision timing identified in the process of step S15, whether the vehicle 1000 has rolled over determined in step S16, the collision direction of the vehicle 1000 estimated in the process of step S19, the object of collision identified in the process of step S20, and map data to output information indicating the situation of the accident caused by the vehicle 1000 (S22). The information indicating the situation of the accident also includes the information shown in FIG. 12. The information indicating the situation of the accident includes, for example, multiple items (which may be referred to as tags), and the situation of the accident is identified by combining the items.

[0090] Next, the accident analysis device 10 searches the accident case database using each item as a key to acquire an accident case corresponding to the situation of the accident caused by the vehicle 1000 (S23). This completes the accident situation analysis process.

[0091] Steps S21 to S23 will be described in detail later.

[0092] The order of the processing steps described above can be changed as desired as long as no contradictions occur in the processing. For example, the processing steps of steps S12 to S20 can be changed as appropriate. Furthermore, some of the processing (steps S21, S23, etc.) may be executed as separate processing.

[0093] FIG. 12 is a diagram showing an example of items included in information indicating the accident situation. The information indicating the accident situation includes multiple items as shown in FIG. 12. "Details" shown in FIG. 12 indicate specific content for each item. "A: Contact object (automobile, obstacle, etc.)" shown in FIG. 12 is information indicating the object with which the vehicle 1000 made contact (collided). "B: Contact location" indicates the location where the vehicle 1000 made contact (collided) with the object (e.g., the front, right side, left front bumper, etc.). "C: Road type" indicates the type of road (straight, curved, intersection, T-junction, expressway, etc.) on which the vehicle 1000 was traveling at the time of the accident. "D: Signal color of own vehicle and other vehicle" indicates the signal color of the vehicle 1000 and the other vehicle in the case of an accident at an intersection. "E: Positional relationship between pedestrian and crosswalk / safety zone" indicates whether the pedestrian was involved in the accident on a crosswalk or safety zone, or whether the pedestrian was involved in the accident at a location other than a crosswalk or safety zone.

[0094] "F: Direction of travel of own vehicle and other vehicles at intersection" indicates whether vehicle 1000 and other vehicles were traveling straight, changing lanes, turning left, turning right, or making a quick right turn at the time of the accident. "G: Lane position on the expressway" indicates the lane (main lane, passing lane) in which vehicle 1000 was traveling when the accident occurred on the expressway. "H: Priority relationship at intersection" indicates which of vehicle 1000 and other vehicles should have had priority to pass the intersection when an accident occurred at an intersection. "I: Whether or not vehicle 1000 and other vehicles violated stop signs or red lights" indicates whether or not vehicle 1000 and other vehicles violated stop signs or red lights. "J: Whether or not or not speed limit was violated" indicates whether vehicle 1000 and other vehicles were observing the speed limit immediately before the accident. "K: Speed ​​of own vehicle and other vehicles before collision" indicates the speeds of vehicle 1000 and other vehicles immediately before the accident. "L: Presence or absence of obstacles on the road" indicates whether or not there were any obstacles on the road on which vehicle 1000 was traveling at the time of the collision. "M: Opening or closing of the door of the collided vehicle" indicates whether or not the door of the other vehicle was open at the time of the collision. "N: Presence or absence of turn signal before collision" indicates whether or not vehicle 1000 had its turn signal on before the collision.

[0095] "Whether or not attention to the road ahead was detected" indicates whether or not the driver of vehicle 1000 was looking ahead at the time of the collision. "Whether or not drinking alcohol was detected" indicates whether or not drinking alcohol was detected in the driver of vehicle 1000 at the time of the collision. "Braking timing" indicates whether the brakes were applied early, normal, or late before the collision. "Steering operation at the time of impact" indicates whether or not there was steering operation before the collision, whether or not there was sudden steering, etc. These pieces of information are determined and analyzed based on information about the driver included in the vehicle data. In this embodiment, information indicating the accident situation is analyzed based on information about the driver, such as brake operation information, steering operation information, information about the driver's line of sight (gaze direction), and information about the driver's breath (whether or not they had drunk alcohol), but information indicating the accident situation may also be analyzed based on information about other drivers.

[0096] Next, a detailed description will be given of the processing procedure (steps S10 to S14, S19, and S21 to S23 in FIG. 6) when the accident analysis device 10 analyzes the circumstances of an accident caused by the vehicle 1000. In the following description, it is assumed that the accident analysis device 10 has already acquired vehicle data and video data from the vehicle 1000 that caused the accident. It is also assumed that the vehicle data and video data each contain time information or synchronization information. That is, in this embodiment, when analyzing video data at a certain point in time, it is possible to perform the analysis using vehicle data corresponding to that point in time, and conversely, when analyzing vehicle data at a certain point in time, it is possible to perform the analysis using video data corresponding to that point in time.

[0097] (Detection and coordinate determination of surrounding objects) 13 is a diagram for explaining the processing procedure when the accident analysis device 10 detects surrounding objects and identifies their coordinates. This processing procedure corresponds to the processing procedure of step S12 in FIG.

[0098] The analysis unit 102 analyzes each image obtained by breaking down the video data into frames, thereby identifying one or more surrounding objects that appear in the image, including vehicles, people, bicycles, traffic signs, traffic lights (including traffic light colors and arrow signals), structures around the road (electric poles, street lights, guardrails, etc.), fallen objects, road markings, crosswalks, and lanes. Furthermore, the analysis unit 102 identifies the coordinates of the area in the image in which the surrounding object appears, for each of the one or more identified surrounding objects.

[0099] The example in FIG. 13 shows an example of an X-th frame image among images obtained by breaking down video data into frames. The X-axis indicates the number of pixels in the left-right direction (X-coordinate), with the left end at 0, and the Y-axis indicates the number of pixels in the up-down direction (Y-coordinate), with the bottom end at 0. In the example in FIG. 13, a truck is shown at the back of the lane to the left of the lane in which the vehicle 1000 is traveling (the second lane from the left), and a passenger car is shown in the lane to the right of the lane in which the vehicle 1000 is traveling. The analysis unit 102 includes a trained model that has learned the ability to recognize recognition objects (e.g., other vehicles, people, bicycles, road signs, traffic lights, structures around the road (electric poles, streetlights, guardrails, etc.), fallen objects, lanes, etc.), and by inputting an image into the trained model, the type and area of ​​one or more surrounding objects shown in the image may be identified. Such processing can be achieved by using existing technologies such as YOLO (Your Only Look Once) and Mask R-CNN (Mask Regional Convolutional Neural Network).

[0100] In the example of A in Fig. 13, the analysis unit 102 has determined that a truck is captured in area C1 (a rectangular area with X=700, Y=300 at the top left and X=900, Y=300 at the bottom right), and that a passenger car is captured in area C2 (a rectangular area with X=1400, Y=250 at the top left and X=1750, Y=50 at the bottom right).Similarly, the analysis unit 102 has determined that utility poles are captured in areas C3 to C6 (coordinates not shown).

[0101] The example in FIG. 13B shows an example of the results of the analysis unit 102 identifying surrounding objects for each frame.

[0102] Furthermore, the analysis unit 102 may collate the analysis results of the images for each frame to determine whether the vehicle's turn signals are flashing, whether the vehicle doors have opened or closed, and whether a person is standing, sitting, or lying down. For example, the analysis unit 102 may determine whether the turn signals are flashing by determining whether the color of the turn signal portion of the vehicle recognized in each image changes periodically.

[0103] The analysis unit 102 may also determine whether a vehicle door has opened or closed by determining changes in the vehicle door portion recognized in each image. The analysis unit 102 may also determine whether a person is standing, sitting, or lying down based on the ratio of the vertical length to the horizontal length of the area in which the person is captured. The person's pose can be determined using conventional techniques, such as Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields.

[0104] (Estimation of the relative position between the vehicle and surrounding objects) 14 is a diagram for explaining the processing procedure when the accident analysis device 10 estimates the relative positional relationship between the vehicle 1000 and surrounding objects. This processing corresponds to the processing procedure of step S13 in FIG.

[0105] The analysis unit 102 estimates, for each peripheral object, a relative positional relationship between the vehicle 1000 and peripheral objects present around the vehicle 1000, based on an area in which one or more peripheral objects identified in the image are captured. More specifically, the analysis unit 102 estimates a distance (d1 and d2 in B of FIG. 14) from the vehicle 1000 to the peripheral object, based on the size of the area in which the peripheral object is captured. Furthermore, the analysis unit 102 estimates a lateral angle (θ1 and θ2 in B of FIG. 14) at which the peripheral object exists, with the traveling direction of the vehicle 1000 (toward the center of the image) being set as the reference (0 degrees), based on the difference in lateral direction between the central coordinates of the area in which the peripheral object is captured and the central coordinates of the image.

[0106] [Example of calculating distance to surrounding objects] A specific example will be shown using A in FIG. 14. First, the analysis unit 102 is assumed to have stored data indicating how many pixels an object 1 meter in length occupies in an image captured by a camera included in the drive recorder 1100, placed vertically (or horizontally) 1 meter away. Because this data varies depending on the camera's angle of view (lens angle), it may be stored in association with each model of camera included in the drive recorder 1100. Furthermore, depending on the model of camera included in the drive recorder 1100, a wide-angle lens or a fisheye lens is often used. Therefore, the analysis unit 102 may perform distortion correction on the video data captured by the camera included in the drive recorder 1100 in accordance with the lens characteristics of the camera included in the drive recorder 1100, and then calculate the distance to the surrounding object and the horizontal angle at which the surrounding object is located, as described below. Distortion correction can be achieved using conventional techniques such as cubic interpolation.

[0107] In the example of A in Figure 14, if a 1m long object is placed 1m away in the vertical direction and photographed, it is assumed that it will occupy 100 pixels in the vertical direction. Similarly, if a 1m long object is placed 10m away in the vertical direction (or horizontal direction) and photographed, it is determined in advance how many pixels it will occupy in the image. In the example of Figure 14, if a 1m long object is placed 10m away in the vertical direction and photographed, it is assumed that it will occupy 10 pixels in the vertical direction.

[0108] In addition, the vertical (or horizontal) size of each surrounding object is determined in advance. For example, the vertical length (height) of a truck may be set to 2 m, and that of a passenger car may be set to 1.5 m.

[0109] As mentioned above, if a 1m long object 1m away has 100 vertical pixels, and a 1m long object 10m away has 10 vertical pixels, then if a 2m high truck is 1m away, the vertical pixel count can be calculated as 200, and if a 2m high truck is 10m away, the vertical pixel count can be calculated as 20. More specifically, if the number of pixels is X and the distance is Y, the following formula holds: Y = 200 ÷ X (Equation 1)

[0110] Next, the analysis unit 102 calculates the distance between the vehicle 1000 and the truck using Equation 1. In the example of A in FIG. 14, the vertical length of the area C1 is 50((400-300)÷2)=50 pixels. Therefore, according to Equation 1, it can be calculated that Y=200÷50=4 m.

[0111] Note that, in a situation where the height (height from the ground) at which the camera equipped in the drive recorder 1100 is attached to the vehicle 1000 and the direction in which the camera equipped in the drive recorder 1100 should be pointed are fixed with high precision, it is possible to determine the distance to the surrounding object using only the value of the Y axis. However, according to the processing procedure described above, even if the attachment position of the camera equipped in the drive recorder 1100 varies depending on the driver, such as when a drive recorder is rented, it is possible to determine the distance between the vehicle 1000 and the surrounding object.

[0112] [Example of calculating the angle in the left and right direction when surrounding objects are present] A specific example will be shown using A in Fig. 14. First, it is assumed that analysis unit 102 stores data indicating the angle of view (lens angle) of the camera equipped in drive recorder 1100. Since this data differs depending on the model of the camera equipped in drive recorder 1100, the data may be stored in association with each model of the camera equipped in drive recorder 1100.

[0113] Here, by dividing the angle of view of the camera equipped in the drive recorder 1100 by the number of pixels in the horizontal direction of the screen, it is possible to determine how many degrees of the angle of view one pixel corresponds to. For example, if the total number of pixels in the horizontal direction of the screen is 2000 pixels and the angle of view of the camera equipped in the drive recorder 1100 is 80 degrees, 200 pixels correspond to 8 degrees. In other words, by calculating the difference in the number of pixels in the horizontal direction between the center position of the area in which the peripheral object is captured in the image and the center position of the image, it is possible to calculate the horizontal angle (angle in the horizontal plane) at which the peripheral object exists, assuming that the direction of travel of the vehicle 1000 is 0 degrees. Note that if the angle of view corresponding to one pixel increases or decreases depending on the position on the screen, the angle per pixel may be corrected based on the image coordinates.

[0114] For example, in A of FIG. 14, the center position in the X direction of the area in which the truck is captured is X=800. Also, assume that the angle of view of the camera equipped in the drive recorder 1100 is 80 degrees, and the total number of pixels in the horizontal direction of the screen is 2000 pixels. Also, the center position of the image is X=1000. Therefore, the center position of the area in which the truck is captured and the center position of the image are separated by 200 pixels (1000-800). As described above, 200 pixels correspond to 8 degrees, so the horizontal angle between the vehicle 1000 and the truck (θ1 in B of FIG. 14) can be calculated to be 8 degrees.

[0115] It should be noted that with the calculation method described above, if a peripheral object comes extremely close to the vehicle 1000 and part of the peripheral object is outside the image, it becomes impossible to calculate the distance and angle. However, if part of the peripheral object is outside the image, it is possible to estimate that the distance between the peripheral object and the vehicle 1000 is within a certain distance. It is also possible to estimate the position of a peripheral object that has disappeared from the image based on changes in the position of the peripheral object in images of previous and subsequent frames.

[0116] For example, assume that in an image of a certain frame X, a portion of a peripheral object is outside the image, but in the image of the previous frame, frame X-1, the entire peripheral object is captured, and the distance between the vehicle 1000 and the peripheral object is 10 m. Furthermore, assume that in the image of frame X-2, which is the frame before that, the distance between the vehicle 1000 and the peripheral object is 11 m. In this case, the distance between the vehicle 1000 and the peripheral object in the image of frame X can be estimated to be 9 m. Furthermore, when the peripheral object is a vehicle, distance estimation is not performed by image recognition of the entire vehicle, but image recognition may be limited to a portion of the vehicle (such as a license plate). This makes it possible to estimate the distance and angle to the vehicle 1000, even if a portion of the vehicle is outside the image, as long as the license plate is captured in the image.

[0117] (Estimate the absolute position of the vehicle and surrounding objects) 15 is a diagram for explaining the processing procedure when the accident analysis device 10 estimates the absolute positions of the vehicle 1000 and surrounding objects. This processing corresponds to the processing procedure of step S14 in FIG.

[0118] First, the analysis unit 102 estimates the absolute position of the vehicle 1000. The analysis unit 102 may estimate the absolute position of the vehicle 1000 using position information of the vehicle 1000 acquired by a positioning unit 1130 (GPS) provided in the drive recorder 1100, which is included in the vehicle data of the vehicle 1000. Alternatively, the analysis unit 102 may more accurately estimate the absolute position of the vehicle 1000 by combining the absolute position of the vehicle 1000 obtained by analyzing video data at the time of the accident using, for example, a conventional technology called Structure From Motion or Simultaneous Localization and Mapping (hereinafter referred to as "SFM" for convenience) with the absolute position of the vehicle 1000 estimated by the positioning unit 1130.

[0119] [Estimation of the absolute position of the vehicle 1000 based on SFM] By using SFM, feature points included in the image of each frame of video data can be extracted, corresponding points (corresponding feature points) in the image of each frame can be identified from the extracted feature points, and the movement of the identified corresponding points can be tracked to reproduce the camera movement. The extraction of feature points can be achieved, for example, by using a conventional technology called SIFT feature values. Furthermore, the identification of corresponding points can be achieved, for example, by using a conventional technology called FLANN (Fast Library for Approximate Nearest Neighbors).

[0120] Here, since the camera that captured the video data is a camera provided in the drive recorder 1100 mounted on the vehicle 1000, the reproduced camera movement can be considered to be the movement of the vehicle 1000. Furthermore, since the vehicle data of the vehicle 1000 includes vehicle speed data, the distance traveled by the vehicle 1000 between each frame may be estimated by comparing the vehicle speed data with the frame rate of the video data. For example, if the vehicle 1000 is traveling at a speed of 54 km / h and the frame rate of the video data is 15 frames per second, the distance traveled by the vehicle 1000 per frame will be approximately 1 meter.

[0121] In order to determine the movement of the vehicle 1000 relative to the surrounding environment, rather than other vehicles traveling parallel to it, the analysis unit 102, when extracting feature points, excludes feature points of moving surrounding objects and extracts feature points of fixedly installed surrounding objects. For example, the analysis unit 102 may extract feature points of traffic signs, traffic lights, and structures around the road (electric poles, streetlights, guardrails, etc.). The type of surrounding object can be identified by using the trained model described above.

[0122] For example, A in Fig. 15 shows image data in frame N, and B in Fig. 15 shows image data in frame N+1. In A and B in Fig. 15, areas C3 to C6 corresponding to fixedly installed surrounding objects move backward as the vehicle 1000 moves, but the position of area C1 of the moving truck remains almost unchanged, and the position of area C2 of the passenger car traveling in the oncoming lane toward the vehicle 1000 changes significantly.

[0123] By using SFM, the analysis unit 102 can calculate information indicating the relative position of the vehicle 1000 based on the first frame of the video data, such as, for example, that the position of the vehicle 1000 in the second frame has moved 2 m forward and 1 m to the left compared to the first frame, and that the position of the vehicle 1000 in the third frame has moved 2.5 m forward and 0.5 m to the left compared to the second frame.

[0124] Next, the analysis unit 102 selects the location information corresponding to the time when the first frame of the video data was captured from the location information indicating the absolute position of the vehicle 1000 acquired by the positioning unit 1130. The location information acquired by the positioning unit 1130 includes time information, and the video data in this embodiment also includes time information indicating the time when it was recorded. Therefore, the analysis unit 102 can select the GPS location information corresponding to the first frame of the video data by comparing the time included in the video data with the time included in the location information.

[0125] Next, the analysis unit 102 uses the GPS position information corresponding to the first frame of the video data to identify the absolute position of the vehicle 1000 in the second and subsequent frames of the video data. As described above, the analysis unit 102 calculates information indicating relative movement between frames. Furthermore, the vehicle data in this embodiment includes orientation data indicating the direction in which the front of the video is facing, and the orientation data makes it possible to determine which direction the front is pointing. Therefore, by using the selected GPS position information as the absolute position of the vehicle 1000 in the first frame, the analysis unit 102 can calculate the absolute position (latitude and longitude) of the vehicle 1000 corresponding to the relative position of the vehicle 1000 in the second frame (the relative position calculated by SFM). For example, if the latitude and longitude of the first frame acquired by the positioning unit 1130 are 134.45 degrees and 32.85 degrees, the latitude and longitude corresponding to a position 2 m forward in the north direction and 1 m westward from the latitude and longitude as the starting point will be the absolute position of the vehicle 1000 in the second frame. The analysis unit 102 repeats this process for each frame to calculate the absolute position (latitude and longitude) of the vehicle 1000 based on the SFM for all frames.

[0126] [Estimation of the absolute position of vehicle 1000] As shown in FIG. 16, the analysis unit 102 estimates the absolute position of the vehicle 1000 for each frame using the absolute position of the vehicle 1000 based on the SFM and the absolute position of the vehicle 1000 determined by GPS. It is assumed that points f11 to f16 shown on the left side of FIG. 16 are the absolute positions of the vehicle 1000 obtained by analyzing frames 1 to 6 of the video data, respectively. It is also assumed that points f21 to f26 shown in the center of FIG. 16 are the absolute positions of the vehicle 1000 at the times corresponding to frames 1 to 6 of the video data, among the absolute positions of the vehicle 1000 obtained by GPS. Points f31 to f36 shown on the right side of FIG. 16 indicate the estimated absolute positions of the vehicle 1000. As mentioned above, point f11, which indicates the absolute position of the vehicle 1000 corresponding to one frame of the video data, is the same as point f21.

[0127] The analysis unit 102 optimizes the absolute position of the vehicle 1000 based on the SFM (points f11 to f16) and the absolute position of the vehicle 1000 based on the GPS (points f21 to f26) so that there is no large discrepancy between them. For example, the absolute position of the vehicle 1000 may be estimated as an absolute position obtained by simply averaging the absolute position of the vehicle 1000 based on the SFM and the absolute position of the vehicle 1000 based on the GPS. For example, for a speed of 30 km / h or more, the sum of these values ​​may be divided by 2 to be used as the absolute position of the vehicle 1000. If the speed of the vehicle 1000 at points f13 and f23 is 30 km / h or more, the latitude of the absolute position of the vehicle 1000 (point f33) can be calculated by (latitude of point f13 + latitude of point f23) ÷ 2. Similarly, the longitude of the absolute position (point f33) of the vehicle 1000 can be calculated by (longitude of point f13+longitude of point f23) / 2.

[0128] Furthermore, the analysis unit 102 may determine the absolute position of the vehicle 1000 by averaging the absolute position of the vehicle 1000 based on the SFM and the absolute position of the vehicle 1000 based on the GPS with a predetermined weight depending on the vehicle speed of the vehicle 1000. For example, for a speed of less than 30 km / h (e.g., 5 km / h), the absolute position of the vehicle 1000 based on the SFM may be considered to be more accurate, and the absolute position of the vehicle 1000 may be determined by averaging the absolute position of the vehicle 1000 based on the SFM and the absolute position of the vehicle 1000 based on the GPS at a ratio of, for example, 4:1. If the vehicle speed of the vehicle 1000 at points f16 and f26 is 5 km or faster, the latitude of the absolute position of the vehicle 1000 (point f36) can be calculated by (latitude of point f16 × 4 + latitude of point f26 × 1) ÷ 5. Similarly, the longitude of the absolute position (point f36) of the vehicle 1000 can be calculated by (longitude of point f16×4+longitude of point f26×1)÷5.

[0129] The analysis unit 102 may estimate the absolute position of the vehicle 1000 for each frame using the absolute position of the vehicle 1000 based on the SFM that has been subjected to noise removal processing and the absolute position of the vehicle 1000 based on the GPS that has been subjected to noise removal processing. For example, a Kalman filter may be used for noise removal. The Kalman filter used to remove noise from the absolute position of the vehicle 1000 based on the SFM and the Kalman filter used to remove noise from the absolute position of the vehicle 1000 based on the GPS may be different Kalman filters. This makes it possible to further improve the accuracy of the estimated absolute position of the vehicle 1000.

[0130] [Estimate the absolute position of surrounding objects] The analysis unit 102 calculates the absolute position of the peripheral object for each frame based on the estimated absolute position of the vehicle 1000 for each frame and the relative positional relationship between the vehicle 1000 and the peripheral object for each frame. As described above, the relative positional relationship between the vehicle 1000 and the peripheral object is indicated by the distance between the vehicle 1000 and the peripheral object and the angle to the left or right at which the peripheral object is located, with the traveling direction of the vehicle 1000 (toward the center of the image) being set as the reference (0 degrees), as shown in B of FIG. 14. The analysis unit 102 determines the moving direction of the absolute position of the vehicle 1000 obtained by taking the difference between the absolute positions of the vehicle 1000 for each frame as the traveling direction of the vehicle 1000, and calculates the latitude and longitude corresponding to the relative position of the peripheral object based on the estimated traveling direction, thereby obtaining the absolute position (latitude, longitude) of the peripheral object.

[0131] (Generating images showing accident situations) The generation unit 103 generates an image showing the accident situation by mapping the absolute position of the vehicle 1000 and the absolute positions of the surrounding objects for each frame, which are estimated by the processing procedure described above, onto a road map. This processing corresponds to the processing procedure of step S21 in FIG. 6.

[0132] FIG. 17 is a diagram showing an example of an image generated by the accident analysis device 10. The Y-axis and X-axis in FIG. 17 correspond to latitude and longitude, respectively. In the roads shown in A to E of FIG. 17, the center line indicates a median strip. The two lanes on the right are lanes where traffic travels from bottom to top, and the two lanes on the left are lanes where traffic travels from top to bottom. It is assumed that A to E of FIG. 17 correspond to frame X, frame X+1, frame X+2, frame X+3, and frame X+4, respectively, but this is merely an example and is not limiting. The accident situation can be reproduced by mapping the absolute position of the vehicle 1000 and the absolute positions of surrounding objects for each frame on a road map. For example, at time A of FIG. 17, a passenger car V2 is approaching the vehicle 1000 from the opposite lane, and at time B of FIG. 17, the vehicle 1000 and the passenger car V2 are approaching each other and colliding. The display of whether or not there is a collision and the location of the collision may be based on the results of estimation performed by a processing procedure related to "estimating the collision location" described later.

[0133] 17C to 17E, passenger car V2 is not present. This is because passenger car V2 has moved behind vehicle 1000 and is no longer captured by the camera equipped in drive recorder 1100. If the camera equipped in drive recorder 1100 is capable of capturing images of the rear of the vehicle, it is possible to estimate the movement of passenger car V2 after the collision by analyzing video data captured behind vehicle 1000 and reflect this in an image showing the accident situation.

[0134] (Estimation of collision direction) The analysis unit 102 estimates the direction in which the vehicle 1000 collides with another vehicle, an obstacle, etc., based on the acceleration generated in the vehicle 1000 at the moment of the collision. This process corresponds to the processing procedure of step S19 in FIG. 6.

[0135] More specifically, the analysis unit 102 determines the direction of the collision based on the acceleration pattern generated in the vehicle 1000 at the moment of the collision. When a collision occurs, the vehicle 1000 experiences negative acceleration in the direction of the object of collision, and also experiences acceleration in the opposite direction to the collision direction due to the reaction of the collision. In other words, the vehicle 1000 generally experiences acceleration that reciprocates on an axis connecting the direction of the object of collision and a direction 180 degrees opposite to that direction.

[0136] Therefore, the analysis unit 102 sums up the output values ​​from the acceleration measurement unit 1160 for a predetermined period from the time of the collision for each direction, and estimates the direction with the largest sum as the collision direction. 18A, when the front direction of the vehicle 1000 is set to 0 degree, the acceleration directions are summed for eight directions: front (D1: 337.5 degrees to 22.5 degrees), right front (D2: 22.5 degrees to 67.5 degrees), right (D3 direction: 67.5 degrees to 112.5 degrees), right rear (D4: 112.5 degrees to 157.5 degrees), rear (D5: 157.5 degrees to 202.5 degrees), left rear (D6: 202.5 degrees to 247.5 degrees), left (D7: 247.5 degrees to 292.5 degrees), and left front (D8: 292.5 degrees to 337.5 degrees), and the direction with the largest sum is regarded as the collision direction. The acceleration measured by the acceleration measurement unit 1160 is acceleration according to the law of inertia. Therefore, in the event of a collision, a positive acceleration is measured in the direction of the collision.

[0137] The analysis unit 102 may regard the time when the acceleration measurement unit 1160 detects the greatest acceleration as the time of the collision. If the camera provided in the drive recorder 1100 is a drive recorder, the analysis unit 102 may regard the time of the collision recorded in the drive recorder as the time of the collision.

[0138] 18B, when vehicle 1000 collides with vehicle V at the right front, it is assumed that acceleration data is acquired from the acceleration sensor in chronological order, for example, at time 1 (45 degree direction, 3.0 G), time 2 (47 degree direction, 1.0 G), time 3 (225 degree direction, 2.5 G), time 4 (227 degree direction, 0.5 G), and time 5 (40 degree direction, 2.0 G). In other words, vehicle 1000 collides with vehicle V at time 1, causing it to suddenly decelerate, and then experiencing acceleration that reciprocates between the right front and left rear.

[0139] In this case, the total acceleration corresponding to the right front (D2) is 3.0 G + 1.0 G + 2.0 G = 6.0 G, and the total acceleration corresponding to the left rear (D6) is 2.5 G + 0.5 G = 3.0 G. Therefore, the analysis unit 102 estimates that the right front (D2) direction is the collision direction.

[0140] Furthermore, the analysis unit 102 may analyze the video data to estimate the direction in which the vehicle 1000 collided with the pedestrian. For example, in the process of detecting surrounding objects described above, if a pedestrian appears in an image larger than a predetermined size, it may be estimated that the vehicle 1000 and the pedestrian collided in the head-on (D1) direction. When the vehicle 1000 and the pedestrian collide, the acceleration detected by the acceleration sensor is smaller than when there is a collision with another vehicle, and when the vehicle 1000 and the pedestrian collide, the collision is almost always a head-on collision. Therefore, when detecting a collision between the vehicle 1000 and the pedestrian, it is preferable to analyze the video data.

[0141] (Output of information showing the accident situation) The analysis unit 102 outputs information indicating the circumstances of the accident caused by the vehicle 1000. This process corresponds to the processing procedure of step S22 in FIG. 6. As described above, the analysis unit 102 outputs each item shown in FIG. 12 as information indicating the circumstances of the accident. The processing procedure for identifying each item shown in FIG. 12 will be described in detail below. In the following description, "another vehicle" means the vehicle with which the vehicle 1000 collided.

[0142] "A: Object of contact (car, obstacle, etc.)" The analysis unit 102 identifies the object that has collided (come into contact) by the collision object identification process shown in FIG. 8. A specific example will be described using FIG. 19. In FIG. 19, it is assumed that the vehicle 1000 has collided with a surrounding object on the right front side. In the case of A in FIG. 19, the object closest to the right front direction (collision direction) of the vehicle 1000 is the passenger car V1. Therefore, the analysis unit 102 identifies that the vehicle 1000 has collided with the passenger car V1. Similarly, in the case of B in FIG. 19, the object closest to the right front direction (collision direction) of the vehicle 1000 is the passenger car V5. Therefore, the analysis unit 102 identifies that the vehicle 1000 has collided with the passenger car V5.

[0143] If the collision direction of the vehicle 1000 is a direction not captured by the camera provided in the drive recorder 1100 (i.e., if only the front of the vehicle 1000 is captured), it is not possible to estimate the absolute value of the surrounding objects in that direction, and therefore it is not possible to identify the object that has come into contact. In this case, the analysis unit 102 may determine that a certain vehicle has collided.

[0144] "B: Contact site" The analysis unit 102 identifies the contact area according to the estimated collision direction of the vehicle 1000. Specifically, if the collision direction is to the front (D1) in FIG. 18, the contact area is the front bumper shown in FIG. 20. If the collision direction is to the right front (D2), the contact area is the right front bumper shown in FIG. 20. If the collision direction is to the right (D3), the contact area is the right side surface shown in FIG. 20. If the collision direction is to the right rear (D4), the contact area is the right rear bumper shown in FIG. 20. If the collision direction is to the rear (D5), the contact area is the rear bumper shown in FIG. 20. If the collision direction is to the left rear (D6), the contact area is the left rear bumper shown in FIG. 20. If the collision direction is to the left (D7), the contact area is the left side surface shown in FIG. 20. If the collision direction is to the left front (D8), the contact area is the left front bumper shown in FIG. 20.

[0145] "C: Road type" The analysis unit 102 identifies the road type of the road on which the vehicle 1000 was traveling at the time of the collision by comparing the absolute position of the vehicle 1000 at the time of the collision with the map data. The road type identified by the analysis unit 102 may include information such as a straight road, a curve, the inside of an intersection, or the inside of a T-junction, in addition to information such as an ordinary road or an expressway.

[0146] "D: Signal color of your vehicle and other vehicles" The analysis unit 102 identifies the traffic light color using the traffic light color and arrow signal identified by image analysis of the video data for each frame. For example, if the road type of the road on which the vehicle 1000 was traveling at the time of the collision was an intersection, the analysis unit 102 identifies the traffic light color on the vehicle 1000's side (either green, yellow, or red) as the color of the traffic light in the image of the frame closest to the time of the collision and identified before the vehicle 1000 entered the intersection, among the traffic light colors identified by analyzing the video data. Whether the traffic light color is before the vehicle 1000 entered the intersection can be determined by comparing the absolute position of the vehicle 1000 with map data.

[0147] It is assumed that the color of the traffic light on the intersecting side at an intersection is not visible in the video data or is visible only for a very short time, making it difficult to determine. Therefore, the analysis unit 102 may be configured to determine that the color of the traffic light on the intersecting side is red when the traffic light on the vehicle 1000 side is green. If the other vehicle that collided with the vehicle 1000 was traveling on a road that intersects the intersection, the traffic light on the other vehicle side will be determined to be red (i.e., the other vehicle entered the intersection on a red light). Similarly, the analysis unit 102 may be configured to determine that the color of the traffic light on the intersecting side is green when the traffic light on the vehicle 1000 side is red. If the other vehicle that collided with the vehicle 1000 was traveling on a road that intersects the intersection, the traffic light on the other vehicle side will be determined to be green (i.e., the other vehicle entered the intersection on a green light).

[0148] "E: Positional relationship between pedestrians, crosswalks, and safety zones" If the object that has come into contact with the vehicle 1000 is a pedestrian, the analysis unit 102 identifies the positional relationship between the pedestrian and the crosswalk / safety zone using the absolute position of the pedestrian and the absolute position of the crosswalk (or safety zone) identified by image analysis of the video data for each frame. Note that the analysis unit 102 may acquire the absolute position of the crosswalk from map data.

[0149] The analysis unit 102 calculates the point at which a line connecting the absolute position of the vehicle 1000 and the absolute position of the pedestrian collided with, obtained by analyzing an image of a frame before the time when the vehicle 1000 collided with the pedestrian, collides with the side of the crosswalk or safety zone farthest from the vehicle 1000. Next, if the distance between the absolute position of the person and this point is within a predetermined distance (e.g., 7 m), the analysis unit 102 determines that the pedestrian involved in the accident was on a crosswalk or safety zone. Furthermore, if the distance between the absolute position of the person and this point exceeds the predetermined distance (e.g., 7 m), the analysis unit 102 determines that the pedestrian involved in the accident was not on a crosswalk or safety zone.

[0150] 21 is a diagram illustrating the process of identifying the positional relationship between a pedestrian and a crosswalk / safety zone. For example, in FIG. 21, the analysis unit 102 calculates a point P2 where a line L connecting the absolute position P1 of the vehicle 1000 (the absolute position of the center of the vehicle 1000) and the absolute position P3 of the pedestrian meets the side of the crosswalk farthest from the vehicle 1000, and if the distance between the pedestrian's absolute position P3 and point P2 is a predetermined distance, it is determined that the pedestrian R was on the crosswalk.

[0151] In the example of A in Figure 21, the pedestrian is on a crosswalk, but in the example of B in Figure 21, the pedestrian is not on a crosswalk. However, in both examples, the analysis unit 102 determines that the pedestrian is on a crosswalk if the distance between the pedestrian's absolute value P3 and point P2 is within a predetermined distance. This is because, in accident cases, it is not particularly important whether the pedestrian involved in the accident was definitely walking on a crosswalk, and it is often determined that the pedestrian was walking on a crosswalk even if they were slightly off the crosswalk.

[0152] "F: Trajectories of own and other vehicles at intersections" If the accident occurs near an intersection (if the absolute position of vehicle 1000 at the time of collision is within a predetermined range from the center position of the intersection), analysis unit 102 identifies the travel trajectory of vehicle 1000 by arranging the absolute position of vehicle 1000 for each frame on map data of the intersection. Analysis unit 102 also identifies the travel trajectory of other vehicles by arranging the absolute positions of other vehicles for each frame on map data of the intersection. More specifically, analysis unit 102 identifies whether vehicle 1000 and other vehicles went straight through the intersection, turned right, turned left, or made a quick right turn, based on the magnitude and direction of the curvature of the travel trajectories.

[0153] 22 is a diagram illustrating the traveling directions of the vehicle 1000 and the other vehicle. For example, the analysis unit 102 may determine that the vehicle 1000 or the other vehicle has traveled straight through the intersection if the curvature of the travel trajectory within a predetermined area including the intersection is less than a predetermined value, and may determine that the vehicle 1000 or the other vehicle has turned left or right at the intersection if the curvature of the travel trajectory within the predetermined area is equal to or greater than a predetermined value. The analysis unit 102 may also determine that the vehicle 1000 or the other vehicle has made a quick right turn if the travel trajectories of the host vehicle and the other vehicle making a right turn pass through the inside of the center of the intersection. The analysis unit 102 may also determine that the vehicle 1000 and the other vehicle have changed lanes if the curvature changes direction within a predetermined time (e.g., 2 seconds).

[0154] "G: Lane position on the highway" If the accident occurs on a highway (if the absolute position of the vehicle 1000 at the time of the collision is on the highway), the analysis unit 102 identifies whether the vehicle 1000 was traveling on a main lane or an overtaking lane by analyzing the video data frame by frame or by comparing the absolute position of the vehicle 1000 with map data. For example, if the vehicle 1000 is traveling in the rightmost lane, it is identified as traveling in the overtaking lane, and if the vehicle is traveling in a lane other than the rightmost lane, it is identified as traveling on a main lane.

[0155] "H: Priority at intersections" When an accident occurs near an intersection, the analysis unit 102 extracts from the map data the presence or absence of a stop sign at the intersection and the road width to determine whether the vehicle 1000 had priority to enter the intersection. More specifically, the analysis unit 102 identifies the vehicle traveling on the side of the roads intersecting the intersection that does not have a stop sign as having priority. Furthermore, if the difference in width between the roads intersecting the intersection is more than twice as large, the analysis unit 102 identifies the vehicle traveling on the wider road as having priority. Furthermore, if there is no stop sign and there is no difference in road width, the analysis unit 102 identifies the vehicle as not having priority.

[0156] "I: Stop sign, red light violation?" The analysis unit 102 determines whether the vehicle 1000 and the other vehicle passed a stop sign without stopping by comparing the travel trajectory of the vehicle 1000, obtained by arranging the absolute positions of the vehicle 1000 and the other vehicle for each frame, with the map data. Furthermore, when the vehicle 1000 or the other vehicle passes a stop sign without stopping, the analysis unit 102 determines that the vehicle 1000 or the other vehicle has violated a stop sign if the vehicle speed of the vehicle 1000 or the other vehicle at the time of passing the stop sign is equal to or greater than a predetermined speed (e.g., 5 km) in a predetermined section (e.g., 3 m before and after) before and after the stop sign. On the other hand, the analysis unit 102 determines that the vehicle 1000 or the other vehicle has not violated a stop sign if the vehicle speed of the vehicle 1000 or the other vehicle at the time of passing is less than the predetermined speed.

[0157] Furthermore, the analysis unit 102 detects whether the vehicle 1000 has passed through a traffic light by comparing the travel trajectory of the vehicle 1000, obtained by arranging the absolute positions of the vehicle 1000 for each frame, with the map data. If the vehicle 1000 has passed through a traffic light, the analysis unit 102 obtains the color of the traffic light by analyzing the image of the frame before the time the vehicle 1000 passed through the traffic light and that is the last frame in which the traffic light is captured. If the color of the traffic light is red, the analysis unit 102 determines that the vehicle 1000 ran a red light when passing through the traffic light.

[0158] "J: Speed ​​limit violation or not?" The analysis unit 102 identifies the speed limit set for the road on which the vehicle 1000 traveled by comparing the travel trajectory of the vehicle 1000, obtained by arranging the absolute position of the vehicle 1000 for each frame, with the map data. The analysis unit 102 also compares the vehicle speed data included in the vehicle data of the vehicle 1000, which is the vehicle speed at a timing a predetermined time before (for example, 5 seconds before) the time of the collision of the vehicle 1000, with the speed limit. If the vehicle speed exceeds the speed limit, the analysis unit 102 identifies that the vehicle 1000 was speeding. If the vehicle speed is equal to or less than the speed limit, the analysis unit 102 identifies that the vehicle 1000 was not speeding.

[0159] "K: Speed ​​before collision between own vehicle and other vehicle" The analysis unit 102 identifies the speed of the vehicle 1000 before the collision from the vehicle speed data included in the vehicle data of the vehicle 1000. The analysis unit 102 also calculates the vehicle speed of the other vehicle from the change in the absolute position of the other vehicle estimated for each frame of the video data. For example, if the distance traveled by the other vehicle per frame is 1 m and the frame rate of the video data is 15 frames per second, it can be determined that the other vehicle was traveling at 54 km / h (15 x 60 x 60 frames per hour).

[0160] "L: Are there any obstacles on the road?" The analysis unit 102 identifies the presence or absence of an obstacle on the road by performing image analysis on the video data for each frame. The analysis unit 102 may determine that an object that cannot be determined as a road is present on the lane on which the vehicle 1000 is traveling, and that the object is not a person, car, bicycle, or motorcycle, and that its absolute position does not move between frames, as an obstacle.

[0161] "M: Opening and closing the door of the other vehicle involved in the collision" The analysis unit 102 performs image analysis on the video data for each frame to identify whether the door of the other vehicle was open or closed.

[0162] "N: Whether or not the turn signal was activated before the collision" The analysis unit 102 determines whether or not the turn signal of the vehicle 1000 was operating before the collision based on the turn signal operation information included in the vehicle data of the vehicle 1000. The analysis unit 102 also determines whether or not the turn signal was operating before the collision by performing image analysis on the video data for each frame.

[0163] "Whether or not attention to the road ahead is detected" The analysis unit 102 determines whether or not the driver was inattentive to the road ahead based on information about the driver's line of sight (gaze direction) included in the vehicle data of the vehicle 1000. For example, if the driver's line of sight was not directed forward from a predetermined time before the accident occurred until the time the accident occurred, it is determined that the driver was inattentive to the road ahead.

[0164] "Whether or not drinking was detected" The analysis unit 102 determines whether drinking has been detected (drinking driving) based on information about the driver's breath (whether or not drinking has occurred) included in the vehicle data of the vehicle 1000. For example, if the alcohol concentration indicated by the information about the driver's breath exceeds a predetermined reference value, it is determined that drinking has been detected. Note that if there is a passenger, this item may be excluded because there is a possibility that the passenger's drinking may be detected. The presence or absence of a passenger may also be determined from video data inside the vehicle.

[0165] "Brake timing" The analysis unit 102 determines whether the braking timing is early, normal, or late (sudden braking) based on the brake operation information included in the vehicle data of the vehicle 1000. For example, by comparing the brake operation timing from the time of the accident with a predetermined reference value, it can determine whether the braking timing is early, normal, or late (sudden braking).

[0166] "Steering operation during impact" The analysis unit 102 determines whether or not the steering wheel has been operated or whether or not the steering wheel has been turned suddenly, based on steering wheel operation information included in the vehicle data of the vehicle 1000. For example, if the steering wheel operation speed immediately before the accident is faster than a predetermined reference value, it may be determined that the steering wheel has been turned suddenly.

[0167] The items of information indicating the accident situation listed here are only examples, and other information may be included.

[0168] (Search for corresponding accident cases) The analysis unit 102 searches for and outputs an accident case corresponding to the analyzed accident situation. This processing corresponds to the processing procedure of step S23 in FIG. 6. The analysis unit 102 searches for an accident case and fault ratio corresponding to the situation of the accident caused by the vehicle 1000 by comparing with an accident case DB (accident case data) that associates the accident situation with past accident cases. FIG. 23 is a diagram showing an example of the accident case DB. In step S23, a screen of the post-incident case DB as shown in FIG. 23 is output. As shown in FIG. 23, the accident case includes "search conditions" for searching for the accident case, "accident details" that indicate the details of the accident, "fault ratio (basic)" that indicates the basic fault ratio, and "correction factors" that are conditions that require correction of the fault ratio.

[0169] In the example of Figure 23, if the object that vehicle 1000 collides with is a car (corresponding to "passenger car or truck" in item A), the accident occurs in an intersection where vehicle 1000 collides with a car (corresponding to "in the intersection" in item C), the accident occurs between a straight-moving vehicle and a right-turning vehicle (corresponding to item F where vehicle 1000 is going straight and the other vehicle is turning right, or the other vehicle is going straight and vehicle 1000 is turning right), and the light is red for the straight-moving vehicle and yellow for the right-turning vehicle (corresponding to item D where vehicle 1000 is red and the other vehicle is yellow, or vehicle 1000 is yellow and the other vehicle is red), it is shown that this corresponds to accident example 1 and the fault ratio between the straight-moving vehicle and the right-turning vehicle is 70:30.

[0170] In addition, as a correction factor, if the vehicle going straight violates the speed limit by 15 km or more (speed limit by 15 km or more in items J and K), the fault ratio between the vehicle going straight and the vehicle turning right will be 75:25. Similarly, if a vehicle going straight violates the speed limit by 30 km / h or more (speeding limit of 30 km / h or more in items J and K), the fault ratio between the vehicle going straight and the vehicle turning right is 80:20. Also, if a vehicle turning right turns right without using its turn signal (no turn signal in item N), the fault ratio between the vehicle going straight and the vehicle turning right is 60:40.

[0171] In addition, the correction factors may also include correction factors that are difficult for the accident analysis device 10 to determine, such as ``other significant negligence'' (e.g., inattention to the road, drunk driving, sudden steering, sudden braking, etc.) among the fault ratios.

[0172] The search unit 104 searches for accident cases that match the identified content by comparing the content identified for each item of information indicating the accident situation with the accident case DB. The search unit 104 also obtains the fault ratio corresponding to the searched accident case from the accident case DB. The search unit 104 also searches for the presence or absence of a corresponding correction element by comparing the correction element of the searched accident case with each item indicating the accident situation. If a corresponding correction element is present, the search unit 104 corrects the fault ratio according to the correction ratio of the corresponding correction element.

[0173] Furthermore, the output unit 105 outputs the accident details and the fault ratio of the accident case searched by the search unit 104 to the terminal 20. Furthermore, if the correction elements include correction elements that are difficult to determine by the accident analysis device 10, the output unit 105 may be configured to output the number of the accident case, the determined fault ratio, and the wording of the correction element that is difficult to determine to the terminal 20.

[0174] When searching the accident case DB, the search unit 104 may search for accident cases that meet some of the search conditions in addition to accident cases that meet all of the search conditions. When searching for accident cases that meet some of the search conditions, the search unit 104 may rank the accident cases according to the number of mismatched items, and the output unit 105 may output the accident details in order of rank.

[0175] Furthermore, the accident cases may be further ranked in descending order of the number of occurrences, and the search unit 104 may search the accident case DB in descending order of rank.

[0176] The above-described embodiments are intended to facilitate understanding of the present invention and are not intended to limit the present invention. The flowcharts, sequences, elements included in the embodiments, and their arrangements, materials, conditions, shapes, sizes, etc., described in the embodiments are not limited to those illustrated and can be modified as appropriate. Furthermore, configurations shown in different embodiments can be partially substituted or combined with each other. [Explanation of symbols]

[0177] 1...accident analysis system, 10...accident analysis device, 11...processor, 12...storage device, 13...communication IF, 14...input device, 15...output device, 20...terminal, 100...storage unit, 101...acquisition unit, 102...analysis unit, 103...generation unit, 104...search unit, 105...output unit, 1000...vehicle, 1100...drive recorder, 1110...first communication unit, 1120...second communication unit, 1130...positioning unit, 1140...recording unit, 1150...sound recording unit, 1160...acceleration measurement unit, 1170...automatic diagnosis data acquisition unit, 1180...control unit, 1190...sensor unit, 1200...data storage unit, 1300...mirror, 1400...automatic diagnosis system

Claims

1. an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and the acquired video data, The analysis unit If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs; A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed. An accident analysis device characterized by:

2. An accident analysis method using an accident analysis device, an acquisition step of acquiring vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis step of analyzing the circumstances of the accident caused by the vehicle based on the acquired vehicle data and the acquired video data, In the analyzing step, If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs; A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed. An accident analysis method characterized by:

3. Computer, an acquisition unit that acquires vehicle data measured by a sensor equipped in the vehicle and video data captured by a camera equipped in the vehicle; an analysis unit that analyzes the circumstances of an accident caused by the vehicle based on the acquired vehicle data and video data; The analysis unit If there is a timing when the volume of the audio data included in the video data is higher than a predetermined judgment value, the timing is judged to be the timing when an accident occurs; A frequency corresponding to a human scream is removed from the audio data included in the video data, and the timing of the occurrence of an accident is determined based on the audio data from which the frequency corresponding to a human scream has been removed. A program characterized by:

Images