Near miss determination system, near miss analysis system, near miss notification system, and near miss determination method

Abstract

To make it possible to highly precisely determine a near miss even in a situation where an ambient moving body cannot be detected because the moving body cannot be seen unobstructedly from a moving body terminal or because of a tentative occlusion, and to properly determine a near miss to which plural moving bodies are related.SOLUTION: A near miss determination system that determines a near miss to which a moving body (vehicle, pedestrian, or bicycle) on a road is related includes an onboard terminal 1 installed in a vehicle, a roadside machine 2 installed on or near the road, a sensor 21 incorporated in the roadside machine, and a server 3 that determines a near miss on the basis of detection information of the sensor. The roadside machine 2 detects a moving body, which exists on a road around the roadside machine, on the basis of a result of detection by the sensor, and acquires moving target information relating to the moving body. The server 3 determines an abnormal condition on the basis of the moving target information, and determines a near miss on the basis of a result of the determination.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a near-miss determination system that determines near-misses involving moving objects on a road, a near-miss analysis system that analyzes the causes of near-misses, a near-miss notification system that notifies people on the road of information regarding the occurrence of past near-misses, and a near-miss determination method in which processing related to the determination of near-misses is executed by a processor. [Background technology]

[0002] In recent years, from the perspective of traffic accident prevention, attention has been focused on near misses, which are events in which an abnormal condition with a high risk of an accident occurred but did not result in an accident. Collecting such near miss cases can provide knowledge for preventing the recurrence of near misses and reducing accidents. Therefore, a technology that can accurately and efficiently identify near misses is desired so that a large number of near miss cases can be collected.

[0003] As a technology for determining such near misses, a technology for determining near misses based on information acquired through vehicle-to-vehicle communication is known (see Patent Document 1). Also, a technology for determining near misses based on information acquired as probe information is known (see Patent Document 2). Also, a technology for determining near misses based on information recorded in a drive recorder is known (see Patent Document 3). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-113275 [Patent Document 2] Japanese Patent Publication No. 2022-173340 [Patent Document 3] Japanese Patent Application Laid-Open No. 2012-164131 Summary of the Invention [Problem to be solved by the invention]

[0005] A sensor installed on a moving object such as a vehicle may not be able to detect other moving objects in the vicinity due to being out of the line of sight or due to temporary occlusion. Also, when a near miss is determined based on a sensor installed on a moving object, in a situation where there is a high risk of a collision between multiple moving objects, it may be determined that only some of the moving objects have experienced a near miss.

[0006] Therefore, the main object of the present invention is to provide a near-miss determination system, near-miss analysis system, near-miss notification system, and near-miss determination method that can accurately determine near-misses even in situations where a mobile terminal is out of the line of sight or cannot detect nearby moving objects due to temporary occlusion, and that can appropriately determine near-misses involving multiple moving objects. [Means for solving the problem]

[0007] The near-miss detection system of the present invention is a near-miss detection system that detects near-misses involving moving objects on a road, and is equipped with a mobile terminal carried by the moving object, a roadside unit installed on a road or in its vicinity, a sensor provided in the roadside unit, and one or more processors that execute processing related to determining the near-miss based on the detection results of the sensor, wherein the processor detects moving objects present on the road around the roadside unit based on the detection results of the sensor, obtains moving target information related to the moving object, determines an abnormal condition based on the moving target information, and determines the near-miss based on the determination results.

[0008] In addition, the near-miss analysis system of the present invention is a near-miss analysis system that analyzes the factors of near-misses involving moving objects on a road, and is equipped with a mobile terminal carried by the moving object, a roadside unit installed on or near a road, a sensor provided in the roadside unit, and one or more processors that perform processing related to determining the near-miss based on detection information from the sensor, wherein the processor detects moving objects present on the road around the roadside unit based on the detection results of the sensor, acquires moving target information related to the moving object, determines an abnormal state based on the moving target information, determines the near-miss based on the determination result, stores the near-miss determination result in a memory unit, and analyzes the factors that caused the near-miss to occur based on the near-miss determination result stored in the memory unit.

[0009] In addition, the near-miss notification system of the present invention is a near-miss notification system that notifies people on the road of information regarding the occurrence of past near-misses, and is equipped with a mobile terminal carried by a mobile body, a roadside unit installed on or near a road, a sensor provided in the roadside unit, and one or more processors that perform processing related to determining the near-miss based on the detection information of the sensor, wherein the processor detects mobile bodies present on the road around the roadside unit based on the detection results of the sensor, acquires moving target information related to the mobile body, determines an abnormal state based on the moving target information, determines the near-miss based on the determination result, stores the near-miss determination result in a memory unit, generates near-miss guidance information suitable for the mobile terminal to which it is to be distributed based on the near-miss determination result stored in the memory unit, and distributes the near-miss guidance information to the mobile terminal.

[0010] In addition, the near-miss determination method of the present invention is a near-miss determination method in which processing related to determining near-misses involving moving objects on a road is executed by one or more processors, and is configured to detect moving objects present on the road around a roadside unit based on the detection results of a sensor installed in a roadside unit installed on or near the road, obtain moving target information related to the moving object, determine an abnormal state based on the moving target information, and determine the near-miss based on the determination results. [Effects of the Invention]

[0011] According to the present invention, even in situations where a nearby moving object cannot be detected due to a lack of line of sight from the mobile terminal or due to temporary occlusion, the sensor installed in the roadside device can properly detect the nearby moving object. Therefore, near misses can be accurately determined based on target object information collected using the sensor in the roadside device. [Brief explanation of the drawings]

[0012] [Figure 1] Overall configuration diagram of the near-miss incident determination system according to the first embodiment [Figure 2] FIG. 1 is a block diagram showing the schematic configuration of an in-vehicle terminal, a roadside device, and a server according to a first embodiment. [Figure 3] FIG. 1 is a block diagram showing an overview of processing performed by an in-vehicle terminal, a roadside device, and a server according to a first embodiment; [Figure 4] FIG. 1 is a flowchart showing the procedure of processing performed by the in-vehicle terminal, roadside device, and server according to the first embodiment. [Figure 5] FIG. 10 is a block diagram showing an overview of processing performed by a roadside device and a server according to a first modification of the first embodiment. [Figure 6] FIG. 10 is a block diagram showing an overview of processing performed by an in-vehicle terminal, a roadside device, and a server according to a second modification of the first embodiment. [Figure 7] A block diagram showing an overview of the processes performed by the in-vehicle terminal, the roadside device, and the server according to the second embodiment. [Figure 8]FIG. 10 is a flowchart showing the procedure of processing performed by the in-vehicle terminal, roadside device, and server according to the second embodiment. [Figure 9] Overall configuration diagram of a near-miss incident determination system according to the third embodiment [Figure 10] FIG. 10 is a block diagram showing an overview of the processes performed by the in-vehicle terminal, the roadside device, and the server according to the third embodiment. [Figure 11] FIG. 10 is a flowchart showing the procedure of processing performed by the in-vehicle terminal, roadside device, and server according to the third embodiment. [Figure 12] FIG. 10 is a block diagram showing an outline of processing performed by an in-vehicle terminal, a roadside device, and a server according to a modification of the third embodiment. [Figure 13] FIG. 10 is a flowchart showing the procedure of processing performed by the in-vehicle terminal, the roadside device, and the server according to a modification of the third embodiment. [Figure 14] FIG. 10 is a block diagram showing an overview of the processes performed by the in-vehicle terminal, the roadside device, and the server according to the fourth embodiment. [Figure 15] FIG. 10 is a flowchart showing the procedure of processing performed by the in-vehicle terminal, the roadside device, and the server according to the fourth embodiment. [Figure 16] FIG. 13 is a block diagram showing an overview of the processes performed by the in-vehicle terminal, the roadside device, and the server according to the fifth embodiment. [Figure 17] FIG. 10 is a flowchart showing the procedure of processing performed by the in-vehicle terminal, the roadside device, and the server according to the fifth embodiment. [Figure 18] Overall configuration diagram of a near-miss notification system according to the sixth embodiment [Figure 19] FIG. 13 is a flowchart showing the procedure of processing performed by the in-vehicle terminal and the server according to the sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] The first invention made to solve the above problem is a near-miss detection system that determines near-misses involving moving objects on a road, and includes a mobile terminal carried by the moving object, a roadside unit installed on or near the road, a sensor provided in the roadside unit, and one or more processors that execute processing related to determining the near-miss based on the detection results of the sensor, wherein the processor detects moving objects present on the road around the roadside unit based on the detection results of the sensor, obtains moving object information related to the moving object, determines an abnormal condition based on the moving object information, and determines the near-miss based on the determination results.

[0014] This means that even in situations where a nearby moving object cannot be detected due to line-of-sight or temporary occlusion from the mobile terminal, the sensor installed in the roadside unit can properly detect the nearby moving object. Therefore, near-misses can be accurately determined based on target information collected using the sensor in the roadside unit. Note that moving objects include vehicles, pedestrians, bicycles, etc., and mobile terminals include in-vehicle terminals installed in vehicles, pedestrian terminals carried by pedestrians, bicycle terminals installed on bicycles, etc.

[0015] In addition, the second invention is configured such that the processor detects multiple moving objects present on the road around the roadside unit based on the detection results of the sensor, acquires the moving target information related to the multiple moving objects, and determines the near misses involving the multiple moving objects based on the moving target information.

[0016] This means that the sensors installed in the roadside units can simultaneously detect multiple moving objects that have experienced an abnormal condition due to a common cause, making it possible to appropriately determine near misses involving multiple moving objects.

[0017] Furthermore, a third invention is configured to further include a sensor provided on the moving body, and the processor detects another moving body on a road around the moving body based on a detection result of the sensor provided on the moving body, acquires the moving target information related to the other moving body, and integrates the moving target information based on the detection result of the sensor provided on the roadside device and the moving target information based on the detection result of the sensor provided on the moving body.

[0018] This allows for accurate determination of near misses based on highly accurate moving target information. In this case, the targets included in the moving target information based on the detection results of the sensor installed in the roadside device and the moving target information based on the detection results of the sensor installed in the moving object are associated as the same moving object (target identification process), and the target information is integrated.

[0019] In addition, a fourth invention is configured such that the processor acquires location information of the moving body acquired by a positioning unit in the mobile terminal, and integrates the location information of the moving body as moving target information with the moving target information based on the detection result of the sensor provided in the roadside device.

[0020] This allows for accurate determination of near misses based on highly accurate moving target information. In this case, the target included in the moving target information based on the detection results of the sensor installed in the roadside device and the moving object that transmitted the location information are associated as the same moving object (target identification process), and the target information can be integrated.

[0021] In addition, a fifth invention is configured such that the processor detects stationary objects on and around the road based on the detection results of the sensor provided in the roadside device, obtains stationary target information regarding the stationary objects, and determines the abnormal state based on the stationary target information and the moving target information.

[0022] This makes it possible to accurately determine an abnormal state based on not only moving objects on the road but also stationary objects on and around the road.

[0023] In addition, a sixth invention is configured such that the processor determines that the abnormal state is at least one of a state in which there is a high risk of collision between moving bodies, a state in which there is a high risk of collision between a moving body and a stationary object, a state in which the trajectory of a moving body is abnormal, a state in which an object blocks a person's view, and a state in which the person's body is abnormal.

[0024] This allows for accurate detection of abnormal conditions that may cause near misses.

[0025] In addition, the seventh invention is configured such that the processor determines, based on the moving target information, whether or not the target person has exhibited a behavioral change in response to an abnormal condition, and if the abnormal condition exists and the behavioral change exists, determines that the near miss has occurred.

[0026] This allows for accurate determination of near misses based on behavioral changes that occur when a person encounters an abnormal situation. In this case, behavioral changes in response to an abnormal situation include sudden driving operations, such as sudden braking, sudden steering, sudden acceleration, etc. Furthermore, sudden driving operations by the driver can be determined based on changes in the vehicle state that can be determined based on target information, such as sudden deceleration, sudden stopping, sudden acceleration, and sudden lane changes of the vehicle.

[0027] In addition, the eighth invention is configured such that, when there is a behavioral change, the processor determines whether or not the abnormal condition has occurred, and when the abnormal condition has occurred, determines that the near miss has occurred.

[0028] According to this, an abnormal state is determined only when a behavioral change occurs, and therefore, the abnormal state determination is performed to a minimum, thereby reducing the processing load.

[0029] In addition, the ninth invention is configured such that the processor determines whether the target person has a physical reaction to an abnormal condition based on the detection results of a sensor that detects the physical condition of the target person, and if the abnormal condition exists and the physical reaction is present, it determines that the near miss has occurred.

[0030] This makes it possible to accurately determine near misses based on the physical reactions that occur when a person encounters an abnormal situation. In this case, physical reactions to an abnormal situation include, for example, a sudden head movement, a sudden shift in gaze, sweating, vocalization (e.g., shouting "It's dangerous"), dilation of the pupils, etc.

[0031] In addition, the 10th invention is further configured to include a sensor provided on the moving body, and the processor determines whether the target person has a physical reaction to an abnormal condition based on the detection result of the sensor provided on the moving body.

[0032] This allows for accurate determination of the physical reaction of the target person to an abnormal condition. In this case, if the mobile object is a vehicle, the sensor provided on the mobile object may be a camera provided on the vehicle for driver monitoring to capture an image of the driver. The sensor may also be a biosensor that detects the physical condition of the person.

[0033] In addition, an eleventh invention is configured such that the sensor provided in the roadside unit includes a camera capable of zooming in to photograph a person on the road, and the processor determines whether or not the person in question is showing a physical reaction to an abnormal condition based on the zoomed image of the camera.

[0034] This makes it possible to determine the physical reaction to an abnormal condition even if a moving object (vehicle, pedestrian, bicycle) does not have a sensor suitable for detecting the physical condition of a person (driver, pedestrian) on the road.

[0035] In addition, a twelfth invention is a near-miss analysis system that analyzes the causes of near-misses involving moving objects on a road, comprising: a mobile terminal carried by the moving object; a roadside unit installed on or near a road; a sensor provided in the roadside unit; and one or more processors that execute processing related to determining the near-miss based on detection information from the sensor, wherein the processor detects moving objects present on the road around the roadside unit based on the detection results of the sensor, acquires moving target information related to the moving object, determines an abnormal condition based on the moving target information, determines the near-miss based on the determination result, stores the near-miss determination result and collected information related to the near-miss in a memory unit, and analyzes the causes of the near-miss based on the near-miss determination result and the collected information stored in the memory unit.

[0036] According to this, the results of the near miss factor analysis provide knowledge for preventing the recurrence of near misses and reducing accidents, and this knowledge is utilized for driving assistance to prevent accidents before they occur and road maintenance to reduce accidents. In this case, for example, the near miss analysis results may be transmitted to a viewing terminal and presented to the user. The collected information related to near misses includes, for example, target information, physical reaction assessment results, and behavioral change assessment results.

[0037] In addition, a thirteenth invention is a near-miss notification system that notifies people on the road of information regarding the occurrence of past near-misses, and comprises a mobile terminal carried by a mobile body, a roadside unit installed on or near a road, a sensor provided in the roadside unit, and one or more processors that execute processing related to determining the near-miss based on detection information from the sensor, wherein the processor detects mobile bodies present on the road around the roadside unit based on the detection results of the sensor, acquires moving target information related to the mobile body, determines an abnormal state based on the moving target information, determines the near-miss based on the determination result, stores the near-miss determination result in a memory unit, generates near-miss guidance information suitable for the mobile terminal to which it is to be distributed based on the near-miss determination result stored in the memory unit, and distributes the near-miss guidance information to the mobile terminal.

[0038] This allows people on the road (such as vehicle drivers, pedestrians, and cyclists) to be alerted to near misses. In this case, for example, information on locations where near misses frequently occur may be distributed to a mobile terminal as near miss guidance information. The near miss guidance information may also be displayed on a display device provided on the mobile terminal. The near miss guidance information may also be distributed to a mobile terminal only if the current conditions (such as the time of day or road conditions) match.

[0039] In addition, a fourteenth invention is a near-miss determination method in which processing related to determining a near-miss involving a moving object on a road is executed by one or more processors, and is configured to detect a moving object on the road around a roadside unit based on the detection results of a sensor installed in the roadside unit installed on or near the road, obtain moving target information related to the moving object, determine an abnormal state based on the moving target information, and determine the near-miss based on the determination result.

[0040] According to this, as with the first invention, even in situations where a nearby moving object cannot be detected from the mobile terminal due to out-of-sight or temporary occlusion, the sensor installed in the roadside device can properly detect the nearby moving object. Therefore, near misses can be accurately determined based on target object information collected using the sensor in the roadside device.

[0041] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0042] (First embodiment) FIG. 1 is a diagram showing the overall configuration of a near-miss incident determination system according to the first embodiment.

[0043] The near-miss incident detection system detects near-miss incidents involving moving objects such as vehicles, pedestrians, and bicycles on the road. The near-miss incident detection system comprises an in-vehicle terminal 1 (mobile terminal), a roadside device 2, a server 3, and a viewing terminal 4. The in-vehicle terminal 1 and the roadside device 2 can communicate using a communication method specified by ITS (Intelligent Transport Systems). The roadside device 2 and the server 3 can communicate via a network.

[0044] The vehicle is equipped with a driving assistance system 5. The driving assistance system 5 includes a sensor 51 and an ECU 52. The ECU 52 controls a steering ECU, a drive ECU, and a braking ECU, which are not shown. The driving assistance system 5 may be, for example, an autonomous driving (AD) system or an advanced driver assistance system (ADAS). The sensor 51 is, for example, a camera, radar, or lidar. The driving assistance system 5 may be provided with a plurality of sensors 51 of the same type or different types.

[0045] The in-vehicle terminal 1 is mounted on a vehicle. The in-vehicle terminal 1 uses a sensor 51 of the driving assistance system 5 to detect moving objects present on the road around the in-vehicle terminal 1, and generates target information (moving target information) related to the moving objects. The target information is added to a message of ITS communication (road-to-vehicle communication) and transmitted from the in-vehicle terminal 1 to the roadside unit 2. Furthermore, in an in-vehicle terminal 1 mounted on a vehicle that does not have the driving assistance system 5, detection of moving objects present on the road around the in-vehicle terminal 1 is not performed, and the position information of the in-vehicle terminal 1 is transmitted to the roadside unit 2 as target information.

[0046] The roadside device 2 is installed on a road or in its vicinity. The roadside device 2 includes a sensor 21. The roadside device 2 uses the sensor 21 to detect moving objects present on the road around the device and generate target information (moving target information) related to the moving objects. This allows the roadside device 2 to acquire target information including, as targets, moving objects (vehicles, pedestrians, bicycles, etc.) that do not possess a mobile terminal (in-vehicle terminal 1, pedestrian terminal 8, bicycle terminal 9, etc.). In the example shown in FIG. 1, there is a pedestrian that does not possess a pedestrian terminal 8. The sensor 21 is, for example, a camera, radar, lidar, etc. The roadside device 2 may be provided with a plurality of sensors 21 of the same type or different types.

[0047] Furthermore, the roadside device 2 integrates the two pieces of target information by associating the targets included in the target information acquired by itself and the target information acquired from the in-vehicle terminal 1 as the same moving object. The integrated target information is transmitted from the roadside device 2 to the server 3.

[0048] The server 3 uses the target information collected by the in-vehicle terminal 1 and the roadside device 2 to determine an abnormal state of the vehicle, i.e., a state in which an accident is likely to occur (abnormal state), and determines whether the driver of the vehicle has experienced a near-miss based on the determination result. The server 3 also registers and manages the near-miss determination result in a near-miss database.

[0049] The viewing terminal 4 accesses the server 3 to display the near-miss judgment results distributed from the server 3. This allows the administrator to view the near-miss judgment results.

[0050] There are two types of near misses. The first type of near miss occurs when an abnormal condition occurs in the vehicle that poses a high risk of an accident, but does not result in an accident. The second type of near miss occurs when an abnormal condition occurs in the vehicle that poses a high risk of an accident, causing the driver to change their behavior, but does not result in an accident.

[0051] The first type of near miss does not involve a change in the behavior of the vehicle driver. For example, if the driver does not recognize the abnormal condition, or if the driver recognizes the abnormal condition but the other vehicle takes evasive action before the driver can change his / her behavior, or if the incident is too sudden and the driver is unable to react or change his / her behavior, the vehicle driver will not change his / her behavior. Even in such cases, the present embodiment determines the near miss to be the first type.

[0052] When an evaluation is made for each near miss case, the second type may be evaluated higher than the first type. For example, when a score for a near miss is assigned for each location where the near miss occurred, the second type may be evaluated higher than the first type.

[0053] In the example shown in FIG. 1, a pedestrian carries a pedestrian terminal 8 (mobile terminal). A bicycle terminal 9 (mobile terminal) is mounted on a bicycle. The pedestrian terminal 8 and bicycle terminal 9 have the same configuration as an in-vehicle terminal 1 mounted on a vehicle (automobile). That is, similar to the ITS communication (V2I communication) between the in-vehicle terminal 1 and the roadside unit 2, ITS communication (P2I communication) is performed between the pedestrian terminal 8 and the roadside unit 2, and ITS communication (B2I communication) is performed between the bicycle terminal 9 and the roadside unit 2.

[0054] In this case, pedestrians and bicycle drivers are the targets of near-miss detection. That is, the pedestrian terminal 8 operates in the same way as the in-vehicle terminal 1 to detect near-misses related to pedestrians. The bicycle terminal 9 operates in the same way as the in-vehicle terminal 1 to detect near-misses related to bicycle drivers. The following mainly describes the case where the target of near-miss detection is a vehicle (automobile) driver.

[0055] Next, a description will be given of the schematic configuration of the in-vehicle terminal 1, roadside device 2, and server 3 according to the first embodiment. Fig. 2 is a block diagram showing the schematic configuration of the in-vehicle terminal 1, roadside device 2, and server 3. Fig. 3 is a block diagram showing an overview of the processing performed by the in-vehicle terminal 1, roadside device 2, and server 3.

[0056] As shown in FIG. 2, the in-vehicle terminal 1 includes a positioning unit 11, a wireless communication unit 12, an input / output unit 13, a storage unit 14, and a processor 15.

[0057] The positioning unit 11 uses a satellite positioning system such as GPS to detect the current position of the device itself and acquires position information of the device itself.

[0058] The wireless communication unit 12 performs ITS communication (road-to-vehicle communication) with the roadside device 2.

[0059] The input / output unit 13 inputs and outputs information to and from the ECU 52 of the driving assistance system 5. Specifically, detection data from the sensor 51 of the driving assistance system 5 is input to the input / output unit 13 via the ECU 52.

[0060] The storage unit 14 stores programs to be executed by the processor 15 and the like.

[0061] The processor 15 performs various processes by executing programs stored in the storage unit 14. As shown in Fig. 3, in this embodiment, the processor 15 performs a moving body detection process and the like.

[0062] In the moving object detection process, the processor 15 detects moving objects (other vehicles, pedestrians, etc.) present on the road around the device based on the detection results of the sensor 51 of the driving assistance system 5, and generates target information related to the moving objects. The target information is added to an ITS communication message and transmitted to the roadside device 2. Note that the moving object detection process may not be performed, and instead the position information acquired by the positioning unit 11 may be transmitted to the roadside device 2 as target information.

[0063] As shown in FIG. 2, the roadside device 2 includes a sensor 21, a wireless communication unit 22, a network communication unit 23, a storage unit 24, and a processor 25.

[0064] The sensor 21 detects objects present around the roadside unit 2 .

[0065] The wireless communication unit 22 performs ITS communication (road-to-vehicle communication) with the in-vehicle terminal 1.

[0066] The network communication unit 23 communicates with the server 3 via a network such as the Internet.

[0067] The storage unit 24 stores programs executed by the processor 25 and the like.

[0068] The processor 25 performs various processes by executing programs stored in the storage unit 24. As shown in Fig. 3, in this embodiment, the processor 25 performs a moving object detection process, a target information integration process, and the like.

[0069] In the moving object detection process, the processor 25 detects moving objects that exist on the roads around the own device based on the detection results of the sensor 21 of the own device, and generates target information related to the moving objects.

[0070] In the target information integration process, the processor 25 associates the targets contained in the target information acquired by its own device, i.e., target information relating to moving objects existing on the roads around the roadside unit 2, and the target information acquired from the in-vehicle terminal 1, i.e., target information relating to moving objects existing on the roads around the vehicle, as the same moving object (target identification process), and integrates the two pieces of target information.

[0071] At this time, the vehicle position information received from the in-vehicle terminal 1 via an ITS communication message may also be treated as target information, and the vehicle position information may be integrated with target information relating to moving objects present on the roads around the vehicle and target information relating to moving objects present on the roads around the road-side unit 2. Furthermore, the in-vehicle terminal 1 may not perform a process of acquiring target information relating to moving objects present on the roads around the vehicle (moving object detection process), and may instead integrate the vehicle position information with target information relating to moving objects present on the roads around the road-side unit 2 in the target information integration process.

[0072] In addition, the roadside unit 2 may receive location information transmitted from a mobile terminal (vehicle terminal 1, pedestrian terminal 8, bicycle terminal 9) and identify the mobile unit that transmitted the location information with the mobile unit detected using the sensor 21 of the device itself.

[0073] As shown in FIG. 2, the server 3 includes a network communication unit 31, a storage unit 32, and a processor 33.

[0074] The network communication unit 31 communicates with the roadside device 2 via a network such as the Internet.

[0075] The storage unit 32 stores programs and the like to be executed by the processor 33. The storage unit 32 also stores target object information collected from the in-vehicle terminal 1 and the roadside unit 2. The storage unit 32 also stores near-miss determination results acquired by the processor 33. The processor 33 performs abnormality determination processing, behavior change determination processing, and near-miss determination processing at appropriate times using the target object information accumulated in the storage unit 32. At this time, the near-miss determination results are registered in a database.

[0076] The processor 33 performs various processes by executing programs stored in the storage unit 32. As shown in Fig. 3, in this embodiment, the processor 33 performs an abnormality determination process, a behavioral change determination process, a near-miss determination process, and the like.

[0077] In the abnormality determination process, the processor 33 determines whether an abnormal state has occurred in the vehicle based on the target object information stored in the storage unit 32. At this time, the processor 33 can determine the abnormal state of the moving object based on the position, speed, etc. of the moving object obtained from the target object information. The abnormality determination process may be performed using a determination model constructed by machine learning such as deep learning.

[0078] In the behavior change determination process, the processor 33 determines whether or not there is a behavior change of the driver in response to an abnormal state, based on the target information stored in the storage unit 32. The processor 33 can determine the behavior change of the driver based on a state change (behavior) of the moving object obtained from the target information. The behavior change determination process may be performed using a determination model constructed by machine learning such as deep learning.

[0079] In the near-miss determination process, the processor 33 determines whether or not a near-miss has occurred based on the determination result of the abnormality determination process and the determination result of the behavioral change determination process.

[0080] In this embodiment, the target information integration process for integrating the target information collected by the in-vehicle terminal 1 and the target information collected by the roadside unit 2 is performed by the roadside unit 2, but the target information integration process may also be performed by the server 3.

[0081] Next, a description will be given of the processing procedures performed by the in-vehicle terminal 1, the roadside device 2, and the server 3 according to the first embodiment. Fig. 4 is a flow diagram showing the processing procedures performed by the in-vehicle terminal 1, the roadside device 2, and the server 3.

[0082] 4(C), in the in-vehicle terminal 1, the processor 15 detects moving objects (other vehicles, pedestrians, etc.) present on the road around the device based on the detection results of the sensor 51 of the driving assistance system 5, and generates target information (moving target information) related to the moving objects (moving target information) (moving object detection process) (ST101). Next, the wireless communication unit 12 transmits the generated target information to the roadside device 2 (ST102). Note that the moving object detection process (ST101) may not be performed, and instead the position information acquired by the positioning unit 11 may be transmitted to the roadside device 2 as target information.

[0083] The target information includes information about the detected moving object, specifically, information such as the target ID, time of presence, position, speed, traveling direction, and attributes (size). The target information is also added to a message of ITS communication (road-to-vehicle communication) and transmitted. The ITS communication message includes the vehicle position information acquired by the positioning unit 11.

[0084] As shown in FIG. 4(B), in the roadside device 2, the processor 25 detects a moving object present on the road around the device based on the detection result of the sensor 21 of the device, and generates target information (moving target information) regarding the moving object (moving object detection process) (ST201).

[0085] In addition, in the roadside device 2, the wireless communication unit 22 receives the target information transmitted from the in-vehicle terminal 1 (ST202). Next, the processor 25 associates the targets included in the target information acquired by the roadside device itself and the target information acquired from the in-vehicle terminal 1 as the same object (target identification processing), and integrates the two sets of target information (target information integration processing) (ST203). Next, the network communication unit 23 transmits the integrated target information to the server 3 (ST204).

[0086] In the target information integration process (ST203), vehicle position information transmitted from the in-vehicle terminal 1 to the roadside unit 2 by an ITS communication message may also be treated as target information. That is, the vehicle position information, target information related to moving objects present on the roads around the vehicle, and target information related to moving objects present on the roads around the roadside unit 2 may be integrated. Furthermore, the in-vehicle terminal 1 may not perform a process of acquiring target information related to moving objects present on the roads around the vehicle (moving object detection process), and the target information integration process (ST203) may instead integrate the vehicle position information and target information related to moving objects present on the roads around the roadside unit 2.

[0087] 4(A-1), in the server 3, the network communication unit 31 receives the target information transmitted from the roadside device 2 (ST301). Next, the processor 33 stores the received information in the storage unit 32 (ST302).

[0088] Next, in the server 3, as shown in FIG. 4(A-2), the processor 33 determines whether or not the vehicle is in an abnormal state based on the target object information accumulated in the storage unit 32 (abnormality determination process) (ST311). The processor 33 also determines whether or not the driver of the vehicle has exhibited a behavioral change based on the target object information (behavioral change determination process) (ST312). Next, the processor 33 determines whether or not a near miss has occurred based on the abnormality determination result and the behavioral change determination result (near miss determination process) (ST313). Next, the processor 33 stores the near miss determination result in the storage unit 32 (ST314). At this time, the near miss determination result and the like are registered in the near miss database.

[0089] In response to a viewing request from the viewing terminal 4, the server 3 displays a viewing screen for the information registered in the near-miss incident database on the viewing terminal 4. This allows the user to view information about near-miss incidents that have occurred in the past.

[0090] Note that video of a near miss occurring may be stored in server 3 or the like. For example, video from a camera serving as sensor 51 of driving assistance system 5 may be stored. By displaying such video on viewing terminal 4, the user can check whether the near miss determination process was performed appropriately, and can perform verification to improve the accuracy of the near miss determination process. In addition, video captured at an angle of view in the same direction as the line of sight of a person (driver, pedestrian), specifically video captured by a camera in a wearable device worn by a person, may be stored. By displaying such video on viewing terminal 4, the user can check the details of the near miss situation from the driver's perspective.

[0091] As described above, in this embodiment, near misses are determined based on target object information collected using the sensor 21 provided in the roadside unit 2, thereby improving the accuracy of near miss determination. For example, a moving object that cannot be detected by the sensor 51 on the vehicle side because it is out of the line of sight (i.e., in the shadow of a nearby building) or because occlusion occurs (i.e., it is temporarily in the shadow of another moving object) can be detected by the sensor 21 on the roadside unit 2. This reduces the number of missed detections of surrounding moving objects, and also makes it possible to simultaneously detect multiple moving objects, allowing for appropriate near miss determination.

[0092] Here, in this embodiment, based on the target information collected by the vehicle-mounted terminal 1 and the roadside unit 2, it is determined whether an abnormal condition such as that shown below, i.e., a condition with a high risk of accident, has occurred in the moving object (abnormality determination processing).

[0093] First, in this embodiment, a state where there is a high risk of collision between multiple moving objects (vehicles, pedestrians, bicycles) is determined as an abnormal state (moving object collision determination). Specifically, whether there is a high risk of collision between the moving objects is determined using TTC (Time To Collision), i.e., the time to collision (the time remaining until collision), as an index. In this case, an abnormal state can be determined by the overlap of the movement prediction circles of each moving object. Note that if the target moving object is a pedestrian, the pedestrian will be excluded from the determination if they are in a safe area such as a sidewalk protected by a guardrail or the like.

[0094] In this embodiment, an abnormal state is determined to be a state in which there is a high risk of the vehicle colliding with a stationary object on the roadside or at the side of the road due to the vehicle deviating from the lane (stationary object collision determination). For example, a state in which there is a high risk of the vehicle colliding with a utility pole or guardrail installed at the side of the road, or a fence or building installed along the road, is determined. In this case, an abnormal state can be determined by the overlap of the vehicle's movement prediction circle with the obstacle.

[0095] In this embodiment, an abnormal state is determined to be an abnormal state of the vehicle trajectory (abnormal trajectory determination). For example, a state in which the vehicle has slipped is determined. In this case, the determination may be made taking into account weather conditions (snowfall, etc.) and road surface conditions (snow accumulation, ice, etc.). Furthermore, the vehicle trajectory may be determined to be in an abnormal state when the vehicle deviates significantly from the linearly predicted position.

[0096] In this embodiment, an abnormal state is determined when an object obstructs the view of a person (driver, pedestrian) (visibility obstruction determination). Specifically, an abnormal state is determined when an object such as a fallen object from a vehicle or building, raindrops during heavy rain, or a smartphone obstructs the view of a person. For example, in the case of a vehicle driver, such a state can be determined based on an image captured by a camera such as a driver monitor that captures the driver.

[0097] In this embodiment, an abnormal state is determined to be an abnormal physical state of a person (driver, pedestrian) (physical abnormality determination). Specifically, physical discomfort such as dizziness or lightheadedness, or abnormal behavior such as falling or running out into the street, is determined to be an abnormal state. For example, in the case of a vehicle driver, such a state can be determined based on an image captured by a camera capturing an image of the driver. In the case of a pedestrian, such a state can be determined based on an image of the pedestrian captured by a camera serving as the sensor 21 provided in the roadside device 2. In the case of a pedestrian, such a state can also be determined based on the detection results of a sensor provided in the pedestrian terminal 8.

[0098] Note that a complex abnormal state may also occur. For example, a state in which there is a high risk of the vehicle colliding with another moving object may be combined with a state in which there is a high risk of the vehicle colliding with a stationary object. Specifically, a driver may perform a sudden steering maneuver to avoid a collision with another moving object, resulting in a high risk of the vehicle colliding with a stationary object (e.g., a utility pole) on the roadside. Even in such a case, the present embodiment appropriately determines whether a near miss has occurred.

[0099] (First Modification of the First Embodiment) Next, a near-miss incident determination system according to a first modified example of the first embodiment will be described. It should be noted that the points not specifically mentioned here are the same as those in the above embodiment. Fig. 5 is a block diagram showing an outline of the processing performed by the roadside unit 2 and the server 3.

[0100] In the first embodiment (see FIGS. 3 and 4), near-miss judgment is performed in the server 3 using both the target object information collected by the in-vehicle terminal 1 and the target object information collected by the roadside device 2. On the other hand, in the present embodiment, the near-miss judgment performed in the server 3 does not use the target object information collected by the in-vehicle terminal 1, and the near-miss judgment is performed in the server 3 using only the target object information collected by the roadside device 2.

[0101] (Second Modification of the First Embodiment) Next, a near-miss incident determination system according to a second modified example of the first embodiment will be described. Note that points not specifically mentioned here are the same as those in the above embodiment. Figure 6 is a block diagram showing an outline of the processing performed by the vehicle-mounted terminal 1, the roadside unit 2, and the server 3.

[0102] In the first embodiment (see FIGS. 3 and 4), the abnormality determination process, behavior change determination process, and near-miss determination process are performed in the server 3. On the other hand, in the present embodiment, the abnormality determination process, behavior change determination process, and near-miss determination process are performed in the roadside unit 2, and near-miss information including the near-miss determination result is transmitted to the server 3.

[0103] The server 3 stores the near-miss information received from the roadside device 2 in the storage unit 32. That is, the near-miss information is registered in the near-miss database. In addition, in response to a viewing request from the viewing terminal 4, the server 3 causes the viewing terminal 4 to display a viewing screen for the near-miss information.

[0104] (Second embodiment) Next, a second embodiment will be described. It should be noted that the points not specifically mentioned here are the same as those in the previous embodiment. Fig. 7 is a block diagram showing an outline of the processing performed by the vehicle-mounted terminal 1, roadside device 2, and server 3 according to the second embodiment.

[0105] In the first embodiment (see Figures 3 and 4), both the in-vehicle terminal 1 and the roadside unit 2 detect moving objects (other vehicles, pedestrians, etc.) present on the road around the device, and generate target information regarding the moving objects (moving object detection processing).

[0106] On the other hand, in this embodiment, in addition to the moving object detection process, the roadside device 2 detects stationary objects present on and around the road surrounding the device based on the detection results of the device's sensor 21, and generates target information regarding the stationary objects (stationary target information) (stationary object detection process). Specifically, roadside objects (electric poles, guardrails, roadside trees, etc.), temporary construction structures (cones, fences, etc.), objects fallen from vehicles or buildings, snow or water covering the road surface, etc. are detected as stationary objects. This allows near-miss judgment based on stationary objects that are out of the line of sight from the in-vehicle terminal 1 or stationary objects not included in the map information.

[0107] The target information relating to stationary objects may be point cloud information generated based on the detection results of a three-dimensional sensor such as a LiDAR as the sensor 21 provided in the roadside device 2.

[0108] Next, a description will be given of the processing procedures performed by the vehicle-mounted terminal 1, roadside device 2, and server 3 according to the second embodiment. Fig. 8 is a flow diagram showing the processing procedures performed by the vehicle-mounted terminal 1, roadside device 2, and server 3.

[0109] The processing (ST101, ST102) performed by the vehicle-mounted terminal 1 shown in FIG. 8(C) is the same as that in the first embodiment (see FIG. 4(C)).

[0110] 8(B), in the roadside device 2, the processor 25 detects stationary objects present on the road around the device and in its vicinity based on the detection results of the sensor 21 of the device, and generates target information related to the stationary objects (stationary target information) (stationary object detection process) (ST211). Next, as in the first embodiment, the processor 25 performs the moving object detection process (ST201), the receiving of target information from the in-vehicle terminal 1 (ST202), and the target information integration process (ST203). Next, the network communication unit 23 transmits the target information related to the stationary objects and the integrated target information related to the moving objects to the server 3 (ST204).

[0111] 8(A-1), in the server 3, the network communication unit 31 receives the target information related to the moving object and the target information related to the stationary object transmitted from the roadside device 2 (ST301). Next, the processor 33 stores the received information in the storage unit 32 (ST302).

[0112] The procedures of the processes (ST311 to ST314) performed by the server 3 shown in Fig. 8(A-2) are the same as those in the first embodiment (see Fig. 4(A-2)). In the abnormality determination process (ST311) and the behavioral change determination process (ST312), the processor 33 performs the processes based on target information related to stationary objects in addition to target information related to moving objects.

[0113] (Third embodiment) Next, a near-miss incident determination system according to a third embodiment will be described. Note that points not specifically mentioned here are the same as those in the above-described embodiments. Fig. 9 is an overall configuration diagram of the near-miss incident determination system. Fig. 10 is a block diagram showing an outline of the processing performed by the in-vehicle terminal 1, roadside unit 2, and server 3.

[0114] In the first embodiment (see Figures 3 and 4) and the second embodiment (see Figures 7 and 8), the server 3 determines the driver's behavioral change (sudden driving operation (e.g., braking operation, steering operation, etc.)) in response to an abnormal condition based on target information collected by the in-vehicle terminal 1 and the roadside unit 2 (behavioral change determination process).

[0115] On the other hand, in this embodiment, the physical reaction of the driver to an abnormal state is determined (physical reaction determination process) using a sensor 61 provided in a driver monitoring system (DMS) 6 in the in-vehicle terminal 1. The driver monitoring system 6 includes the sensor 61 and an ECU 62.

[0116] In this embodiment, the roadside device 2 determines whether the driver has changed his / her behavior in response to the abnormal condition based on target information about nearby moving objects collected using the sensor 21 of the device itself (behavior change determination process).

[0117] In addition, in this embodiment, when the server 3 determines that there is either a physical reaction or a change in the driver's behavior, it determines an abnormal state (abnormality determination process), and determines a near miss depending on the determination result (near miss determination process). Note that in this embodiment, the abnormality determination process is performed after the behavior determination process, so the order of the abnormality determination process and the behavior determination process is reversed from that in the first embodiment.

[0118] Here, in the physical reaction determination process performed by the in-vehicle terminal 1, the driver's physical reactions to the abnormal condition, such as a sudden turn of the face, a sudden shift of the gaze, sweating, vocalization (e.g., a cry of "Danger!"), and pupil dilation, are determined based on the detection results of the sensor 61 of the driver monitoring system 6. At this time, for example, the driver's sudden turn of the face, a sudden shift of the gaze, and pupil dilation are determined based on images from a camera serving as the sensor 61. Furthermore, the driver's sweating is determined by using a biosensor provided in a wearable device worn by the driver as the sensor 61. Furthermore, the driver's vocalization is determined based on sounds picked up by a microphone serving as the sensor 61.

[0119] Meanwhile, in the behavior change determination process performed by the roadside device 2, a sudden driving operation (e.g., a sudden braking operation, a sudden steering operation, a sudden acceleration operation, etc.) is determined as a behavior change of the driver in response to an abnormal condition, based on target information related to stationary objects and target information related to moving objects. When the driver performs a sudden driving operation, the sudden driving operation appears as a sudden change in the state (behavior) of the vehicle, so a sudden driving operation can be determined as a behavior change of the driver based on the target information. For example, a sudden braking operation appears as a sudden deceleration or sudden stop of the vehicle. Furthermore, a sudden steering operation appears as a sudden change in the vehicle's course. Furthermore, a sudden acceleration operation appears as a sudden acceleration of the vehicle. Furthermore, a behavior change is something that causes a change in the state of a moving object, and if the target moving object is a pedestrian, for example, a stopping state is determined as a behavior change of the pedestrian.

[0120] In addition, if a sound (e.g., braking sound) caused by a sudden change in the vehicle's state (behavior) is detected based on the audio picked up by a microphone serving as a sensor 21 installed in the roadside unit 2, it may be determined that a sudden driving operation (e.g., sudden braking) has been performed as a change in the driver's behavior.

[0121] In addition, the events (physical reactions, behavioral changes) contained in the judgment results of the physical reaction judgment process performed in the vehicle-mounted terminal 1 and the judgment results of the behavioral change judgment process performed in the roadside unit 2 may be matched as events related to the same near miss (event identification process), and the two judgment results may be integrated.

[0122] In the first embodiment, there are two types of near misses: a first type that does not involve a change in a person's behavior, and a second type that involves a change in a person's behavior. On the other hand, in the present embodiment, a near miss is determined to have occurred when there is either a physical reaction or a change in the driver's behavior and an abnormal state. However, there may be types of near misses that involve both a change in a person's behavior and a physical reaction, a type that involves only a change in a person's behavior, a type that involves only a physical reaction, and a type that involves neither a change in a person's behavior nor a physical reaction.

[0123] Next, a description will be given of the processing procedures performed by the in-vehicle terminal 1, roadside device 2, and server 3 according to the third embodiment. Fig. 11 is a flow diagram showing the processing procedures performed by the in-vehicle terminal 1, roadside device 2, and server 3.

[0124] As shown in FIG. 11(C), the in-vehicle terminal 1 performs a moving object detection process (ST101) in the same manner as in the first embodiment (see FIG. 4(C)). The processor 15 determines the driver's physical reaction to the abnormal condition based on the detection result of the sensor 61 of the driver monitoring system 6 (physical reaction determination process) (ST121). Next, the wireless communication unit 12 transmits the generated target information and the physical reaction determination result to the roadside device 2 (ST102). The physical reaction determination result may be transmitted only when there is a physical reaction.

[0125] The physical reaction determination result transmitted from the vehicle-mounted terminal 1 to the roadside device 2 is added to the ITS communication message together with the target information. In this case, the physical reaction determination result may be transmitted using an extension area of the message.

[0126] 11(B), the roadside device 2 performs stationary object detection processing (ST211) and moving object detection processing (ST201) in the same manner as in the first embodiment (see FIG. 4(B)). In addition, the processor 25 determines a behavioral change of the driver in response to the abnormal condition (behavioral change determination processing) (ST221) based on target information related to the stationary object and target information related to the moving object.

[0127] Furthermore, in the roadside device 2, as in the first embodiment (see FIG. 4(B)), reception of target information from the in-vehicle terminal 1 (ST202) and target information integration processing (ST203) are performed. Next, the network communication unit 23 transmits the target information related to the stationary object, the integrated target information related to the moving object, the physical reaction determination result, and the behavioral change determination result to the server 3 (ST204).

[0128] 11(A-1), in the server 3, the network communication unit 31 receives the target information on stationary objects, the target information on moving objects, the physical reaction determination result, and the behavioral change determination result transmitted from the roadside device 2 (ST301). Next, the processor 33 stores the received information in the storage unit 32 (ST302).

[0129] Next, in the server 3, as shown in Figure 11 (A-2), the processor 33 determines whether or not there is at least one of a physical reaction and a behavioral change based on the physical reaction determination results and the behavioral change determination results stored in the memory unit 32 (ST321).

[0130] Here, if there is at least one of a physical reaction and a behavioral change (Yes in ST321), processor 33 then determines whether or not the vehicle is in an abnormal state based on the target information accumulated in memory unit 32 (abnormality determination process) (ST311). Next, processor 33 determines whether or not a near miss has occurred based on the abnormality determination result (near miss determination process) (ST313). Next, processor 33 stores the near miss determination result in memory unit 32 (ST314).

[0131] On the other hand, if the driver has neither a physical reaction nor a change in behavior (No in ST321), the abnormality determination process (ST311), the near-miss determination process (ST313), and the storage of the determination results (ST314) are omitted.

[0132] Furthermore, in this embodiment, similar to the above-described embodiment, a user can view, on the viewing terminal 4, information registered in the near-miss database in the server 3, i.e., information about past near-misses. In particular, in this embodiment, for each near-miss incident, the driver's physical reactions to the abnormal condition (such as a sudden head turn, a sudden shift in gaze, sweating, or vocalization) and behavioral changes (sudden driving maneuvers) are displayed. The driver's physical reactions and behavioral changes may be displayed as a list for each near-miss incident, or may be displayed for each near-miss occurrence point on a map. The user may also be able to view the driver's physical reactions and behavioral changes through video. In this case, video captured by a camera in a wearable device worn by the driver or a zoomed image captured by a camera serving as the sensor 21 installed in the roadside unit 2 may be displayed. The video captured by the camera in the wearable device can be collected by the server 3 via the in-vehicle terminal 1.

[0133] (Modification of the third embodiment) Next, a near-miss incident determination system according to a modified example of the third embodiment will be described. Note that the points not specifically mentioned here are the same as those in the above-described embodiment. Fig. 12 is a block diagram showing an outline of the processing performed by the vehicle-mounted terminal 1, the roadside unit 2, and the server 3.

[0134] In the third embodiment, the abnormality determination process and the near-miss determination process are performed in the server 3. On the other hand, in this modification, the abnormality determination process and the near-miss determination process are performed in the road-side unit 2, as shown in FIG.

[0135] In this case, the roadside unit 2 determines whether or not there is at least one of a physical reaction and a behavioral change, and if there is at least one of a physical reaction and a behavioral change, an abnormality determination process and a near-miss determination process are performed. Also, similar to the second modification of the first embodiment (see FIG. 6), near-miss information including the near-miss determination result and the like is transmitted to the server 3, and the server 3 stores the received near-miss information in the storage unit 32.

[0136] Next, a description will be given of the processing procedures performed by the in-vehicle terminal 1, the roadside device 2, and the server 3 according to a modified example of the third embodiment. Fig. 13 is a flow diagram showing the processing procedures performed by the in-vehicle terminal 1, the roadside device 2, and the server 3.

[0137] The processing (ST101, ST121, ST102) performed by the vehicle-mounted terminal 1 shown in FIG. 13(C) is the same as that in the third embodiment (see FIG. 11(C)).

[0138] As shown in FIG. 13(B), the roadside unit 2 performs stationary object detection processing (ST211), moving object detection processing (ST201), behavior change determination processing (ST221), reception of target information from the vehicle-mounted terminal 1 (ST202), and target information integration processing (ST203), similar to the third embodiment (see FIG. 11(B)).

[0139] Next, in the roadside device 2, the processor 25 determines whether or not at least one of a physical reaction and a behavioral change has occurred based on the physical reaction determination result and the behavioral change determination result (ST231).

[0140] Here, if there is at least one of a physical reaction and a behavioral change (Yes in ST231), processor 25 then determines whether or not the vehicle is in an abnormal state based on the target information (abnormality determination process) (ST232). Next, processor 25 determines whether or not a near miss has occurred based on the abnormality determination result (near miss determination process) (ST233). Next, network communication unit 23 transmits near miss information including the near miss determination result and the like to server 3 (ST234).

[0141] 13(A), in the server 3, the network communication unit 31 receives the near-miss information transmitted from the roadside unit 2 (ST301). Next, the processor 33 stores the received information in the storage unit 32 (ST302).

[0142] (Fourth embodiment) Next, a near-miss incident determination system according to a fourth embodiment will be described. The points not specifically mentioned here are the same as those in the above-described embodiments. Fig. 14 is a block diagram showing an outline of the processing performed by the vehicle-mounted terminal 1, the roadside unit 2, and the server 3.

[0143] In the third embodiment, the vehicle-mounted terminal 1 uses a sensor 61 of the driver monitoring system 6 to determine the driver's physical reaction to an abnormal condition (for example, a sudden change in the direction of the face, a sudden movement of the viewpoint, sweating, vocalization) (physical reaction determination process).

[0144] On the other hand, in this embodiment, the roadside device 2 uses its own sensor 21 to determine the driver's physical reaction to the abnormal condition (physical reaction determination process). At this time, the driver's physical reaction, specifically, a sudden head turning motion, etc. is determined based on the zoomed image of the camera serving as the sensor 21.

[0145] Furthermore, in this embodiment, similar to the third embodiment, the roadside unit 2 uses its own sensor 21 to acquire target information regarding surrounding moving objects, and based on the target information, the driver's behavioral change in response to the abnormal condition (sudden driving operation (e.g., braking operation, steering operation, etc.)) is determined (behavioral change determination process).

[0146] In this embodiment, similarly to the third embodiment, the physical reaction determination process is also performed in the vehicle-mounted terminal 1.

[0147] Furthermore, the behavior change determination process performed in the roadside device 2 may also use zoomed images from a camera serving as the sensor 21 to determine a behavior change of the driver, specifically, a sudden steering operation or the like.

[0148] In addition, the events (physical reactions, behavioral changes) contained in the judgment results of the physical reaction judgment process performed in the vehicle-mounted terminal 1, the judgment results of the physical reaction judgment process performed in the roadside unit 2, and the judgment results of the behavioral change judgment process performed in the roadside unit 2 may be matched as events related to the same near miss (event identification process), and the three judgment results may be integrated.

[0149] Next, a description will be given of the processing procedures performed by the in-vehicle terminal 1, the roadside device 2, and the server 3 according to the fourth embodiment. Fig. 15 is a flow diagram showing the processing procedures performed by the in-vehicle terminal 1, the roadside device 2, and the server 3.

[0150] The processing (ST101, ST121, ST102) performed by the vehicle-mounted terminal 1 shown in FIG. 15(C) is the same as that in the third embodiment (see FIG. 11(C)).

[0151] As shown in FIG. 15(B), the roadside unit 2 performs stationary object detection processing (ST211), moving object detection processing (ST201), and behavior change determination processing (ST221) in the same manner as in the third embodiment (see FIG. 11(B)).

[0152] Furthermore, in the roadside device 2, the processor 25 determines the driver's physical reaction to the abnormal condition based on the zoomed image acquired by the camera serving as the sensor 21 of the device itself (ST241).

[0153] At this time, a zoomed image of the target person (vehicle driver, pedestrian, or cyclist) is acquired based on the position information of the moving object (vehicle, pedestrian, or cyclist) included in the target information. Specifically, if the camera serving as sensor 21 has a PTZ (pan, tilt, and zoom) function, the direction and magnification of sensor 21 (camera) are adjusted with respect to the target person based on the position information of the moving object, thereby acquiring a zoomed image of the target person. Furthermore, an image of the target person may be cut out from the image of the high-resolution camera serving as sensor 21 based on the position information of the moving object.

[0154] Next, in the roadside device 2, as in the third embodiment (see FIG. 11(B)), reception of target information from the in-vehicle terminal 1 (ST202) and target information integration processing (ST203) are performed. Next, the network communication unit 23 transmits the target information related to the stationary object, the integrated target information related to the moving object, the physical reaction determination result, and the behavioral change determination result to the server 3 (ST204).

[0155] The processing performed by the server 3 shown in FIGS. 15(A-1) and (A-2) is the same as that in the third embodiment (see FIGS. 11(A-1) and (A-2)).

[0156] (Fifth embodiment) Next, a near-miss analysis system according to a fifth embodiment will be described. The points not specifically mentioned here are the same as those in the above-described embodiments. Fig. 16 is a block diagram showing an outline of the processing performed by the vehicle-mounted terminal 1, the roadside unit 2, and the server 3.

[0157] In this embodiment, the server 3 performs a process of analyzing the factors that caused the near miss (factor analysis process). Here, the factors of the near miss are the factors that caused the abnormal state (state with a high risk of accident) that led to the near miss. The results of the near miss factor analysis provide knowledge for preventing the recurrence of near misses and reducing accidents, and this knowledge is utilized for driving assistance to prevent accidents before they occur, road maintenance to reduce accidents, and the like. In particular, the results of the near miss factor analysis are effective in considering methods of alerting drivers in the event of an abnormality (the content and timing of the notification).

[0158] The in-vehicle terminal 1 and the roadside device 2 collect information necessary for near-miss incident determination processing as well as information necessary for factor analysis processing. The collected information is registered in the near-miss incident database of the server 3.

[0159] Specifically, as in the above embodiment, target information regarding nearby moving objects, target information regarding nearby stationary objects, judgment results regarding the driver's physical response to the abnormal condition, and judgment results regarding the driver's behavioral changes in response to the abnormal condition are collected.

[0160] Furthermore, in this embodiment, the in-vehicle terminal 1 collects driver information related to the attributes of the driver, etc. (driver information collection process). Specifically, the driver's age, etc. are collected as the driver's attributes based on the image of the sensor 61 (camera) of the driver monitoring system 6. In addition, information related to the driver's behavior of turning his or her face, etc. is collected based on the image of the sensor 61 of the driver monitoring system 6.

[0161] Furthermore, the roadside device 2 collects environmental information about the surrounding road environment (environment detection process). Specifically, information about the weather (rainfall, snowfall, etc.) and road surface conditions (snow accumulation, flooding, etc.) is collected based on the detection results of the sensor 21 of the device itself.

[0162] The server 3 determines whether a near miss has occurred based on the target information, the physical reaction determination result, and the behavioral change determination result.

[0163] The server 3 also identifies factors causing near misses based on the target object information, driver information, and environmental information. For example, the server 3 identifies vehicle speeding and violation of stop signs as factors causing near misses based on the target object information. The server 3 also identifies, based on the driver information, particularly the driver's attributes, that the driver is elderly as a factor causing near misses. The server 3 also identifies, based on the driver information, particularly information about the driver's behavior of turning their head, insufficient safety checks, such as checking left and right when entering an intersection, and inattention to the road ahead while driving (distracted driving), as factors causing near misses based on the environmental information. The server 3 also identifies, based on the environmental information, bad weather (such as snowfall) and time of day (such as dusk) as factors causing near misses.

[0164] Furthermore, in response to a viewing request from the viewing terminal 4, the server 3 causes the viewing terminal 4 to display a viewing screen of the cause analysis results. This allows the user to view the cause analysis results. Furthermore, when the cause analysis results are delivered to the in-vehicle terminal 1 and the locations where near misses have occurred in the past are presented to the driver, the causes of the near misses may also be presented to the driver.

[0165] Next, a description will be given of the processing procedures performed by the in-vehicle terminal 1, roadside device 2, and server 3 according to the fifth embodiment. Fig. 17 is a flow diagram showing the processing procedures performed by the in-vehicle terminal 1, roadside device 2, and server 3.

[0166] In the in-vehicle terminal 1, as shown in FIG. 17(C), the processor 15 collects driver information regarding the driver's attributes, etc. based on the image of the sensor 61 (camera) of the driver monitoring system 6 (driver information collection process) (ST151).

[0167] In addition, in the in-vehicle terminal 1, moving object detection processing (ST101) and physical reaction determination processing (ST121) are performed as in the third embodiment (see FIG. 11(C)). Next, the wireless communication unit 12 transmits the acquired driver information, physical reaction determination results, and target information to the server 3 (ST102).

[0168] As shown in FIG. 17(B), the roadside unit 2 performs stationary object detection processing (ST211), moving object detection processing (ST201), and behavior change determination processing (ST221) in the same manner as in the third embodiment (see FIG. 11(B)).

[0169] Furthermore, in the roadside device 2, the processor 25 detects the road environment around the device based on the detection result of the sensor 21 of the device, and acquires environmental information about the road environment around the device (environment detection process) (ST251). At this time, the acquired environmental information includes information about weather conditions (sunny, rainy, cloudy, snowfall, fog, wind speed, temperature, etc.) and information about road surface conditions (frozen, snowy, etc.).

[0170] Next, in the roadside device 2, the network communication unit 23 transmits the target information relating to the stationary object, the driver information relating to the moving object, the target information, the physical reaction determination result, the behavioral change determination result, and the environmental information to the server 3 (ST204).

[0171] 17(A-1), in the server 3, the network communication unit 31 receives the driver information, target information, physical reaction determination result, behavioral change determination result, and environmental information transmitted from the in-vehicle terminal 1 (ST301). Next, the processor 33 stores the received information in the storage unit 32 (ST302).

[0172] Next, in the server 3, as shown in Figure 17 (A-2), similar to the third embodiment (see Figure 4 (A-2)), a judgment regarding the physical reaction results and behavioral change results (ST321), an abnormality judgment process (ST311), and a near-miss judgment process (ST313) are performed.

[0173] Next, in the server 3, the processor 33 identifies the cause of the near miss (speeding, violation of stop signs, etc.) based on the driver information, the target information, the physical reaction determination result, the behavioral change determination result, and the environmental information (cause analysis process) (ST351). Next, the processor 33 stores the near miss determination result and the cause analysis result in the memory unit 32 (ST352). At this time, the near miss determination result and the cause analysis result are registered in the near miss database.

[0174] On the other hand, if the driver has neither a physical reaction nor a change in behavior (No in ST321), the abnormality determination process (ST311), the near-miss determination process (ST313), the factor analysis process (ST351), and the storage of the determination results (ST352) are omitted.

[0175] Furthermore, in this embodiment, as in the previous embodiment, a user can view, on the viewing terminal 4, information registered in the near-miss database in the server 3, i.e., information about near-misses that have occurred in the past. In particular, in this embodiment, the cause analysis results are displayed for each near-miss case. At this time, the cause analysis results may be displayed as a list for each near-miss case, or the cause analysis results may be displayed for each near-miss occurrence point displayed on a map.

[0176] (Sixth embodiment) Next, a near-miss notification system according to a sixth embodiment will be described. Note that points not specifically mentioned here are the same as those in the above-described embodiments. Figure 18 is an overall configuration diagram of the near-miss notification system.

[0177] In this embodiment, the driving assistance system 5 provided in the vehicle includes a display device 53 in addition to the sensor 51 and the ECU 52. The display device 53 displays near-miss guidance information provided by the server 3. This allows the driver to be informed of locations where near-misses caused by abnormal conditions such as the vehicle slipping or a child running out into the road frequently have occurred in the past (near-miss frequent locations).

[0178] In this embodiment, location information regarding the current location of the vehicle is transmitted from the in-vehicle terminal 1 to the server 3, and the server 3 generates near-miss guidance information regarding near-miss frequent occurrence points in the vicinity of the current location of the vehicle, and distributes the near-miss guidance information to the in-vehicle terminal 1. In this way, near-miss frequent occurrence points in the vicinity of the current location of the vehicle are presented to the driver.

[0179] 18, the vehicle position information is transmitted to the server 3 via a base station of the mobile communication network, and the near-miss notification information is distributed to the in-vehicle terminal 1 via a base station of the mobile communication network. On the other hand, the vehicle position information and the near-miss notification information may be transmitted and received between the in-vehicle terminal 1 and the server 3 via the roadside unit 2.

[0180] Near misses occur due to location-specific circumstances, such as the shape of the road and the condition of surrounding buildings, but they can also occur only at certain times of the day (for example, at dusk) or in certain road conditions, such as bad weather (for example, when there is snow).

[0181] Therefore, in this embodiment, based on the information registered in the near-miss database, the server 3 selects near-miss frequent locations that are near the vehicle's current location and whose occurrence conditions regarding the time of day and road environment match those of the current location, and generates near-miss guidance information regarding these near-miss frequent locations (guidance information generation process), and delivers this near-miss guidance information to the in-vehicle terminal 1.

[0182] In this case, the server 3 registers the near-miss determination results in a database, as in the above embodiment. For each case where a near-miss is determined to have occurred, the near-miss determination results include information such as the location information of the occurrence point, the time of occurrence, and the road environment at the time of occurrence. As a result, in the guidance information generation process, near-miss frequent occurrence points that are near the current vehicle location and whose occurrence conditions related to the time of day and road environment match those of the current situation are identified.

[0183] Display device 53 may be a liquid crystal display provided on a dashboard or the like, or a head-up display that projects an image onto the windshield. Display device 53 may also be a wearable device such as a head-mounted display or smart glasses. Display device 53 may also be a device used in a car navigation system.

[0184] If the display device 53 is a liquid crystal display, for example, a near-miss occurrence map is displayed on the display device 53. In the near-miss occurrence map, marks indicating near-miss frequent occurrence points and marks indicating the position and traveling direction of the vehicle are superimposed on a map of the area around the vehicle. The near-miss occurrence map may be displayed on a screen that provides guidance on the planned driving route in a car navigation system. In addition, the near-miss occurrence map may be superimposed with a message guiding the near-miss frequent occurrence points, such as a message saying "Approaching a near-miss frequent occurrence point!" or a caution message, such as a message saying "Vehicle approaching from the right, check both sides." In addition, near-miss frequent occurrence points may be displayed together with accident frequent occurrence points.

[0185] Furthermore, when the display device 53 is a head-up display, for example, a mark representing a near-miss frequent occurrence point is superimposed on the position of the near-miss frequent occurrence point in the actual field of vision of the driver by the display device 53. As a result, when the vehicle approaches a near-miss frequent occurrence point, the driver is notified that a near-miss frequent occurrence point exists ahead of the vehicle, and the driver is alerted.

[0186] Next, a description will be given of the procedure of the process performed by the in-vehicle terminal 1 and the server 3 according to the sixth embodiment. Fig. 19 is a flow chart showing the procedure of the process performed by the in-vehicle terminal 1 and the server 3.

[0187] In the in-vehicle terminal 1, first, the positioning unit 11 detects the position of the device itself and acquires current position information (ST161). Next, the wireless communication unit 12 transmits the position information acquired by the positioning unit 11 to the server 3 (ST162).

[0188] In the server 3, the network communication unit 31 receives the location information transmitted from the in-vehicle terminal 1 (ST361). Next, the processor 33 selects a near-miss frequent occurrence point that is located near the current location of the vehicle and occurred in the same time period as the present, based on the received location information and the current time, and generates near-miss guidance information related to the near-miss frequent occurrence point (guidance information generation process) (ST362). Next, the network communication unit 31 transmits the near-miss guidance information to the in-vehicle terminal 1 (ST363).

[0189] In the in-vehicle terminal 1, the wireless communication unit 12 receives the near-miss notification information transmitted from the server 3 (ST171). Next, the processor 15 displays the near-miss notification information on the display device 53. At this time, the display device 53 displays near-miss notification frequent locations that are in the vicinity of the current position of the vehicle and occurred in the same time period as the present.

[0190] 19, a near-miss frequent occurrence point that is in the vicinity of the current position of the vehicle and occurred in the same time period as the present is selected, and near-miss guidance information related to that near-miss frequent occurrence point is generated. On the other hand, a near-miss frequent occurrence point may also be selected by including occurrence conditions related to the road environment (for example, when there is snowfall) in the selection criteria.

[0191] In this case, similarly to the fifth embodiment (see FIGS. 16 and 17), the roadside unit 2 collects environmental information about the surrounding road environment, specifically, about the weather (rainfall, snowfall, etc.) and road surface conditions (snow accumulation, flooding, etc.), based on the detection results of its own sensor 21 (environment detection process). The server 3 registers the environmental information acquired from the roadside unit 2 in a near-miss database for each near-miss incident. Furthermore, based on the information registered in the near-miss database, the server 3 selects near-miss frequent points that are in the vicinity of the current location of the vehicle and whose occurrence conditions related to the time period and road environment match those of the current situation, and generates near-miss guidance information about the near-miss frequent points.

[0192] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. Furthermore, it is also possible to combine the components described in the above embodiments to create new embodiments. [Industrial Applicability]

[0193] The near-miss determination system, near-miss analysis system, near-miss notification system, and near-miss determination method of the present invention can accurately determine near-misses even in situations where nearby moving objects cannot be detected from a mobile terminal due to being out of the line of sight or due to temporary occlusion, and further have the effect of being able to appropriately determine near-misses involving multiple moving objects.Therefore, they are useful as a near-miss determination system that determines near-misses involving moving objects on the road, a near-miss analysis system that analyzes the factors behind near-misses that have occurred in the past, a near-miss notification system that notifies people on the road of information regarding the occurrence of past near-misses, and a near-miss determination method in which processing related to determining near-misses involving moving objects on the road is executed by one or more processors. [Explanation of symbols]

[0194] 1: In-vehicle terminal (mobile terminal) 2: Roadside machine 3: Server 4: Viewing device 5: Driving assistance systems 6: Driver monitoring system 8: Pedestrian terminal (mobile terminal) 9: Bicycle terminal (mobile terminal) 15: Processor 21: Sensor 25: Processor 32: Storage part 33: Processor 51: Sensor 53: Display device 61: Sensor

Claims

1. A near-miss incident determination system that determines near-miss incidents involving moving objects on a road, a mobile terminal possessed by a mobile entity; a roadside unit installed on or near a road; a sensor provided in the roadside unit; one or more processors that execute processing related to determining the near miss based on the detection results of the sensors; the processor: detecting a moving object present on a road around the roadside device based on the detection result of the sensor, and acquiring moving target information relating to the moving object; A near-miss determination system characterized by determining an abnormal state based on the moving target information and determining the near-miss based on the determination result.

2. the processor: detecting a plurality of moving objects present on a road around the roadside device based on the detection results of the sensor, and acquiring the moving target information relating to the plurality of moving objects; The near-miss incident determination system according to claim 1, characterized in that the near-miss incidents involving a plurality of moving objects are determined based on the moving target information.

3. Further, a sensor provided on the moving body is provided, the processor: detecting another moving object on a road around the moving object based on the detection result of the sensor provided on the moving object, and acquiring the moving target information relating to the other moving object; The near-miss detection system according to claim 1, characterized in that the moving target information based on the detection results of the sensor installed in the roadside device and the moving target information based on the detection results of the sensor installed in the moving body are integrated.

4. the processor: acquiring location information of the mobile object acquired by a positioning unit in the mobile terminal; The near-miss determination system according to claim 1, characterized in that the position information of the moving body is integrated as moving target information with the moving target information based on the detection results of the sensor installed in the roadside device.

5. the processor: detecting stationary objects on and around the road based on the detection results of the sensors provided in the roadside devices, and acquiring stationary target information relating to the stationary objects; 2. The near-miss incident determination system according to claim 1, wherein the abnormal state is determined based on the stationary target information and the moving target information.

6. the processor: The near-miss detection system described in claim 1, characterized in that the abnormal state is determined to be at least one of a state in which there is a high risk of collision between moving objects, a state in which there is a high risk of collision between a moving object and a stationary object, a state in which the trajectory of a moving object is abnormal, a state in which an object obstructs a person's view, and a state in which a person's body is abnormal.

7. the processor: determining whether or not the target person has exhibited a behavioral change in response to the abnormal state based on the moving target information; 2. The near-miss incident determination system according to claim 1, wherein when the abnormal state exists and the behavioral change exists, it is determined that the near-miss incident has occurred.

8. the processor: If the behavioral change is detected, it is determined whether the abnormal condition has occurred; 8. The near-miss incident determination system according to claim 7, wherein when the abnormal state occurs, it is determined that the near-miss incident has occurred.

9. the processor: determining whether the target person has a physical reaction to an abnormal condition based on the detection result of the sensor that detects the physical condition of the target person; 2. The near-miss incident determination system according to claim 1, wherein when the abnormal state exists and the physical reaction occurs, it is determined that the near-miss incident has occurred.

10. Further, a sensor provided on the moving body is provided, the processor: The near-miss detection system according to claim 9, characterized in that it determines whether a target person has a physical reaction to an abnormal condition based on the detection results of the sensor installed in the mobile body.

11. the sensor provided in the roadside device includes a camera capable of zooming in to photograph a person on a road, The processor: The near-miss detection system according to claim 9, characterized in that it determines whether or not a target person has a physical reaction to an abnormal condition based on the zoomed image of the camera.

12. A near-miss analysis system that analyzes factors of near-misses involving moving objects on a road, a mobile terminal possessed by a mobile entity; a roadside unit installed on or near a road; a sensor provided in the roadside unit; one or more processors that execute processing related to determining the near miss based on the detection information of the sensor; the processor: detecting a moving object present on a road around the roadside device based on the detection result of the sensor, and acquiring moving target information relating to the moving object; An abnormal state is determined based on the moving target information, and the near miss is determined based on the determination result, and the near miss determination result and collected information related to the near miss are stored in a storage unit, A near-miss analysis system characterized by analyzing the causes of near-miss occurrences based on the near-miss judgment results and the collected information stored in the memory unit.

13. A near-miss notification system that notifies people on the road of information regarding the occurrence of near-misses in the past, a mobile terminal possessed by a mobile entity; a roadside unit installed on or near a road; a sensor provided in the roadside unit; one or more processors that execute processing related to determining the near miss based on the detection information of the sensor; the processor: detecting a moving object present on a road around the roadside device based on the detection result of the sensor, and acquiring moving target information relating to the moving object; Determine an abnormal state based on the moving target information, determine the near miss based on the determination result, and store the near miss determination result in a storage unit; Based on the near-miss determination result stored in the storage unit, near-miss guidance information suitable for the mobile terminal to which it is to be distributed is generated; A near-miss notification system characterized in that the near-miss guidance information is delivered to the mobile terminal.

14. A near-miss determination method in which a process related to determining a near-miss involving a moving object on a road is executed by one or more processors, Detecting a moving object present on a road around the roadside device based on a detection result of a sensor provided in the roadside device provided on or near the road, and acquiring moving target information relating to the moving object; A near-miss determination method characterized by determining an abnormal state based on the moving target information, and determining the near-miss based on the determination result.

Images