Hazard factor estimation device, hazard factor estimation system, and hazard factor estimation program

Abstract

To provide a dangerous factor estimation device, a dangerous factor estimation system, and a dangerous factor estimation program capable of estimating a factor of a dangerous situation.SOLUTION: An estimation unit 50 acquires a shot image of the periphery of a vehicle taken by an imaging unit 24, and at least one of vehicle information concerning traveling of the vehicle detected by a vehicle information detection unit 22 and a shot image of a driver taken by the imaging unit 24. An estimation unit 50 estimates a dangerous scene as a dangerous situation from the shot image of the periphery of the vehicle, and estimates a factor of the dangerous scene from at least one of the vehicle information and the shot image of the driver.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a risk factor estimation device, a risk factor estimation system, and a risk factor estimation program that estimate factors of dangerous situations while driving. [Background technology]

[0002] Patent Document 1 discloses a driving diagnosis device that generates and outputs screen information including a distribution of safe drivers and dangerous drivers determined using driving history information of drivers stored in a memory unit, with the horizontal axis representing the degree of risky driving behavior of the driver and the vertical axis representing the number of drivers according to the degree of risky driving behavior, and a diagnosis result regarding the risky driving behavior of the driver to be diagnosed.

[0003] Patent document 2 discloses a danger sign determination device that includes a behavior acquisition unit that acquires behavior including the driver's head swing angle, a danger sign determination unit that determines danger signs by referring to the driver's behavior, and an output unit that outputs the result of the danger sign determination. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-197308 A [Patent Document 2] JP 2019-79150 A Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, screen information is generated that shows the driver's own diagnostic results in relation to the distribution of diagnostic results for the risky driving behavior of other drivers. However, this does not identify the causes of risky driving, and there is room for improvement in terms of suggesting ways to improve driving.

[0006] In addition, in Patent Document 2, danger signs are determined using only images of the driver, but since the environment outside the vehicle is not taken into consideration, the causes of dangerous situations cannot be identified, and there is room for improvement in terms of suggesting ways to improve driving.

[0007] Therefore, an object of the present invention is to provide a risk factor estimation device, a risk factor estimation system, and a risk factor estimation program capable of estimating the cause of a dangerous situation. [Means for solving the problem]

[0008] In order to achieve the above object, the risk factor estimation device according to claim 1 includes an acquisition unit that acquires a surrounding image captured from the vehicle, and at least one of vehicle information related to the traveling of the vehicle and a driver image captured of the driver, a risk estimation unit that estimates a dangerous situation from the surrounding image acquired by the acquisition unit, and a factor estimation unit that estimates a factor of the dangerous situation estimated by the risk estimation unit from the at least one of the images acquired by the acquisition unit. The acquisition unit acquires the surrounding photographed image and the vehicle information, and the factor estimation unit estimates that aggressive driving is the factor by determining whether or not a deviation between the vehicle information at the time of estimating the dangerous situation and an average value of the vehicle information of other vehicles is equal to or greater than a predetermined threshold. .

[0009] According to the invention recited in claim 1, the acquisition unit acquires at least one of a surrounding captured image captured from the vehicle around the vehicle, and vehicle information related to the vehicle's travel and a driver captured image captured of the driver.

[0010] The danger estimation unit estimates a dangerous situation from the surrounding photographed image acquired by the acquisition unit.

[0011] The factor estimation unit estimates the factor of the dangerous situation estimated by the risk estimation unit from at least one of the data acquired by the acquisition unit. This estimates a dangerous situation and estimates the factor of the estimated dangerous situation, thereby providing a risk factor estimation device capable of estimating the factor of a dangerous situation.

[0012] Also The acquisition unit acquires the surrounding photographed image and the vehicle information, and the cause estimation unit estimates the vehicle information at the time of estimating the dangerous situation. and determining whether the deviation between the vehicle information of the vehicle and the average value of the vehicle information of the other vehicles is equal to or greater than a predetermined threshold. Aggressive driving is presumed to be the cause. do This allows the driver's operating conditions to be understood from vehicle information, making it possible to estimate that reckless driving is a factor in dangerous situations.

[0013] In addition The factor estimation unit may estimate the driver's inattention as the factor from the image of the driver captured at the time of estimating the dangerous situation. In this way, it is possible to know whether the driver is inattentive or not from the image of the driver captured at the time of estimating the dangerous situation, and therefore it is possible to estimate the driver's inattention as the factor from the image of the driver captured at the time of estimating the dangerous situation.

[0014] The factor estimation unit also estimates the surrounding captured image Evaluate the risk to When the gaze direction obtained from the driver's photographed image does not overlap with an area of ​​the obtained risk heat map that is equal to or larger than a predetermined threshold, the driver's inattention may be estimated as the cause. In this way, when the driver is not looking in the direction of a dangerous situation in the case of inattention, it is possible to estimate that the driver's inattention is the cause when the gaze direction obtained from the driver's photographed image does not overlap with an area of ​​the obtained risk heat map that is equal to or larger than a predetermined threshold in the case of inattention.

[0015] The risk factor estimation device and the surrounding captured image and the vehicle information, or the surrounding captured image, the vehicle information, The driver's photograph Statue and a vehicle that transmits the information to the risk factor estimation device. In this way, a risk factor estimation system that estimates a dangerous situation and estimates the cause of the estimated dangerous situation can be provided.

[0016] The computer also includes a process of acquiring at least one of a surrounding image captured from the vehicle and vehicle information related to the traveling of the vehicle and a driver image captured of the driver, estimating a dangerous situation from the surrounding image, and estimating a cause of the dangerous situation from at least one of the surrounding image and the driver image. and acquiring the surrounding photographed image and the vehicle information, and judging whether a deviation between the vehicle information at the time of estimating the dangerous situation and an average value of the vehicle information of other vehicles is equal to or greater than a predetermined threshold value, thereby estimating that aggressive driving is the cause of the dangerous situation.In this way, a dangerous situation is estimated and the cause of the estimated dangerous situation is estimated, so that a dangerous situation estimation program capable of estimating the cause of the dangerous situation can be provided. Effect of the Invention

[0017] As described above, according to the present invention, it is possible to provide a risk factor estimation device, a risk factor estimation system, and a risk factor estimation program capable of estimating the cause of a dangerous situation. [Brief description of the drawings]

[0018] [Figure 1] 1 is a diagram showing a schematic configuration of a driving evaluation system according to an embodiment of the present invention; [Diagram 2] 2 is a functional block diagram showing the functional configuration of an in-vehicle device and a cloud server in the driving evaluation system according to the present embodiment. FIG. [Diagram 3] 2 is a block diagram showing the configuration of a control unit of an in-vehicle device and a central processing unit of a cloud server. [Figure 4] 4 is a block diagram showing a detailed functional configuration of an estimation unit. FIG. [Diagram 5] 13 is a flowchart showing an example of a flow of processing performed by the function of an estimation unit of a central processing unit in a cloud server of the driving evaluation system according to the present embodiment. [Figure 6] 10 is a flowchart showing an example of the flow of a driver's state estimation process. [Figure 7] 10 is a flowchart showing an example of the flow of a vehicle behavior estimation process. [Figure 8] 13 is a flowchart showing an example of the flow of a lack of attention determination process. [Figure 9] 11A to 11C are diagrams illustrating an example of a frame image, a region of a gaze direction, and a region where a degree of danger is equal to or greater than a threshold, and an example of calculation of a degree of dangerous gaze. [Figure 10] 10 is a flowchart showing an example of the flow of an aggressive driving determination process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] An example of an embodiment of the present invention will be described in detail below with reference to the drawings. In this embodiment, a driving evaluation system that evaluates the driving of a driver will be described as an example of a risk factor estimation system. Fig. 1 is a diagram showing a schematic configuration of the driving evaluation system according to this embodiment.

[0020] In the driving evaluation system 10 according to the present embodiment, an in-vehicle device 16 mounted on a vehicle 14 and a cloud server 12 as an example of a risk factor estimation device are connected via a communication network 18. In the driving evaluation system 10 according to the present embodiment, image information obtained by photographing a plurality of in-vehicle devices 16 and vehicle information indicating the state of each vehicle 14 are transmitted to the cloud server 12, and the cloud server 12 accumulates the image information and the vehicle information. Furthermore, based on the accumulated information, the cloud server 12 performs processes such as detecting dangerous scenes and estimating factors of dangerous situations in dangerous scenes, and evaluating the driver.

[0021] FIG. 2 is a functional block diagram showing the functional configuration of the vehicle-mounted device 16 and the cloud server 12 in the driving evaluation system 10 according to this embodiment.

[0022] The vehicle-mounted device 16 includes a control unit 20, a vehicle information detection unit 22, an image capture unit 24, a communication unit 26, and a display unit .

[0023] The vehicle information detection unit 22 detects vehicle information related to the vehicle 14. Examples of the vehicle information include vehicle information such as the position information of the vehicle 14, vehicle speed, acceleration, steering angle, accelerator opening, distance to obstacles around the vehicle, and route. Specifically, the vehicle information detection unit 22 can apply a plurality of types of sensors and devices that acquire information indicating the state of the surrounding environment of the vehicle 14. Examples of the sensors and devices include sensors mounted on the vehicle 14 such as a vehicle speed sensor and an acceleration sensor, a GNSS (Global Navigation Satellite System) device, an in-vehicle communication device, a navigation system, and a radar device. The GNSS device receives GNSS signals from a plurality of GNSS satellites to measure the position of the vehicle 14. The positioning accuracy of the GNSS device improves as the number of GNSS signals that the GNSS device can receive increases. The in-vehicle communication device is a communication device that performs at least one of vehicle-to-vehicle communication with another vehicle 14 and road-to-vehicle communication with a roadside device via the communication unit 26. The navigation system includes a map information storage unit that stores map information, and performs processing such as displaying the position of the vehicle 14 on a map and guiding the route to the destination based on the position information obtained from the GNSS device and the map information stored in the map information storage unit. The radar device includes multiple radars with different detection ranges, detects objects such as pedestrians and other vehicles 14 present around the vehicle 14, and acquires the relative position and relative speed of the detected objects and the vehicle 14. The radar device also includes a processing device that processes the detection results of surrounding objects. The processing device excludes noise, roadside objects such as guardrails, etc. from the monitoring target based on changes in the relative position and relative speed with respect to each object included in the most recent multiple detection results, and tracks and monitors pedestrians, other vehicles 14, etc. as the monitoring target objects. The radar device then outputs information such as the relative position and relative speed with respect to each monitoring target object. In this embodiment, at least the vehicle speed is detected as the vehicle information.

[0024] In this embodiment, the photographing unit 24 has an outward-facing camera that photographs the outside of the vehicle and an inward-facing camera that photographs the inside of the vehicle cabin. The photographing unit 24 photographs the surroundings of the vehicle, such as the front, using the outward-facing camera, and generates outward-facing video data representing the photographed images of the video as image information. The photographing unit 24 also photographs the driver in the vehicle cabin using the inward-facing camera, and generates inward-facing video data representing the photographed images of the video as image information. For example, a drive recorder or the like can be applied as the outward-facing camera, and a driver monitor camera or the like can be applied as the inward-facing camera. The outward-facing camera may further photograph at least one of the surroundings of the vehicle on the side and rear of the vehicle 14.

[0025] The communication unit 26 establishes communication with the cloud server 12 via the communication network 18, and transmits and receives information such as image information obtained by the photographing unit 24 and vehicle information detected by the vehicle information detection unit 22 as CAN (Controller Area Network) data.

[0026] The display unit 28 provides various information to the occupant by displaying information. In the present embodiment, for example, information provided by the cloud server 12 is displayed.

[0027] As shown in FIG. 3, the control unit 20 is composed of a general microcomputer including a CPU (Central Processing Unit) 20A, a ROM (Read Only Memory) 20B, a RAM (Random Access Memory) 20C, a storage 20D, an interface (I / F) 20E, and a bus 20F.

[0028] The control unit 20 controls uploading to the cloud server 12 image information of a moving image showing an image captured by the capture unit 24, and vehicle information detected by the vehicle information detection unit 22 when the image was captured. When uploading the image information and vehicle information, identification information for identifying at least one of the vehicle and the driver is added and transmitted. The information for identifying the driver may be, for example, a captured image of the driver, identification information of a smart key carried by the driver, or other information capable of identifying the driver.

[0029] On the other hand, the cloud server 12 includes a central processing unit 30, a central communication unit 36, and a DB (database) 38.

[0030] As shown in FIG. 3, the central processing unit 30 is configured by a general microcomputer including a CPU 30A, a ROM 30B, a RAM 30C, a storage 30D, an interface (I / F) 30E, and a bus 30F.

[0031] The central processing unit 30 has the functions of a vehicle information collecting unit 44, a video data collecting unit 48, and an estimating unit 50 by the CPU 30A expanding a program stored in the ROM 30B into the RAM 30C and executing it.

[0032] The vehicle information collection unit 44 collects vehicle information detected by each of the vehicle-mounted devices 16 of the multiple vehicles 14, and stores the collected information in the DB 38. As the vehicle information, information detected by various sensors mounted on the vehicles 14 is collected.

[0033] The video data collection unit 48 performs a process of collecting video data captured by each of the vehicle-mounted devices 16 of the multiple vehicles 14 as image information and storing the collected data in the DB 38. In this embodiment, each of the outward-facing video data and the inward-facing video data captured by the imaging unit 24 is stored in the DB 38 as image information.

[0034] The estimation unit 50 acquires at least one of an image of the periphery of the vehicle 14 captured by the image capture unit 24, vehicle information related to the traveling of the vehicle 14 detected by the vehicle information detection unit 22, and an image of the driver captured by the image capture unit 24. The estimation unit 50 also performs a process of estimating a danger sheet as a dangerous situation from the captured image of the periphery of the vehicle, and estimating the cause of the dangerous scene as the dangerous situation from at least one of the vehicle information and the captured image of the driver. For example, the estimation unit 50 performs a process of estimating a dangerous scene, estimating the state of the driver in the dangerous scene, and estimating the vehicle behavior, thereby estimating the risk factor.

[0035] The central communication unit 36 ​​establishes communication with the vehicle-mounted device 16 via the communication network 18, and transmits and receives information such as image information and vehicle information.

[0036] The DB 38 stores the vehicle information collected by the vehicle information collecting unit 44, the video data collected by the video data collecting unit 48, and identification information for identifying at least one of the vehicle and the driver, in association with each other.

[0037] The cloud server 12 performs processes such as detecting dangerous scenes, estimating the causes of dangerous scenes, and evaluating drivers based on the information stored in the DB 38. The cloud server 12 provides various services such as a service of feeding back the driving evaluation results to the driver.

[0038] Next, a detailed description will be given of the functional configuration of the above-mentioned estimation unit 50 in the central processing unit 30 of the cloud server 12. FIG.

[0039] 4, the estimation unit 50 has the functions of an outward video acquisition unit 52, an outward video frame reading unit 54, and a dangerous scene estimation unit 56. The estimation unit 50 also has the functions of an inward video acquisition unit 58, an inward video frame reading unit 60, a driver state estimation unit 62, a vehicle information acquisition unit 64, a vehicle behavior estimation unit 66, and a dangerous driving factor estimation unit 68. The outward video acquisition unit 52, the inward video acquisition unit 58, and the vehicle information acquisition unit 64 correspond to an example of an acquisition unit. The dangerous scene estimation unit 56 corresponds to an example of a risk estimation unit, and the dangerous driving factor estimation unit 68 corresponds to an example of a factor estimation unit.

[0040] The outward facing video acquisition unit 52 acquires outward facing video data captured by an outward facing camera from the image information collected from the vehicle-mounted device 16 and stored in the DB 38. In detail, the outward facing video acquisition unit 52 acquires outward facing video data of a vehicle or a driver of interest.

[0041] The outward moving image frame reading section 54 extracts frame images from the outward moving image data acquired by the outward moving image acquisition section 52.

[0042] The dangerous scene estimation unit 56 determines whether or not the frame image extracted by the outward video frame reading unit 54 corresponds to a dangerous scene, and estimates the dangerous scene. For example, the dangerous scene estimation uses a model determined in advance by machine learning or the like to estimate dangerous scenes such as failure to stop temporarily, obstructing pedestrians, and not maintaining a safe distance between vehicles.

[0043] The inward-facing video acquisition unit 58 acquires inward-facing video data captured by the inward-facing camera from among the captured images collected from the vehicle-mounted device 16 and stored in the DB 38. In detail, the inward-facing video acquisition unit 58 acquires inward-facing video data of a vehicle or driver of interest.

[0044] The inward-facing video frame reading unit 60 extracts a frame image corresponding to the time of the dangerous scene estimated by the dangerous scene estimation unit 56 from the inward-facing video data acquired by the inward-facing video acquisition unit 58. If there is no frame image at the same time as the dangerous scene, a frame image close to the time of the dangerous scene may be extracted.

[0045] Driver state estimation unit 62 estimates the driver state by estimating the driver's facial direction, line of sight, alertness, etc. from the frame images extracted by outward video frame reading unit 54. This makes it possible to determine whether the driver is paying inattention or not from the frame images of the outward video when a dangerous scene is estimated, making it possible to estimate that the driver's inattention at the time a dangerous scene is estimated is a factor.

[0046] The vehicle information acquisition unit 64 acquires vehicle information corresponding to the time of the dangerous scene estimated by the dangerous scene estimation unit 56 from among the vehicle information stored in the DB 38. In detail, the vehicle information acquisition unit 64 acquires vehicle information corresponding to the time of the dangerous scene of the vehicle or driver of interest. Note that, if there is no vehicle information at the same time as the dangerous scene, the vehicle information acquisition unit 64 may acquire vehicle information close to the time of the dangerous scene.

[0047] The vehicle behavior estimation unit 66 estimates vehicle behavior such as sudden acceleration, sudden deceleration, and sudden steering in the dangerous scene estimated by the dangerous scene estimation unit 56 from the vehicle information acquired by the vehicle information acquisition unit 64. As a result, the driving operation status can be known from the vehicle information, so it becomes possible to estimate reckless driving as a cause of the dangerous scene.

[0048] The dangerous driving factor estimation unit 68 estimates the factors of dangerous driving based on the estimation results of the driver state estimation unit 62 and the vehicle behavior estimation unit 66. Specifically, the dangerous driving factor estimation unit 68 estimates the presence or absence of looking away from the driver's line of sight estimated by the driver state estimation unit 62. The estimation method is performed using well-known techniques such as pattern matching and a predetermined machine learning model. The dangerous driving factor estimation unit 68 also estimates the presence or absence of reckless driving such as sudden acceleration, sudden deceleration, and sudden steering by determining whether the deviation between the vehicle information at the time corresponding to the dangerous scene of the vehicle of interest or the vehicle information of the driver and the average value of the vehicle information of other vehicles is equal to or greater than a predetermined threshold. For example, the dangerous driving factor estimation unit 68 estimates the presence or absence of reckless driving such as sudden acceleration, sudden deceleration, and sudden steering when looking away is estimated and normal vehicle behavior is estimated. When the driver state is estimated to be normal and the vehicle behavior is estimated to be sudden acceleration, the dangerous driving factor estimation unit 68 also estimates the presence or absence of reckless driving.

[0049] Next, a description will be given of processing specifically performed by the function of the estimation unit 50 of the central processing unit 30 in the cloud server 12 of the driving evaluation system 10 according to this embodiment configured as described above. Fig. 5 is a flowchart showing an example of the flow of processing performed by the function of the estimation unit 50 of the central processing unit 30 in the cloud server 12 of the driving evaluation system 10 according to this embodiment. Note that the processing in Fig. 5 may be started, for example, at predetermined intervals or when instructed by an operator of the cloud server 12 or the like.

[0050] In step 100, CPU 30A acquires outward facing video, and proceeds to step 102. That is, outward facing video acquisition unit 52 acquires outward facing video data of a vehicle or driver of interest from the outward facing video data stored in DB 38 and captured by an outward facing camera.

[0051] In step 102, the CPU 30A reads an outward moving image or a moving image frame, and proceeds to step 104. That is, the outward moving image frame reading section 54 extracts a frame image of interest from the outward moving image data acquired by the outward moving image acquisition section 52.

[0052] In step 104, CPU 30A estimates a dangerous scene from the frame image of interest, and proceeds to step 106. That is, dangerous scene estimation unit 56 determines whether or not the frame image extracted by outward video frame reading unit 54 corresponds to a dangerous scene, and estimates the dangerous scene. The estimation of the dangerous scene estimates dangerous scenes such as failure to stop temporarily, obstructing pedestrians, and not maintaining a safe distance between vehicles, using a model determined in advance by machine learning or the like.

[0053] In step 106, the CPU 30A determines whether or not there is a dangerous scene. If the determination is affirmative, the process proceeds to step 108, and if not, the process proceeds to step 114.

[0054] In step 108, the CPU 30A executes a driver's state estimating process for estimating the state of the driver, and then proceeds to step 110. The driver's state estimating process will be described in detail later.

[0055] In step 110, the CPU 30A executes a vehicle behavior estimation process for estimating the behavior of the vehicle, and then proceeds to step 112. The vehicle behavior estimation process will be described in detail later.

[0056] In step 112, the CPU 30A executes a risk factor estimation process for estimating a risk factor of a risk scene, and then proceeds to step 114. The risk factor estimation process will be described in detail later.

[0057] In step 114, the CPU 30A judges whether or not there is a next frame. If the judgment is negative, the process returns to step 102 to repeat the above-mentioned processing, and if the judgment is positive, the series of processing is terminated.

[0058] The above-mentioned driver's state estimating process will now be described in detail. Fig. 6 is a flowchart showing an example of the flow of the driver's state estimating process.

[0059] In step 200, CPU 30A acquires an inward-facing video, and proceeds to step 202. That is, inward-facing video acquisition section 58 acquires inward-facing video data of a vehicle or driver of interest from the inward-facing video data stored in DB .

[0060] In step 202, CPU 30A acquires a video frame of inward-facing video data corresponding to an image frame of the outward-facing video data being watched, and proceeds to step 204. That is, inward-facing video frame reading unit 60 extracts a frame image corresponding to the time of the dangerous scene estimated by dangerous scene estimation unit 56, from the inward-facing video data acquired by inward-facing video acquisition unit 58. Note that if there is no frame image at the same time as the dangerous scene, a frame image close to the time of the dangerous scene may be extracted.

[0061] In step 204, CPU 30A estimates the driver's state such as the driver's line of sight and alertness, terminates a series of driver state estimation processes, and proceeds to step 110 in Fig. 5. That is, driver state estimation unit 62 estimates the driver's state by estimating the driver's facial direction, line of sight, alertness, etc. from the frame images extracted by outward video frame reading unit 54, and estimates whether the driver is looking away from the road from the direction of the driver's line of sight, etc.

[0062] Next, the above-mentioned vehicle behavior estimation process will be described in detail with reference to a flowchart of FIG 7.

[0063] In step 300, the CPU 30A acquires vehicle information, and the process proceeds to step 302. That is, the vehicle information acquisition unit 64 acquires, from the vehicle information stored in the DB 38, the vehicle information of the noted vehicle or driver.

[0064] In step 302, CPU 30A acquires vehicle information corresponding to a frame image of the outward video data, and proceeds to step 304. That is, vehicle information acquisition unit 64 acquires, from the acquired vehicle information, vehicle information corresponding to the time of the dangerous scene estimated by dangerous scene estimation unit 56. Note that, when there is no vehicle information at the same time as the dangerous scene time, vehicle information acquisition unit 64 may acquire vehicle information close to the time of the dangerous scene.

[0065] In step 304, the CPU 30A estimates vehicle behavior such as sudden acceleration, sudden deceleration, and sudden steering, ends the vehicle behavior estimation process, and proceeds to step 112 in Fig. 5. That is, the vehicle behavior estimation unit 66 extracts vehicle information at a time corresponding to a dangerous scene from the vehicle information of the noted vehicle or driver, calculates an average value of the vehicle information of other vehicles, and determines whether the deviation from the vehicle information of the noted vehicle is equal to or greater than a predetermined threshold. In this way, vehicle behavior such as sudden acceleration, sudden deceleration, and sudden steering is estimated.

[0066] Next, the risk factor estimation process will be described in detail. In this embodiment, the risk factor estimation process includes a lack of attention determination process for determining whether the driver is inattentive, and a reckless driving determination process for determining whether the driver is driving recklessly.

[0067] First, the insufficient attention determination process will be described in detail with reference to a flowchart of FIG 8, which shows an example of the flow of the insufficient attention determination process.

[0068] In step 400, CPU 30A maps the gaze direction within the outward image frame, and proceeds to step 402. Specifically, dangerous driving factor estimation unit 68 identifies the gaze direction area in the frame image by matching the position of the driver's gaze direction estimated by vehicle behavior estimation unit 66 with the frame image of the outward video of interest. In the example of the frame image shown in Fig. 9, a dotted rectangular area is identified as the gaze direction area.

[0069] In step 402, CPU 30A creates a risk heat map, and proceeds to step 404. Specifically, dangerous driving factor estimation unit 68 evaluates the risk of a frame image of the focused outward video, creates a risk heat map, and identifies an area that is above a predetermined threshold. In the example of a frame image shown in Fig. 9, a rectangular area indicated by a solid line in the frame image is identified as an area that is above the threshold.

[0070] In step 404, the CPU 30A calculates the degree of dangerous attention and proceeds to step 406. Specifically, the dangerous driving factor estimation unit 68 calculates the IoU (Intersection over Union) of the region in the line of sight and the region where the danger level is equal to or higher than the threshold. That is, an index is calculated that indicates the degree of overlap between the region in the line of sight and the region where the danger level is equal to or higher than the threshold. For example, as shown in FIG. 9, the degree of dangerous attention is calculated by dividing the region where the region in the line of sight and the region where the danger level is equal to or higher than the threshold overlap by the region including both the line of sight and the region where the danger level is equal to or higher than the threshold. As a result, in the case of insufficient attention, since the driver does not look in the direction of the dangerous scene, when the driver's line of sight does not overlap with the region of the danger level heat map that is equal to or higher than the predetermined threshold, it is possible to estimate that the driver's inattention is the cause.

[0071] In step 406, the CPU 30A judges whether the dangerous attention degree is equal to or greater than a threshold. This judgement is made by the dangerous driving factor estimation unit 68 judging whether the IoU value calculated in step 404 is equal to or less than a predetermined threshold. If the judgement is negative, the attention insufficiency judgement process is terminated and the process proceeds to step 114 in FIG. 5, whereas if the judgement is positive, the process proceeds to step 408.

[0072] In step 408, CPU 30A determines that the driver is looking away and that the driver is not paying attention, ends the inattention determination process, and proceeds to step 114 in Fig. 5. That is, dangerous driving factor estimation unit 68 determines that the driver is looking away and that the driver is not paying attention, as a cause of the dangerous scene.

[0073] Next, the reckless driving determination process will be described in detail with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the flow of the reckless driving determination process.

[0074] In step 500, the CPU 30A calculates an average sensor value at the dangerous scene observation point based on the position information, and proceeds to step 502. In detail, the dangerous driving factor estimation unit 68 calculates an average sensor value of the other vehicles 14 at the dangerous scene observation point based on the position information of the target vehicle.

[0075] In step 502, the CPU 30A judges whether the difference between the sensor value of the target vehicle and the average sensor value of the other vehicles is equal to or greater than a predetermined threshold value. If the judgment is negative, the aggressive driving judgment process is terminated and the process proceeds to step 114 in FIG. 5. If the judgment is positive, the process proceeds to step 504.

[0076] In step 504, the CPU 30A determines that the driving has been reckless, ends the reckless driving determination process, and proceeds to step 114 in Fig. 5. That is, the dangerous driving factor estimating unit 68 determines that reckless driving is a factor in the dangerous scene.

[0077] By performing the processing in this manner, the cloud server 12 can estimate a dangerous scene as a dangerous situation, and estimate the causes of the dangerous scene, such as lack of attention by the driver or reckless driving.

[0078] In addition, by being able to infer the causes of dangerous situations, it will be possible to suggest ways to improve driving.

[0079] In the above embodiment, the cloud server 12 acquires the vehicle information and the inward-facing video frames to estimate the cause of the dangerous situation, but the present invention is not limited to this. For example, the cloud server 12 may acquire the vehicle information or the inward-facing video frames and perform the above-mentioned lack of attention determination process or the reckless driving determination process.

[0080] In the above embodiment, the detection of a dangerous scene is performed on the cloud server 12 side, but the present invention is not limited to this. For example, the vehicle-mounted device 16 may detect a dangerous scene and transmit the detection result to the cloud server 12, or another server may detect a dangerous scene and transmit the detection result to the cloud server 12.

[0081] In addition, the processing performed by the cloud server 12 in each of the above embodiments has been described as software processing performed by executing a program, but is not limited to this. For example, the processing may be performed by hardware such as a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field-Programmable Gate Array). Alternatively, the processing may be a combination of both software and hardware. In addition, when the processing is performed by software, the program may be stored in various storage media and distributed.

[0082] Furthermore, the present invention is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0083] 10 Driving Evaluation System 12 Cloud Server 14 Vehicles 16 Onboard equipment 22 Vehicle information detection unit 24 Photography Department 30 Central Processing Unit 38 DB 44 Vehicle Information Collection Department 48 Video Data Collection Department 50 Estimation part 52 Outward video acquisition unit (acquisition unit) 54 Outgoing video frame reader 56 Danger scene estimation unit (danger estimation unit) 58 Inward Video Acquisition Unit (Acquisition Unit) 60 Inward video frame reader 62 Driver state estimation unit 64 Vehicle information acquisition unit (acquisition unit) 66 Vehicle behavior estimation unit 68 Dangerous driving factor estimation unit (factor estimation unit)

Claims

1. an acquisition unit that acquires at least one of a surrounding captured image captured from the vehicle around the vehicle, vehicle information related to the traveling of the vehicle, and a driver captured image captured of the driver; a risk estimation unit that estimates a dangerous situation from the surrounding photographed image acquired by the acquisition unit; a factor estimation unit that estimates a factor of the dangerous situation estimated by the risk estimation unit from at least one of the two acquired by the acquisition unit; Including, The acquisition unit acquires the surrounding captured image and the vehicle information, The factor estimation unit is a risk factor estimation device that estimates aggressive driving as the factor by determining whether the deviation between the vehicle information at the time the dangerous situation is estimated and the average value of the vehicle information of other vehicles is greater than or equal to a predetermined threshold.

2. The risk factor estimation device according to claim 1 , wherein the factor estimation unit estimates, as the factor, insufficient attention of the driver from the image of the driver captured at the time when the dangerous situation is estimated.

3. 3. The risk factor estimation device according to claim 2, wherein the factor estimation unit estimates that the factor is lack of attention when the gaze direction determined from the driver's captured image does not overlap with an area of ​​a risk heat map determined by evaluating the risk level for the surrounding captured image that is equal to or greater than a predetermined threshold.

4. A risk factor estimation device according to any one of claims 1 to 3, a vehicle that transmits the surrounding photographed image and the vehicle information, or the surrounding photographed image, the vehicle information, and the driver photographed image to the risk factor estimation device; A hazard estimation system including:

5. On the computer, Acquire at least one of a surrounding image captured from the vehicle and vehicle information related to the running of the vehicle and a driver image captured of the driver; A process of estimating a dangerous situation from the surrounding captured image and estimating a cause of the dangerous situation from at least one of the above, Acquire the surrounding photographed image and the vehicle information, A risk factor estimation program for executing a process of estimating reckless driving as the cause by determining whether the deviation between the vehicle information at the time the dangerous situation is estimated and the average value of the vehicle information of other vehicles is greater than or equal to a predetermined threshold.

Images