Driving assistance device, driving assistance method, and recording medium

Abstract

A driving assistance device, a driving assistance method and a recording medium are provided. The driving assistance device (1) comprises: a controlled vehicle movement information acquisition unit (11) for acquiring the controlled vehicle movement information; a peripheral object movement information acquisition unit (12) for acquiring the peripheral object movement information; a risk occurrence time prediction unit (13) for predicting the risk occurrence time when a predetermined risk occurs due to the approach of the controlled vehicle (100) and the peripheral object (120) based on the controlled vehicle movement information and the peripheral object movement information; a driver state recognition unit (14) for recognizing the state of the driver of the controlled vehicle (100); and a caution reminder control unit for determining the execution time of the caution reminder and the content of the caution reminder based on the remaining time until the risk occurrence time and the driver's state recognized by the driver state recognition unit (14).

Description

Technical Field

[0001] The present invention relates to a driving assistance device, a driving assistance method and a recording medium. Background Art

[0002] In the past, the following vehicle-mounted surrounding condition warning device has been proposed: based on the monitoring range of the camera and radar device of the vehicle, the blind spot range that is blocked by other vehicles and cannot be monitored is calculated, and the blind spot range is displayed on the map image to remind the driver to pay attention (for example, refer to Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2007-233770 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, the above-mentioned conventional devices remind the driver to pay attention based on the surrounding conditions of the vehicle detected by the camera and radar device of the vehicle. Therefore, if the vehicle side does not recognize the behavior of other vehicles and pedestrians moving around the vehicle, it may not be possible to remind the driver to pay attention at an appropriate time.

[0008] The present application aims to solve the above-mentioned problems, and its purpose is to provide a driving assistance device, a driving assistance method and a recording medium that can remind the driver of objects around the vehicle through appropriate content at an appropriate time, thereby improving safety. Moreover, it can further improve traffic safety and contribute to the development of a sustainable transportation system.

[0009] Means for solving problems

[0010] As a first method for achieving the above-mentioned purpose, there can be cited a driving assistance device, which comprises: a controlled vehicle movement information acquisition unit, which acquires controlled vehicle movement information including position information of the controlled vehicle for a controlled vehicle; a surrounding object movement information acquisition unit, which acquires surrounding object movement information including position information of surrounding objects existing in the periphery of the controlled vehicle; a risk occurrence time prediction unit, which predicts the risk occurrence time when a specified risk occurs due to the proximity of the controlled vehicle to the surrounding objects based on the controlled vehicle movement information and the surrounding object movement information; a driver state recognition unit, which recognizes the state of the driver of the controlled vehicle; and an attention reminder control unit, which determines the execution timing of the attention reminder performed by the reporting device of the controlled vehicle and the content of the attention reminder based on the remaining time until the risk occurrence time and the state of the driver recognized by the driver state recognition unit.

[0011] In the above-mentioned driving assistance device, it can also be configured that the driver state recognition unit recognizes the direction of the driver's line of sight as the driver's state, and the surrounding object movement information includes information indicating whether the controlled vehicle is recognized at the surrounding object. When it can be judged that the controlled vehicle is not recognized at the surrounding object based on the surrounding object movement information, the attention reminder control unit also executes the attention reminder even if the driver state recognition unit recognizes that the driver's line of sight is directed toward the surrounding object.

[0012] In the driving assistance device described above, the attention calling control unit may be configured to increase the level of the attention calling as the remaining time until the occurrence time of the predetermined risk becomes shorter.

[0013] In the above-mentioned driving assistance device, it can also be configured that the driver state recognition unit recognizes the driving state of the driver driving the controlled vehicle as the state of the driver, and the attention reminder control unit does not execute the attention reminder when it can be judged that the driver is in a state capable of avoiding the specified risk based on the driving state of the driver driving the controlled vehicle recognized by the driver state recognition unit.

[0014] In the above-mentioned driving assistance device, it can also be configured that the driver state recognition unit recognizes the health state of the driver and the direction of the driver's line of sight. When the driver state recognition unit recognizes that the driver's line of sight is directed toward the surrounding object, the attention reminder control unit decides whether to perform the attention reminder based on the health state of the driver recognized by the driver state recognition unit.

[0015] As a second method for achieving the above-mentioned purpose, a driving assistance method executed by a computer is cited, comprising: a controlled vehicle movement information acquisition step, for a controlled vehicle, acquiring the controlled vehicle movement information including the position information of the controlled vehicle; a surrounding object movement information acquisition step, for a surrounding object existing in the periphery of the controlled vehicle, acquiring the surrounding object movement information including the position information of the surrounding object; a risk occurrence time prediction step, for predicting the risk occurrence time when a specified risk occurs due to the proximity of the controlled vehicle to the surrounding object based on the controlled vehicle movement information and the surrounding object movement information; a driver state recognition step, for recognizing the state of the driver of the controlled vehicle; and an attention reminder control step, for determining the execution timing of the attention reminder executed by the reporting device of the controlled vehicle and the content of the attention reminder based on the remaining time until the risk occurrence time and the state of the driver recognized by the driver state recognition step.

[0016] As a third scheme for achieving the above-mentioned purpose, a non-volatile recording medium is cited, which records a program executed by a computer, wherein the program causes the computer to function as the following parts: a controlled vehicle movement information acquisition unit, which acquires controlled vehicle movement information including position information of the controlled vehicle for a controlled vehicle; a surrounding object movement information acquisition unit, which acquires surrounding object movement information including position information of surrounding objects existing in the periphery of the controlled vehicle; a risk occurrence time prediction unit, which predicts the risk occurrence time when a specified risk occurs due to the proximity of the controlled vehicle to the surrounding objects based on the controlled vehicle movement information and the surrounding object movement information; a driver state recognition unit, which recognizes the state of the driver of the controlled vehicle; and an attention reminder control unit, which determines the execution time of the attention reminder performed by the reporting device of the controlled vehicle and the content of the attention reminder based on the remaining time until the risk occurrence time and the state of the driver recognized by the driver state recognition unit.

[0017] Effects of the Invention

[0018] According to the driving assistance device, it is possible to alert the driver to objects existing around the host vehicle with appropriate content and at an appropriate timing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a structural diagram of a vehicle equipped with a driving assistance device.

[0020] Figure 2 This is a structural diagram of a driving assistance device.

[0021] Figure 3 This is a first flowchart of the driving assistance process.

[0022] Figure 4 This is a second flowchart of the driving assistance process.

[0023] Figure 5 This is the third flowchart of the driving assistance process.

[0024] Figure 6 The following is an explanatory diagram of a driving assistance process when the host vehicle turns right at an intersection.

[0025] Figure 7 This is an explanatory diagram of the timing of executing a caution warning corresponding to a right turn at an intersection and the content of the caution warning.

[0026] Figure 8 This is an explanatory diagram of an execution status of a caution warning corresponding to a right turn at an intersection.

[0027] Fig. 9 This is an explanatory diagram of the timing of executing a caution reminder and the content of the caution reminder corresponding to driving on a main road.

[0028] Description of Reference Numerals

[0029] 1: driving assistance device, 10: processor, 11: controlled vehicle movement information acquisition unit, 12: surrounding object movement information acquisition unit, 13: risk occurrence time prediction unit, 14: driver state recognition unit, 15: attention reminder control unit, 16: ADAS control unit, 20: memory, 21: program, 30: communication unit, 31: vehicle control device, 32: navigation device, 33: GNSS sensor, 34: map data, 40: front camera, 41: rear camera, 42: front radar, 43: rear radar, 44: foot camera, 45: driver seat camera, 46: speed sensor, 47: vital sign sensor, 48: grip sensor, 49: pedal contact sensor, 50: display, 51: speaker, 52: seat vibration device, 100: controlled vehicle (this vehicle), 110, 120: other vehicles, 200: communication network, 210: traffic information server, D: driver. DETAILED DESCRIPTION

[0030] [1. Vehicle structure]

[0031] Reference Figure 1 Next, a structure of a controlled vehicle 100 equipped with the driving assistance device 1 according to the present embodiment will be described. The controlled vehicle 100 is a four-wheeled vehicle, and hereinafter, the controlled vehicle 100 is referred to as the host vehicle 100 .

[0032] The front portion of the vehicle 100 is provided with a front camera 40 for photographing the front of the vehicle 100 and a front radar 42 for detecting the position of an object existing in front of the vehicle 100. The rear portion of the vehicle 100 is provided with a rear camera 41 for photographing the rear of the vehicle 100 and a rear radar 43 for detecting the position of an object existing in the rear of the vehicle 100.

[0033] The vehicle 100 has a display 50 corresponding to the notification device of the present disclosure, a speaker 51 , a seat vibration device 52 , and a driver's seat camera 45 for photographing the driver D in the vehicle cabin. The vehicle 100 also has a communication unit 30 , a vehicle control device 31 , and a navigation device 32 .

[0034] The communication unit 30 communicates with a server or the like outside the vehicle 100. The vehicle control device 31 controls the operation of the vehicle 100. The navigation device 32 includes a GNSS (Global Navigation Satellite System) sensor 33 for detecting the position of the vehicle 1 by receiving radio waves transmitted from a plurality of navigation satellites to the ground, and map data 34, and performs route guidance to a destination, etc.

[0035] [2. Structure of driving assistance device]

[0036] Reference Figure 2 , the structure of the driving assistance device 1 will be described. The driving assistance device 1 is a control unit composed of a processor 10, a memory 20, an interface circuit (not shown), and the like.

[0037] The driving assistance device 1 is connected to the front camera 40, the rear camera 41, the front radar 42, the foot camera 44, the driver's seat camera 45, the speed sensor 46, the vital sign sensor 47, the grip sensor 48, the pedal contact sensor 49, the display 50, the speaker 51, the seat vibration device 52, the vehicle control device 31 and the navigation device 32 of the vehicle 100.

[0038] The foot camera 44 captures the image of the driver D (see Figure 1 ) and inputs the captured images of the operation status of the driver D on the brake pedal and the accelerator pedal to the driving assistance device 1. The driver's seat camera 45 captures the face of the driver D and inputs the captured images to the driving assistance device 1. The speed sensor 46 detects the driving speed of the vehicle 100 and inputs the speed detection signal to the driving assistance device 1. The vital sign sensor 47 detects the biological information (heart rate, pulse wave, body temperature, blood pressure, respiratory rate, etc.) of the driver D and inputs the detection signal to the driving assistance device 1.

[0039] The grip sensor 48 detects the grip of the steering wheel by the driver D, and inputs the detection signal to the driving assistance device 1. The pedal contact sensor 49 detects the contact between the driver D's foot and the brake pedal, and inputs the detection signal to the driving assistance device 1. The display 50 displays various information according to the control signal input from the driving assistance device 1. The display 50 is, for example, a flat panel display such as a liquid crystal display and an organic EL display. In addition, the display 50 may also be a HUD (Head-Up Display). The speaker 51 amplifies and outputs the sound signal output from the driving assistance device 1.

[0040] The seat vibration device 52 is installed on the driver's seat and vibrates according to the control signal input from the driving assistance device 1, thereby alerting the driver D. The vehicle control device 31 performs driving assistance control based on ADAS (Advanced Driving Assistant System) based on the images captured by the front camera 40 and the rear camera 41 and the position detection signals of the front radar 42 and the rear radar 43. ADAS includes collision mitigation braking, pedestrian accident mitigation steering, etc.

[0041] The driving assistance device 1 communicates with the traffic information server 210 via the communication network 200 through the communication unit 30. The traffic information server 210 receives the movement information (including the position, speed, driving condition, etc.) sent from a plurality of vehicles and portable terminals registered in the traffic information server 210 in real time and records it in the object DB (database) 211. Figure 2 , an example is shown in which the traffic information server 210 receives movement information transmitted from a portable terminal 131 (smartphone, mobile phone, tablet terminal, etc.) of a four-wheeled vehicle 110, a two-wheeled vehicle 120, and a pedestrian 130. The four-wheeled vehicle 110, the two-wheeled vehicle 120, and the pedestrian 130 correspond to the surrounding objects of the present disclosure.

[0042] In addition, the traffic information server 210 receives and obtains captured images sent from a surveillance camera 150 installed on the roadside, analyzes the captured images, identifies objects existing near the installation location of the surveillance camera 150, and records the information of the identified objects in the object DB 211.

[0043] The processor 10 functions as a controlled vehicle movement information acquisition unit 11, a surrounding object movement information acquisition unit 12, a risk occurrence time prediction unit 13, a driver state recognition unit 14, a caution reminder control unit 15, and an ADAS control unit 16 by reading and executing a program 21 stored in a memory 20. The program 21 may be read by the driving assistance device 1 from a recording medium (a CD, a flash memory, etc.) and stored in the memory 20, or may be downloaded by the driving assistance device 1 from an external server not shown and stored in the memory 20.

[0044] The processing performed by the controlled vehicle movement information acquisition unit 11 is equivalent to the controlled vehicle movement information acquisition step in the driving assistance method of the present disclosure, and the processing performed by the surrounding object movement information acquisition unit 12 is equivalent to the surrounding object movement information acquisition step in the driving assistance method of the present disclosure. The processing performed by the risk occurrence time prediction unit 13 is equivalent to the risk occurrence time prediction step in the driving assistance method of the present disclosure, and the processing performed by the driver state recognition unit 14 is equivalent to the driving state recognition step in the driving assistance method of the present disclosure. The processing performed by the attention reminder control unit 15 is equivalent to the attention reminder control step in the driving assistance method of the present disclosure.

[0045] The controlled vehicle movement information acquisition unit 11 acquires controlled vehicle movement information including the position of the host vehicle 100 detected by the GNSS sensor 33 and the speed of the host vehicle 100 detected by the speed sensor 46. The peripheral object movement information acquisition unit 12 transmits the host vehicle position information Cpi indicating the position of the host vehicle 100 to the traffic information server 210, and requests the traffic information server 210 to transmit the peripheral object movement information Evi including the position and speed of the object (peripheral object) existing around the host vehicle 100, thereby acquiring the peripheral object movement information Evi. In addition, when the speed of the object is calculated based on the change in the position of the object, the speed does not need to be included in the peripheral object movement information Evi.

[0046] The risk occurrence time prediction unit 13 determines the possibility of a specified risk (emergency stop, contact, etc.) occurring due to the proximity of the vehicle 100 to the surrounding objects based on the movement information of the controlled vehicle, the movement information of the surrounding objects, and the map data 34, and predicts the time when the specified risk will occur (risk occurrence time) if there is a possibility of the specified risk occurring.

[0047] The driver state recognition unit 14 recognizes the driving state of the driver D driving the vehicle 100 based on the captured image of the foot camera 44, the captured image of the driver seat camera 45, the detection signal of the grip sensor 48, and the detection signal of the pedal contact sensor 49. In addition, the driver state recognition unit 14 recognizes the health state of the driver D based on the captured image of the driver seat camera 45 and the detection signal of the vital sign sensor 47.

[0048] When the risk occurrence time prediction unit 13 recognizes the occurrence of a predetermined risk and predicts the risk occurrence time, the attention reminder control unit 15 determines the execution timing of reminding the driver D to pay attention and the content of the attention reminder based on the remaining time until the risk occurrence time, the driving state of the driver D, the health state of the driver D, the state of the surrounding objects, etc. The ADAS control unit 16 performs ADAS control by outputting a control signal to the vehicle control device 31.

[0049] [3. Attention reminder processing]

[0050] according to Figure 3 to Figure 5 The flowchart shown in FIG. 1 illustrates the execution procedure of the attention calling process executed by the driving assistance device 1 .

[0051] exist Figure 3 In step S1, the controlled vehicle movement information acquisition unit 11 acquires the controlled vehicle movement information based on the detection signal of the GNSS sensor 33, the detection signal of the speed sensor 46, and the map data 34. In the next step S2, the surrounding object movement information acquisition unit 12 accesses the traffic information server 210 to acquire the surrounding object movement information related to the surrounding objects existing around the host vehicle 100.

[0052] In the next step S3, the risk occurrence time prediction unit 13 determines whether there is a predetermined risk that may occur due to the proximity of the host vehicle 100 to the surrounding objects based on the movement information of the controlled vehicle and the movement information of the surrounding objects. In the next step S4, the risk occurrence time prediction unit 13 advances the processing to step S5 if there is a predetermined risk, and advances the processing to step S1 if there is no predetermined risk.

[0053] In step S5, the risk occurrence time prediction unit 13 predicts the risk occurrence time of the host vehicle 100 contacting the surrounding objects based on the position and speed of the host vehicle 100 identified from the controlled vehicle movement information and the position and speed of the surrounding objects identified from the surrounding object movement information. In the next step S6, the controlled vehicle movement information acquisition unit 11 acquires the controlled vehicle movement information, and in step S7, the surrounding object movement information acquisition unit 12 acquires the surrounding object movement information.

[0054] In the next step S8, the attention reminder control unit 15 calculates the remaining time until the risk occurs based on the movement information of the controlled vehicle and the movement information of the surrounding objects. In the next step S9, the attention reminder control unit 15 determines whether the remaining time until the risk occurs is more than 10 seconds. If the remaining time is more than 10 seconds, the process proceeds to step S6. If the remaining time is less than 10 seconds, the process proceeds to step S10.

[0055] In step S10, the driver state recognition unit 14 obtains the driver state information. Here, the driver state recognition unit 14 recognizes the direction of the driver D's line of sight from the image of the driver D captured by the driver's seat camera 45. In the next step S11, the attention reminder control unit 15 determines whether the driver D is visually checking the surrounding objects based on the direction of the driver D's line of sight. Then, the attention reminder control unit 15 enters the process when the driver D is visually checking the surrounding objects. Figure 4 In step S12, no reminder is given in this case.

[0056] On the other hand, when the driver D is not visually checking surrounding objects, the attention reminder control unit 15 advances the processing to step S30 to execute the attention reminder, in which static location information (such as a right turn at an intersection described later) is reported, and the processing advances to step S40. Figure 4 Specifically, the attention reminder control unit 15 reminds the driver D to pay attention by displaying the attention reminder on the display 50, outputting the attention reminder sound from the speaker 51, and operating the seat vibration device 52. In addition, the driver D may be reminded to pay attention by any one or both of the display of the attention reminder on the display 50, the output of the attention reminder sound from the speaker 51, and the operation of the seat vibration device 52.

[0057] exist Figure 4 In step S12, the controlled vehicle movement information acquisition unit 11 acquires the controlled vehicle movement information, and in step S13, the surrounding object movement information acquisition unit 12 acquires the surrounding object movement information. In the next step S14, the attention reminder control unit 15 calculates the remaining time until the risk occurs based on the controlled vehicle movement information and the surrounding object movement information. In the next step S15, the attention reminder control unit 15 determines whether the remaining time until the risk occurs is less than 10 seconds. If the remaining time is less than 10 seconds, the process proceeds to step S16, and if the remaining time is more than 10 seconds, the process proceeds to step S12.

[0058] In step S16, the driver state recognition unit 14 obtains the driver state information. Here, the driver state recognition unit 14 also recognizes the direction of the driver D's line of sight based on the image of the driver D captured by the driver's seat camera 45. In the next step S17, the attention reminder control unit 15 determines whether the driver D is visually checking the surrounding objects based on the direction of the driver D's line of sight. Then, the attention reminder control unit 15 enters the processing when the driver D is visually checking the surrounding objects. Figure 5 In step S18, no warning is given in this case.

[0059] On the other hand, when the driver D is not visually checking the surrounding objects, the attention reminder control unit 15 causes the processing to enter step S40, executes the attention reminder, in which the dynamic location information (such as the blind spot of the oncoming vehicle when turning right at the intersection described later) is reported, and causes the processing to enter step S41. Figure 5 Step S18.

[0060] exist Figure 5 In step S18, the controlled vehicle movement information acquisition unit 11 acquires the controlled vehicle movement information, and in step S19, the surrounding object movement information acquisition unit 12 acquires the surrounding object movement information. In the next step S20, the attention reminder control unit 15 calculates the remaining time until the risk occurs based on the controlled vehicle movement information and the surrounding object movement information. In the next step S21, the attention reminder control unit 15 determines whether the remaining time until the risk occurs is less than 5 seconds. If the remaining time is less than 5 seconds, the processing proceeds to step S22. If the remaining time is more than 5 seconds, the processing proceeds to step S18.

[0061] In step S22, the driver state recognition unit 14 obtains the driver state information. Here, the driver state recognition unit 14 also recognizes the direction of the driver D's line of sight based on the image of the driver D captured by the driver's seat camera 45. In the next step S23, the attention reminder control unit 15 determines whether the driver D is visually checking the surrounding objects based on the direction of the driver D's line of sight. Then, the attention reminder control unit 15 causes the process to enter S24 when the driver D is visually checking the surrounding objects, and in this case, no attention reminder is performed.

[0062] On the other hand, when the driver D is not visually confirming the surrounding objects, the attention reminder control unit 15 causes the process to enter step S50, performs attention reminder, and reports the identification of the object (such as a straight-moving two-wheeled vehicle when turning right at an intersection described later), and causes the process to enter step S24. In step S24, the ADAS control unit 16 appropriately performs ADAS control such as collision mitigation actions based on the distance between the host vehicle 100 and the surrounding objects recognized from the captured image of the front camera 40 and the position detection signal of the front radar 42.

[0063] [4. Example of how to implement the reminder]

[0064] Reference Figure 6 to Figure 8 , a method of the attention reminder processing executed by the driving assistance device 1 when the host vehicle 100 turns right at an intersection is described. Figure 6 The vehicle 100 is shown to be traveling along C1 → C2 → C3 and turning right at the intersection. In the opposite lane, other vehicles 110a and 110b are stopped, and the two-wheeled vehicle 120 is moving straight. In addition, pedestrians 130a and 130b are present on the road after the right turn.

[0065] exist Figure 6 In the situation shown, the attention reminder control unit 15 follows Figure 7 The setting conditions shown are used to call the attention of the driver D of the vehicle 100. Figure 6 In the illustrated situation, the contact between the host vehicle 100 and the two-wheeled vehicle 120 at the intersection is recognized as a predetermined risk, and the predicted time when the host vehicle 100 and the two-wheeled vehicle 120 arrive at the intersection is predicted as the risk occurrence time by the risk occurrence time prediction unit 13.

[0066] exist Figure 7 In the setting conditions, the aggravating conditions for determining whether to execute the attention reminder are set by AND (combination) with the conditions of the remaining time until the risk occurs (more than 10 seconds, less than 10 seconds, less than 5 seconds). That is, in the state of C1 where the remaining time is more than 10 seconds, the attention reminder is executed in the case of "the driver neglects to monitor the surroundings (the driver's line of sight is not directed towards the surrounding objects)". In addition, in the state of C2 where the remaining time is less than 10 seconds, the attention reminder is executed in the case of "there is a car waiting to turn right in the opposite lane and there is a two-wheeled vehicle going straight". In addition, in the state of C3 where the remaining time is less than 5 seconds, the attention reminder is executed in the case of "a two-wheeled vehicle going straight".

[0067] Figure 8 The time axis t shows the Figure 7 The setting conditions of t1 to t3 are related to the situation in which the driver D of the vehicle 100 is reminded of his attention. Figure 6 , Figure 7 According to the remaining time until the occurrence time of the specified risk, that is, t4, a reminder of "please turn right at the next intersection" is given at t1, a reminder of "please pay attention to the blind spot of the oncoming car" is given at t2, and a reminder of "motorcycle is coming" is given at t3.

[0068] then, Fig. 9 The setting conditions of the attention reminder corresponding to the situation where the vehicle 100 is traveling straight on the main road are shown. In this case, the contact between the vehicle 100 and other vehicles at the connection position of the main road and the auxiliary road is identified as a specified risk, and the predicted time when the vehicle 100 reaches the connection position of the main road and the auxiliary road is predicted by the risk occurrence time prediction unit 13 as the risk occurrence time.

[0069] exist Fig. 9 In the setting conditions of , the aggravating condition for determining whether to execute the attention reminder is set by ANDing the remaining time (more than 10 seconds, less than 10 seconds, less than 5 seconds) until the risk occurs. That is, in the state of C1 where the remaining time is more than 10 seconds, the attention reminder is executed in the case of "the driver neglects to monitor the surroundings (the driver's line of sight is not directed toward the surrounding objects)". In addition, in the state of C2 where the remaining time is less than 10 seconds, the attention reminder is executed in the case of "the brakes of other vehicles parked on the auxiliary road are released". The situation where the brakes of other vehicles parked on the auxiliary road are released is identified based on the movement information of the surrounding objects related to other vehicles.

[0070] [5. Other embodiments]

[0071] In the above embodiment, when it is recognized that the driver D of the vehicle 100 is looking toward a peripheral object, it is determined that there is no need to call the attention of the driver D. It is possible to determine that the driver D is recognizing the peripheral object by other determination factors such as the following determination factors 1 to 3.

[0072] Judgment factor 1: The grip sensor 48 detects that the driver D is gripping the steering wheel, and the pedal contact sensor 49 detects that the driver D has placed his foot on the brake pedal.

[0073] Judgment factor 2: When it is known from the history of images captured by the foot camera 44 that the driver D normally places his feet on the floor and steps on the brake pedal just before braking, it is recognized from the images captured by the foot camera 44 that the driver D moves his feet from the floor to the brake pedal.

[0074] Judgment factor 3: For the initial reminder ( Figure 3In step S30 of the caution reminder), the speed sensor 46 detects a deceleration of a predetermined level or more.

[0075] In the above-described embodiment, when it is recognized that the sight line of the driver D of the host vehicle 100 is directed toward a surrounding object, it is determined that there is no need to remind the driver D of his attention. As another embodiment, when it is estimated that the attention of the driver D is reduced based on the health status of the driver D recognized by the driver state recognition unit 14, even when it is recognized that the sight line of the driver D of the host vehicle 100 is directed toward a surrounding object, the attention reminder may be performed.

[0076] For example, the stress, fatigue, and psychological state of the driver D are estimated based on the physical information of the driver D detected by the vital sign sensor 47 or the behavior of the driver D captured by the driver's seat camera 45, thereby judging whether the driver D's attention has been reduced.

[0077] It should be noted that Figure 1 , Figure 2 This is a schematic diagram that shows the structure of the vehicle 100 and the driving assistance device 1 according to the main processing content in order to facilitate understanding of the present invention. The structure of the vehicle 100 and the driving assistance device 1 can also be configured by other division methods. In addition, the processing of each component can be performed by one hardware unit or by multiple hardware units. In addition, Figure 3 , Figure 4 The processing of each component of the flowchart shown may be executed by one program or by a plurality of programs.

[0078] [6. Structure supported by the above-mentioned embodiment]

[0079] The above-mentioned embodiment is a specific example of the following structure.

[0080] (Structure 1) A driving assistance device, comprising: a controlled vehicle movement information acquisition unit, which acquires controlled vehicle movement information including position information of the controlled vehicle for a controlled vehicle; a surrounding object movement information acquisition unit, which acquires surrounding object movement information including position information of surrounding objects existing in the periphery of the controlled vehicle; a risk occurrence time prediction unit, which predicts a risk occurrence time when a specified risk occurs due to the proximity of the controlled vehicle to the surrounding objects based on the controlled vehicle movement information and the surrounding object movement information; a driver state recognition unit, which recognizes the state of the driver of the controlled vehicle; and a caution reminder control unit, which determines the execution timing of a caution reminder by a reporting device of the controlled vehicle and the content of the caution reminder based on the remaining time until the risk occurrence time and the state of the driver recognized by the driver state recognition unit.

[0081] According to the driving assistance device of Configuration 1, it is possible to call the driver's attention to objects existing around the controlled vehicle (host vehicle) with appropriate content and at an appropriate timing.

[0082] (Structure 2) A driving assistance device according to Structure 1, wherein the driver state recognition unit recognizes the direction of the driver's line of sight as the driver's state, and the surrounding object movement information includes information indicating whether the controlled vehicle is recognized at the surrounding object, and when it can be determined based on the surrounding object movement information that the controlled vehicle is not recognized at the surrounding object, the attention reminder control unit also performs the attention reminder even if the driver state recognition unit recognizes that the driver's line of sight is toward the surrounding object.

[0083] According to the driving assistance device of structure 2, when the controlled vehicle is not recognized by surrounding objects and the behavior of the surrounding objects cannot be predicted, the driver is reminded to pay attention even when the driver of the controlled vehicle is visually confirming the surrounding objects, thereby improving the effect of preventing the occurrence of specified risks.

[0084] (Configuration 3) In the driving assistance device according to Configuration 1 or Configuration 2, the attention calling control unit increases the level of the attention calling as the remaining time until the occurrence time of the predetermined risk becomes shorter.

[0085] According to the driving assistance device of the third configuration, the driver of the controlled vehicle can be made aware that the possibility of a predetermined risk occurring is increasing, and can be urged to avoid the risk.

[0086] (Structure 4) A driving assistance device according to any one of Structures 1 to 3, wherein the driver state recognition unit recognizes the driving state of the driver driving the controlled vehicle as the driver's state, and the attention reminder control unit does not execute the attention reminder when it is able to judge that the driver is in a state capable of avoiding the specified risk based on the driving state of the driver driving the controlled vehicle recognized by the driver state recognition unit.

[0087] According to the driving assistance device of the fourth configuration, it is possible to avoid the driver being annoyed by the warning of caution when the driver is taking measures to avoid a predetermined risk.

[0088] (Structure 5) A driving assistance device according to any one of Structures 1 to 4, wherein the driver state recognition unit recognizes the driver's health state and the direction of the driver's line of sight, and when the driver state recognition unit recognizes that the driver's line of sight is directed toward the surrounding object, the attention reminder control unit determines whether to perform the attention reminder based on the driver's health state recognized by the driver state recognition unit.

[0089] According to the driving assistance device of structure 5, when the driver's cognitive ability may be reduced due to the driver's health condition (stress, fatigue, psychological state, etc.), the driver is reminded to pay attention even when the driver's line of sight is directed toward surrounding objects, thereby improving the effect of preventing the occurrence of specified risks.

[0090] (Structure 6) A driving assistance method, which is executed by a computer, the driving assistance method comprising: a controlled vehicle movement information acquisition step, for a controlled vehicle, acquiring the controlled vehicle movement information including the position information of the controlled vehicle; a surrounding object movement information acquisition step, for a surrounding object existing in the periphery of the controlled vehicle, acquiring the surrounding object movement information including the position information of the surrounding object; a risk occurrence time prediction step, for predicting the risk occurrence time when a specified risk occurs due to the proximity of the controlled vehicle to the surrounding object based on the controlled vehicle movement information and the surrounding object movement information; a driver state recognition step, for recognizing the state of the driver of the controlled vehicle; and an attention reminder control step, for determining the execution timing of the attention reminder executed by the reporting device of the controlled vehicle and the content of the attention reminder based on the remaining time until the risk occurrence time and the driver's state recognized by the driver state recognition step.

[0091] By executing the driving assistance method of Configuration 6 on a computer, the same operational effects as those of the driving assistance device of Configuration 1 can be obtained.

[0092] (Structure 7) A non-volatile recording medium having recorded thereon a program executed by a computer, wherein the program causes the computer to function as: a controlled vehicle movement information acquisition unit, which acquires controlled vehicle movement information including position information of the controlled vehicle for a controlled vehicle; a surrounding object movement information acquisition unit, which acquires surrounding object movement information including position information of surrounding objects existing in the periphery of the controlled vehicle; a risk occurrence time prediction unit, which predicts a risk occurrence time when a specified risk occurs due to the proximity of the controlled vehicle to the surrounding objects based on the controlled vehicle movement information and the surrounding object movement information; a driver state recognition unit, which recognizes the state of the driver of the controlled vehicle; and a caution reminder control unit, which determines the execution timing of a caution reminder by a reporting device of the controlled vehicle and the content of the caution reminder based on the remaining time until the risk occurrence time and the state of the driver recognized by the driver state recognition unit.

[0093] The configuration of the driving assistance device of Configuration 1 can be realized by executing the program of Configuration 7 on a computer.

Claims

1. A driving assistance device comprising: a controlled vehicle movement information acquisition unit, which acquires controlled vehicle movement information including position information of the controlled vehicle for the controlled vehicle; a surrounding object movement information acquisition unit for acquiring surrounding object movement information including position information of surrounding objects existing around the controlled vehicle; a risk occurrence time prediction unit that predicts a risk occurrence time when a predetermined risk occurs due to the approach of the controlled vehicle to the surrounding object based on the controlled vehicle movement information and the surrounding object movement information; a driver state recognition unit for recognizing the state of the driver of the controlled vehicle; as well as The attention reminder control unit determines the execution timing of the attention reminder by the reporting device of the controlled vehicle and the content of the attention reminder based on the remaining time until the risk occurs and the state of the driver recognized by the driver state recognition unit.

2. The driving assistance device according to claim 1, wherein: The driver state recognition unit recognizes the direction of the driver's line of sight as the driver's state, The peripheral object movement information includes information indicating whether the controlled vehicle is recognized at the peripheral object. When it can be determined based on the surrounding object movement information that the controlled vehicle is not identified at the surrounding object, the attention reminder control unit also performs the attention reminder even if the driver state recognition unit recognizes that the driver's line of sight is directed toward the surrounding object.

3. The driving assistance device according to claim 1 or 2, wherein: The attention calling control unit increases the level of the attention calling as the remaining time until the occurrence time of the predetermined risk becomes shorter.

4. The driving assistance device according to claim 1 or 2, wherein: The driver state recognition unit recognizes a driving state of the driver driving the controlled vehicle as the driver's state, The attention calling control unit does not execute the attention calling when it can be determined that the driver is in a state where the predetermined risk can be avoided based on the driving state of the driver driving the controlled vehicle recognized by the driver state recognition unit.

5. The driving assistance device according to claim 1 or 2, wherein: The driver state recognition unit recognizes the health state of the driver and the direction of the driver's line of sight, When the driver state recognition unit recognizes that the driver's line of sight is directed toward the peripheral object, the attention calling control unit determines whether to perform the attention calling according to the health state of the driver recognized by the driver state recognition unit.

6. A driving assistance method, which is executed by a computer, the driving assistance method comprising: A controlled vehicle movement information acquisition step, for a controlled vehicle, acquiring controlled vehicle movement information including position information of the controlled vehicle; A peripheral object movement information acquisition step of acquiring peripheral object movement information including position information of peripheral objects existing around the controlled vehicle; a risk occurrence time prediction step, predicting a risk occurrence time when a predetermined risk occurs due to the approach of the controlled vehicle to the surrounding object based on the movement information of the controlled vehicle and the movement information of the surrounding object; A driver status identification step, identifying the status of the driver of the controlled vehicle; as well as The attention reminder control step determines the execution time of the attention reminder performed by the reporting device of the controlled vehicle and the content of the attention reminder based on the remaining time until the risk occurs and the state of the driver identified by the driver state identification step.

7. A recording medium, which is a non-volatile recording medium, recording a program executed by a computer, wherein the program causes the computer to function as: a controlled vehicle movement information acquisition unit, which acquires controlled vehicle movement information including position information of the controlled vehicle for the controlled vehicle; a surrounding object movement information acquisition unit for acquiring surrounding object movement information including position information of surrounding objects existing around the controlled vehicle; a risk occurrence time prediction unit that predicts a risk occurrence time when a predetermined risk occurs due to the approach of the controlled vehicle to the surrounding object based on the controlled vehicle movement information and the surrounding object movement information; a driver state recognition unit for recognizing the state of the driver of the controlled vehicle; as well as The attention reminder control unit determines the execution timing of the attention reminder by the reporting device of the controlled vehicle and the content of the attention reminder based on the remaining time until the risk occurs and the state of the driver recognized by the driver state recognition unit.

Images