Driver assistance control system

Abstract

To enable accurate prediction of dangers a vehicle could encounter while traveling.SOLUTION: A driving assistance control device 20 provided herein is configured to predict a dangerous situation a vehicle may encounter while traveling and notify a vehicle occupant of the dangerous situation, and comprises: a first information acquisition unit 32 configured to acquire first information including elements indicative of an external situation of the vehicle; a second information acquisition unit 33 configured to acquire second information including elements indicative of an internal situation of another vehicle and information on acquisition date / time and acquisition location of each element from the another vehicle; and a dangerous situation prediction unit 34 configured to combine the first information and the second information corresponding to a current date and time and travel route ahead in a travel direction of the vehicle to predict a dangerous situation for the vehicle. The dangerous situation prediction unit 34 predicts a vehicle situation that may be encountered in the travel route ahead in the travel direction of the vehicle in the future on the basis of a relation between elements of the first information and the second information, thereby, predicting an encounter with a dangerous situation.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a driving support control device.

Background Art

[0002] There is known a technique for notifying a driver of a distracted state from an image of the driver of a vehicle (see, for example, Patent Document 1). There is known a technique for extracting and providing vehicle information collected from vehicles in which the time and position at which a predetermined operation event is detected among the vehicle information of the host vehicle are common (see, for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the techniques described in Patent Document 1 and Patent Document 2, it is difficult to estimate and notify a danger that may occur in the future on the traveling route of a vehicle. Further, when the accuracy of the estimation is low, there is a risk of erroneously estimating that there is a danger or erroneously estimating that there is no danger. Therefore, it is desired to accurately estimate a danger that may occur when a vehicle is traveling.

[0005] The present invention has been made in view of the above, and an object thereof is to accurately estimate a danger that may occur when a vehicle is traveling.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objective, the driver assistance control device according to the present invention is a driver assistance control device that predicts dangerous conditions occurring while a vehicle is in motion and notifies the occupant, and comprises: a first information acquisition unit that acquires first information including elements indicating the conditions inside the vehicle of the vehicle; a second information acquisition unit that acquires second information including at least one of elements indicating the conditions outside the vehicle and elements indicating the conditions inside the vehicle of another vehicle, the date and time of acquisition of each element from the other vehicle, and acquisition location information; and a dangerous condition prediction unit that predicts a dangerous condition of the vehicle by combining the first information acquired by the first information acquisition unit with the second information acquired by the second information acquisition unit, which corresponds to the current date and time and the road in the direction of travel of the vehicle, and the dangerous condition prediction unit predicts the occurrence of the dangerous condition by predicting the conditions of the vehicle that may occur in the future on the road in the direction of travel of the vehicle, based on the relationship between the elements of the first information and the second information. [Effects of the Invention]

[0007] The present invention has the effect of being able to predict with high accuracy the dangers that may occur while a vehicle is in motion. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a schematic diagram showing an example of the configuration of a driver assistance system according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing an example configuration of a driver assistance system equipped with a driver assistance control device according to the first embodiment. [Figure 3] Figure 3 is a flowchart showing an example of the processing flow in the driver assistance control device according to the first embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the driver assistance control device according to the present invention will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the following embodiments.

[0010] [First Embodiment] <Driving assistance system> Figure 1 is a schematic diagram showing an example of the configuration of the driver assistance system 1 according to the first embodiment. The driver assistance system 1 has driver assistance devices 10 installed in multiple vehicles that are connected to each other via a network NW.

[0011] <Driving support system> Figure 2 is a block diagram showing an example configuration of a driver assistance device 10, which includes a control unit 20, a driver assistance control device according to the first embodiment. The driver assistance device 10 predicts dangerous conditions that may occur while the vehicle is in motion and notifies the occupants.

[0012] In this embodiment, the driver assistance device 10 is a so-called drive recorder. In addition to being mounted on a vehicle, the driver assistance device 10 may also be a portable device that can be used in a vehicle. The driver assistance device 10 includes a camera 11, a recording unit 12, a display unit 13, a GPS (Global Positioning System) receiver 14 which is an example of a GNSS (Global Navigation Satellite System), a map information storage unit 15, an inference information storage unit 16, a communication unit 17, and a control unit 20 which serves as a driver assistance control device.

[0013] Camera 11 is a camera that photographs the area around the vehicle. Camera 11 may photograph not only the outside of the vehicle but also the inside of the vehicle. In this embodiment, camera 11 is described as an integrated unit of two cameras positioned on the front windshield inside the vehicle's interior, which photograph the front and rear, and each is an ultra-wide-angle camera capable of photographing approximately 178° horizontally and approximately 60° vertically, thereby enabling photography of almost the entire area around the vehicle and inside the vehicle. However, it is not limited to this, and may be composed of, for example, a camera with a fisheye lens capable of photographing 360° in all directions, or a group of multiple cameras that photograph each direction. Camera 11 continuously photographs images from the time the engine is started until it is stopped, that is, while the vehicle is in operation. In this embodiment, camera 11 continuously photographs images while the vehicle's accessory power is ON. Camera 11 outputs the captured video data to the video data acquisition unit 21 of the control unit 20. The video data is, for example, a moving image composed of 30 frames per second.

[0014] The recording unit 12 is used for temporary storage of data in the driver assistance device 10. The recording unit 12 is, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a memory card. Alternatively, it may be an external recording unit wirelessly connected via a communication device (not shown). The recording unit 12 records loop recording data based on control signals output from the recording control unit 24 of the control unit 20.

[0015] The display unit 13 is, for example, a display device unique to the driving support device 10 or a display device shared with another system including a navigation system. The display unit 13 is, for example, a display including a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display. The display unit 13 displays an image based on the video signal output from the display control unit 25 of the control unit 20. The display unit 13 displays the video captured by the camera 11 or the video recorded in the recording unit 12. The display unit 13 displays information for notifying the driver of the vehicle that a surrounding vehicle is driving dangerously. The driving support device 10 may include an audio output unit (not shown) such as a speaker that outputs ambient sounds recorded together with the video, voices for notifying the driver of the vehicle, warning sounds such as buzzers, etc.

[0016] The GPS receiver 14 receives radio waves from a GPS satellite (not shown). The GPS receiver 14 outputs the signal of the received radio wave to the position information acquisition unit 26 of the control unit 20.

[0017] The map information storage unit 15 stores map information. The map information storage unit 15 outputs the stored map information to the map information acquisition unit 28. The map information storage unit 15 may be configured to use the map information of a navigation system (not shown), or may be a storage device such as an external server that acquires map information via the communication unit 17.

[0018] The estimation information storage unit 16 stores the first information and the second information.

[0019] The communication unit 17 is a communication unit. The communication unit 17 may communicate by any method such as the Internet or a mobile phone line. The communication unit 17 communicates data with the driving support device 10 mounted on another vehicle.

[0020] The control unit 20 is an arithmetic processing unit (control device) composed of, for example, a CPU (Central Processing Unit). The control unit 20 loads the stored program into the memory and executes the instructions included in the program. The control unit 20 includes an internal memory not shown in the figure, and the internal memory is used for temporarily storing data in the control unit 20. The control unit 20 is connected to the bus 20x and includes a video data acquisition unit 21, a buffer memory 22, a video data processing unit 23, a recording control unit 24, a display control unit 25, a position information acquisition unit 26, a vehicle information acquisition unit 27, a map information acquisition unit 28, a playback control unit 29, a second information reception control unit 31, a first information acquisition unit 32, a second information acquisition unit 33, a danger state estimation unit 34, a notification control unit 35, a transmission information generation unit 36, and a first information transmission control unit 37.

[0021] The video data acquisition unit 21 acquires video data obtained by photographing the periphery of the vehicle. More specifically, the video data acquisition unit 21 acquires the video data output by the camera 11 and outputs it to the buffer memory 22. The video data acquisition unit 21 may also acquire audio data inside and outside the vehicle using a microphone not shown in the figure.

[0022] The buffer memory 22 is an internal memory included in the control unit 20, and is a memory that temporarily stores the video data for a certain period of time acquired by the video data acquisition unit 21 while updating it.

[0023] The video data processing unit 23 encodes the video data temporarily stored in the buffer memory 22 using any codec, such as H.264 or MPEG-4 (Moving Picture Experts Group), and converts it into any file format, such as MP4. The conversion of video data may include audio data, or the audio data may be stored in a separate file. The video data processing unit 23 generates video data as a file for a certain period of time from the video data temporarily stored in the buffer memory 22. As a specific example, the video data processing unit 23 generates 60 seconds of video data as one file from the video data temporarily stored in the buffer memory 22 in recording order. The video data processing unit 23 outputs the generated video data to the recording control unit 24. The video data processing unit 23 also outputs the video data to the display control unit 25. The duration of the video data generated as a file is 60 seconds as an example, but is not limited to this.

[0024] The recording control unit 24 controls the recording unit 12 to record the video data that has been filed by the video data processing unit 23. When the vehicle's accessory power is ON, or during the period when loop recording processing is performed, the recording control unit 24 records the video data that has been filed by the video data processing unit 23 to the recording unit 12 as overwritable video data. More specifically, during the period when loop recording processing is performed, the recording control unit 24 continues to record the video data generated as a file by the video data processing unit 23 to the recording unit 12, and when the capacity of the recording unit 12 becomes full, it overwrites the oldest video data with new video data.

[0025] If the recording control unit 24 predicts that the road ahead of the vehicle is in a dangerous condition, it may have the recording control unit 24 record the video data captured while the dangerous condition was determined to be present, along with information indicating the dangerous condition, to the recording unit 12 with overwriting protection disabled.

[0026] The playback control unit 29 outputs video data, such as loop recording data recorded in the recording unit 12, to the display control unit 25 in response to user input from a touch panel (not shown) located on the display unit 13, and plays it back.

[0027] The display control unit 25 controls the display of video data on the display unit 13. The display control unit 25 outputs a video signal that causes the video data to be output to the display unit 13. More specifically, the display control unit 25 displays video data captured by the camera 11, or video data played back by the playback control unit 29, on the display unit 13.

[0028] If the display control unit 25 determines, based on the hazard prediction unit 34, that there is a hazard in the road ahead of the vehicle's direction of travel, it displays information on the display unit 13 to inform the vehicle driver that a hazard has been predicted.

[0029] The location information acquisition unit 26 calculates the vehicle's current location information using a known method based on the radio waves received by the GPS receiver unit 14. The location information calculated by the location information acquisition unit 26 may be recorded together with video data such as loop recordings.

[0030] The vehicle information acquisition unit 27 acquires various vehicle information via CAN (Controller Area Network) or the like. For example, the vehicle information acquisition unit 27 acquires vehicle information including information about the vehicle's speed. However, it may also acquire vehicle information including sensor data acquired by various sensors installed on the vehicle. For example, the vehicle information acquisition unit 27 outputs the acquired vehicle information to the hazardous condition estimation unit 34. The vehicle information acquired by the vehicle information acquisition unit 27 may be recorded together with video data such as loop recordings.

[0031] The map information acquisition unit 28 acquires the map information stored in the map information storage unit 15.

[0032] The second information receiving control unit 31 controls the system to receive second information from a driver assistance device 10 installed in another vehicle via the communication unit 17.

[0033] Secondary information is obtained, for example, from video data captured by cameras installed on other vehicles, or from sensor data acquired by various sensors installed on other vehicles.

[0034] The first information acquisition unit 32 acquires first information that includes at least one of the following: an element indicating the situation outside the vehicle and an element indicating the situation inside the vehicle. The first information acquisition unit 32 may also acquire first information that includes an element indicating the situation outside the vehicle. The first information acquisition unit 32 may also acquire first information that includes an element indicating the situation inside the vehicle.

[0035] The primary information is obtained, for example, from video data captured by cameras installed on the vehicle, or from sensor data acquired by various sensors installed on the vehicle.

[0036] The first information includes information indicating the road path in the direction of travel of the vehicle. This information is obtained, for example, from location information acquired by the location information acquisition unit 26 and map information acquired by the map information acquisition unit 28. This information may also be obtained, for example, from the vehicle's navigation system, and including user-set planned route information is preferable for obtaining a wider range of road paths.

[0037] The first information acquisition unit 32, for example, performs image processing on the video data acquired by the video data acquisition unit 21 to recognize elements indicating the situation outside the vehicle and elements indicating the situation inside the vehicle, thereby acquiring first information. The image processing can be any known method and is not limited to that. The first information acquisition unit 32, for example, acquires first information from sensor data acquired by various sensors (not shown) installed on the vehicle, by recognizing elements indicating the situation outside the vehicle and elements indicating the situation inside the vehicle.

[0038] The elements of the first information indicating the situation outside the vehicle include, for example, the road surface condition of the vehicle's route, traffic congestion information for the vehicle's route, information indicating visibility for the vehicle's route, information indicating brightness around the vehicle, and weather information. The road surface condition of the route may be acquired, for example, by performing image processing on video data acquired by the video data acquisition unit 21 to recognize the road surface of the route. Traffic congestion information for the vehicle's route may be acquired, for example, from an external device via the communication unit 17. Traffic congestion information for the vehicle's route may also be acquired, for example, by the vehicle information acquisition unit 27, based on sensor data acquired from a speed sensor installed on the vehicle, when the vehicle's speed has been below a speed threshold for a predetermined period of time or longer, and the system recognizes that congestion is occurring. Information indicating brightness around the vehicle may be acquired, for example, by the vehicle information acquisition unit 27 from sensor data acquired by an illuminance sensor installed on the vehicle.

[0039] The elements indicating the conditions inside the vehicle in the first information include, for example, the vehicle's speed, location information, current date and time, the actions of the vehicle's occupants, and information indicating the vehicle's continuous driving time. The vehicle's speed may be obtained, for example, by the vehicle information acquisition unit 27 from sensor data acquired by a speed sensor installed in the vehicle. The vehicle's location information is, for example, location information acquired by the location information acquisition unit 26. Information indicating the actions of the vehicle's occupants may be obtained, for example, by performing image processing on the in-vehicle video data acquired by the video data acquisition unit 21 to recognize the actions of the vehicle's occupants.

[0040] The second information acquisition unit 33 acquires second information including at least one element indicating the external conditions of another vehicle and an element indicating the internal conditions of another vehicle, the date and time the element was acquired, and the location information of the element. The second information acquisition unit 33 may also acquire second information including an element indicating the external conditions of another vehicle, the date and time the element was acquired, and the location information of the element. The second information acquisition unit 33 may also acquire second information including an element indicating the internal conditions of another vehicle, the date and time the element was acquired, and the location information of the element.

[0041] More specifically, the second information acquisition unit 33 performs image processing on the video data received by the second information reception control unit 31, for example, to recognize elements indicating the external conditions of the other vehicle and elements indicating the internal conditions of the other vehicle, thereby acquiring second information. The second information acquisition unit 33 also acquires second information by recognizing elements indicating the external conditions of the other vehicle and elements indicating the internal conditions of the other vehicle from sensor data acquired by various sensors (not shown) installed on the other vehicle.

[0042] Elements of the second information indicating the situation outside the vehicle include, for example, the road surface condition of the other vehicle's route, traffic congestion information of the other vehicle's route, information indicating visibility of the other vehicle's route, information indicating brightness around the other vehicle, and information regarding the weather around the other vehicle. The road surface condition of the route may be obtained, for example, by performing image processing on the video data received by the second information receiving control unit 31 to recognize the road surface of the route. Traffic congestion information of the other vehicle's route may be obtained, for example, by recognizing that congestion is occurring based on sensor data received by the second information receiving control unit 31 from a speed sensor installed on the other vehicle, where the speed of the other vehicle has been below a speed threshold for a predetermined period of time or longer. Information indicating brightness around the other vehicle may be obtained, for example, from sensor data received by the second information receiving control unit 31 from an illuminance sensor installed on the other vehicle.

[0043] The elements of the second information indicating the situation inside the vehicle include, for example, the speed of other vehicles, the location information of other vehicles, the current date and time, the actions of the occupants of other vehicles, and information indicating the continuous driving time of other vehicles. The speed of other vehicles may be obtained, for example, from sensor data acquired by a speed sensor installed in the other vehicle, which is received by the second information receiving control unit 31. The location information of other vehicles is, for example, the location information of other vehicles received by the second information receiving control unit 31. Information indicating the actions of the occupants of other vehicles may be obtained, for example, by performing image processing on the video data of the interior of the vehicle from a camera installed in the other vehicle, which is received by the second information receiving control unit 31, to recognize the actions of the occupants of the other vehicle.

[0044] The hazardous state estimation unit 34 estimates the hazardous state of the vehicle by combining the first information acquired by the first information acquisition unit 32 with the second information acquired by the second information acquisition unit 33, which corresponds to the current date and time and the road in the direction of travel of the vehicle. More specifically, the hazardous state estimation unit 34 estimates the hazardous state by combining elements of the first information indicating the situation outside the vehicle and elements indicating the situation inside the vehicle with at least one of the elements of the second information indicating the situation outside the vehicle and elements indicating the situation inside the vehicle of another vehicle, which corresponds to the current date and time and the road in the direction of travel of the vehicle. The hazardous state estimation unit 34 may also estimate the hazardous state by combining elements of the first information indicating the situation outside the vehicle with elements of the second information indicating the situation outside the vehicle of another vehicle, which corresponds to the current date and time and the road in the direction of travel of the vehicle. The hazardous state estimation unit 34 may also estimate the hazardous state by combining elements of the first information indicating the situation inside the vehicle with the situation inside the vehicle of another vehicle, which corresponds to the current date and time and the road in the direction of travel of the vehicle. The hazard state estimation unit 34 can estimate a hazard state by combining first information obtainable from the vehicle itself with second information corresponding to the current date and time and the road in the direction the vehicle is traveling. For example, the hazard state estimation unit 34 estimates "hazard present" by combining the road surface condition of the road in the direction of travel, "dry," included in the first information, with the road surface condition of the road in the direction of travel, "wet," included in the second information. For example, the hazard state estimation unit 34 estimates "no hazard" by combining the road surface condition of the road in the direction of travel, "dry," and the weather, "sunny," included in the first information, with the road surface condition of the road in the direction of travel, "dry," and the weather, "sunny," included in the second information.

[0045] For example, if the road surface is "wet," it is slippery and dangerous. For example, if the road is congested, or if you are at the end of a traffic jam, rear-end collisions are more likely and dangerous. For example, if visibility is poor on the road, dangerous situations are likely. For example, if the surrounding light is dark, or if the light changes abruptly, such as at the entrance or exit of a tunnel, dangerous situations are likely. For example, if the weather is rainy, snowy, or foggy, dangerous situations are likely. For example, if the speed is high, the distance from braking to stopping becomes longer, making dangerous situations more likely. For example, if the passengers are asleep, the driver is more likely to feel drowsy, making dangerous situations more likely. For example, if the continuous driving time is long, the driver's fatigue accumulates and they become more prone to feeling drowsy, making dangerous situations more likely. In this way, the dangerous state of your vehicle can be estimated from each of these factors. It is also possible to make estimations by combining multiple factors.

[0046] Secondary information corresponding to the current date and time and the road in the direction of travel of the vehicle includes, for example, the following: For example, secondary information within a predetermined period of a few days prior. For example, secondary information for the same day and time. Various information on the road in the direction of travel of the vehicle obtained at the same time on the same day of the week may be information with similar trends, such as natural congestion during rush hour. For example, secondary information within a predetermined period of a few minutes from the current date and time. For example, secondary information for a predetermined section within a few kilometers from the current location on the road in the direction of travel of the vehicle.

[0047] In this embodiment, the hazardous state estimation unit 34 estimates the occurrence of a hazardous state by estimating the conditions outside or inside the vehicle that may occur in the future on the road in the direction of travel of the vehicle, based on the relationship between the elements of the first information and the second information. The hazardous state estimation unit 34 can estimate a hazardous state by estimating the conditions that may occur on the road in the direction of travel of the vehicle, based on the first information that can be obtained in the vehicle and the second information that corresponds to the current date and time and the road in the direction of travel of the vehicle. For example, the hazardous state estimation unit 34 estimates that the weather will become rainy and the road surface condition will change to "wet" on the road in the direction of travel of the vehicle, based on the road surface condition "dry" and weather "sunny" included in the first information and the road surface condition "wet" and weather "rain" included in the second information, and estimates that there is a "hazard". The hazard state estimation unit 34, for example, infers that the road surface condition in the direction of travel will remain "dry" and not change, based on the road surface condition of the road included in the first information ("dry") and the road surface condition of the road in the direction of travel ("dry") included in the second information, and thus infers that there is "no danger" in the road in the direction of travel. The hazard state estimation unit 34, for example, infers that the road ahead of the vehicle's current position will have poor visibility, based on the weather ("sunny") and time ("nighttime") of the road included in the first information and the weather ("sunny"), time ("nighttime") and visibility ("poor") of the road in the direction of travel included in the second information, and thus infers that there is "danger".

[0048] In this embodiment, the hazardous state estimation unit 34 estimates the occurrence of a hazardous state by replacing elements indicating external conditions in the second information with elements indicating external conditions in the first information. The elements indicating external conditions that are replaced are those that are highly likely to occur as they are on the road in the direction of travel of the vehicle in the future. Elements indicating internal conditions in the vehicle may also be used for replacement, but external elements in particular are less affected by conditions specific to the vehicle, such as the behavior of the occupants and the driving performance of the vehicle, and are highly likely to occur in the future. For example, the hazardous state estimation unit 34 replaces the road surface condition of the road included in the first information, "dry," with the road surface condition of the road in the direction of travel, "wet," in the second information, and estimates that there is a "hazard" because the road surface condition will deteriorate. For example, the hazardous state estimation unit 34 replaces the traffic congestion information of the road included in the first information, "no congestion," with the traffic congestion information of the road in the direction of travel, "congestion present," in the second information, and estimates that there is a "hazard" because congestion will occur and the driving conditions will change. For example, by comparing the brightness of the surrounding area of ​​the road ("bright") included in the first piece of information with the brightness of the surrounding area of ​​the road in the direction of travel ("dark") included in the second piece of information, it can be inferred that there is "danger" because the brightness is darker.

[0049] In this embodiment, the hazardous state estimation unit 34 estimates the occurrence of a hazardous state by comparing elements of second information with corresponding elements of first information. The elements of second information and corresponding elements of first information to be compared are elements that contain information useful for estimating the occurrence of a hazardous state from the correlation between the conditions of other vehicles and the vehicle itself. Here, comparing elements within the vehicle is particularly useful, but it is not limited to this, and elements outside the vehicle may also be added to estimate the occurrence of a hazardous state comprehensively. For example, the hazardous state estimation unit 34 considers the speed of the vehicle itself, "80 km / h (exceeding the speed limit)," included in the first information, and the speed of other vehicles, "60 km / h (below the speed limit)," included in the second information, as average speeds, compares them, and estimates that there is a "hazard" because the vehicle is driving slightly above the speed limit. For example, the hazardous state estimation unit 34 compares the continuous driving time, "4 hours (long driving)," included in the first information, with the continuous driving time, "1 hour (no long driving)," included in the second information, and estimates that there is a "hazard" from the perspective of fatigue due to long driving time because the continuous driving time is long.

[0050] The notification control unit 35 controls the system to notify the prediction result of the hazard state prediction unit 34. In this embodiment, the notification control unit 35 controls the system to notify the prediction result of the hazard state prediction unit 34 in a manner that includes the difference between the first information and the second information. The notification control unit 35 causes the display unit 13 to display the notification, for example, via the display control unit 25. The notification control unit 35 may also notify the system, for example, by voice, buzzer, or warning light. The notification control unit 35 may also display an image on the display unit 13 showing a message such as "Hazard has been predicted. Please pay attention to your speed," based on the elements used to predict the danger, such as the difference in speed between the first information and the second information.

[0051] The transmission information generation unit 36 ​​generates transmission information for transmitting to other driver assistance devices 10 installed in other vehicles, which is first information acquired by the first information acquisition unit 32, and which includes at least one of elements indicating the situation outside the vehicle and elements indicating the situation inside the vehicle.

[0052] The first information transmission control unit 37 controls the transmission of first information to a driver assistance device 10 installed in another vehicle via the communication unit 17.

[0053] <Processing in driver assistance systems> Next, with reference to Figure 3, the information processing in the driver assistance system 1 will be described. Figure 3 is a flowchart showing an example of the processing flow in the control unit 20, which is a driver assistance control device according to the first embodiment. When the power of the own vehicle is turned ON and in use, the driver assistance device 10 of the own vehicle is activated. When another vehicle is in use, the driver assistance device 10 of the other vehicle is activated. When the control unit 20 of the other vehicle's driver assistance device 10 is driving near the own vehicle, it transmits transmission information, which is second information for the own vehicle. While the driver assistance device 10 is activated, the camera 11 is constantly taking pictures, and various sensors are constantly operating.

[0054] The control unit 20 acquires first information (step S11). More specifically, the control unit 20 acquires first information from the first information acquisition unit 32, which includes at least one of the elements indicating the situation outside the vehicle and the elements indicating the situation inside the vehicle. The control unit 20 then proceeds to step S12.

[0055] The control unit 20 acquires second information (step S12). More specifically, the control unit 20 acquires second information from the second information acquisition unit 33, which includes at least one of the elements indicating the external situation of another vehicle and the internal situation of another vehicle. The control unit 20 then proceeds to step S13.

[0056] The control unit 20 determines whether or not there is a hazard on the road in the direction of travel of its own vehicle (step S13). More specifically, the control unit 20 uses the hazard state estimation unit 34 to estimate a hazard state by combining at least one of the elements indicating the external situation and internal situation of the vehicle in the first information and the elements indicating the external situation and internal situation of other vehicles in the second information, which correspond to the current date and time and the road in the direction of travel of the vehicle. If the estimation result is determined to be "hazardous" (Yes in step S13), the control unit 20 proceeds to step S14. If the estimation result is not determined to be "hazardous" (No in step S13), the control unit 20 proceeds to step S15.

[0057] If a danger is determined (Yes in step S13), the control unit 20 notifies the danger (step S14). More specifically, the control unit 20 controls the notification control unit 35 to notify the danger state prediction result of the danger state prediction unit 34 in a manner that includes the difference between the first information and the second information. The control unit 20 proceeds to step S15.

[0058] The control unit 20 determines whether or not to terminate the process (step S15). The control unit 20 determines to terminate the process if, for example, the power to the vehicle is turned off, or if a termination operation of the vehicle's driver assistance device 10 is detected. If the control unit 20 determines to terminate the process (Yes in step S15), it terminates the process. If the control unit 20 does not determine to terminate the process (No in step S15), it repeats the process in step S11.

[0059] <Effects> As described above, in this embodiment, in addition to first information obtainable from the vehicle itself, a second piece of information corresponding to the current date and time and the road path in the direction of travel of the vehicle can be combined to estimate a dangerous situation. In this embodiment, it is possible to predict with high accuracy the dangers that may occur to the vehicle. According to this embodiment, it is possible to estimate with high accuracy the dangers that may occur while the vehicle is in motion.

[0060] This embodiment allows for the estimation of a dangerous situation by combining first information obtainable from the vehicle itself with second information corresponding to the current date and time and the vehicle's direction of travel.

[0061] This embodiment can infer a dangerous situation by using first information obtainable from the vehicle itself and second information corresponding to the current date and time and the road in the direction the vehicle is traveling, thereby inferring a situation that may occur on the road in the direction the vehicle is traveling.

[0062] In this embodiment, the occurrence of a dangerous situation can be inferred by replacing the elements indicating the external conditions of the vehicle in the second information with the elements indicating the external conditions of the vehicle in the first information.

[0063] This embodiment allows for the prediction of the occurrence of a dangerous situation by comparing elements of second information with corresponding elements of first information.

[0064] This embodiment allows the system to control the notification of the hazard state prediction unit 34's prediction results in a manner that includes the difference between the first information and the second information. According to this embodiment, when a hazard is predicted, the passenger can understand the situation in more detail.

[0065] [Second Embodiment] The basic configuration of the driver assistance device 10 in this embodiment is the same as that of the driver assistance device 10 in the first embodiment. In the following description, components that are the same as those in the driver assistance device 10 are denoted by the same reference numerals or corresponding reference numerals, and their detailed descriptions are omitted. The driver assistance device 10 differs from the first embodiment in the processing in the hazardous condition prediction unit 34.

[0066] The second set of information further includes hazard information indicating the occurrence of dangerous situations while other vehicles are in motion. This hazard information includes, for example, accidents or collisions encountered while other vehicles are in motion, as well as information indicating incidents that did not result in an accident or collision but which the driver of the other vehicle felt were dangerous, such as sudden braking, sudden steering, or so-called "near misses." Image processing may be performed on video data captured by the camera 11 of the other vehicle to recognize accidents or collisions and incidents that the driver of the other vehicle felt were dangerous. Accidents or collisions and incidents that the driver of the other vehicle felt were dangerous may be recognized by the function of a so-called drive recorder installed in the other vehicle. Hazard information may also be arbitrarily registered by the passengers of the other vehicle, such as by voice input, as accidents or collisions and incidents that the driver of the other vehicle felt were dangerous.

[0067] The hazard state estimation unit 34 estimates the occurrence of a hazardous state based on the hazard information contained in the second information. More specifically, the hazard state estimation unit 34 estimates a hazardous state based on the first information and the hazard information contained in the second information, which corresponds to the current date and time and the road in the direction of travel of the vehicle. When the second information contains hazard information, the hazard state estimation unit 34 increases the weighting of the hazard state estimation judgment, for example, assuming that a hazard recognized by another vehicle is likely to occur strongly in the vehicle as well. Alternatively, for example, if the elements of the first and second information match and hazard information exists on the road in the direction of travel of the vehicle, the hazard state estimation unit 34 may estimate "hazard present" on the grounds that a hazard acquired by another vehicle is likely to occur strongly in the vehicle as well.

[0068] <Effects> As described above, this embodiment can infer that there is a "danger" if hazard information exists on the road ahead of the vehicle's direction of travel. According to this embodiment, it is possible to predict with high accuracy the dangers that may occur while the vehicle is in motion.

[0069] The driver assistance device according to the present invention may be implemented in various different forms other than those described above.

[0070] The components of the illustrated driver assistance system are functionally conceptual and do not necessarily have to be physically configured as shown. In other words, the specific form of each device is not limited to those shown, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage conditions of each device.

[0071] The configuration of the driver assistance system is implemented, for example, as software, such as a program loaded into memory. In the above embodiment, these were described as functional blocks implemented through the cooperation of hardware or software. That is, these functional blocks can be implemented in various forms using hardware alone, software alone, or a combination thereof.

[0072] The above-described components include those that are easily conceivable by those skilled in the art, and those that are substantially identical. Furthermore, the above-described components can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the components are possible without departing from the spirit of the present invention.

[0073] For example, if the elements of the first information and the elements of the second information match by a predetermined percentage or more, it can be determined that the information is highly related and that the surrounding conditions of the vehicle are close. The hazard state estimation unit 34 may, for example, adopt the estimation result of "danger present" obtained by combining and comparing the elements of the first information and the elements of the second information if they match by "80%" or more.

[0074] Furthermore, when combining elements of primary and secondary information to infer a dangerous situation, it is not necessary to have all the information for each element. It is also possible to infer by comparing the relationships between elements outside and inside the vehicle, such as considering the vertical movement that may occur inside the vehicle based on the unevenness of the road surface outside the vehicle. For example, it is also possible to infer using only some of the elements. Additionally, processing such as standardizing the granularity of different information, such as differences in the field of view or resolution of the video, or substituting similar or equivalent information, may be performed.

[0075] For example, the first information acquisition unit 32, the second information acquisition unit 33, and the hazard state estimation unit 34 may be implemented on an external device to the driver assistance system 10, such as a server or cloud. By using hardware resources on the cloud side, it is also possible to retain and refer to vehicle-specific information on external storage for a long period of time. Furthermore, it becomes possible to handle large amounts of information at high speed, and to improve the accuracy of hazard state estimation by using known techniques such as machine learning.

[0076] Secondary information about other vehicles acquired by one's own vehicle may be transmitted voluntarily or periodically by the other vehicle, or the vehicle may be requested to transmit information from the other vehicle as needed. When requesting transmission, it is preferable to specify the date and time of acquisition, acquisition location information, and the information to be acquired, as this will enable efficient communication between vehicles. In addition, if an accident is detected by the event detection function of the drive recorder installed in the vehicle's driver assistance device 10, the vehicle may be requested to transmit video data of the accident to other nearby vehicles. By receiving video data taken by other vehicles, the vehicle involved in the accident can collect and confirm the circumstances of the accident, which may be difficult to confirm from the vehicle's blind spot, etc., by collecting video data from other vehicles. Furthermore, if the transmission and reception between vehicles is configured in a cloud as described above, it may be controlled on the cloud side. [Explanation of Symbols]

[0077] 10. Driving support systems 11 Cameras 12 Records Section 13 Display section 14 GPS receiver 15 Map Information Storage Unit 16 Inference information storage unit 17 Communications Department 20 Control Unit (Driving Support Control Unit) 21 Video data acquisition unit 22 buffer memory 23 Video Data Processing Unit 24 Recording Control Unit 25 Display Control Unit 26 Location information acquisition section 27 Vehicle Information Acquisition Unit 28 Map Information Acquisition Unit 29 Regeneration Control Unit 31 Second Information Reception Control Unit 32 First Information Acquisition Department 33 Second Information Acquisition Department 34 Hazardous Situation Prediction Section 35 Notification Control Unit 36 Transmission Information Generation Unit 37 First Information Transmission Control Unit

Claims

1. A driver assistance control device that predicts dangerous situations that may occur while the vehicle is in motion and notifies the occupants, A first information acquisition unit acquires first information including elements indicating the conditions inside the vehicle, such as the vehicle's speed, location information, current date and time, the actions of the vehicle's occupants, and the vehicle's continuous driving time. A second information acquisition unit acquires second information including at least one of the following elements that indicate the external conditions of another vehicle: road surface conditions of the other vehicle's route, traffic congestion information of the other vehicle's route, visibility information of the other vehicle's route, brightness information of the surrounding area of ​​the other vehicle, and weather information of the surrounding area of ​​the other vehicle, as well as at least one of the following elements that indicate the internal conditions of the vehicle: the other vehicle's speed, location information of the other vehicle, current date and time, actions of the other vehicle's occupants, and continuous driving time of the other vehicle, along with the date and time of acquisition of each element from the other vehicle and the location information of acquisition. A hazard state estimation unit combines the first information acquired by the first information acquisition unit with the second information acquired by the second information acquisition unit, which corresponds to the current date and time and the direction of travel based on the vehicle's planned route information, to estimate the vehicle's hazard state. Equipped with, The aforementioned hazardous condition estimation unit acquires second information corresponding to the road in the direction of travel based on the current date and time and the vehicle's planned travel route information, for a predetermined period of time such as a few days prior to the current date and time, the same day of the week and time of day as the current date and time, or a predetermined period of time such as a few minutes from the current date and time, and within a predetermined section such as a few kilometers from the current location on the road in the direction of travel of the vehicle. The hazardous condition estimation unit estimates the situation of the vehicle that may occur in the future on the road in the direction of travel based on the planned route information of the vehicle, based on the relationship between the elements of the first information and the second information, and estimates the occurrence of the hazardous condition. Driver assistance control system.

2. The hazardous condition estimation unit estimates the occurrence of the hazardous condition by replacing the elements indicating the conditions inside the vehicle in the second information with the elements indicating the conditions inside the vehicle in the first information. The driver assistance control device according to claim 1.

3. The aforementioned second information further includes hazard information indicating the occurrence of a dangerous condition while the other vehicle is in motion, The hazard state estimation unit estimates the occurrence of the hazard state based on the hazard information. The driver assistance control device according to claim 1 or 2.

4. A notification control unit controls the notification of the prediction result of the danger state prediction unit in a manner that includes the difference between the first information and the second information. A driver assistance control device according to any one of claims 1 to 3, comprising:

Images