Driving assistance systems

Abstract

To provide a driving support device which allows a driver to grasp existence of a notification object even in a state in which distance between a vehicle and the notification object is far apart from one another.SOLUTION: A driving support device comprises: an image acquisition unit which acquires a satellite image of a travel scheduled route of a vehicle among satellite images photographed by an artificial satellite; a mobile object detection unit which detects an object advancing toward the travel scheduled route from the acquired satellite image; and a notification unit which notifies crew members of the vehicle of the object when the object satisfies a predetermined condition.SELECTED DRAWING: Figure 5

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , ,

[0003] , , , , ,

[0001] The present invention relates to a driving support device.

Background Art

[0002] Patent Document 1 discloses a vehicle peripheral monitoring device that detects an object protruding onto a traveling road and alerts a driver. In the device of this Patent Document 1, when a pedestrian is detected from a side road based on a captured image captured by an infrared camera mounted on the vehicle, an alarm sound is output from a speaker.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

[0007] In the driver assistance device according to claim 1, the image acquisition unit acquires satellite images of the vehicle's planned route. The moving object detection unit detects objects heading towards the planned route from the satellite images acquired by the image acquisition unit. Furthermore, the notification unit notifies the vehicle occupants if the object detected by the moving object detection unit meets predetermined conditions. In this way, by notifying the occupants of information about objects detected based on satellite images, the driver can be aware of the presence of an object even when the vehicle is far from the object. Note that the term "object" here is not limited to people and vehicles, but also includes animals and fallen objects.

[0011] Also This allows you to know in advance if large wild animals might suddenly appear on your planned route.

[0013] moreover By inputting image data into the learning model, it can determine whether the size of a wild animal is greater than or equal to a predetermined size, resulting in more accurate determination than when comparing pre-stored data.

[0014] Claim 2 The driving assistance device relating to the claim 1 The system includes the described driving assistance device and a vehicle control unit that controls the vehicle, and when the object detected by the moving object detection unit satisfies predetermined conditions, the vehicle control unit sets an upper limit speed for the vehicle.

[0015] Claim 2 In the driver assistance system related to this, collisions between the vehicle and objects can be suppressed by setting an upper speed limit for the vehicle. [Effects of the Invention]

[0016] As explained above, the driver assistance device according to the present invention allows the driver to be aware of the presence of a notification target even when the vehicle is at a distance from the notification target. [Brief explanation of the drawing]

[0017] [Figure 1] This is a schematic diagram showing the entire system, including the driver assistance device according to the embodiment. [Figure 2] This is a block diagram showing the hardware configuration of the driver assistance device according to the embodiment. [Figure 3] This is a block diagram showing the hardware configuration of an in-vehicle device in an embodiment. [Figure 4] This is a block diagram showing the functional configuration of the driver assistance device according to the embodiment. [Figure 5] This flowchart shows an example of the notification process flow in the embodiment. [Modes for carrying out the invention]

[0018] A driver assistance system S equipped with a driver assistance device 10 according to the embodiment will be described with reference to the drawings.

[0019] As shown in Figure 1, the driver assistance system S according to this embodiment comprises a driver assistance device 10, a server 12, and a vehicle 11. The driver assistance device 10, the server 12, and the vehicle 11 are connected via a network N and are able to communicate with each other.

[0020] The driver assistance device 10 is installed, for example, outside the vehicle 11 and is configured to receive instructions from the vehicle 11 and transmit requested information. The driver assistance device 10 is also configured to acquire various information from the server 12 via the network N. The driver assistance device 10 may be connected to multiple vehicles 11 via the network N.

[0021] Vehicle 11 includes an in-vehicle device 28 as a vehicle control unit. The in-vehicle device 28 of the present embodiment is, for example, an ECU (Electronic Control Unit) that performs various controls. The server 12 is installed outside the vehicle 11 and is configured to be able to receive data from the artificial satellite 13. Therefore, satellite images taken by the artificial satellite 13 are stored in the server 12. In addition, information such as map information, dangerous locations, and dangerous sections is stored in the server 12, and necessary information is transmitted to the in-vehicle device 28 via the network N in response to a request from the in-vehicle device 28 (vehicle 11).

[0022] The driving support system S of the present embodiment is configured as described above. The driving support device 10 is configured to detect an object heading toward the planned route from a satellite image captured by the artificial satellite 13 and notify the vehicle occupants when this object meets a predetermined condition.

[0023] (Hardware Configuration of Driving Support Device 10) As shown in FIG. 2, the driving support device 10 includes a CPU (Central Processing Unit: processor) 30, a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 34, a storage 36, and a communication interface (communication I / F) 38. Each component is connected to be communicable with each other via an internal bus 39.

[0024] The CPU 30 is a central arithmetic processing unit that executes various programs and controls each part. That is, the CPU 30 reads a program from the ROM 32 or the storage 36 and executes the program using the RAM 34 as a work area. In addition, the CPU 30 performs control of each of the above components and various arithmetic processes according to the program recorded in the ROM 32 or the storage 36.

[0025] ROM32 stores various programs and data. RAM34 temporarily stores programs or data as a working area. Storage36 is a non-temporary recording medium consisting of an HDD (Hard Disk Drive) or SSD (Solid State Drive) that stores various programs, including the operating system, and various data. In this embodiment, ROM32 or storage36 stores programs for performing various processes.

[0026] The communication interface 38 is a interface for the driver assistance system 10 to communicate with the server 12, the in-vehicle unit 28, and other devices, and standards such as CAN (Controller Area Network), Ethernet (registered trademark), LTE (Long Term Evolution), FDDI (Fiber Distributed Data Interface), and Wi-Fi (registered trademark) are used.

[0027] (Hardware configuration of the in-vehicle unit 28) As shown in Figure 3, the in-vehicle unit 28 consists of a CPU 40, ROM 42, RAM 44, storage 46, a communication interface (communication I / F) 48, and an input / output interface (input / output I / F) 50. Each component is connected to the others via an internal bus 52 so that they can communicate with each other.

[0028] The CPU 40 is a central processing unit that executes various programs and controls various components. Specifically, the CPU 40 reads programs from the ROM 42 or storage 46 and executes them using the RAM 44 as a working area. The CPU 40 also controls the above components and performs various calculations according to the programs recorded in the ROM 42 or storage 46.

[0029] ROM 42 stores various programs and data. RAM 44 temporarily stores programs or data as a working area. Storage 46 is a non-temporary recording medium consisting of an HDD or SSD that stores various programs, including the operating system, and various data.

[0030] The communication interface 48 is the interface through which the in-vehicle unit 28 communicates with the server and other devices, and standards such as CAN (Controller Area Network), Ethernet (registered trademark), LTE (Long Term Evolution), FDDI (Fiber Distributed Data Interface), and Wi-Fi (registered trademark) are used.

[0031] The input / output interface 50 is connected to a navigation system 54, a GPS (Global Positioning System) device 56, and a display device 58. The navigation system 54 sets the planned route from the vehicle 11's current location to the destination and provides various guidance to ensure the vehicle 11 follows the planned route. At this time, the navigation system 54 acquires map information from the server 12 as appropriate. If some or all of the map information is stored in the storage 46, the navigation system 54 may set the planned route without acquiring map information from the server 12.

[0032] The GPS device 56 is a device that determines the current position of the vehicle 11 by receiving GPS signals from GPS satellites. The display device 58 displays various information on a display unit (not shown) installed inside the vehicle. For example, the display device 58 displays information about the planned driving route set by the navigation system 54, and information about objects (moving bodies) moving toward the planned driving route.

[0033] Furthermore, the on-board unit 28 is electrically connected to an accelerator control unit (not shown), and is configured to set an upper limit on the vehicle speed by controlling the accelerator control unit based on signals from the on-board unit 28.

[0034] (Functional configuration of the driver assistance system 10) The driver assistance system 10 implements various functions using the hardware resources described above. The functional configuration implemented by the driver assistance system 10 will be explained with reference to Figure 4.

[0035] As shown in Figure 4, the driver assistance device 10 is configured as follows: an image acquisition unit 62, a moving object detection unit 64, a moving object identification unit 66, a notification determination unit 68, a notification unit 70, and an upper limit speed setting unit 72. Each of these functional configurations is realized by the CPU 30 reading and executing a program stored in the ROM 32 or storage 36.

[0036] The image acquisition unit 62 acquires satellite images of the planned route from among the satellite images taken by the artificial satellite 13. For example, if the satellite images taken by the artificial satellite 13 are stored in the server 12 for each shooting location, the image acquisition unit 62 acquires satellite images of the planned route of the vehicle 11 from the server 12.

[0037] The image acquisition unit 62 may acquire satellite images of only some of the roads along the planned route of the vehicle 11. For example, the image acquisition unit 62 may not acquire satellite images of expressways, but only of roads other than expressways. When acquiring satellite images of only some of the roads, image processing such as detection of moving objects can be performed quickly, even if the planned route is long-distance.

[0038] Alternatively, the image acquisition unit 62 may acquire only satellite images of the vehicle 11's planned route within a predetermined distance from the vehicle 11's current position.

[0039] The moving object detection unit 64 detects objects heading towards the planned travel route from satellite images acquired by the image acquisition unit 62. For example, the moving object detection unit 64 may identify objects heading towards the planned travel route by estimating the direction of movement of an object based on multiple images taken at the same location at predetermined time intervals. The term "object" here is not limited to people and vehicles, but also includes animals and fallen objects.

[0040] In this embodiment, the moving object detection unit 64 may also detect objects using satellite images taken immediately before the vehicle 11 travels along the road in question. If the satellite 13 is a geostationary satellite, it can photograph the same location at any time. If the satellite 13 is a low Earth orbit satellite, the photographic frequency is based on the number of return days, so if satellite images taken immediately before the vehicle travels along the road cannot be obtained, the most recent satellite image is used to detect obstacles.

[0041] The moving object identification unit 66 identifies the object detected by the moving object detection unit 64. Specifically, the moving object identification unit 66 identifies the type and size of the object. For example, if the object is a living organism, the moving object identification unit 66 may identify whether it is a person or an animal other than a person.

[0042] In this embodiment, the moving object identification unit 66 identifies an object by inputting satellite image data into a learning model that has been trained using machine learning to identify information about an object. For example, the learning model in this embodiment consists of a convolutional neural network (CNN) that has been trained using training data to perform deep learning in order to identify the type and size of an object based on the data of each pixel in the satellite image.

[0043] The notification determination unit 68 determines whether an object is a notification object that meets predetermined conditions. For example, if the notification determination unit 68 identifies the object as a pedestrian, it determines whether the pedestrian is a child of an age that drivers should pay attention to. If it determines that the pedestrian is a child of an age that drivers should pay attention to, the notification determination unit 68 designates the object as a notification object.

[0044] Furthermore, the notification determination unit 68 determines that if the moving object identification unit 66 of this embodiment determines that an object is an animal, and the size of the animal is greater than a predetermined size, then the object will be designated as a notification target object. In addition, the notification determination unit 68 determines that an object is a notification target object if the object is an animal and is of a type that the driver should pay attention to. Animals of a type that the driver should pay attention to include animals designated as endangered species and animals that may pose a danger to human health.

[0045] The notification unit 70 notifies the occupants of information about an object if the notification determination unit 68 determines that the object is a target object for notification. In this embodiment, as an example, the notification unit 70 notifies the occupants by displaying a predetermined information on a display unit installed in the vehicle interior via the display device 58. The information to be notified may include, for example, the type and size of the object, the distance from the vehicle 11 to the object, and its direction.

[0046] The upper speed limit setting unit 72 sets the upper speed limit for the vehicle 11 when an object detected by the moving object detection unit 64 meets predetermined conditions. For example, the upper speed limit setting unit 72 may set the upper speed limit for the vehicle 11 when the distance to the object to be notified falls below a predetermined distance, and prohibit driving at a speed exceeding the upper speed limit. The upper speed limit setting unit 72 may also change the upper speed limit set depending on the type of object. When the upper speed limit setting unit 72 sets the upper speed limit, the notification unit 70 notifies the occupants in advance.

[0047] (action) Next, the operation of this embodiment will be explained.

[0048] An example of notification processing by the driver assistance device 10 of this embodiment will be explained using the flowchart shown in Figure 5. These display processes are performed by the CPU 30 reading a display program from the ROM 32 or storage 36, loading it into the RAM 34, and executing it. In this embodiment, as an example, the notification process is performed after the planned route of the vehicle 11 has been set.

[0049] (An example of notification processing) In step S102, the CPU 30 acquires satellite images. Specifically, the CPU 30 acquires satellite images of roads that meet the specified conditions along the planned route from the server 12 using the functions of the image acquisition unit 62.

[0050] In step S104, the CPU 30 detects and identifies the moving object. Specifically, the CPU 30 uses the functions of the moving object detection unit 64 to detect an object heading towards the planned travel path from the satellite image acquired in step S102. The CPU 30 also uses the functions of the moving object identification unit 66 to identify the object detected by the moving object detection unit 64. That is, the CPU 30 identifies the type and size of the object. Furthermore, if the object is a living organism, the CPU 30 may also identify whether it is a person or an animal other than a person.

[0051] In step S106, the CPU 30 determines whether the identified object is a notification object. Specifically, the CPU 30 determines, based on the function of the notification determination unit 68, that an object heading towards the planned route is a notification object if it is a pedestrian child of an age that the driver should pay attention to. The CPU 30 also determines that an object is a notification object if it is an animal and its size is above a predetermined size. Furthermore, the CPU 30 determines that an object is a notification object if it is an animal and it is of a type that the driver should pay attention to.

[0052] If the CPU 30 determines in step S106 that the object is subject to notification, it proceeds to the processing in step S108. On the other hand, if the CPU 30 determines in step S106 that the object is not subject to notification, it terminates the notification process because there is no longer a need to notify the crew.

[0053] In step S108, the CPU 30 determines whether the identified object is subject to the setting of an upper speed limit. In this embodiment, the determination conditions differ for objects to be notified and objects for which an upper speed limit is set. Therefore, there are objects that are subject to notification but do not need to have an upper speed limit set. For example, even if an object is heading along the planned route, if it is unlikely to affect the movement of the vehicle 11, an upper speed limit is not set.

[0054] If the CPU 30 determines in step S108 that an upper speed limit should be set for the object, it proceeds to step S110. In step S110, the CPU 30 sets the upper speed limit for the vehicle 11. Specifically, the CPU 30 sends a signal to an accelerator control unit (not shown) to control the vehicle 11 so that it does not exceed the set upper speed limit. Then, the CPU 30 proceeds to step S112.

[0055] On the other hand, if the CPU 30 determines in step S108 that the object is not one for which an upper speed limit should be set, it proceeds to the process in step S112.

[0056] The CPU 30 notifies the occupant in step S112. Specifically, the CPU 30 notifies the occupant by displaying a predetermined information on a display unit installed in the vehicle interior via the function of the notification unit 70, via the display device 58. The information to be notified may include, for example, the type and size of the object, the distance from the vehicle 11 to the object, and its direction.

[0057] Furthermore, if an upper speed limit is set in step S110, the CPU 30 notifies the occupants in step S112 that the upper speed limit has been set using the function of the notification unit 70. At this time, the CPU 30 may also notify the occupants of the specific upper speed limit and the reason for setting the upper speed limit. Then, the CPU 30 terminates the notification process.

[0058] As described above, according to the driving support device 10 of this embodiment, the image acquisition unit 62 acquires satellite images of the planned route of the vehicle 11. The moving object detection unit 64 detects objects heading towards the planned route from the satellite images acquired by the image acquisition unit 62. Furthermore, the notification unit 70 notifies the occupants of the vehicle 11 when an object detected by the moving object detection unit 64 and identified by the moving object identification unit 66 meets predetermined conditions. In this way, by notifying the occupants of information about objects detected based on satellite images, the driver can be aware of the presence of an object even when the distance between the vehicle 11 and the object is great.

[0059] Furthermore, in this embodiment, the driver can obtain information in advance about children heading along the planned route, allowing them to prepare for situations where children might run into the road.

[0060] Furthermore, in this embodiment, it is possible to know in advance if a large wild animal might suddenly appear on the planned route.

[0061] Furthermore, in this embodiment, since the size of wild animals is determined by inputting satellite image data into the learning model, it is possible to make a more accurate determination than when comparing pre-stored data.

[0062] Furthermore, in this embodiment, by setting an upper speed limit for the vehicle 11 using the upper speed limit setting unit 72, collisions between the vehicle 11 and objects can be suppressed.

[0063] Although the driving assistance device 10 according to the embodiment has been described above, it goes without saying that it can be implemented in various forms without departing from the spirit of the present invention. In the above embodiment, the device is configured to provide notifications to alert the occupants, but it is not limited to this. For example, the notification unit 70 may be used to notify the occupants if an object heading towards the planned driving route is an unusual animal. In this case, the moving object identification unit 66 may be used to notify if the animal is of a specific type, regardless of its size.

[0064] Furthermore, in the above embodiment, a forward-facing camera mounted on the vehicle 11 may be used to identify the moving object. For example, if an object heading towards the planned travel route has been identified in advance from satellite imagery, and the moving object appears in front of the vehicle 11, the forward-facing camera may be used to image the moving object, and it may be determined whether the information of the moving object matches the information of the moving object identified by the moving object identification unit 66. If the moving object imaged by the forward-facing camera and the moving object identified by the moving object identification unit 66 are different, the identification accuracy of the moving object identification unit 66 can be improved by correcting the processing of the moving object identification unit 66 based on the image data of the moving object imaged by the forward-facing camera.

[0065] Furthermore, in the above embodiment, the notification unit 70 notified the occupants of information about the moving object by displaying it on a display unit inside the vehicle, but this is not limited to this. For example, the notification unit 70 may also notify the occupants of information about the moving object by voice from a speaker (not shown) installed inside the vehicle.

[0066] Furthermore, the processing that the CPU 30 reads and executes in the above embodiment may also be executed by various processors other than the CPU 30. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processing, such as ASICs (Application Spec Integrated Circuits). The above processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types, for example, by multiple FPGAs, or by a combination of a CPU and an FPGA. More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0067] Furthermore, although the above embodiment uses a configuration in which various data is stored in the storage 36, the system is not limited to this. For example, non-temporary recording media such as CDs (Compact Disks), DVDs (Digital Versatile Disks), and USB (Universal Serial Bus) memory may be used as the storage unit. In this case, various programs and data will be stored in these recording media.

[0068] Furthermore, the processing flow described in the above embodiment is merely an example, and unnecessary steps may be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose. [Explanation of Symbols]

[0069] 10. Driving support systems 11 vehicles 13 Satellite 28. On-board equipment (vehicle control unit) 62 Image acquisition unit 64 Moving object detection unit 70 Notification Department S Driving Assistance System

Claims

1. An image acquisition unit that acquires satellite images of the vehicle's planned route from among satellite images taken by artificial satellites, A moving object detection unit that detects objects heading towards the planned travel route from the acquired satellite imagery, A notification unit that notifies the occupants of a vehicle when the aforementioned object meets predetermined conditions, It has, A driving assistance device that, by inputting image data of the satellite image into a machine learning model that has been trained to determine the size of the object based on the satellite image, determines whether the object is a wild animal of a predetermined size or larger, and if the object is a wild animal of a predetermined size or larger, notifies the occupant via the notification unit.

2. The driving support device according to claim 1, A vehicle control unit that controls the vehicle, It has, A driving assistance system in which, when the object detected by the moving object detection unit satisfies predetermined conditions, the vehicle control unit sets an upper limit speed for the vehicle.

Images