Notification system, notification method, and computer program

Abstract

To make it possible for a driver to easily grasp a notification content and switching thereof, when many kinds of notifications are carried out using vibration.SOLUTION: A notification 100 comprises: a vibration unit 21 for giving vibration stimulation to a vehicle occupant; and a control device 3 for controlling the vibration unit 21. The control device 3 is configured to perform first notification of giving notice of a degree of distance between a vehicle and a stationary object around the vehicle, and second notification of giving notice of presence of a moving body approaching the vehicle by vibrating the vibration unit 21, based on vehicle-peripheral information in a manner of varying a vibration level of the vibration unit 21 according to the degree of distance from the stationary object while vibrating the vibration unit 21 in a vibration pattern of continuously vibrating the vibration unit 21 regardless of the degree of distance from the stationary object when performing the first notification, meanwhile, vibrating the vibration unit 21 in the vibration pattern of intermittently vibrating the vibration unit 21 when performing the second notification.SELECTED DRAWING: Figure 1

Description

Technical Field

[0006] , ,

[0001] The present invention relates to an alert system, an alert method, and a computer program.

Background Art

[0002] Patent Document 1 discloses a vehicle notification device configured to notify a driver in the vehicle by vibrating a vibrating body according to a vibration rhythm pattern corresponding to the type of an alarm sound when the alarm sound such as a siren is generated outside the vehicle.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When performing a plurality of types of notifications with different contents by vibration, there is a method of changing the vibration rhythm pattern to perform each content notification as in the above-described conventional vehicle notification device. If an attempt is made to execute a plurality of types of notifications only by changing the vibration rhythm pattern, the amount of information that must be transmitted by vibration increases as the number of notification types increases, and the vibration rhythm pattern becomes complicated. And, as the vibration rhythm pattern becomes more complicated, it becomes more difficult for the driver to grasp what kind of notification is being executed.

[0005] Also, when the notification content is continuously switched to another content, the vibration rhythm pattern will also change continuously. However, as the vibration rhythm pattern becomes more complicated, it becomes more difficult to grasp the change at that time, and it becomes more difficult to grasp the switching of the notification content.

[0006] This invention addresses these problems and aims to enable drivers to easily understand the content of multiple types of notifications with different content and how they switch when vibration is used. [Means for solving the problem]

[0007] To solve the above problems, a notification system according to one aspect of the present invention comprises a vibration unit that provides vibration stimuli to the occupants of a vehicle, and a control device that controls the vibration unit. The control device provides the occupants of a vehicle by vibrating the vibration unit, thereby providing a first notification that informs the occupants of the degree of distance between the vehicle and stationary objects around it, and a second notification that informs the occupants of the presence of a moving object approaching the vehicle, based on information about the surroundings of the vehicle. When providing the first notification, the control device vibrates the vibration unit in a vibration pattern that vibrates continuously regardless of the degree of distance to the stationary object, and changes the vibration level of the vibration unit according to the degree of distance to the stationary object. When providing the second notification, the control device is configured to vibrate the vibration unit in a vibration pattern that vibrates intermittently.

[0008] Furthermore, a notification method according to one aspect of the present invention is a notification method using a control device that controls a vibration unit that provides vibration stimuli to the occupants of a vehicle, wherein, based on information about the surroundings of the vehicle, a first notification is given to the occupants of the vehicle by vibrating the vibration unit, informing them of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification is given to the occupants of the vehicle by vibrating the vibration unit, wherein when the first notification is given, the vibration unit is vibrated in a vibration pattern that causes it to vibrate continuously regardless of the degree of distance to the stationary object, and the vibration level of the vibration unit is changed according to the degree of distance to the stationary object, and when the second notification is given, the vibration unit is vibrated in a vibration pattern that causes it to vibrate intermittently.

[0009] Furthermore, a computer program according to one aspect of the present invention is a computer program for a control device that controls a vibration unit that provides vibration stimuli to the occupants of a vehicle, and provides a first notification to the occupants of the vehicle by vibrating the vibration unit, based on information about the surroundings of the vehicle, to inform them of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification to inform them of the presence of a moving object approaching the vehicle, and when providing the first notification, the control device is instructed to vibrate the vibration unit in a vibration pattern that vibrates the vibration unit continuously regardless of the degree of distance to the stationary object, and to change the vibration level of the vibration unit according to the degree of distance to the stationary object, and when providing the second notification, the control device is instructed to vibrate the vibration unit in a vibration pattern that vibrates the vibration unit intermittently. [Effects of the Invention]

[0010] According to these aspects of the present invention, even when multiple types of notifications with different content are transmitted by vibration, the driver can easily understand the content of the notifications and when they are switched. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of a notification system according to one embodiment of the present invention. [Figure 2] This figure shows examples of vibration rhythm patterns and vibration levels for a notification (first notification) that indicates the distance to stationary objects around the vehicle, and for a notification (second notification) that indicates the presence of a moving object approaching the vehicle. [Figure 3] This flowchart explains the details of the control content related to the first notification. [Figure 4] This flowchart explains the details of the control system for the second notification. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, similar components will be given the same reference numerals.

[0013] Figure 1 is a schematic diagram of a notification system 100 according to one embodiment of the present invention. The notification system 100 can be installed in vehicles such as autonomous vehicles and manually driven vehicles.

[0014] The notification system 100 comprises a peripheral information acquisition device 1, a notification device 2, and a control device 3. The peripheral information acquisition device 1, the notification device 2, and the control device 3 are each connected to communicate via an in-vehicle network 9 that conforms to standards such as a controller area network.

[0015] The surrounding information acquisition device 1 is a device for acquiring data on objects around the vehicle as vehicle surrounding information. The surrounding information acquisition device 1 according to this embodiment includes one or more external cameras 11 for photographing objects around the vehicle, and a distance measuring sensor 12 for measuring the distance to objects around the vehicle.

[0016] The external camera 11 captures images of the area around the vehicle at a predetermined frame rate (e.g., 10Hz to 40Hz) and generates a camera image showing the area around the vehicle. Each time the external camera 11 generates a camera image, it transmits the generated camera image (vehicle surroundings information) to the control device 3 via the in-vehicle network 9.

[0017] The distance measurement sensor 12 irradiates a plurality of laser beams or radio waves with different irradiation angles horizontally and in parallel to a distance measurement area around the host vehicle, and receives the reflected light of each irradiated laser beam or the reflected wave of each irradiated radio wave. Then, based on the received reflected light or reflected wave, the distance measurement sensor 12 measures the distances to various obstacles (e.g., people, vehicles, walls, etc.) existing in the distance measurement area. The distances to various obstacles are acquired for each reflection point (irradiation point) of the laser beam or radio wave, respectively. The distance measurement sensor 12 associates the distance to each reflection point with the coordinate information of each reflection point, and then transmits them as distance measurement data (vehicle surrounding information) to the control device 3 via the in-vehicle network 9. Examples of the distance measurement sensor 12 include a lidar (LiDAR: Light Detection and Ranging), a millimeter-wave radar sensor, an ultrasonic sensor, and the like.

[0018] The notification device 2 is a device for notifying various information to the driver of the host vehicle. The notification device 2 according to the present embodiment includes at least a vibration unit 21 for applying a vibration stimulus to the driver so as to be able to notify through the driver's sense of touch. The vibration unit 21 can be provided, for example, on the steering wheel or the driver's seat, and in this embodiment, it is built into the driver's seat. When the notification device 2 receives a vibration signal from the control device 3 via the in-vehicle network 9, the notification device 2 vibrates the vibration unit 21 based on the content of the received vibration signal.

[0019] The control device 3 is an ECU (Electronic Control Unit) including a communication unit 31, a storage unit 32, and a processing unit 33.

[0020] The communication unit 31 includes an interface circuit for connecting the control device 3 to the in-vehicle network 9. The communication unit 31 supplies data received from the outside (such as vehicle surrounding information) to the processing unit 33. Also, the communication unit 31 outputs the vibration signal output from the processing unit 33 to the vibration unit 21.

[0021] The memory unit 32 has a storage medium such as a HDD (Hard Disk Drive), SSD (Solid Disk Drive), or semiconductor memory, and stores various computer programs, data, etc. used in the processing by the processing unit 33.

[0022] The processing unit 33 has one or more CPUs (Central Processing Unit) and its peripheral circuits. The processing unit 33 executes various computer programs stored in the memory unit 32 and is, for example, a processor.

[0023] To give an example of the processing performed by the processing unit 33, for example, the processing unit 33 outputs a first vibration signal corresponding to the degree of the distance between the host vehicle and stationary objects around it (e.g., "long distance", "medium distance", "short distance", etc.) based on the vehicle surrounding information so that the driver can grasp the distance between the host vehicle and stationary objects around it (e.g., walls, parked vehicles, etc.). Thereby, for example, even for a driver with a hearing impairment, the degree of the distance between the host vehicle and stationary objects around it can be notified by vibration.

[0024] Further, the processing unit 33 outputs a second vibration signal to the vibration unit 21 when detecting the presence of a moving object approaching the host vehicle based on the vehicle surrounding information in order to prevent a collision with a moving object (e.g., a person or a moving vehicle) that the driver may not recognize. Thereby, for example, even for a driver with a hearing impairment, the presence of a moving object approaching the host vehicle can be notified by vibration.

[0025] By the way, when performing a plurality of types of notifications with different contents by vibration, there is a method of changing the vibration rhythm pattern to perform notifications of each content as in the conventional art. An arbitrary vibration rhythm pattern can be created by appropriately changing the combination (order) of the time (on time) for vibrating the vibration unit 21 and the time (off time) for not vibrating it and the length of each time.

[0026] In this embodiment, the distance between the vehicle and surrounding stationary objects is indicated to the driver in three stages: "long distance," "medium distance," and "short distance." Therefore, three types of vibration rhythm patterns are required to notify the driver of these distances through changes in the vibration rhythm pattern. In addition, one more type of vibration rhythm pattern is required to inform the driver of the presence of a moving object approaching the vehicle.

[0027] Thus, attempting to implement multiple types of notifications with different content solely through changes in the vibration rhythm pattern results in a complex vibration rhythm pattern. However, the more complex the vibration rhythm pattern becomes, the more difficult it becomes to understand what the currently running notification is about.

[0028] Furthermore, when the notification content changes continuously, the vibration rhythm pattern will also change continuously. However, the more complex the vibration rhythm pattern becomes, the more difficult it is to perceive the change. In other words, there is a risk that the driver may not be able to immediately perceive the change in notification content.

[0029] Therefore, in this embodiment, as shown in Figure 2, when providing notification (first notification) to indicate the distance between the vehicle and surrounding stationary objects, the vibration unit 21 is vibrated in a vibration rhythm pattern that causes the vibration unit 21 to vibrate continuously regardless of the distance to the stationary objects, and the vibration level (vibration intensity) of the vibration unit 21 is changed according to the distance to the stationary objects. Specifically, in this embodiment, the vibration level is set to "weak" when the distance between the vehicle and surrounding stationary objects is "long distance", to "medium" when it is "medium distance", and to "strong" when it is "short distance".

[0030] Furthermore, when issuing a notification (second notification) to alert the vehicle of the presence of a moving object approaching it, the vibration unit 21 is vibrated in a vibration rhythm pattern that causes it to vibrate intermittently. Specifically, in this embodiment, three short vibrations are repeated periodically. In the example shown in Figure 2, the vibration level of the second notification is set to "medium," but the vibration level of the second notification is not particularly limited.

[0031] Thus, according to this embodiment, the vibration rhythm pattern is common for the first notification that informs the driver of the distance between the vehicle and surrounding stationary objects, making it easy to recognize that the currently running notification is the first notification. Furthermore, the degree of distance to stationary objects can be communicated to the driver through the strength of the vibration level, allowing for an intuitive and immediate understanding of whether the distance to stationary objects is close or far. Additionally, when the notification content changes continuously, the change can be easily perceived through the change in vibration level.

[0032] Furthermore, regarding the second notification, which alerts the vehicle to the presence of a moving object approaching it, the vibration rhythm pattern is set to intermittent vibration, making it clear that it differs from the first notification, which uses continuous vibration. This makes it easy to recognize that the notification currently being given is the second notification, and also makes it easy to recognize when the notification content switches continuously from, for example, the first notification to the second notification.

[0033] Figure 3 is a flowchart illustrating the details of the control content related to the first notification. The control device 3 repeatedly executes this routine at a predetermined calculation cycle.

[0034] In step S11, the control device 3 determines the distance to stationary objects around the vehicle based on the vehicle surroundings information. In this embodiment, the control device 3 recognizes stationary objects around the vehicle based on the distance measurement data and determines the distance to the recognized stationary objects. Recognition of stationary objects can be performed, for example, by grouping reflection points that satisfy predetermined conditions as reflection points of laser light, etc., reflected from the same object, based on the distance to each reflection point and the coordinate information of each reflection point. Camera images can also be used for stationary object recognition.

[0035] In step S12, the control device 3 determines whether the distance to the stationary object is less than a predetermined proximity threshold D1. If the distance to the stationary object is less than the proximity threshold D1, the control device 3 proceeds to step S13. On the other hand, if the distance to the stationary object is equal to or greater than the proximity threshold D1, the control device 3 proceeds to step S14.

[0036] In step S13, the control device 3 outputs a vibration signal to the vibration unit 21 so that the vibration rhythm pattern of the vibration unit 21 becomes "continuous vibration" and the vibration level becomes "strong," causing the vibration unit 21 to vibrate.

[0037] In step S14, the control device 3 determines whether the distance to the stationary object is less than a predetermined medium-distance threshold D2. The medium-distance threshold D2 is greater than the short-distance threshold D1. If the distance to the stationary object is less than the medium-distance threshold D2, the control device 3 proceeds to step S15. On the other hand, if the distance to the stationary object is equal to or greater than the medium-distance threshold D2, the control device 3 proceeds to step S16.

[0038] In step S15, the control device 3 outputs a vibration signal to the vibration unit 21 so that the vibration rhythm pattern of the vibration unit 21 becomes "continuous vibration" and the vibration level becomes "medium," causing the vibration unit 21 to vibrate.

[0039] In step S16, the control device 3 determines whether the distance to the stationary object is less than a predetermined long-distance threshold D3. The long-distance threshold D3 is greater than the medium-distance threshold D2. If the distance to the stationary object is less than the long-distance threshold D3, the control device 3 proceeds to step S17. On the other hand, if the distance to the stationary object is greater than or equal to the long-distance threshold D3, the control device 3 terminates the process without providing vibration-based notification.

[0040] In step S17, the control device 3 outputs a vibration signal to the vibration unit 21 so that the vibration rhythm pattern of the vibration unit 21 becomes "continuous vibration" and the vibration level becomes "weak," causing the vibration unit 21 to vibrate.

[0041] Figure 4 is a flowchart illustrating the details of the control content related to the second notification. The control device 3 repeatedly executes this routine at a predetermined calculation cycle.

[0042] In step S21, the control device 3 detects objects present around its own vehicle based on information about the vehicle's surroundings. In this embodiment, the control device 3 sequentially inputs camera images received from the external camera 11 into a classifier to detect the region in the camera image where an object is represented and the type of object represented in that region. The classifier can be, for example, a convolutional neural network (CNN) having multiple convolutional layers connected in series from the input side to the output side. The control device 3 then estimates the distance from the external camera 11 to the object using the standard size of each object stored in the memory unit 32 for each type of object and the size of the object detected in the camera image, and calculates the position and velocity of the object by tracking the object detected in the camera image over time. Note that the object detection method is not limited to this method, and various known methods may be used for detection.

[0043] In step S22, the control device 3 determines whether any moving objects, such as other vehicles or pedestrians approaching the vehicle, are located within a predetermined range around the vehicle. If the control device 3 determines that a moving object approaching the vehicle is located within the predetermined range around the vehicle, it proceeds to step S23 to prevent a collision. On the other hand, if the control device 3 determines that there is no risk of collision, it terminates the process without issuing a vibration notification.

[0044] In step S23, the control device 3 outputs a vibration signal to the vibration unit 21 so that the vibration rhythm pattern of the vibration unit 21 becomes "intermittent vibration" and the vibration level becomes "medium," causing the vibration unit 21 to vibrate.

[0045] The notification system 100 according to this embodiment, as described above, comprises a vibration unit 21 that provides vibration stimuli to the vehicle occupants and a control device 3 that controls the vibration unit 21. The control device 3 provides a first notification to the vehicle occupants by vibrating the vibration unit 21, based on information about the vehicle's surroundings, to inform them of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification to inform them of the presence of a moving object approaching the vehicle. When providing the first notification, the control device 3 vibrates the vibration unit 21 in a vibration rhythm pattern that vibrates the vibration unit continuously, regardless of the degree of distance to the stationary object, and changes the vibration level of the vibration unit 21 according to the degree of distance to the stationary object. When providing the second notification, the control device 3 is configured to vibrate the vibration unit 21 in a vibration rhythm pattern that vibrates the vibration unit 21 intermittently.

[0046] In this embodiment, the control device 3 is configured to increase the vibration level when the distance to the vehicle and surrounding stationary objects is greater when the distance to the stationary object is close compared to when it is far away, in order to inform the vehicle of the degree of distance between the vehicle and the stationary object in the first notification. The vehicle surrounding information also includes camera images (image data) from an external camera 11 that photographs objects around the vehicle, and distance measurement data from a distance measuring sensor 12 that measures the distance to objects around the vehicle.

[0047] Thus, according to this embodiment, the vibration rhythm pattern is common for the first notification that informs the driver of the distance between the vehicle and surrounding stationary objects, making it easy to recognize that the currently running notification is the first notification. Furthermore, the degree of distance to stationary objects can be communicated to the driver through the strength of the vibration level, allowing for an intuitive and immediate understanding of whether the distance to stationary objects is close or far. Additionally, when the notification content changes continuously, the change can be easily perceived through the change in vibration level.

[0048] Furthermore, the second notification, which alerts the vehicle to the presence of a moving object approaching it, uses a vibration rhythm pattern, clearly distinguishing it from the first notification, which uses continuous vibration. This makes it easy to recognize that the currently running notification is the second notification, and also makes it easy to detect when the notification content switches continuously from the first notification to the second notification.

[0049] Therefore, with the notification system 100 according to this embodiment, even when multiple types of notifications with different content (the "strong," "medium," and "weak" levels of the first notification, and the second notification) are transmitted by vibration, the driver can easily grasp the content of the notification and its transitions through vibration. For this reason, even a driver with a hearing impairment, for example, can easily grasp the content of multiple types of notifications through vibration.

[0050] Although embodiments of the present invention have been described above, these embodiments only represent a part of the application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific configurations of the above embodiments.

[0051] For example, in the above embodiment, the notification device 2 was equipped with a vibration unit 21, but it may also be equipped with a display for providing notification through the driver's sight, or a speaker for providing notification through the driver's hearing. In that case, in addition to notification by vibration from the vibration unit 21, notification may also be provided by displaying text information on the display or by providing sound from the speaker.

[0052] Furthermore, in the above embodiment, the computer program executed in the control device 3 may be provided in the form of a computer-readable portable recording medium, such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. [Explanation of Symbols]

[0053] 3. Control device 21 Vibration Unit 100 Notification System

Claims

1. A vibration unit that provides vibration stimulation to the vehicle occupants, A control device for controlling the vibration unit, A notification system comprising, The control device is Based on the surrounding information of the vehicle, a first notification informing the occupants of the vehicle of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification informing the occupants of the vehicle of the presence of a moving object approaching the vehicle, are provided by vibrating the vibration unit. When the first notification is made, the vibration unit is vibrated in a vibration pattern that causes the vibration unit to vibrate continuously, regardless of the distance from the stationary object, and the vibration level of the vibration unit is changed according to the distance from the stationary object. When the second notification is performed, the vibration unit is configured to vibrate with a vibration pattern that causes the vibration unit to vibrate intermittently. Notification system.

2. The control device is In the first notification, the system is configured to indicate the degree of distance between the vehicle and stationary objects in its vicinity, and to increase the vibration level when the distance to the stationary objects is close compared to when they are far away. The notification system according to claim 1.

3. The surrounding information includes image data from a camera that photographs objects around the vehicle and distance measurement data from a distance measuring sensor that measures the distance to objects around the vehicle. The notification system according to claim 1 or claim 2.

4. A notification method using a control device that controls a vibration unit that provides vibration stimuli to the occupants of a vehicle, Based on the surrounding information of the vehicle, a first notification informing the occupants of the vehicle of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification informing the occupants of the vehicle of the presence of a moving object approaching the vehicle, are provided by vibrating the vibration unit. When the first notification is made, the vibration unit is vibrated in a vibration pattern that causes the vibration unit to vibrate continuously, regardless of the distance from the stationary object, and the vibration level of the vibration unit is changed according to the distance from the stationary object. When the second notification is performed, the vibration unit is vibrated in a vibration pattern that causes the vibration unit to vibrate intermittently. Notification method.

5. A computer program for a control device that controls a vibration unit that provides vibration stimuli to the occupants of a vehicle, Based on the surrounding information of the vehicle, a first notification informing the occupants of the vehicle of the degree of distance between the vehicle and stationary objects in its vicinity, and a second notification informing the occupants of the vehicle of the presence of a moving object approaching the vehicle, are provided by vibrating the vibration unit. When the first notification is made, the vibration unit is vibrated in a vibration pattern that causes the vibration unit to vibrate continuously, regardless of the distance from the stationary object, and the vibration level of the vibration unit is changed according to the distance from the stationary object. When the second notification is performed, the control device is instructed to vibrate the vibration unit in a vibration pattern that causes the vibration unit to vibrate intermittently. Computer program.

Images