Notification device and notification method

Abstract

A notification device is applied to a vehicle, the vehicle including one or more cameras that capture images of at least an area in front of the vehicle, and a human-machine interface device that performs a notification to draw the attention of a driver of the vehicle. The notification device includes a processor. The processor controls the human-machine interface device so that when the one or more cameras detect a no-U-turn sign in front of the vehicle and the one or more cameras detect a right-turn pavement marking in the vehicle's travel lane, the human-machine interface device notifies the driver that a U-turn is not permitted.

Description

Technical Field

[0001] The present disclosure relates to a notification device and a notification method applied to a vehicle. Background Art

[0002] Japanese Unexamined Patent Application Publication No. 2018-088188 (JP 2018-088188 A) discloses a driver assistance device. This driver assistance device uses an onboard camera to detect a traffic sign indicating that the vehicle's entry is regulated. When the vehicle exhibits behavior of moving toward a regulated lane where the vehicle's progress is regulated by the traffic sign, the driver assistance device suppresses this behavior or warns the driver using a device such as a display. Furthermore, a no-U-turn sign is described as an example of a traffic sign.

[0003] Japanese Unexamined Patent Application Publication No. 2019-212188 (JP2019-212188A) discloses a technology for notifying a driver of a road sign recognized by a camera using a notification unit such as a monitor device. Summary of the Invention

[0004] If the driver is notified every time the camera detects a no-U-turn sign, unnecessary notification may be performed even if the vehicle is actually traveling in a lane where a U-turn is not possible, such as a through lane.

[0005] The present disclosure is made in view of the above problems, and an object of the present disclosure is to inform the driver that U-turn is not allowed in appropriate scenarios.

[0006] A notification device according to the present disclosure is applied to a vehicle, including one or more cameras that capture images of at least an area in front of the vehicle, and a human-machine interface device that performs a notification to draw the attention of the vehicle's driver. The notification device includes a processor. The processor controls the human-machine interface device so that when one or more cameras detect a no-U-turn sign in front of the vehicle and one or more cameras detect a right-turn road surface marking in the vehicle's travel lane, the human-machine interface device notifies the driver that a U-turn is not permitted.

[0007] The one or more cameras may include a first camera and a second camera, the second camera being capable of capturing an image of a road surface area closer to the vehicle than the road surface area captured by the first camera. The processor may perform processing for detecting a right turn road surface marking by the second camera after the first camera detects a no-U-turn sign.

[0008] The processor may control the human-machine interface device so that even if a no-U-turn sign is not detected, the human-machine interface device notifies the driver that a U-turn is not permitted when one or more cameras detect a no-U-turn pavement marking in the vehicle's travel lane.

[0009] A notification method according to the present disclosure is applied to a vehicle, including one or more cameras that capture images of at least an area in front of the vehicle, and a human-machine interface device that performs a notification to draw the attention of a driver of the vehicle. The notification method includes controlling the human-machine interface device so that, when the one or more cameras detect a no-U-turn sign in front of the vehicle and the one or more cameras detect a right-turn road surface marking in the vehicle's travel lane, the human-machine interface device notifies the driver that a U-turn is not permitted.

[0010] According to the present disclosure, the driver is notified that a U-turn is not allowed only when a right-turn road marking is detected in the vehicle's driving lane, that is, when the vehicle is traveling in the right-turn lane. This suppresses unnecessary notifications and allows notifications for appropriate scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, wherein like numerals represent like elements, and wherein:

[0012] Figure 1 A diagram schematically showing an example of a configuration of a vehicle according to an embodiment;

[0013] Figure 2 is a diagram showing an example of an intersection that a vehicle is approaching;

[0014] Figure 3 A flowchart showing a process related to safe driving assistance according to an embodiment;

[0015] Figure 4 A flowchart showing a process related to safe driving assistance according to a first modification of the embodiment;

[0016] Figure 5 is a diagram showing an example of a right turn lane on which a no-U-turn pavement marking is painted; and

[0017] Figure 6 This is a flowchart showing a process related to safe driving assistance according to a second modification of the present embodiment. DETAILED DESCRIPTION

[0018] The embodiments of the present disclosure will be described with reference to the accompanying drawings. However, when the quantity, number, amount, range, etc. of each element are mentioned in the embodiments shown below, the technical ideas of the present disclosure are not limited to the quantities mentioned herein unless it has been clearly stated in principle or clearly specified in the quantity.

[0019] 1. Vehicle Configuration Example

[0020] Figure 11 is a diagram schematically showing an example of a configuration of a vehicle 1 according to an embodiment of the present invention. The vehicle 1 includes an Electronic Control Unit (ECU) 10 , a first camera 20 , a second camera 22 , and a Human Machine Interface (HMI) device 30 .

[0021] The ECU 10 is a computer that controls the vehicle 1. The ECU 10 includes one or more processors (hereinafter referred to as processors) 12 and one or more storage devices (hereinafter referred to as storage devices) 14. The processor 12 performs various processes. For example, the processor 12 includes a central processing unit (CPU). The storage device 14 stores various information required for the processing performed by the processor 12. Examples of the storage device 14 include volatile memory, non-volatile memory, a hard disk drive (HDD), and a solid state drive (SSD). The processor 12 executes various programs, thereby realizing the various processes performed by the processor 12. Various programs are stored in the storage device 14. A plurality of ECUs 10 can be provided.

[0022] The first camera 20 and the second camera 22 each capture an image of the area in front of the vehicle 1. Each of the "first camera" and the "second camera" according to the present disclosure can capture an image of at least the area in front of the vehicle, and in addition to capturing the area in front of the vehicle 1, can also capture images of areas in other directions (e.g., side areas).

[0023] The first camera 20 is mounted on the vehicle 1, for example, to recognize various targets such as road signs (e.g., Figure 2 The first camera 20 captures images of the area in front of the vehicle 1, such as the No U-turn sign 54 shown in FIG. 2 , vehicles in front of the vehicle 1 (such as the vehicle ahead and oncoming vehicles), and pedestrians. The first camera 20 is, for example, mounted in the cabin of the vehicle 1 (e.g., behind the rearview mirror) and captures images of the area in front of the vehicle 1 through the windshield. For example, the "first camera" according to the present disclosure can be mounted on the roof of the vehicle 1.

[0024] The second camera 22 is, for example, one of a plurality of cameras. Each of the plurality of cameras is located on the front, rear, right, or left side of the vehicle 1 and serves the basic purpose of assisting the driver in confirming the surroundings of the vehicle 1 when the vehicle 1 is traveling at a low speed, such as when the driver parks and starts the vehicle. The second camera 22 is, for example, a wide-angle fisheye camera and is mounted on the front bumper. The "second camera" according to the present disclosure may be, for example, at least one of the two cameras located on the right and left sides of the vehicle 1 (e.g., the lower side of a door mirror for rearward confirmation) for the above-mentioned purpose.

[0025] The second camera 22 is disposed vertically below the first camera 20. In addition, the second camera 22 has a field of view capable of capturing an image further down than the image captured by the first camera 20. Figure 1 As shown in FIG, the road surface region R2 of the image captured by the second camera 22 includes a road surface region R2A that is closer to the vehicle 1 than the road surface region R1 of the image captured by the first camera 20 .

[0026] The HMI device 30 has the function of issuing a notification to attract the attention of the driver of the vehicle 1. The HMI device 30 includes, for example, a display for visually attracting the driver's attention. The display is, for example, a display mounted on the dashboard of the vehicle 1 or a heads-up display (HUD) that displays information on the windshield of the vehicle 1. Furthermore, the HMI device 30 may include one or both of a buzzer and a speaker to aurally attract the driver's attention, either in place of or in addition to the display.

[0027] The HMI device 30 is controlled by the ECU 10 (processor 12 ). As an example, the “notification device” according to the present disclosure is configured to include the processor 12 , the first camera 20 , the second camera 22 , and the HMI device 30 .

[0028] 2. Safe driving assistance

[0029] Figure 2 1 is a diagram showing an example of an intersection that the vehicle 1 is approaching. Figure 2 A driving path 40 on which vehicle 1 is traveling and an opposing path 44 opposite driving path 40 via a median 42 are shown. As an example, driving path 40 has two lanes; and near intersection 46, driving path 40 has a through and left turn lane 48, a through lane 50, and a right turn lane 52.

[0030] A no-U-turn sign 54 is installed on the center median 42 near the intersection 46. In addition, on lanes 48, 50, and 52 near the intersection 46, pavement markings 56, 58, and 60 are painted respectively as pavement markings painted on the road surface (arrow markings indicate possible travel directions).

[0031] As one of the safety driving assistance measures for vehicle 1, processor 12 performs a notification to draw the driver's attention to the fact that U-turns are not permitted at intersection 46. It is conceivable that such a notification is performed each time the camera detects a no-U-turn sign 54. However, according to this method, there is a possibility that unnecessary notification may be performed even when vehicle 1 is actually traveling in a lane where U-turns are not permitted (such as through lane 50).

[0032] Therefore, in this embodiment, when the No U-Turn sign 54 is detected in front of the vehicle 1 and the right-turn road marking 60 is detected in the driving lane of the vehicle 1, the processor 12 notifies the driver that the U-turn is not permitted. Here, when this method is adopted and only the first camera 20 for detecting the No U-Turn sign 54 and the road marking 60 is used, there is an additional problem as follows.

[0033] That is, the first camera 20 has a field of view suitable for recognizing various targets including road signs such as the No U-turn sign 54, vehicles in front of the vehicle 1, and pedestrians, and is mounted in a position suitable for recognizing various targets. Therefore, when the right turn road marking 60 is located in the road surface area R1 of the first camera 20 (see FIG. Figure 1 ) and the road surface marking 60 is blocked by the vehicle ahead, even if the vehicle ahead moves away from the road surface marking 60 when the vehicle 1 later approaches the road surface marking 60, the road surface marking 60 may be beyond the field of view of the first camera 20. In other words, the first camera 20 may not be able to detect the right turn road surface marking 60.

[0034] Therefore, in this embodiment, the processor 12 is configured to execute the process of detecting the right turn road marking 60 by the second camera 22 after the no-U-turn sign 54 is detected by the first camera 20 .

[0035] Figure 3 1 is a flowchart showing the process related to safe driving assistance according to the present embodiment. When the vehicle 1 is traveling, the process of this flowchart is repeatedly executed.

[0036] exist Figure 3 In step S100, the ECU 10 (processor 12) determines whether the no-U-turn sign 54 is detected by image recognition using the first camera 20. As a result, when the no-U-turn sign 54 is not detected, the process proceeds to return.

[0037] On the other hand, if the No-U-Turn sign 54 is detected in step S100, the process proceeds to step S102. In step S102, the processor 12 determines whether a right-turn road surface marking 60 is detected in the host lane (the lane in which the vehicle 1 is currently traveling) by using image recognition by the first camera 20. For example, detection of the right-turn road surface marking 60 in the host lane can be performed as follows. Specifically, the processor 12 detects the white lines on the right and left sides of the vehicle 1 using the first camera 20, identifies the host lane using the detected white lines on the right and left sides of the vehicle 1, and detects the right-turn road surface marking 60 present in the identified host lane.

[0038] If the determination result in step S102 is "Yes" (i.e., if the first camera 20 detects the right-turn road surface marking 60 in the host lane), the process proceeds to step S106. On the other hand, if the determination result is "No" (i.e., if the first camera 20 does not detect the right-turn road surface marking 60 in the host lane), the process proceeds to step S104.

[0039] In step S104, the processor 12 determines whether a right turn road surface marking 60 is detected in the own lane by image recognition using the second camera 22. Even when the second camera 22 is used, detection of the right turn road surface marking 60 in the own lane can be performed, for example, by the above-described method.

[0040] When the determination result in step S104 is "No" (i.e., when neither the first camera 20 nor the second camera 22 detects the right turn road marking 60 in the own lane), the process proceeds to return. Figure 2 In the example shown in FIG, when traveling in the straight and left turn lane 48 or the straight lane 50, no right turn road surface marking 60 is detected in the vehicle's own lane. When the vehicle 1 is traveling in a driving lane other than the right turn lane 52, it can be determined that there is no or at least a low probability of making a U-turn. Therefore, if the determination result in step S104 is "No," even if the no-U-turn sign 54 is detected, notification by the HMI device 30 is not performed. As a result, the driver can be prevented from being annoyed by excessive notifications.

[0041] On the other hand, when the determination result in step S104 is “YES” (ie, when the second camera 22 detects the right-turn road surface marking 60 in the own lane), the process proceeds to step S106 .

[0042] As described above, when the first camera 20 or the second camera 22 detects the right-turn road surface marking 60 in the driver's lane, that is, when it is determined that the vehicle 1 is traveling in the right-turn lane 52, the process proceeds to step S106. In this case, it can be determined that there is a possibility of making a U-turn. Therefore, in step S106, the processor 12 controls the HMI device 30 to cause the HMI device 30 to issue a notification to the driver to alert him that the U-turn is not permitted.

[0043] 3. Effect

[0044] As described above, according to this embodiment, notification to the driver that a U-turn is not permitted is performed when not only the No-U-Turn Sign 54 is detected but also the Right-Turn Road Marking 60 is detected in the lane in which the vehicle 1 is traveling. Therefore, notification is performed only when the Right-Turn Road Marking 60 is detected in the lane in which the vehicle 1 is traveling, in other words, only when the vehicle 1 is traveling in the Right-Turn Lane 52. This suppresses unnecessary notifications and allows notifications tailored to appropriate scenarios.

[0045] Furthermore, according to the present embodiment, in order to detect the right-turn road surface marking 60, not only the first camera 20 but also the second camera 22 is used when necessary. The second camera 22 can capture an image of a road surface area R2A closer to the vehicle 1 than the road surface area R1 of the image captured by the first camera 20 (see FIG. Figure 1 ). Therefore, even in a case where the right-turn road surface marking 60 cannot be detected by the first camera 20 due to the presence of a preceding vehicle when the vehicle 1 (host vehicle) is approaching the road surface marking 60, the road surface marking 60 can be detected by the second camera 22 when the vehicle 1 approaches the road surface marking 60. Therefore, compared with an example in which the road surface marking 60 is detected using only the first camera 20, the right-turn road surface marking 60 can be detected more reliably.

[0046] 4. First Modification

[0047] In the above configuration, the second camera 22 is used together with the first camera 20 to detect the right-turn road surface marking 60. However, as in the first modified example to be described below, according to the present disclosure, the detection of the right-turn road surface marking 60 for determining whether the driver should be notified that a U-turn is not allowed may not necessarily be performed using both the first camera 20 and the second camera 22.

[0048] Figure 4 This is a flowchart showing the processing related to safe driving assistance according to the first modified example of this embodiment. The processing in this flowchart is the same as that in the first modified example except that the processing in step S104 is omitted. Figure 3 The processing is the same as that in the flowchart shown in . That is, according to Figure 4 In the process shown in , only the first camera 20 is used to detect the right turn road surface marking 60. As described in the first modification, only the first camera 20 for detecting the no-U-turn sign 54 may be used to detect the right turn road surface marking 60.

[0049] 5. Second Modification

[0050] In the above embodiment, when the no-U-turn sign 54 is not detected, the driver is not notified that the U-turn is not allowed. Here, for example, it may be difficult to detect (recognize) the no-U-turn sign 54 due to obstacles such as trees or backlight from the sun. Therefore, the notification that the U-turn is not allowed can be performed as follows.

[0051] Figure 5 FIG is a diagram showing an example of a right turn lane on which a no-U-turn pavement marking 72 is painted. Figure 5 A right turn lane 70 shown in FIG. 1 is painted with a no-U-turn pavement marking 72 as well as the right turn pavement marking 60. In the second modification, the processor 12 controls the HMI device 30 so that even if the no-U-turn sign 54 is not detected, when the first camera 20 or the second camera 22 detects the no-U-turn pavement marking 72 in the driving lane of the vehicle 1, the HMI device 30 notifies the driver that a U-turn is not permitted.

[0052] Figure 6 This is a flowchart showing the processing related to safe driving assistance according to the second modified example of this embodiment. The processing in this flowchart is the same as that in the second modified example except that the processing in step S200 and step S202 is added. Figure 3 The processing is the same as in the flowchart shown in .

[0053] exist Figure 6 In step S100, when the determination result is "No" (i.e., when the No U-turn sign 54 is not detected), the process proceeds to step S200. In step S200, the processor 12 determines whether a No U-turn road surface marking 72 is detected in the own lane by using image recognition by the first camera 20. Detection of the No U-turn road surface marking 72 in the own lane can be performed in the same manner, for example, using the method described above for detecting the right turn road surface marking 60 in the own lane.

[0054] If the determination result in step S200 is "Yes" (i.e., if the first camera 20 detects the no-U-turn road marking 72 in the host lane), the process proceeds to step S106. On the other hand, if the determination result is "No" (i.e., if the first camera 20 does not detect the no-U-turn road marking 72 in the host lane), the process proceeds to step S202.

[0055] In step S202, the processor 12 determines whether a no-U-turn road marking 72 is detected in the host lane by image recognition using the second camera 22. Even when the second camera 22 is used, detection of the no-U-turn road marking 72 in the host lane can be performed, for example, by the above-described method.

[0056] When the determination result in step S202 is “No” (ie, when neither the first camera 20 nor the second camera 22 detects the no-U-turn road marking 72 in the own lane), the process proceeds to RETURN. That is, the HMI device 30 does not perform the notification.

[0057] On the other hand, when the determination result in step S202 is “YES” (ie, when the second camera 22 detects the no-U-turn road surface marking 72 in the own lane), the process proceeds to step S106 .

[0058] As mentioned above, according to Figure 6 In the process shown in FIG, not only when the determination result in step S102 or step S104 is "Yes," but also when the determination result in step S200 or step S202 is "Yes," the process proceeds to step S106, and HMI device 30 performs notification. As a result, even if no-U-turn sign 54 cannot be detected due to an obstacle, etc., a notification can be performed when it can be determined that vehicle 1 is traveling in a lane where U-turns are not permitted by detecting no-U-turn pavement markings 72 applied to the road surface. This can increase the chances of notification in appropriate scenarios where notification should be performed.

[0059] For example, only the first camera 20 may be used to detect the no-U-turn road marking 72. That is, the process of step S200 may be combined with Figure 4 The processing in the flowchart shown in FIG.

[0060] In addition, the above embodiment (as well as the first and second variations) has been described for countries or regions where vehicles travel on the left side of the road (such as Japan). However, each of the "notification device" and "notification method" according to the present disclosure can be applied in the same manner to vehicles traveling in countries or regions that require right-hand driving. Specifically, when the "notification device" or "notification method" is applied to a vehicle traveling in a country or region that requires right-hand driving, the "right turn" in the above description can be interpreted as a "left turn."

Claims

1. A notification device, applied to a vehicle, the vehicle comprising: one or more cameras that capture images of at least an area in front of the vehicle, the one or more cameras including a first camera and a second camera, the second camera being capable of capturing images of an area of ​​the road surface closer to the vehicle than the area of ​​the road surface captured by the first camera; as well as a human-machine interface device that performs a notification to draw the attention of a driver of said vehicle, Wherein, the notification device includes a processor, The processor is configured to, after the first camera detects a no-U-turn sign in front of the vehicle, execute a process of detecting a right-turn road surface marking in a driving lane of the vehicle by the second camera, and control the human-machine interface device so that, when the second camera detects a right-turn road surface marking in the driving lane of the vehicle, the human-machine interface device notifies the driver that a U-turn is not permitted.

2. The notification device according to claim 1, wherein: The processor controls the human-machine interface device so that the human-machine interface device notifies the driver that a U-turn is not permitted when the one or more cameras detect a no-U-turn pavement marking in the vehicle's driving lane even if the no-U-turn sign is not detected.

3. A notification method, applied to a vehicle, the vehicle including one or more cameras that capture images of at least an area in front of the vehicle, and a human-machine interface device that performs notification to draw the attention of a driver of the vehicle, the one or more cameras including a first camera and a second camera, the second camera being capable of capturing an image of an area of ​​a road surface closer to the vehicle than an area of ​​the road surface captured by the first camera, in, The notification method includes: After the first camera detects a no-U-turn sign in front of the vehicle, the second camera detects a right-turn road marking in the vehicle's driving lane, and The human-machine interface device is controlled so that when a right-turn road surface marking is detected in the driving lane of the vehicle by the second camera, the human-machine interface device notifies the driver that a U-turn is not permitted.

Images