Driver assistance device and driver assistance procedure

Abstract

Driving assistance device (20) which assists in driving a self-propelled vehicle (40), wherein the driving assistance device comprises: - a character recognition unit (21) that recognizes a traffic sign in an image captured by an on-board imaging device (11); and - a notification execution unit (23) which informs a driver of sign information, which is the information displayed by the traffic sign recognized by the sign recognition unit, wherein - the notification execution unit stops providing the sign information to the driver based on the vehicle speed of the own vehicle and operating information of a direction indicator (14) of the own vehicle under a condition that the sign information is reported to the driver, - the notification execution unit stops providing the sign information to the driver when a predetermined condition is met, in a state where the driver is being informed of the sign information, wherein the predetermined condition cumulatively comprises: a first condition that the vehicle speed falls below a target vehicle speed (Vth); and a second condition as operational information that the direction indicator is activated; and a third condition relating to the steering angle of the vehicle. The notification execution unit continues to provide the sign information to the driver before the vehicle turns left or right, until the vehicle has turned left or right due to the fulfillment of the predetermined condition. wherein the third condition, which concerns the steering angle of the own vehicle, is the exceeding of a determined value (θth) by an integrated yaw rate value (Σθ), wherein the integrated yaw rate value (Σθ) is calculated by integrating the yaw rates in the same direction as the actuation of the direction indicator.

Description

BACKGROUND OF THE INVENTION [Field of Invention]

[0001] The present invention relates to a driver assistance device and a driver assistance method. The present invention relates in particular to a method for assisting the driver in operating a vehicle. [Previous state of the art]

[0002] A known method involves recognizing a traffic sign in an image of the vehicle's surroundings captured by an onboard camera. Information displayed by the recognized traffic sign is shown on a monitor inside the vehicle's passenger compartment (see, for example, JP 4604088 B2). A driver can thus access information about regulations affecting a road, guidance information, and the like simply by glancing at the information displayed on the monitor.

[0003] However, in the above procedure according to JP 4 604 088 B2, only the detection result of an area in front of a vehicle from an onboard camera is displayed. Consequently, even if the road has changed, for example as a result of the vehicle turning right or left, the detection result before the road change may still be displayed on the monitor. In this case, the driver may be provided with sign information (i.e., traffic sign information) that differs from the traffic sign applicable to the road on which the vehicle is actually traveling. Consequently, there are concerns that accurate driver assistance cannot be provided.

[0004] Further relevant prior art is known from publication US 2015 / 0035 981 A1.

[0005] Document DE 10 2004 042 382 A1 is aimed at the detection of traffic signs by means of suitable sensors and the display of the traffic sign information only for a specific period of time, whereby the display time increases with higher driving speeds.

[0006] Document DE 10 2010 039 634 A1 discloses that if a vehicle is no longer in the previous traffic situation, the displayed speed limit can be deleted.

[0007] Document DE 10 2011 081 456 A1 finally discloses a device and a method for traffic sign recognition, in which, in assessing whether the display of a recognized traffic sign should be suppressed or stopped, the vehicle speed is taken into account in addition to the operation of the direction indicator and a steering angle. SUMMARY

[0008] It is therefore an object of the present invention to provide a driver assistance device that accurately informs a driver about information on a traffic sign.

[0009] The problem is solved by a driver assistance device having the features of claim 1 and by a driver assistance method having the features of claim 2. Further developments according to the invention are the subject of the dependent claims.

[0010] The present invention uses, among other things, the following means to solve the problem described above.

[0011] An exemplary embodiment of the present invention provides a driver assistance device that assists in driving a self-driving vehicle. The driver assistance device comprises: a sign recognition unit that recognizes a traffic sign in an image captured by an on-board imaging device, and a notification execution unit that reports sign information to the driver, which is the information displayed by the traffic sign recognized by the sign recognition unit. Furthermore, when the sign information is reported to the driver, the notification execution unit stops reporting the sign information to the driver based on the vehicle speed and the operating information of a turn signal indicator of the vehicle.

[0012] Based on the vehicle's speed and the turn signal indicator's operating information, the system can detect, for example, a situation in which the road is about to change, or a situation after the road has changed, coinciding with the vehicle's right or left turn. If the road changes, the sign information applicable to the vehicle's lane can also change. As a result of the configuration described above, in which the reporting of sign information based on the vehicle's speed and the turn signal indicator's operating information is stopped, situations in which the driver receives unsafe sign information can be prevented to the greatest extent possible. This ensures that the driver is accurately informed about the information on the traffic sign and receives precise assistance for safe driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The attached drawings show: Fig. 1 a block diagram of an overall configuration of a driver assistance system according to an embodiment; Fig. 2A and Fig. 2B Illustrations of a transition in a monitor display during a left turn according to the embodiment; Fig. 3 a diagram of a flowchart of a processing procedure for a left / right turn determination process according to the embodiment; Fig. 4. A flowchart of a processing procedure for a sign reporting process according to the embodiment; and Fig. 5 a time diagram of a transition in a monitor display of character information according to the embodiment. DESCRIPTION OF THE EXECUTION FORMS

[0014] An embodiment that implements a driver assistance device is described below with reference to the drawings. The driver assistance device is an onboard device that performs driver assistance for a vehicle based on an image of an area in front of the vehicle. The image is captured by an imaging device mounted in the vehicle. First, an overall configuration of a driver assistance system according to the present embodiment is described with reference to the drawings. Fig. 1 described.

[0015] In the Fig. 1 is an imaging device 11, an onboard camera. The imaging device 11 is composed of at least either a CCD (charge-coupled device) camera, a CMOS (complementary metal-oxide-semiconductor) image sensor, a near-infrared camera, or the like. The imaging device 11 is, for example, located near the upper edge of the vehicle's windshield. The imaging device 11 records the surroundings, including the road in front of the vehicle.

[0016] A driver assistance device 20 is a computer comprising a central processing unit (CPU), read-only memory (ROM), read / write memory (RAM), input / output (I / O), and the like. The driver assistance device 20 includes a character recognition unit 21 and a notification execution unit 23. The driver assistance device 20 executes a program stored on a storage medium. The various functions are implemented by the driver assistance device 20 executing the program.

[0017] The character recognition unit 21 captures an image taken by the imaging device 11. The character recognition unit 21 recognizes a traffic sign contained in the captured image by means of pattern matching. In the pattern matching, the character recognition unit 21 uses, for example, pre-prepared patterns for traffic signs. The method for recognizing a traffic sign in an image is not limited to this. For example, a method can also be used in which recognition is based on a linear classifier. The character recognition unit 21 recognizes a number displayed on the traffic sign from features in the recognized traffic sign. The character recognition unit 21 stores the recognized number on the traffic sign in a memory unit 22. The memory unit 22 consists of rewritable, non-volatile memory.

[0018] The notification execution unit 23 receives an input of information (hereinafter referred to as sign information) displayed by the traffic sign, which is recognized by the sign recognition unit 21. The notification execution unit 23 then reports the input sign information to the driver. For example, if a traffic sign indicating a speed limit is recognized, the notification execution unit 23 reports a numerical value to the driver indicating the speed limit displayed on the sign. The notification execution unit 23 performs the notification via a notification unit 30. The notification unit 30 includes a monitor 31 that displays an image, a speaker 32 that emits a warning tone and voice guidance, and the like.According to the present embodiment, the monitor 31 is arranged in a position within the vehicle passenger compartment that is visible to the driver. The monitor 31 displays the numerical value indicating the speed limit. The monitor 31 thus provides the driver with speed limit information applicable to the road on which the vehicle is currently traveling and serves as a warning to exercise caution.

[0019] The notification execution unit 23 receives input from detection signals from various sensors installed in the vehicle. It also receives input from operating signals from various devices operated by the driver. Based on these input signals, the notification execution unit 23 displays the corresponding information to the driver. The various sensors include a vehicle speed sensor 12, which detects the vehicle's speed; a yaw rate sensor 13, which detects the angular velocity (yaw rate) in one direction of rotation of the vehicle; and similar sensors. The yaw rate sensor 13 is equivalent to a sensor that detects the vehicle's steering angle.

[0020] The vehicle also has a direction indicator 14, which indicates the vehicle's direction of travel, located outside the vehicle. The direction indicator 14 has a control lever that serves as a left / right turn indicator unit. The driver operates the control lever to input the direction in which the vehicle will turn, i.e., a left turn or a right turn. The control lever is operated by the driver to set either a left indicator position, a neutral position, or a right indicator position. Based on the position to which the control lever is set, a flashing signal is output to the notification execution unit 23 as operational information.

[0021] In this context, the road changes when the vehicle turns right or left. This may also result in changes to the applicable speed limit or similar restrictions. If the speed limit or similar restriction remains displayed on monitor 31 even after the road changes, the driver will receive incorrect sign information. Furthermore, there are concerns that accurate driver assistance will not be possible.

[0022] Consequently, when the driver is notified of the sign information recognized by the sign recognition unit 21, the notification execution unit 23 stops informing the driver about the sign information currently being provided, based on the vehicle speed of the vehicle and the operating information of the direction indicator 14. In particular, when the driver is notified of the sign information, the notification execution unit 23 stops providing the sign information to the driver if predetermined conditions are met, including the following two conditions (corresponding to a first and a second condition): (1) the vehicle speed falls below a destination vehicle speed, and (2) the direction indicator 14 is activated. Consequently, an unnatural state in which unsafe sign information is provided to the driver can be prevented.

[0023] The Fig. 2A and Fig. Figure 2B shows a transition in the display on monitor 31 when the vehicle 40 turns left, from a state in which the vehicle 40 is traveling straight ahead. In the Fig. 2A and Fig. 2B shows the numerical value indicating the speed limit on road 61 before the left turn as "50". The drawings show the Fig. 2A a state immediately before the own vehicle 40 reaches an intersection. Fig. 2B shows a state in which the company vehicle 40 turns left at the intersection.

[0024] In the Fig. 2A, when an image of a speed limit sign 63, which applies to road 61, is captured by the imaging device 11, the numerical value displayed on the speed limit sign 63 is shown as the sign information on the monitor 31 in the state in which the vehicle 40 is traveling straight ahead on road 61. This is as described in Fig. 2A shows the numerical value “50”, which is displayed on the speed limit sign 63, which applies to the own lane, displayed on monitor 31.

[0025] A situation is considered in which, when the sign information applicable to road 61 is displayed on monitor 31, the vehicle 40 reaches an intersection. The vehicle 40 turns left at the intersection and proceeds on road 62, which extends in a left / right direction with respect to the vehicle's direction of travel prior to the left turn. In this case, the notification execution unit 23 stops reporting the sign information to the driver when conditions are met, including that the vehicle's speed falls below a target vehicle speed Vth and that the turn signal 14 is activated. This is described in Fig. As shown in Figure 2B, the sign information displayed on monitor 31 is deleted and no longer shown. This state, in which the sign information is not displayed, continues until a traffic sign on road 62 is detected. After the left turn is completed, if the imaging device 11 captures an image of a traffic sign on road 62, the sign information is displayed again on monitor 31.

[0026] Below is a driver assistance control unit of the present embodiment with reference to the one described in the Fig. 3 and Fig. The four flowcharts shown describe each process. Fig. 3 and Fig. The four components described below can be implemented as hardware using an electronic circuit. Alternatively, at least one section can be implemented using software, i.e., using a process executed in a microcomputer.

[0027] First, a left / right turn determination process executed by the notification execution unit 23 is performed with reference to the Fig. 3 described. This process takes place in a predetermined cycle by the notification execution unit 23.

[0028] In the Fig. In step S100, the notification execution unit 23 determines whether the turn signal input from the turn signal indicator 14 is set to ON, i.e., whether the turn signal indicator 14's operating lever is actuated and switched from the neutral position to the left or right indicator position. If it is determined that the turn signal is set to ON, the notification execution unit 23 proceeds to step S101. The notification execution unit 23 calculates an integrated value (hereinafter referred to as the integrated yaw rate value Σθ) of the yaw rates detected by the yaw rate sensor 13. Here, the measurement period, or integration period, is set to a time span from the moment the turn signal 14 is actuated by the driver, who changes the position of the operating lever and switches, until the present time.The notification execution unit 23 calculates the integrated yaw rate value Σθ by integrating the yaw rates in the same direction as the indicated direction shown by the turn signal of the direction indicator 14.

[0029] In the subsequent step S102, the notification execution unit 23 determines whether the integrated yaw rate value Σθ is greater than a specified value θth. If it is determined that the integrated yaw rate value Σθ is less than or equal to the specified value θth, the notification execution unit 23 proceeds to step S105 and determines that the vehicle 40 will neither turn left nor right. Conversely, if it is determined that the integrated yaw rate value Σθ is greater than the specified value θth, the notification execution unit 23 proceeds to step S103.

[0030] In step S103, the notification execution unit 23 determines whether the vehicle speed detected by the vehicle speed sensor 12 is below the target vehicle speed Vth. The target vehicle speed Vth is a vehicle speed determination value used to determine whether the vehicle 40 is attempting to turn right or left. The target vehicle speed Vth is set, for example, to a value of 30 to 40 km / h. According to the present embodiment, the vehicle speed is compared with the target vehicle speed Vth after the point at which the integrated yaw rate value Σθ exceeds the target value θth.

[0031] If it is determined that the vehicle's own speed is greater than or equal to the destination vehicle speed Vth, the notification execution unit 23 advances to step S105 and determines that the vehicle 40 will neither turn right nor left. However, if it is determined that the vehicle's own speed is less than the destination vehicle speed Vth, the notification execution unit 23 advances to step S104. The notification execution unit 23 determines that the vehicle 40 will turn left or right and terminates the current routine.

[0032] If the turn signal changes from ON to OFF before the integrated yaw rate value Σθ exceeds the target value θth or before the vehicle's own speed falls below the target vehicle speed Vth, the notification execution unit 23 determines "NO" in step S100 and advances to step S105. The notification execution unit 23 determines that the vehicle 40 will not turn left or right.

[0033] Below is a sign notification process executed by the notification execution unit 23 with reference to the Fig. 4 described. This is executed by the notification execution unit 23 in a predetermined cycle.

[0034] In the Fig. 4 determines the notification execution unit 23, in step S200, whether or not the own vehicle 40 turns left or right, based on the result of the left / right turn determination process in the Fig. 3. If it is determined that the vehicle 40 will neither turn right nor left, the notification execution unit 23 immediately terminates the current routine. However, if it is determined that the vehicle 40 will turn left or right, the notification execution unit 23 proceeds to step S201. The notification execution unit 23 determines whether or not the driver is informed about the sign information on the traffic sign detected by the sign recognition unit 21. In this step, the notification execution unit 23 determines whether or not the sign information applicable to the lane in which the vehicle is currently traveling is displayed on the monitor 31.

[0035] If it is determined that the traffic sign in the driver's lane is not recognized and the driver is not informed of the sign information, the notification execution unit 23 determines "NO" in step S201 and immediately terminates the current routine. However, if it is determined that the driver is informed of the sign information, the notification execution unit 23 proceeds to step S202. The notification execution unit 23 stops the provision of the sign information by the notification unit 30. In this step, the notification execution unit 23 clears the currently displayed sign information on the monitor 31.

[0036] Below is a specific aspect of the transition on the monitor display of character information with reference to the one in the Fig. The time diagram shown in section 5 describes the scene in which the vehicle travels at 40 km / h. This scene is identical to the one in the... Fig. 2. In the Fig. 5. It is assumed that the driver turns left at an intersection while traveling at a vehicle speed greater than or equal to the destination vehicle speed Vth. At this time, when the driver activates the direction indicator 14, the turn signal is switched from OFF to ON (time t11).

[0037] Furthermore, the driver performs actions such as releasing the accelerator pedal, applying the brakes, and so forth, in order to reduce the speed of the vehicle 40. The speed of the vehicle 40 eventually falls below the target vehicle speed Vth (time t12). When the integrated yaw rate value Σθ in the left direction exceeds the target value θth because the driver is turning the steering wheel, the character information displayed on monitor 31 is cleared and no longer shown (time t13).

[0038] According to the present embodiment, which is described in more detail above, the following advantageous effects are achieved.

[0039] Based on the vehicle speed of the vehicle 40 and the operating information of the turn signal 14, a scene in which the road is about to change because the vehicle 40 is turning left or right, or a scene after the road has changed, can be detected. If the road changes, the sign information applicable to the vehicle's lane can also change accordingly. In this respect, as a result of the configuration described above, a situation in which the driver is presented with unsafe sign information can be prevented to the greatest extent possible. This ensures that the driver receives accurate assistance for safe driving.

[0040] In particular, in a state where the driver is being informed of the sign information, the sign information displayed on monitor 31 is cleared when the predetermined conditions, including the following two conditions, are met: (1) the own vehicle speed falls below the destination vehicle speed Vth; and (2) the turn signal 14 is activated. When the turn signal 14 is activated to indicate the right turn or left turn direction while the own vehicle is traveling at 40, and the own vehicle speed is low, the probability that the own vehicle 40 will turn left or right is high, compared to or rather than changing lanes.Given this probability, the configuration described above allows for a distinction between a left / right turn and a lane change. Consequently, it prevents the provision of unsafe signal information when a left / right turn occurs. Furthermore, during a lane change, it prevents the erroneous termination of signal information when no lane change is taking place.

[0041] The provision of signal information to the driver is stopped based on the vehicle's own speed, the operating information of the turn signal indicator 14, and the steering angle of the vehicle 40. Specifically, after the turn signal is switched to ON, the vehicle's own speed is compared with the target vehicle speed Vth at the time when the integrated yaw rate value Σθ exceeds the target value θth, or immediately thereafter. Provided that the vehicle's own speed is less than the target vehicle speed Vth, it is determined that the vehicle 40 is turning right or left. The provision or transmission of signal information to the driver is stopped. A driver turning left or right usually operates the steering wheel to initiate the turn.

[0042] Consequently, since the steering angle of the vehicle 40 is added to the determination conditions, the accuracy in determining a scene in which the road changes, i.e., where the road changes direction, can be further improved. Furthermore, the reporting of the road sign information 61 prior to the left / right turn continues until the vehicle 40 actually turns left or right. This prevents, for example, the driver from experiencing discomfort resulting from no longer being informed of the sign information while driving straight ahead. (Other embodiments)

[0043] The present invention is not limited to the embodiment described above. The present invention can, for example, be implemented as follows. Sections below are identical to those in the embodiment described above and are designated with the same reference numerals and are not repeated below.

[0044] In the Fig. 2. The sign information displayed on monitor 31 is deleted at the moment when the vehicle 40 changes from road 61, on which it is traveling before the left turn, to the road corresponding to its route after the left turn, i.e., during the left turn. However, the point in time at which the provision of the sign information is to be stopped is not specified here. For example, the provision of the sign information could be stopped after the vehicle 40 has completed the left / right turn.

[0045] According to the embodiment described above, the integrated yaw rate value Σθ is considered a parameter relating to the steering angle of the vehicle 40. The transmission of the signal information to the driver is stopped if the integrated yaw rate value Σθ exceeds the set value θth. However, the parameter relating to the steering angle of the vehicle 40 is not limited to the integrated yaw rate value Σθ. For example, a reading from a steering angle sensor that detects the steering angle of the steering wheel operated by the driver can be used.

[0046] According to the embodiment described above, the transmission of the sign information to the driver is stopped based on the vehicle's own speed, the operating information of the turn signal indicator 14, and the steering angle of the vehicle 40. However, the transmission of the sign information to the driver can also be carried out using the vehicle's own speed and the operating information of the turn signal indicator 14 without taking the steering angle of the vehicle into account.

[0047] In particular, the transmission of sign information is stopped, for example, when two conditions are met: the vehicle's own speed falls below the destination vehicle speed Vth and the turn indicator 14 is activated, regardless of whether or not the integrated yaw rate value Σθ exceeds the destination value θth. As a result of this configuration, the transmission of sign information can be stopped at a point before the vehicle 40 begins the left / right turn. In this case, the transmission of sign information to the driver can be stopped at the earliest possible time before the left / right turn.

[0048] According to the embodiment described above, in step S105 it is determined that no left / right turn occurs if in step S100 the Fig.3. It is determined that the turn signal is set to OFF. However, the period from the time the turn signal changes to ON until a predetermined time interval (such as a few seconds) has elapsed after the turn signal changes to OFF can be defined as the time interval in which the left / right turn determination takes place.

[0049] Specifically, the integrated yaw rate value Σθ is calculated over a period from the moment the turn signal is switched ON until a predetermined time interval has elapsed after the turn signal has been switched OFF. The vehicle is determined to turn left or right at 40° if the calculated integrated yaw rate value Σθ exceeds the target value θth and the vehicle's speed at that time is below the target vehicle speed Vth.

[0050] The conditions for determining whether or not the vehicle 40 turns left / right in the left / right turn determination process may include conditions different from the condition regarding the vehicle's speed, the condition regarding the operating information of the turn signal 14, and the condition regarding the steering angle. For example, the conditions for determining may include a condition indicating that the road will change, based on an image captured by the imaging device 11 or data from a navigation system.

[0051] According to the embodiment described above, a description has been given by way of example in which the information displayed by the traffic sign recognized by the sign recognition unit 21 is displayed on the monitor 31. However, the information can also be output as an acoustic signal from the loudspeaker 32. Alternatively, the display on the monitor 31 and the acoustic output from the loudspeaker 32 can occur simultaneously. Furthermore, the traffic sign whose sign information is to be reported is not limited to speed limit signs. Various traffic signs are applicable.

[0052] The forming elements described above should be understood conceptually and not as limiting the embodiment described above. For example, functions provided by a single forming element can be implemented by distributing them among several forming elements. Conversely, functions provided by several forming elements can be implemented by a single forming element.

Claims

[1] Driving assistance device (20) which assists in driving a self-propelled vehicle (40), wherein the driving assistance device comprises: - a character recognition unit (21) that recognizes a traffic sign in an image captured by an on-board imaging device (11); and - a notification execution unit (23) which informs a driver of sign information, which is the information displayed by the traffic sign recognized by the sign recognition unit, wherein - the notification execution unit stops providing the sign information to the driver based on the vehicle speed of the own vehicle and operating information of a direction indicator (14) of the own vehicle under a condition that the sign information is reported to the driver, - the notification execution unit stops providing the sign information to the driver when a predetermined condition is met, in a state where the driver is being informed of the sign information, wherein the predetermined condition cumulatively comprises: a first condition that the vehicle speed falls below a target vehicle speed (Vth); and a second condition as operational information that the direction indicator is activated; and a third condition relating to the steering angle of the vehicle. The notification execution unit continues to provide the sign information to the driver before the vehicle turns left or right, until the vehicle has turned left or right due to the fulfillment of the predetermined condition. wherein the third condition, which concerns the steering angle of the own vehicle, is the exceeding of a determined value (θth) by an integrated yaw rate value (Σθ), wherein the integrated yaw rate value (Σθ) is calculated by integrating the yaw rates in the same direction as the actuation of the direction indicator. [2] Driving assistance procedure performed by a driving assistance device (20) that assists in driving a self-propelled vehicle (40), wherein the driving assistance procedure comprises the following steps: - Recognition of a traffic sign in an image taken by an on-board imaging device (11); - Reporting sign information, which is the information displayed by the recognized traffic sign, to a driver; - Stopping the instruction of the driver about the sign information based on a vehicle speed of the own vehicle and operating information of a direction indicator (14) of the own vehicle under a condition that the driver is informed about the sign information; - Stopping the instruction of the driver regarding the sign information when a predetermined cumulative condition is met, in a state where the driver is being informed of the sign information, wherein the predetermined condition comprises: a first condition that the vehicle speed falls below a target vehicle speed (Vth); and a second condition as operational information that the direction indicator is activated; and a third condition relating to the steering angle of the vehicle. Continue providing the sign information to the driver before the vehicle turns left or right, until the vehicle has turned left or right due to the fulfillment of the predetermined condition. wherein the third condition, which concerns the steering angle of the own vehicle, is the exceeding of a determined value (θth) by an integrated yaw rate value (Σθ), wherein the integrated yaw rate value (Σθ) is calculated by integrating the yaw rates in the same direction as the actuation of the direction indicator.

Images