Vehicle driving assistance device, vehicle driving assistance method, and program

Abstract

The present invention relates to a driving assistance device for a vehicle, a driving assistance method for a vehicle, and a program. The driving assistance device for a vehicle comprises: a front camera and an image processing ECU, which detect a display object existing in front of the vehicle and recognize the display content of the detected display object; and a driving assistance ECU, which, when the display object is detected by the front camera and the image processing ECU, performs driving assistance control to assist the driver, and when the detected display object is a display object detected multiple times in the past, and the consistency rate of the recognition result as the ratio of the number of times the display content is determined to be the same to the number of detections is greater than a first threshold, the driving assistance ECU increases the assistance level of the driving assistance control compared to the case where the consistency rate of the recognition result is less than the first threshold.

Description

Technical Field

[0001] The present invention relates to a driving assistance device for a vehicle, a driving assistance method for a vehicle, and a program. Background Art

[0002] In the past, a driving assistance device for a vehicle is known, which detects display objects such as road signs existing outside the vehicle and displays the display content of the detected display object to the driver. For example, in Japanese Patent Laid-Open No. 2017-97681, a driving assistance device is disclosed, which has: a camera that detects a specific road sign existing in front of the vehicle that indicates that travel outside the specified direction is prohibited; and a display device and an alarm device, when the vehicle is traveling in a direction different from the specified direction of the specific road sign detected by the camera, the display device performs a reminder display, and the alarm device sounds an alarm. According to this driving assistance device, driving assistance control corresponding to the display content of the detected display object can be provided.

[0003] However, in the driving assistance device disclosed in Japanese Patent Laid-Open No. 2017-97681, when a road sign is erroneously detected by a camera, even if the driver performs a driving operation suitable for the displayed content of the road sign, the display device performs a reminder display and the alarm device sounds an alarm (sounds a warning). Therefore, when the camera erroneously detects a road sign, the driver may be annoyed by the reminder display performed by the display device and the warning sound generated by the alarm device. Summary of the invention

[0004] The present invention is made to address the above-mentioned problems. That is, one of the purposes of the present invention is to provide a driving assistance device, a vehicle driving assistance method, and a program that can reduce the annoyance felt by the driver in the case of a display object outside the vehicle being misdetected (including a case of misidentifying the display content of the display object) or in the case of a high possibility of misdetection.

[0005] In order to solve the above-mentioned problem, the driving assistance device of the vehicle of the present invention comprises: a camera sensor (front camera) which generates image data by photographing the scenery in front of the vehicle; and a control unit (driving assistance ECU) which, when it is determined based on the generated image data that a specific display object exists in front of the vehicle, identifies the display content of the specific display object determined to exist, and executes driving assistance control, which is control for assisting driving performed by the driver of the vehicle and is control corresponding to the display content of the specific display object, wherein the control unit (driving assistance ECU) is configured to: when it is determined that the consistency rate of the recognition result as a ratio of "the number of times the recognition results of the display content of the same specific display object are the same" to "the number of times the same specific display object is determined to exist" is less than a first threshold, perform control with a lower assistance level indicating the degree of influence on the driver as the driving assistance control compared with a case where it is determined that the consistency rate of the recognition result is greater than the first threshold.

[0006] In one aspect of the present invention, the driving assistance device of the vehicle also has a reporting device (display device, sound output device), which reports information related to the specific display object to the driver, and the control unit (driving assistance ECU) is configured to: when it is determined that the consistency rate of the recognition result is above the first threshold, execute a control including a reporting control and a suitable control as the driving assistance control, the reporting control is a control in which the reporting device (display device, sound output device) reports to the driver that the specific display object is determined to exist, and the suitable control is a vehicle control for making the driving state of the vehicle close to a state suitable for the display content of the specific display object, and when it is determined that the consistency rate of the recognition result is less than the first threshold, execute a control including the reporting control but not including the suitable control as the driving assistance control.

[0007] In one aspect of the present invention, the control unit (driving assistance ECU) is configured to: when it is determined that the number of times the same specific display object is present is greater than a prescribed number of times, but it is determined that the consistency rate of the recognition results is less than a second threshold, the control with a low assistance level is performed as the driving assistance control, compared with a case where it is determined that the number of times the same specific display object is present is greater than the prescribed number of times, and the consistency rate of the recognition results is determined to be greater than the second threshold.

[0008] In one aspect of the present invention, a driving assistance device of a vehicle further includes a reporting device (display device, sound output device), which reports information related to the specific display object to the driver, and the control unit (driving assistance ECU) is configured to: when it is determined that the number of times the same specific display object is present is greater than the prescribed number of times and the consistency rate of the recognition result is greater than the second threshold value, execute control including reporting control and adaptation control as the driving assistance control, the reporting control is control for reporting to the driver that the specific display object is determined to be present through the reporting device (display device, sound output device), and the adaptation control is vehicle control for making the driving state of the vehicle close to a state suitable for the display content of the specific display object, and when it is determined that the number of times the same specific display object is present is greater than the prescribed number of times but it is determined that the consistency rate of the recognition result is less than the second threshold value, execute control including the reporting control but not including the adaptation control as the driving assistance control.

[0009] In one aspect of the present invention, the control unit (driving assistance ECU) is configured to: when it is determined that the number of times the same specific display object exists is less than the prescribed number of times, compared with the case where the consistency rate of the recognition result is above the first threshold, perform the control with a low assistance level as the driving assistance control.

[0010] In one aspect of the present invention, the control unit (driving assistance ECU) is configured to: when it is determined that the number of times the same specific display object exists is less than the prescribed number of times, execute a control including the reporting control but not including the adaptation control as the driving assistance control.

[0011] According to the present invention, the assistance level of the driving assistance control is changed (the content of the driving assistance control is changed) according to the recognition accuracy of the display content of a specific display object. Specifically, when the recognition accuracy of the display content of the specific display object is low (that is, when there is a high possibility that the display content of the specific display object is misrecognized), the level of the driving assistance control is lowered compared to when the recognition accuracy is high. With such a configuration, when the display content of the specific display object is misrecognized, the driver's annoyance with the driving assistance control can be reduced.

[0012] In one aspect of the present invention, the driving assistance device of the vehicle also has a database (detection object DB), which registers the position of the specific display object determined to exist by the control unit (driving assistance ECU) and the recognition result of the display content, and the control unit (driving assistance ECU) is configured to: register the position of the specific display object determined to exist and the recognition result of the display content in the database (detection object DB), and calculate the consistency rate of the recognition result regarding the same specific display object with reference to the database (detection object DB).

[0013] By registering the recognition result of the detected display object in the database (detected object DB), the accuracy of the database can be improved. In addition, if the driving assistance device has a database (detected object DB), no communication fee will be generated when referring to the database (detected object DB).

[0014] In one aspect of the present invention, the control unit (driving assistance ECU) is configured to: when it is determined that the number of times the same specific display object exists is more than a prescribed number of times and the consistency rate of the recognition result is more than a second threshold, delete the recognition result about the same specific display object from the database (detection object DB) but retain one recognition result.

[0015] In one aspect of the present invention, the control unit (driving assistance ECU) is configured to delete the recognition result regarding the specific display object that has not been detected within a prescribed period from the database (detected object DB).

[0016] With such a configuration, the capacity of the database (detection object DB) can be reduced.

[0017] In one aspect of the present invention, the control unit (driving assistance ECU) is configured to: calculate the similarity between the image data and a pre-prepared template for determining the existence of the specific display object, and determine that the specific display object exists when the similarity is above a detection threshold; and not register the recognition result regarding the specific display object in the database (detection object DB) when the similarity is above the detection threshold but less than an accuracy threshold which is a value higher than the detection threshold.

[0018] With such a configuration, it is possible to prevent or suppress a decrease in the accuracy of the recognition result registered in the database.

[0019] In one aspect of the present invention, the control unit (driving assistance ECU) is configured to delete the recognition result regarding the same specific display object from the database (detection object DB) when it is determined that the number of times the same specific display object exists is greater than the specified number of times, the consistency rate of the recognition result is greater than the second threshold, and the similarity regarding the same specific display object is greater than the specified threshold.

[0020] With such a configuration, the capacity of the database (detection object DB) can be reduced.

[0021] The vehicle driving assistance method of the present invention comprises the following steps: determining whether a specific display object exists in front of the vehicle based on the image data acquired by photographing the scenery in front of the vehicle; and, if it is determined that the specific display object exists in front of the vehicle, recognizing the display content of the specific display object determined to exist, and executing driving assistance control, which is control for assisting driving performed by the driver of the vehicle and is control corresponding to the display content of the specific display object, wherein the vehicle driving assistance method further comprises the following steps: if it is determined that the specific display object exists in front of the vehicle, determining whether a consistency rate of recognition results as a ratio of "the number of times the recognition results of the display content of the same specific display object are the same" to "the number of times it is determined that the same specific display object exists" is less than a first threshold value, and if it is determined that the consistency rate of the recognition results is less than the first threshold value, in the step of executing the driving assistance control, compared with a case where it is determined that the consistency rate of the recognition results is greater than the first threshold value, executing a control of a lower assistance level indicating the degree of influence on the driver as the driving assistance control.

[0022] The program of the present invention is used to cause a computer of a driving assistance device of a vehicle to execute the following steps: determining whether a specific display object exists in front of the vehicle based on the image data acquired by photographing a scene in front of the vehicle; and, if it is determined that the specific display object exists in front of the vehicle, recognizing the display content of the specific display object determined to exist, and executing driving assistance control, which is control for assisting driving performed by a driver of the vehicle and is control corresponding to the display content of the specific display object, wherein the program causes the computer to further execute the following steps: if it is determined that the specific display object exists in front of the vehicle, determining whether a recognition result consistency rate, which is a ratio of "the number of times the recognition results of the display content of the same specific display object are the same" to "the number of times it is determined that the same specific display object exists", is less than a first threshold value, and if it is determined that the recognition result consistency rate is less than the first threshold value, in the step of executing the driving assistance control, compared with a case where it is determined that the recognition result consistency rate is greater than the first threshold value, executing a control with a lower assistance level indicating the degree of influence on the driver as the driving assistance control.

[0023] According to the present invention, when the display content of a specific display object is erroneously recognized, it is possible to reduce the annoyance that the driver feels towards driving assistance control.

[0024] In the above description, in order to help understand the present invention, the names and / or figure marks used in the embodiments are added in brackets to the configurations of the invention corresponding to the embodiments described later. However, the constituent elements of the present invention are not limited to the embodiments specified by the names and / or figure marks. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Hereinafter, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, wherein like reference numerals denote like elements, and wherein:

[0026] Figure 1 It is a diagram showing the structure of a vehicle.

[0027] Figure 2 This is a diagram showing the contents of a display object database (detection object DB).

[0028] Figure 3 : is a flowchart showing a driving assistance control start routine. DETAILED DESCRIPTION

[0029] Hereinafter, the driving assistance device of the vehicle of the embodiment of the present invention is described. In the following description, the driving assistance device of the vehicle is sometimes referred to as "assistance device". The assistance device is mounted on the vehicle. In each of the following embodiments, a method in which the assistance device is mounted on a vehicle having an engine (internal combustion engine) as a driving force source for driving is shown, but the assistance device can also be mounted on a vehicle having a motor (electric motor) as a driving force source for driving (electric vehicle) and a vehicle having an engine and a motor as a driving force source for driving (hybrid vehicle). In addition, among the specific display objects as the detection object of the auxiliary device, road signs, electronic bulletin boards, road signs, and characters, numbers and marks recorded on the road surface are included. Moreover, among the specific display objects, display objects whose display content does not change and display objects whose display content changes (for example, display objects that display moving images and display objects that switch between display and non-display, etc.) are also included.

[0030] like Figure 1 As shown, the vehicle 100 has an auxiliary device 10, an engine ECU 11, and a brake ECU 12. The auxiliary device 10 has an image processing ECU 13, a driving assistance ECU 14, a front camera 15, a vehicle speed sensor 16, a display device 17, a sound output device 18, and a position detection device 19. Each ECU has a computer, and the computer has a memory 24 such as a CPU, ROM, and RAM. The CPU reads data, performs numerical calculations, and outputs calculation results by sequentially executing a predetermined program (routine). The ROM pre-registers the programs executed by the CPU and the lookup table (map). The RAM can temporarily store data. These ECUs can communicate data (exchange data) with each other via CAN (Controller Area Network). In addition, the vehicle 100 also has an engine 20 as a driving force source, an engine actuator 21 that controls the engine 20, a transmission 22 (speed gear) that adjusts the driving force of the engine 20 and transmits the driving force to the drive wheels, and a brake actuator 23 that applies braking force to each wheel.

[0031] The engine ECU 11 is connected to an accelerator pedal sensor 25, an engine actuator 21, and a transmission 22. The accelerator pedal sensor 25 is configured to detect the operation amount (depression amount) of the accelerator pedal in sequence. The accelerator pedal is operated by the driver and is used to change the driving force of the vehicle 100 by adjusting the force generated by the engine 20. The engine ECU 11 sequentially obtains the detection results of the operation amount of the accelerator pedal obtained by the accelerator pedal sensor 25, and controls the engine 20 and the transmission 22 in such a way that the driving force corresponding to the obtained detection results is transmitted to the driving wheels of the vehicle 100. In addition, the engine ECU 11 controls the engine 20 and the transmission 22 based on the instructions sent from the driving assistance ECU 14. In addition, when the engine ECU 11 receives the driving force change instruction from the driving assistance ECU 14, it controls the engine 20 and the transmission 22 according to the driving force change instruction, thereby increasing or decreasing the driving force transmitted to the driving wheels. That is, the engine ECU 11 can increase or decrease the speed of the vehicle 100 according to the driving force change instruction.

[0032] The brake ECU 12 is connected to the brake actuator 23. The brake actuator 23 adjusts the hydraulic pressure of the working oil of the friction brake mechanism provided at each wheel, and applies the friction braking force corresponding to the hydraulic pressure to each wheel. The brake ECU 12 determines the driver's requested deceleration based on the operation amount of the brake pedal detected by the brake pedal sensor 26, and controls the brake actuator 23 in a manner that the vehicle 100 decelerates according to the driver's requested deceleration. Moreover, when the brake ECU 12 receives a braking command from the driving assistance ECU 14, it controls the operation of the brake actuator 23 in a manner that the vehicle 100 decelerates according to the requested deceleration included in the braking command. That is, the brake ECU 12 drives the brake actuator 23 according to the braking command, thereby adjusting the braking force applied to the vehicle (hereinafter referred to as "vehicle braking force") to reduce the speed of the vehicle 100.

[0033] The front camera 15 sequentially (for example, at a period of 100 msec) photographs the scenery within a specified range in front of the vehicle 100, generates data of the photographed images (image data), and sequentially sends the generated image data to the image processing ECU 13. It should be noted that the "front" in the present embodiment includes not only the front direction of the vehicle 100, but also includes the front oblique upper side and the front oblique lower side, the front oblique right side and the front oblique left side. The front camera 15 is disposed near the interior mirror in the vehicle compartment. However, the installation position of the front camera 15 is not particularly limited, for example, it may also be a "specified position outside the vehicle compartment of the vehicle 100" such as the central part of the front bumper and the front part of the side rearview mirror.

[0034] The image processing ECU 13 determines whether a specific display object pre-set as a detection object exists in front of the vehicle 100 by performing conventionally known image recognition processing on the image data sequentially sent from the front camera 15. In other words, the image processing ECU 13 detects a specific display object existing in front of the vehicle 100. In the image recognition processing, well-known methods such as pattern matching and Hough transform can be applied. Specifically, the image processing ECU 13 performs well-known image processing such as edge detection on the image data acquired from the front camera 15, and extracts the contours of all objects contained in the image. Then, by performing pattern matching processing on the image data subjected to image processing using a plurality of templates prepared in advance for each type of specific display object in order to determine whether a specific display object exists, it is determined whether a specific display object set as a detection object exists in front of the vehicle 100.

[0035] More specifically, the image processing ECU 13 calculates the similarity (SSD (Sum of Squared Difference: sum of squared differences) or SAD (Sum of Absolute Difference: sum of absolute differences)) between the image data generated by the front camera 15 and a plurality of templates. Then, the image processing ECU 13 determines that a specific display object corresponding to a template having a similarity value of at least a predetermined threshold value (hereinafter referred to as "detection threshold value") and having the highest similarity value is detected (more precisely, it is determined that the specific display object corresponding to the template is included in the captured image).

[0036] Then, when the image processing ECU 13 determines that there is a specific display object as a detection object, it recognizes its display content. In particular, when the specific display object is a road sign showing the maximum speed, the maximum speed shown by the specific display object is recognized. For example, the display content of the specific display object is associated with a template. Then, the image processing ECU 13 can recognize the content of the detected specific display object by recognizing the template corresponding to the specific display object determined to exist.

[0037] The display device 17 is configured to display an image (including graphics, text, and marks) based on a display instruction from the driving assistance ECU 14. Then, the display device 17 displays an image, so that various information can be provided to the driver. In particular, the display device 17 displays an image representing the display content of a specific display object that is determined to exist, so that the driver can be informed that a specific display object that is determined to exist in front of the vehicle 100 (in other words, the existence of a specific display object) and the display content of the specific display object, that is, information related to the specific display object. The display device 17, for example, has a liquid crystal display, and is arranged at a position where the driver can visually recognize the display content of the specific display object (for example, a center console in the vehicle compartment). The display device 17 can display a two-dimensional image in full color, for example, and can be composed of a liquid crystal display, an organic EL display, a plasma display, etc. In the present embodiment, the display device 17 is a central display arranged near the center of the vehicle width direction of the instrument panel. However, the display device 17 is not limited to a central display.

[0038] The sound output device 18 is configured to output sound based on the sound command from the driving assistance ECU 14. Specifically, the sound output device 18 has an amplifier and a speaker. When it is determined that a specific display object exists, the sound output device 18 can report information related to the specific display object to the driver by emitting a predetermined sound.

[0039] The vehicle speed sensor 16 is configured to sequentially detect the speed of the vehicle 100 and sequentially output a signal indicating the detected speed of the vehicle 100. The vehicle speed sensor 16 is configured to output a signal that changes according to the speed (i.e., vehicle speed) of the vehicle 100. The driving assistance ECU 14 acquires the vehicle speed based on the signal from the vehicle speed sensor 16. The vehicle speed sensor 16 may be a wheel speed sensor provided at each of the four wheels provided on the vehicle 100.

[0040] When the image processing ECU 13 determines that a specific display object exists, the driving assistance ECU 14 performs driving assistance control, which is a control for assisting the driver in driving the vehicle 100 and is a control corresponding to the display content of the specific display object. The driving assistance control includes: a control for notifying (warning) the driver that a specific display object is determined to exist and the content of the specific display object (hereinafter referred to as "notification control"); and a control for bringing the vehicle 100 into a driving state suitable for the display content of the specific display object (hereinafter referred to as "suitability control").

[0041] The report control includes: control for causing the display device 17 to display an image representing the display content of a specific display object that is determined to exist; and control for causing the sound output device 18 to output a specified sound (a warning sound, a sound notification, etc.). For example, when it is determined that a specific display object (such as a road sign) showing a maximum speed exists, the maximum speed shown by the specific display object is displayed on the display device 17, and the sound output device 18 generates a warning sound. According to the report control, the driver can be informed of the specific display object that is determined to exist and the display content of the display object that is determined to exist. In addition, the driver can be prompted to perform "driving operations such as bringing the driving state of the vehicle 100 close to the driving state suitable for the display content of the specific display object."

[0042] The adaptive control is a vehicle control for making the driving state of the vehicle 100 close to the driving state suitable for the display content of the specific display object or for making the driving state of the vehicle 100 consistent with the driving state suitable for the display content. For example, the adaptive control is a control that automatically decelerates the vehicle 100 by adjusting the vehicle driving force and / or the vehicle braking force when the specific display object is a road sign (restriction sign) showing the maximum speed and the speed of the vehicle 100 is higher than the maximum speed displayed by the specific display object.

[0043] Furthermore, the driving assistance ECU 14 can perform driving assistance control with different assistance levels. The assistance level is the degree of intervention in the driver's driving operation (the intensity of the degree of influence on the driving state of the vehicle 100). The higher the degree of intervention in the driving operation, the higher the assistance level. For example, in the case where the driving assistance control is a report control, the easier it is for the driver to recognize the report (warning), the higher the assistance level.

[0044] Specifically, in the action of causing the display device 17 to display an image, the assistance level of the process of causing the display device 17 to display a moving image (including image flickering and image enlargement and reduction) is higher than that of causing the display device 17 to display a still image. In addition, even in the action of causing the display device 17 to display a moving image, the assistance level of the side with faster or larger image movement (faster flickering speed and enlargement and reduction speed) is higher. In addition, in the case where the notification control is the action of causing the sound output device 18 to generate a warning sound, the assistance level becomes higher as the volume of the warning sound becomes larger. Moreover, in the case where the driving assistance control is the notification control, the assistance level of the combination of the two is higher than that of the process of causing the display device 17 to display an image and the action of causing the sound output device 18 to generate a warning sound. In the case where the driving assistance control is the appropriate control, the larger the control amount, the higher the assistance level. For example, in the case of decelerating the vehicle 100, the larger the deceleration amplitude, the higher the assistance level. Moreover, the assistance level of the combination of the notification control and the appropriate control is higher than that of the single notification control. In this way, when the assistance levels are different from each other, the contents of the driving assistance control are different from each other.

[0045] Then, the driving assistance ECU 14 (assistance device 10) performs the first level driving assistance control, the second level driving assistance control and the third level driving assistance control according to the situation. Among the three driving assistance controls, the first level driving assistance control has the lowest degree of intervention in the driver's driving operation, the third level driving assistance control has the highest degree of intervention in the driver's driving operation, and the second level driving assistance control has a degree of intervention in the driving operation between the first level driving assistance control and the second level driving assistance control.

[0046] More specifically, the driving assistance ECU 14 performs a report control that displays the content of a specific display object on the display device 17 as a first level of driving assistance control. The driving assistance ECU 14 performs a report control that displays the content of the specific display object on the display device 17 and causes the sound output device 18 to output a warning sound as a second level of driving assistance control. The driving assistance ECU 14 performs a report control that displays the content of the specific display object on the display device 17 and causes the sound output device 18 to output a warning sound, and allows the execution of a suitable control that decelerates the vehicle 100 as a third level of driving assistance control. That is, in the present embodiment, only the report control is executed in the lowest first level and the second lowest second level, and in the highest third level, in addition to the execution of the report control, the execution of the suitable control is also allowed. In this way, the contents of the driving assistance control are different from each other in the first level, the second level, and the third level.

[0047] Furthermore, in the present embodiment, the driving assistance ECU 14 changes the assistance level (the content of the driving assistance control) of the driving assistance control according to the consistency rate of the recognition results so far for the specific display object determined to be present (i.e., the consistency rate of the recognition results for the same specific display object). That is, due to factors such as the contamination of the specific display object, the state of light, and the weather (rainfall, snowfall, and snow accumulation), the image processing ECU 13 sometimes misidentifies the display content of the specific display object. Therefore, if it is the same specific display object, the recognition result should be the same each time, but sometimes the recognition result is different (i.e., misidentification) due to the aforementioned factors. Furthermore, in the case where the image processing ECU 13 misidentifies the display content of the specific display object, the driving assistance ECU 14 performs the driving assistance control based on the misidentified display content. For example, in the case where the driving assistance ECU 14 misidentifies the display content of the specific display object showing the maximum speed (i.e., the maximum speed), the misidentified maximum speed is displayed on the display device 17 and the sound output device 18 generates a warning sound. Furthermore, when the speed of the vehicle 100 detected by the vehicle speed sensor 16 is higher than the maximum speed that was erroneously recognized, the driving assistance ECU 14 may perform control to decelerate the vehicle 100. Therefore, even though the speed of the vehicle 100 is lower than the maximum speed indicated by the specific display object, driving assistance control that is not suitable for the display content of the specific display object may be performed due to erroneous recognition. As a result, the driver may feel annoyed by the driving assistance control.

[0048] Therefore, in the present embodiment, when it is determined that there is a high possibility that the display content of a specific display object is misidentified, the assistance level of the driving assistance control is reduced compared to the case where the display content of the specific display object is determined to be correctly identified. Specifically, when the driving assistance ECU 14 detects the same specific display object, it calculates the consistency rate of the recognition result of the same specific display object with reference to the past recognition results of the same specific display object. "The consistency rate of the recognition result" is represented by (the consistency rate of the recognition result) = (the number of times the recognition results of the display content of the same specific display object are the same) / (the number of times it is determined that the same specific display object exists (in other words, the number of times the same specific display object is detected)). It should be noted that the "number of times it is determined that the same specific display object exists" in the above formula is more than two times (i.e., multiple times). That is, there is the following tendency: for the same specific display object, when it is detected multiple times, the higher the consistency rate of the recognition result, the easier it is to be correctly recognized, and the lower the consistency rate of the recognition result, the easier it is to be misidentified. Therefore, it is considered that: when it is determined that the same specific display object exists multiple times and the consistency rate of the recognition results for the same specific display object is greater than a predetermined threshold, the display content of the same specific display object can be correctly recognized, and when the consistency rate of the recognition results is less than the predetermined threshold (the first threshold value x 1 ), there is a high possibility that the display content of the same specific display object is misidentified. Furthermore, when the consistency rate of the recognition result is less than a predetermined threshold, the assistance level of the driving assistance control is reduced compared to the case where the consistency rate of the recognition result is above the predetermined threshold.

[0049] It should be noted that, when a specific display object showing a maximum speed is detected, sometimes different maximum speeds are recognized due to misrecognition. In this case, (the consistency rate of recognition results) will exist for each different maximum speed. For example, when a specific display object showing a maximum speed of 80 km / h is detected ten times, if it is recognized as 80 km / h eight times out of ten and 60 km / h twice, then for 80 km / h, the consistency rate of the recognition result of the same specific display object is 0.8, and for 60 km / h, the consistency rate of the recognition result of the same specific display object is 0.2. In such a case, the consistency rate of the recognition result of the highest maximum speed with respect to the "consistency rate of recognition results" is treated as the "consistency rate of recognition results" of the same specific display object. In the example described, "0.8" which is a higher value is set as the consistency rate of the recognition result of the same specific display object, and it is determined that the maximum speed shown by the display object is 80 km / h.

[0050] In order to realize such driving assistance control, the memory 24 of the driving assistance ECU 14 stores the following information in a computer-readable form: Figure 2 The detection object DB27 (database) shown. In the items of "latitude" and "longitude" in the detection object DB27, the position (latitude and longitude) of the specific display object that is judged to exist is registered. In the item of "high precision flag", a value indicating whether "it is easy to be correctly identified" for each specific display object is registered. That is, the value of the item of "high precision flag" is set to "enabled (ON)" when it is easy to be correctly identified, and is set to "off (OFF)" when it is easy to be misidentified. It should be noted that the initial value of the high precision flag is set to "off (OFF)". That is, the high precision flag for the specific display object that is judged to exist for the first time (the specific display object that is judged to exist once) is "off (OFF)". In the item of "history", the recognition result of the display content of the specific display object is registered. In Figure 2 In the example shown, the specific display object is an example of a road sign (restriction sign) indicating the maximum speed. In this case, the recognition result of the maximum speed is registered in the item of the history. Figure 2 In the figure, the structure of the recognition results of the past five times is registered, but the number of recognition results registered in the detection object DB 27 is not limited to five times. In the item of "Last Update Date", the change of enabling (ON) and closing (OFF) of the high-precision flag of each display object or the last date when the content of the item of "History" was changed is registered.

[0051] When the image processing ECU 13 detects a specific display object, the driving assistance ECU 14 searches whether there is a past recognition history of the detected specific display object. Then, if there is a recognition history of the detected specific display object, the driving assistance ECU 14 determines whether the consistency rate of the recognition results of the display contents of the detected specific display object (i.e., the same specific display object) this time and the past multiple times is equal to a first threshold value X. 1 When the consistency rate of the recognition result is the first threshold value X 1 In the above case, it can be considered that the possibility of accurately recognizing the display content of the same specific display object is high. On the other hand, when the consistency rate of the recognition result is less than the first threshold value X 1 In the case of , it can be considered that there is a possibility that the display content of the same specific display object is misrecognized. Therefore, when the consistency rate of the recognition result is less than the first threshold value X 1 In the case of 1Compared with the above situation, the driving assistance ECU 14 performs driving assistance control with a low assistance level. With such a configuration, when the display content of a specific display object is misrecognized, the driver's annoyance with the driving assistance control can be reduced. It should be noted that the first threshold value X 1 The specific value of is not particularly limited and can be set appropriately.

[0052] It should be noted that the driving assistance ECU 14 uses the values ​​of the items of the position of the vehicle 100 detected by the position detection device 19 and the position of the detection object DB 27 to determine the identity of the detected specific display object and the display object registered in the detection object DB 27. Specifically, when the image processing ECU 13 determines that a specific display object exists, in the case where the record of the display object at the same position as the position of the vehicle 100 detected by the position detection device 19 is registered in the detection object DB 27, the driving assistance ECU 14 determines that the "specific display object determined to exist by the image processing ECU 13" is the same as the detection object at the same position registered in the detection object DB. However, considering the error of the GPS, in the case where there is no record of the display object at the position consistent with the position of the vehicle 100 detected by the position detection device 19, the display object closest to the position of the vehicle 100 detected by the position detection device 19 and less than a predetermined distance from the position of the vehicle 100 detected by the position detection device 19 is determined to be the same display object as the specific display object determined to exist.

[0053] Then, whenever it is determined that there is a specific display object, and the recognition result of the display content of the specific display object is obtained from the image processing ECU 13, the driving assistance ECU 14 updates the value of the item "history" of the display object in the detection object DB 27. However, in the case where the accuracy of the detection object DB 27 may become lower due to the update, the driving assistance ECU 14 performs driving assistance control but does not update the detection object DB 27. Alternatively, the driving assistance ECU 14 updates the detection object DB 27 (specifically, the values ​​of each item of the specific display object determined to be present are recorded in the detection object DB 27) but does not change the final update date.

[0054] For example, when the image processing ECU 13 determines that there is a specific display object, as the distance between the position of the vehicle 100 detected by the position detection device 19 and the position registered in the detection object DB 27 increases, the probability that the specific display object determined to exist and the display object registered in the detection object DB 27 are the same display object becomes lower. Therefore, a "another prescribed distance" smaller than the "prescribed distance" is pre-set, and when the distance from the "position of the vehicle 100 detected by the position detection device 19" to the "position of the display object registered in the detection object DB 27" is greater than the "another prescribed distance" and less than the "prescribed distance", it is considered that there is a possibility that the specific display object determined to exist and the display object registered in the detection object DB 27 are different from each other. And, in this case, the driving assistance ECU 14 performs driving assistance control but does not perform updating of the detection object DB 27.

[0055] In addition, when pattern matching is used to determine whether a specific display object exists, the image processing ECU 13 calculates the similarity between the template and the captured image, and determines that a specific display object exists when the similarity is greater than a detection threshold. In this case, an accuracy threshold higher than the detection threshold is pre-set, and when the similarity is greater than the detection threshold and less than the accuracy threshold, it is considered that the accuracy of the recognition of the display content of the specific display object is low. Therefore, in this case, the image processing ECU 13 does not update the history of the specific display object.

[0056] It should be noted that when the number of times the same specific display object was determined to exist in the past is small, the reliability of the consistency rate of the recognition result for the same specific display object becomes low. Therefore, when it is determined that the same specific display object exists, the driving assistance ECU 14 determines the "number of times the same specific display object has been determined to exist" so far, including this time, for the same specific display object. It should be noted that the "number of times the same specific display object has been determined to exist" is the number of times including this determination. Hereinafter, this number will be recorded as the "number of detections". Then, when the number of detections so far for the same specific display object is more than the specified number, but the consistency rate of the recognition results so far for the same specific display object (the consistency rate of the recognition results calculated including this recognition result) is less than the second threshold value X 2 In the case of the case where the number of detections so far for the same specific display object is greater than the predetermined number of times, and the consistency rate of the recognition results so far for the same specific display object is greater than the second threshold value X 2Compared with the above situation, the assistance level of the driving assistance control is lowered. That is, if the number of detections so far is greater than the specified number, the reliability of the consistency rate of the recognition results so far can be regarded as high, and if the number of detections so far is less than the specified number, the reliability of the consistency rate of the recognition results so far can be regarded as low.

[0057] Then, when the number of detections so far is greater than the predetermined number of times, the consistency rate of the recognition results so far is determined, thereby providing driving assistance control that matches the display content of the same specific display object when it is considered that there is a high probability that the display content of the same specific display object has been correctly recognized. 2 is greater than the first threshold X 1 However, the second threshold value X 2 The specific value of is not limited and can be set appropriately. Similarly, the specific value of the "predetermined number of times" is not limited and can be set appropriately.

[0058] Furthermore, when the number of detections so far for the same specific display object is greater than the predetermined number of times, but the consistency rate of the recognition results so far for the same specific display object is less than the second threshold value X 2 In the case of "the number of detections is less than the specified number of times" and "the number of detections is greater than the specified number of times, and the consistency rate of past recognition results is greater than the second threshold value X", it can be considered that the same specific display object is more likely to be misdetected than in other cases. Therefore, in this case, it is considered that there is a possibility of misrecognition, which is different from the case of "the number of detections is less than the specified number of times" and "the number of detections is greater than the specified number of times, and the consistency rate of past recognition results is greater than the second threshold value X". 2 Specifically, the driving assistance ECU 14 sets the assistance level of the driving assistance control to the lowest first level.

[0059] When the number of detections of the same specific display object so far is less than the prescribed number, as described above, the reliability of the consistency rate of the recognition results so far can be regarded as low. Therefore, in this case, the number of detections of the same specific display object so far is greater than the prescribed number, and the consistency rate of the recognition results so far of the specific display object is equal to the second threshold value X. 2 Compared with the above situation, the driving assistance ECU 14 lowers the level of driving assistance control. In this embodiment, the assistance level of driving assistance control in this case is set to the second level. That is, the driving assistance ECU 14 sets the assistance level of driving assistance control in this case to a level greater than the number of detections so far for the same specific display object is greater than the prescribed number of times, and the consistency rate of the recognition results so far for the same specific display object is greater than the second threshold value X. 2The third level in the above case is low, and the number of detections so far for the same specific display object is greater than the predetermined number of times, but the consistency rate of the recognition results so far for the same specific display object is less than the second threshold value X 2 The first level of the case is high.

[0060] Furthermore, in the present embodiment, the driving assistance ECU 14 changes the assistance level of the driving assistance control according to whether the high-precision flag is enabled (ON) or disabled (OFF). Specifically, as described above, when the number of detections so far for a certain specific display object is greater than the prescribed number of times and the consistency rate of the recognition results so far is greater than the second threshold value X, 2 In the above case, the driving assistance ECU 14 sets the high-precision flag of the same specific display object to be enabled (ON). On the other hand, when the number of detections so far for a certain specific display object is greater than the prescribed number of times, but the consistency rate of the recognition results so far for the same specific display object is less than the first threshold value X 1 , the driving assistance ECU 14 sets the high-precision flag of the same specific display object to OFF. Then, when the high-precision flag is enabled (ON), the driving assistance ECU 14 increases the assistance level of the driving assistance control compared to the case where the high-precision flag is turned off (OFF). Specifically, the driving assistance ECU 14 sets the assistance level of the driving assistance control to the third level when the high-precision flag is enabled (ON), and sets the assistance level of the driving assistance control to the first level or the second level when the high-precision flag is turned off (OFF). It should be noted that, when the number of detections so far is less than the prescribed number of times, it is not possible to regard the consistency rate of the recognition results so far for the same specific display object as an indicator of "whether it is easy to be correctly recognized". Therefore, for a specific display object whose number of detections so far is less than the prescribed number of times, the driving assistance ECU 14 sets the high-precision flag to OFF. In addition, the driving assistance ECU 14 sets the initial value of the high-precision flag to OFF.

[0061] Therefore, when the high-precision flag is OFF for the same specific display object, the number of detections so far is greater than the predetermined number of times and the consistency rate of the recognition results so far is greater than the second threshold value X. 2 In the above case, the driving assistance ECU 14 changes the high-precision flag for the same specific display object from OFF to ON. And after the high-precision flag becomes ON, if the consistency rate of the recognition result does not decrease to less than the first threshold value X 1, the driving assistance ECU 14 maintains the high-precision flag as enabled (ON). Furthermore, when the same specific display object is detected during the period when the high-precision flag is enabled (ON), the driving assistance ECU 14 increases the assistance level of the driving assistance control compared to the case where the high-precision flag is turned off (OFF). In this embodiment, the driving assistance ECU 14 performs the driving assistance control with the highest assistance level.

[0062] Furthermore, in the configuration in which the driving assistance ECU 14 has the detection object DB 27, as the number of specific display objects determined to be present and the number of times determined to be present (i.e., the number of detections) increase, the size of the detection object DB 27 increases. Then, as a result, the usage of the memory 24 required for storing the detection object DB increases. In the case where the driving assistance ECU 14 has the detection object DB 27, that is, in the case where the vehicle 100 has the detection object DB 27, the capacity of the memory 24 is greatly limited compared to the configuration in which the detection object DB 27 is registered in the memory 24 of the device outside the vehicle (e.g., a data center), and therefore it is preferred that the size of the detection object DB 27 is smaller.

[0063] Therefore, the driving assistance ECU 14 integrates the records of specific display objects for which the high-precision flag is ON into one record, and deletes the remaining records. Figure 2 The display object with the "identification ID" of "2" indicates that the records are integrated into one record. Specifically, the driving assistance ECU 14 deletes the number of detections so far exceeding the prescribed number of times and the consistency rate of the recognition results so far exceeding the second threshold value X. 2 The above specific display object recognition records are retained, but only one record with a correct recognition result (a value with a consistency rate of recognition results greater than a predetermined threshold value) is retained. This maintains the level of recognition results using the detection object DB 27 and reduces the usage of the memory 24 of the driving support ECU 14.

[0064] It should be noted that, instead of the high-precision flag being enabled (ON), when "the number of detections of the same specific display object so far is greater than the prescribed number of times, and the consistency rate of the recognition results so far is greater than the second threshold value X" 2 In the case of "more than", the driving assistance ECU 14 deletes the history of the recognition results of the same specific display object so far and integrates them into one history. That is, the recognition result of the display content of the same specific display object should be the same recognition result each time. Therefore, if the number of detections of the same specific display object so far is more than the prescribed number of times (at least multiple times), and the consistency rate of the recognition results so far is greater than the second threshold value X 2The above can be regarded as being able to accurately identify the display content of the same specific display object. Therefore, the driving assistance ECU 14 sets the high-precision flag for such a specific display object to enable (ON), deletes the recognition history but retains one history. In this way, the accuracy of the determination result using the detection object DB 27 can be maintained, and the size of the detection object DB 27 can be reduced. It should be noted that here, the second threshold value X 2 The condition for deleting the records of the recognition results of the same specific display object so far and integrating them into one record may also be determined by using the first threshold value X 1 Moreover, it is also possible to use 1 and the second threshold X 2 Different thresholds.

[0065] Moreover, when the period from the final update date of the resume becomes more than the prescribed number of days, the possibility of detecting a specific display object in the future is low (not passing through the location where the specific display object is set), or the possibility of the specific display object being removed or relocated is high. Therefore, the driving assistance ECU 14 compares the current date with the final update date of each display object registered in the detection object DB 27, and deletes the information about the specific display object that has passed the prescribed period from the final update date (that is, the specific display object that has not been detected after the final update date). Thus, the size of the detection object DB 27 can be reduced, and the accuracy of the detection object DB 27 can be improved. It should be noted that the specific value of the "prescribed period" is not particularly limited and can be set appropriately.

[0066] In addition, when the recognition results so far for the same specific display object are inconsistent for more than a specified number of times, the history of the recognition results for the same specific display object can be deleted from the detection object DB27. For example, sometimes a pattern drawn on a vehicle traveling in front of the vehicle (for example, a pattern drawn on the back of a truck's cargo bed) is detected (i.e., misidentified) as a display object. In such a case, the recognition result this time is inconsistent with the past recognition results. Therefore, in such a case, the recognition history is deleted, thereby suppressing the execution of driving assistance control based on misidentification. It should be noted that the "prescribed number of times" is not particularly limited and can be set appropriately.

[0067] Furthermore, in the configuration using the high-precision flag as described above, when the display content of a specific display object is changed, even if the high-precision flag was enabled (ON) before the change, it becomes disabled (OFF) after the change. Furthermore, since the history of past recognition results is registered in the detection object DB 27, after the flag is changed, the consistency rate of the recognition result for the specific display object is less than the specified value (the second threshold value X) until a certain number of detections are performed. 2 ), when the specific display object is detected, there will be a period during which the assistance level of the driving assistance control becomes low (set to the first level). Therefore, in order to shorten the first level period in such a case, the past recognition results about the specific display object can be deleted from the detection object DB27.

[0068] Specifically, when the number of detections so far for the same specific display object is greater than the predetermined number of times, but the consistency rate of the recognition results so far is less than the second threshold value X 2 In the case of , the driving assistance ECU 14 obtains the similarity of the same specific display object from the image processing ECU 13, and determines whether the similarity is above the accuracy threshold. Then, when the similarity is above the accuracy threshold, it is deemed that the image processing ECU 13 has correctly recognized the content of the same specific display object. Then, the history of the same specific display object except for the most recent one (that is, the one in which the similarity is above the accuracy threshold) is deleted. Therefore, in this case, the number of detections of the same specific display object is set to once. Therefore, when the same specific display object is detected next time, it becomes "the number of detections is less than the prescribed number of times", and therefore, the number of detections of the same specific display object is more than the prescribed number of times, but the consistency rate of the past recognition results is less than the second threshold value X. 2 Therefore, the number of detections of the same specific display object so far is more than the specified number of times, but the consistency rate of the past recognition results is less than the second threshold value X can be shortened. 2 The period of the assistance level of the driving assistance control executed in the case of .

[0069] In addition, sometimes a specific display object is blocked by a large vehicle such as a truck traveling in front of the vehicle 100, and the specific display object is not detected by the image processing ECU 13. In this case, the driving assistance ECU 14 performs driving assistance control based on the history registered in the detection object DB 27. Specifically, although the vehicle 100 passes through a position where a specific display object registered in the detection object DB 27 exists, the image processing ECU 13 does not detect the specific display object. It is determined whether the high-precision flag about the specific display object registered in the detection object DB 27 is enabled (ON) or turned off (OFF). If the high-precision flag is turned off (OFF), it is further determined whether the consistency rate of the past recognition results is a predetermined threshold value (second threshold value X). 2 ) or above. Then, in the case where "the high-precision flag for the specific display object is enabled (ON)" or in the case where "the high-precision flag for the specific display object is turned off (OFF) but the consistency rate of past recognition results is above a specified threshold value (second threshold value)", it is considered that the specific display object cannot be recognized for some reason. Then, the driving assistance ECU 14 performs the driving assistance control in the same way as when the specific display object is detected. However, in this case, the driving assistance ECU 14 can lower the assistance level of the driving assistance control compared to the case where the specific display object is actually detected. It should be noted that "even if the high-precision flag is not enabled (ON), the consistency rate of the recognition results exceeds the second threshold value X". 2 The situation "is equivalent to the situation where the number of detections is less than the prescribed number of times.

[0070] Thus, even when a specific display object is blocked by other vehicles 100 and is not detected, driving assistance control suitable for the display content of the specific display object that should exist can be provided. In addition, in the present embodiment, the driving assistance ECU 14 has a detection object DB 27. If such a configuration is adopted, it is not necessary to communicate with, for example, a detection object DB constructed outside the vehicle 100, and therefore, even when a specific display object is not detected, driving assistance control can be provided without requiring communication fees.

[0071] In addition, when the number of detections so far for the same specific display object is greater than the predetermined number of times, but the consistency rate of the recognition results so far does not reach the second threshold value X 2 In the above case, it can be considered that it is difficult to recognize the same specific display object by the image processing ECU 13 due to contamination and lighting conditions. Therefore, when such a display object is photographed, a configuration can be applied to improve the consistency rate of the recognition result by increasing the magnification of the front camera 15 (seeking to improve the camera performance) or the like.

[0072] Next, the specific operation of the driving support ECU 14 will be described. Every time a predetermined time has passed, the CPU of the driving support ECU 14 (hereinafter, also simply referred to as "CPU") repeatedly executes the following operations: Figure 3 The flowchart of "driving assistance control start routine" is represented. It should be noted that when there is a specific display object in front of the vehicle 100, the CPU will continue to determine that the specific display object exists until the vehicle 100 passes the position of the specific display object. Therefore, the CPU sequentially tracks the specific display object that has been detected through a well-known object tracking method, thereby setting the number of times it is determined that "the specific display object exists" to one during the period from the entry of the specific display object into the front camera 15 to the exit of the specific display object from the front camera 15. The computer program for executing the driving assistance control start routine is pre-stored in the ROM of the driving assistance ECU 14. The CPU of the driving assistance ECU 14 reads the computer program from the ROM and executes it. Thus, the driving assistance control start routine is executed.

[0073] In step S101, the CPU determines whether a specific display object as a detection object exists in front of the vehicle 100. In the case where it is determined that the specific display object does not exist in front of the vehicle 100, the CPU temporarily ends the driving assistance control start routine. Therefore, the driving assistance control is not started until it is determined that the specific display object exists. However, the CPU can continue the driving assistance control if it has already executed the driving assistance control.

[0074] In the case where it is determined that there is a specific display object in step S101, the CPU proceeds to step S102. In step S102, the CPU refers to the detection object DB27 to retrieve whether the specific display object determined to exist is registered in the detection object DB27, and in the case of being registered in the detection object DB27, it is determined that the detected specific display object is the same display object as the multiple display objects registered in the detection object DB27. Specifically, the CPU obtains the detection result of the current position (longitude and latitude) of the vehicle 100 from the position detection device 19, and compares the detection result of the current position with the position (longitude and latitude) of the display object registered in the detection object DB27, thereby determining the identity of the specific display object determined to exist and the display object registered in the detection object DB27. Furthermore, the CPU identifies the display content of the specific display object determined to exist. Then, the CPU proceeds to step S103.

[0075] It should be noted that sometimes the display object at the position consistent with the position detected by the position detection device 19 does not exist in the detection object DB27. In this case, the CPU determines that the "display object that is closest to the position of the vehicle 100 detected by the position detection device 19 and whose distance from the current position of the vehicle 100 is less than a specified value" among the multiple displays registered in the detection object DB27 is the same specific display object as the specific display object determined to exist. In addition, in the case where there is no "display object whose distance from the position of the vehicle 100 at the detection time point of the specific display object is less than a specified value" in the detection object DB27, the CPU determines that the specific display object determined to exist is the display object determined to exist for the first time this time. In this case, the CPU sets the high-precision flag for the specific display object to "OFF" and sets the number of detections to "once".

[0076] In step S103, the CPU determines whether the high-precision flag for the specific display object determined to be present is ON or OFF. If the high-precision flag is ON, the process proceeds to step S104.

[0077] In step S104, the CPU refers to the detection object DB 27 to calculate the consistency rate of the recognition results of the specific display object so far (including the consistency rate of the recognition result this time), and determines whether the calculated consistency rate of the recognition results so far is a first threshold value X. 1 The consistency rate of the recognition results of the same specific display object is the first threshold value X 1 In the above case, the CPU proceeds to step S105. In step S105, the CPU executes the third level of driving assistance control. Then, the CPU proceeds to step S106.

[0078] In step S104, it is determined that the consistency rate of the recognition result is less than the first threshold value X 1 In the case of, the CPU proceeds to step S107. In step S107, the CPU changes the value of the high-precision flag for the same specific display object from ON to OFF. Then, the CPU proceeds to step S108. In step S108, the CPU executes the second level of driving assistance control. Then, the CPU proceeds to step S106.

[0079] In step S106, the CPU adds the recognition result of the specific display object determined to exist in step S101 to the detection object DB 27, and updates the last update date of the specific display object registered in the detection object DB 27. It should be noted that, when entering step S106 after step S107, the CPU changes the value of the high-precision flag of the specific display object registered in the detection object DB 27 from enabled (ON) to disabled (OFF).

[0080] Thus, in the driving assistance control start routine, when the consistency rate of the recognition result is less than the first threshold value X 1 In the case of 1 Compared with the above situation, the CPU lowers the assistance level of the driving assistance control. That is, when the driving assistance control is executed based on the display content that misrecognized the specific display object, the driver may feel uncomfortable with the driving assistance control or feel annoyed with the driving assistance control. Therefore, when the high-precision flag for the specific display object that is determined to exist is enabled (ON) and the consistency rate of the recognition result is equal to the first threshold value X 1 In the above case, it is considered that the possibility of correctly recognizing (not misrecognizing) the display content of the specific display object is high. Then, in this case, the CPU executes the third level of driving assistance control with the highest assistance level. In contrast, even if the high-precision flag is enabled (ON), when the consistency rate of the recognition result is less than the first threshold value X 1 In the case of the high-precision control, the CPU also changes the high-precision flag to OFF and executes the second level of driving assistance control which is lower than the third level.

[0081] According to the driving assistance control start routine, when the display content of a specific display object is correctly recognized, driving assistance control suitable for the display content of the specific display object can be executed. On the other hand, when the display content of the specific display object is misrecognized, the driver's sense of disharmony and annoyance with the driving assistance control can be reduced.

[0082] If the high-precision flag is OFF in step S103, the CPU proceeds to step S109. In step S109, the CPU determines whether the number of detections of the same specific display object determined to exist in step S101 is greater than a threshold. If the number of detections is greater than the threshold, the CPU proceeds to step S110.

[0083] In step S110, the CPU determines whether the consistency rate of the recognition results so far for the same specific display object is equal to the second threshold value X. 2 When it is determined that the consistency rate of the recognition results so far is the second threshold value X 2In the above case, the CPU proceeds to step S111.

[0084] When the number of detections of the same specific display object is greater than the predetermined number of times and the consistency rate of the recognition results of the same specific display object is greater than the second threshold value X 2 In the above case, it can be considered that there is a tendency that the same specific display object is easily correctly recognized by the image processing ECU 13. Therefore, in step S111, the CPU changes the high-precision flag for the same specific display object from off to on. Then, the CPU enters step S112. In step S112, the CPU integrates the past history of the same specific display object. Then, it enters step S105, and the CPU performs the third level of driving assistance control.

[0085] In step S110, it is determined that the consistency rate of the recognition result of the specific display object is less than the second threshold value X 2 In the case of, the CPU proceeds to step S113. In step S113, the CPU executes the first level of driving assistance control. Then, the CPU proceeds to step S106.

[0086] In step S106 , the CPU registers the recognition result of the specific display object determined to be present in step S101 in the detected object DB 27 , and changes the last update date registered in the detected object DB 27 to the date when the specific display object was detected.

[0087] Thus, in the driving assistance control start routine, when the consistency rate of the recognition results for the same specific display object is greater than the second threshold value X, 2 In the above case, the consistency rate with the recognition result is less than the second threshold value X 2 The CPU increases the assistance level of the driving assistance control compared to the case where the display content of the specific display object is correctly recognized. Thus, when the display content of the specific display object is correctly recognized, the driving assistance control suitable for the display content of the specific display object can be executed. Specifically, when the consistency rate of the recognition results for the same specific display object is equal to the second threshold value X 2 In the above case, the CPU executes the third level of driving assistance control, which is the highest level of assistance. On the other hand, when the consistency rate of the recognition results for the same specific display object is less than the second threshold value X 2 In the case of 2 In particular, when the number of detections of the same specific display object is less than the specified number of times and the consistency rate of the recognition results of the same specific display object is less than the second threshold value X 2In the case of the first level of driving assistance control, which is the lowest level of assistance, the CPU executes the first level of driving assistance control. Thus, when the display content of the same specific display object is misrecognized, the driver's sense of disharmony and annoyance with the driving assistance control can be reduced.

[0088] When the number of detections of the same specific display object is small, the consistency rate of the recognition result of the same specific display object is less reliable as an indicator of "whether the same specific display object is easy to be correctly recognized". Therefore, the CPU determines in step S109 whether the number of detections of the specific display object determined to exist in step S101 is more than the prescribed number of times, and if the number of detections is less than the prescribed number of times, the process proceeds to step S108. That is, in this case, the CPU makes the assistance level lower than the third level when the high-precision flag is enabled (ON), and makes the assistance level lower than the consistency rate of the recognition result by less than the second threshold value X. 2 It should be noted that, when the history data of a specific display object does not exist in the detection object DB 27, that is, when the specific display object is detected for the first time, the CPU proceeds from step S102 to step S108 to execute the second level of driving assistance control, similarly to the case where the high-precision flag is OFF and the number of detections is less than the prescribed number of times.

[0089] It should be noted that, when a specific display object is detected, the CPU updates the detection history of the specific display object in the detection object DB27, but when the accuracy of the detection object DB27 may be reduced due to the update, the routine of not updating the detection object DB27 is applied. For example, in at least one of the cases where the similarity calculated during image recognition by the image processing ECU 13 is less than the accuracy threshold and the distance between the specific display object determined to exist and the display object registered in the detection object DB27 is more than a specified distance, the CPU executes the driving assistance control but does not execute the update of the detection object DB27. If it is such a structure, it is possible to prevent or suppress the reduction of the accuracy of the detection object DB.

[0090] In addition, as described above, when the display content of a specific display object is changed, there will be a period during which the assistance level of the driving assistance control becomes low. In order to shorten this period, a step of determining whether the similarity calculated by the image processing ECU 13 to detect a specific display object is above the accuracy threshold can be added between step S110 and step S113. Then, in the case where it is determined in the additional step that the similarity is above the accuracy threshold, the CPU deletes the recognition result (record) of the specific display object. On the other hand, in the case where it is determined in the additional step that the similarity is less than the accuracy threshold, the CPU does not delete the recognition result of the specific display object. According to such a driving assistance control start routine, when the number of detections of the same specific display object is more than the specified number of times ("yes" in step S109), the consistency rate of the recognition results so far for the same specific display object is less than the second threshold value X. 2 (No in step S110), and when the similarity with respect to the same specific display object is equal to or greater than a predetermined threshold, the recognition result with respect to the same specific display object is deleted from the detection object DB 27. Therefore, the period during which the assistance level of the driving assistance control is lowered can be shortened.

[0091] (Other embodiments)

[0092] In the above embodiment, the driving assistance ECU 14 is shown to have a structure of a detection object DB 27, but it is also possible to construct a structure in which the detection object DB 27 is constructed outside the vehicle 100 and the detection object DB is constructed in the data center of the vehicle 100. In this case, it is also possible to have the following structure: the driving assistance ECU 14 can refer to the detection object DB of the data center by wirelessly communicating between the auxiliary device 10 of the vehicle 100 and the data center, and update the detection object DB of the data center by sending the recognition result obtained by the driving assistance ECU 14 to the data center. If it is such a structure, the detection object DB based on the detection results of other vehicles can be shared, so that, for example, the historical information and high-precision mark about a specific display object detected for the first time by the vehicle 100 can be used.

[0093] Furthermore, the following configuration may be adopted: when the period from the last update date of the detected specific display object is more than the prescribed number of days, the reliability of the history of the specific display object is considered to be low, and the auxiliary device 10 communicates with the data center to refer to the detection object DB of the data center. Then, when the last update date of the specific display object registered in the detection object DB of the data center is newer than the last update date of the detection object DB 27 registered in the driving assistance ECU 14 of the vehicle 100, the driving assistance control is performed using the information of the detection object DB registered in the data center. According to such a configuration, compared with the configuration of using only the detection object DB 27 of the driving assistance ECU 14 of the vehicle 100, the accuracy of setting the assistance level of the driving assistance control can be improved.

[0094] In addition, it can also be a structure as follows: when the detected specific display object is registered in the detection object DB27 of the driving assistance ECU14, the auxiliary device 10 refers to the detection object DB27, and when the detected specific display object is not registered in the detection object DB27 of the driving assistance ECU14, the auxiliary device 10 communicates with the data center to refer to the detection object DB of the data center. Moreover, it can also be a structure as follows: when the period from the last update date of the detected specific display object is more than the prescribed period, the auxiliary device 10 communicates with the data center to refer to the detection object DB of the data center. That is, it can also be a structure as follows: when the accuracy (reliability) of the detection object DB27 of the driving assistance ECU14 of the vehicle 100 is considered to be low, the auxiliary device 10 communicates with the data center to refer to the detection object DB27 of the data center. According to such a structure, communication fees can be suppressed and high-precision driving assistance control can be provided.

[0095] Although the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and the following embodiments are also included in the technical scope of the present invention. In addition, in addition to the following, various changes can be made and implemented without departing from the scope of the present invention.

[0096] For example, in the above-described embodiment, a sign indicating the maximum speed is shown as a display object of the detection object, but the display object of the detection object is not limited. For example, various road signs such as a sign prohibiting travel except in a specified direction, a temporary stop sign, and a minimum speed sign can be applied as the display object of the detection object. In addition, in addition to road signs, electronic bulletin boards and various guide displays can also be applied to the display object of the detection object.

[0097] It should be noted that the first, second, and third levels of driving assistance control described above are examples and are not limited thereto. For example, these driving assistance controls may be controls as described below.

[0098] <Combination Example 1>

[0099] First level: The CPU executes notification control to cause the display device 17 to display a still image of a detected specific display object.

[0100] Second level: The CPU executes notification control to cause the display device 17 to display a moving image of a specific display object (for example, a screen that causes the specific display object to flash).

[0101] Third level: The CPU performs both the report control of the second level and the above-mentioned adaptation control.

[0102] <Combination Example 2>

[0103] First level: The CPU executes notification control to display a still image of a specific display object on the display device 17 and to output a warning sound on the sound output device 18 .

[0104] Second level: The CPU executes notification control to display a still image of a specific display object on the display device 17 and to output a warning sound with a volume higher than that of the first level warning sound on the sound output device 18 .

[0105] Third level: The CPU performs both the report control of the second level and the above-mentioned adaptation control.

[0106] <Combination Example 3>

[0107] First level: The CPU executes notification control to display a still image of a specific display object on the display device 17 and to output a warning sound on the sound output device 18 .

[0108] Second level: The CPU executes notification control to display a moving image of a specific display object on the display device 17 and to output a warning sound with a volume higher than that of the first level warning sound on the sound output device 18 .

[0109] Third level: The CPU performs both the report control of the second level and the above-mentioned adaptation control.

[0110] It should be noted that the first to third levels of driving assistance control may include the notification control but not the adaptation control. For example, the following combinations may be possible.

[0111] <Combination Example 4>

[0112] First level: The CPU executes notification control to cause the display device 17 to display a still image of a specific display object.

[0113] Second level: The CPU executes notification control to cause the display device 17 to display a moving image of a specific display object and to cause the sound output device 18 to output a warning sound.

[0114] Level 3: The CPU executes notification control to display a moving image of a specific display object (however, a moving image of more intense motion than that of the level 2) on the display device 17 and to output a warning sound louder than the level 2 warning sound on the sound output device 18 .

[0115] <Combination Example 5>

[0116] First level: The CPU executes notification control to cause the display device 17 to display a still image of a specific display object.

[0117] Second level: The CPU executes notification control to cause the display device 17 to display a moving image of a specific display object (for example, a screen that causes the specific display object to flash).

[0118] Level 3: The CPU executes notification control to cause the display device 17 to display a moving image of a specific display object (for example, a screen in which the specific display object is blinking) and to cause the sound output device 18 to output a warning sound.

[0119] Furthermore, in the above embodiment, the driving assistance control assistance level has three levels, namely, the first level, the third level, but the number of driving assistance control assistance levels is not limited to three. That is, the driving assistance control assistance levels may be two or four or more.

[0120] In addition, in the above-described embodiment, the detected object DB 27 is stored in the memory 24 of the driving assistance ECU 14, but the present invention is not limited to such a configuration. For example, the following configuration may be adopted: the auxiliary device 10 further has a memory or a storage device (e.g., a hard disk, an SSD, etc.), and the detected object DB 27 is stored in the memory or the storage device.

[0121] In addition, in each of the above embodiments, a vehicle having an engine (internal combustion engine) as a driving force source is shown, but the driving force source is not limited to the engine. The present invention can also be applied to a vehicle having a motor (electric motor) as a driving force source (electric vehicle) and a vehicle having an engine and a motor as driving force sources (hybrid vehicle).

Claims

1. A driving assistance device for a vehicle, comprising: a camera sensor that generates image data by photographing scenery in front of the vehicle; and a control unit that, when determining that a specific display object exists in front of the vehicle based on the generated image data, identifies display content of the specific display object determined to exist, and performs driving assistance control, which is control for assisting driving performed by a driver of the vehicle and is control corresponding to the display content of the specific display object, The control unit is configured to: In a case where it is determined that the consistency rate of the recognition result as a ratio of "the number of times the recognition results of the display content of the same specific display object are the same" to "the number of times the same specific display object is determined to exist" is less than a first threshold, a control with a lower assistance level indicating the degree of influence on the driver is executed as the driving assistance control, compared with a case where it is determined that the consistency rate of the recognition result is greater than the first threshold, When the number of detections for the same specific display object is more than the prescribed number of times and the consistency rate of the recognition results is more than the second threshold, the high-precision flag of the same specific display object is set to be enabled; when the number of detections for the same specific display object is more than the prescribed number of times, but the consistency rate of the recognition results for the same specific display object is less than the first threshold, the high-precision flag of the same specific display object is set to be disabled; when the high-precision flag is enabled, the assistance level of the driving assistance control is increased compared to when the high-precision flag is disabled, wherein, The second threshold is a value higher than the first threshold, further comprising a database in which a history of the recognition result of the position of the specific display object determined to exist by the control unit and the display content is registered, The control unit is configured to: registering the position of the specific display object determined to exist and the recognition result of the display content in the database, and calculating the consistency rate of the recognition result about the same specific display object with reference to the database, The control unit is configured to: The similarity between the image data and a template prepared in advance for determining the existence of the specific display object is calculated, and when the similarity is above a detection threshold, it is determined that the specific display object exists; when the similarity is above the detection threshold but below an accuracy threshold which is a value higher than the detection threshold, the recognition result regarding the specific display object is not registered in the database.

2. The driving assistance device for a vehicle according to claim 1, further comprising a reporting device for reporting information related to the specific display object to the driver, The control unit is configured to: When it is determined that the consistency rate of the recognition result is greater than the first threshold value, a control including a reporting control and a suitable control is executed as the driving assistance control, wherein the reporting control is a control for reporting to the driver that the specific display object is determined to be present by the reporting device, and the suitable control is a vehicle control for bringing the driving state of the vehicle close to a state suitable for the display content of the specific display object, If it is determined that the consistency rate of the recognition result is less than the first threshold value, a control including the notification control but not including the adaptation control is executed as the driving assistance control.

3. The driving assistance device for a vehicle according to claim 1, wherein: The control unit is configured to: The recognition result regarding the specific display object that has not been detected within a predetermined period is deleted from the database.

Images