Vehicle display device

Abstract

By displaying traffic icons corresponding to traffic signs recognized by an external camera, along with corresponding driving instructions, on the in-car display, drivers can easily understand traffic signs and how to respond. [Solution] A vehicle display device that displays traffic display icons on an in-vehicle display corresponding to traffic displays in front of the vehicle recognized by an external camera of the vehicle, wherein when displaying the traffic display icons on the display, a driving response method pre-associated with the traffic display is displayed on the display.

Description

[Technical Field]

[0001] The present invention relates to a display device for a vehicle. [Background Art]

[0002] Conventionally, Japanese Unexamined Patent Application Publication No. 2014-120113 is known as a technical document relating to vehicle display devices. This publication discloses a technology for displaying a road sign recognized by a vehicle-mounted camera on an in-vehicle display. [Prior Art Literature] [Patent Literature]

[0003] [Patent Literature 1] Japanese Unexamined Patent Application Publication No. 2014-120113 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] However, merely displaying a road sign on an in-vehicle display poses a problem that it is not easy for an inexperienced driver to associate the sign with appropriate driving behavior. [Means for Solving the Problem]

[0005] One aspect of the present invention is a vehicle display device that displays, on an in-vehicle display, a traffic indication icon corresponding to a traffic indication ahead of the vehicle recognized by an external camera of the vehicle, wherein when the traffic indication icon is displayed on the display, a driving handling method pre-associated with the traffic indication is displayed on the display.

[0006] In a vehicle display device according to one aspect of the present invention, when an external camera of the vehicle recognizes a conditional traffic display, it is determined whether the vehicle's status satisfies the conditions for application of the conditional traffic display. If it is determined that the vehicle's status satisfies the conditions, a traffic display icon corresponding to the conditional traffic display and a driving response method pre-associated with the traffic display are displayed on the display. If it is determined that the vehicle's status does not satisfy the conditions, a traffic display icon and a driving response method are displayed on the display.

[0007] In a vehicle display device according to one aspect of the present invention, when an external camera of the vehicle recognizes a time-conditional traffic display, it is determined whether the current time satisfies the time conditions of the time-conditional traffic display. If it is determined that the current time satisfies the time conditions, the traffic display icon corresponding to the traffic display and the driving response method for when the time conditions are met, which is pre-associated with the traffic display icon, are displayed on the display. If it is determined that the current time does not satisfy the time conditions, the traffic display icon and the driving response method for when the time conditions are not met, which is pre-associated with the traffic display icon, are displayed on the display.

[0008] In a vehicle display device according to one aspect of the present invention, the display includes a first display and a second display having a larger screen display than the first display. When displaying traffic display icons on the first or second display, the first display may display a driving method pre-associated with the traffic display icon, and the second display may display a detailed explanation of the traffic display icon. [Effects of the Invention]

[0009] According to each aspect of the present invention, by displaying traffic display icons corresponding to traffic displays recognized by an external camera and driving response methods pre-associated with the traffic displays on a display inside the vehicle, the driver can easily grasp the traffic displays and driving response methods. [Brief explanation of the drawing]

[0010] [Figure 1] A block diagram of a vehicle display device according to one embodiment. [Figure 2] (a) A diagram showing an example of a time-conditional road sign. (b) A diagram showing an example of the display of driving instructions in the MID (Mid-Indicator). (c) A diagram showing an example of the display of road sign icons and detailed explanations in the center display. [Figure 3] (a) This figure shows an example of a road sign with vehicle type restrictions. (b) This figure shows an example of a display of driving instructions in MID (Mission Control Day). [Figure 4] (a) This figure shows an example of a road sign indicating a road closure and a display of instructions for driving. (b) This figure shows an example of a road sign indicating a road closure and a display of instructions for driving. (c) This figure shows an example of a road sign indicating a no-entry zone and a display of instructions for driving. [Figure 5] This figure shows an example of how the road sign icon and detailed explanation for a road closure are displayed on the center display. [Figure 6] This flowchart shows an example of the display process for traffic display icons and driving response methods in a vehicle display device. [Figure 7] This flowchart shows an example of display processing for conditional traffic displays in vehicle display devices. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described below with reference to the drawings.

[0012] Figure 1 is a block diagram showing a vehicle display device 100 according to one embodiment. The vehicle display device 100 shown in Figure 1 is a device that is mounted on a vehicle such as a passenger car or a truck (hereinafter referred to as "vehicle") and displays information related to traffic signs recognized by an onboard external camera on a display inside the vehicle. Traffic signs refer to signs that indicate information related to traffic. Specifically, traffic signs include at least one of the following: road signs, road markings, traffic information signs, etc.

[0013] The vehicle display device 100 includes an ECU (Electronic Control Unit) 10. The ECU 10 is an electronic control unit having a CPU (Central Processing Unit) and a memory unit. The memory unit consists of, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and EEPROM (Electrically Erasable Programmable Read-Only Memory). The ECU 10 realizes various functions by, for example, executing a program stored in the memory unit with the CPU. The ECU 10 may be composed of multiple electronic units.

[0014] The ECU 10 is connected to an external camera 1, a display 2, and a speaker 3. The external camera 1 is an imaging device that captures images of the external conditions of the vehicle. The external camera 1 includes, for example, a front camera mounted behind the vehicle's windshield to capture images of the area in front of the vehicle. The external camera 1 may also include a rear camera to capture images of the area behind the vehicle and side cameras to capture images of the left and right sides of the vehicle. The external camera 1 transmits the captured images of the external conditions of the vehicle to the ECU 10.

[0015] Display 2 is a device installed inside the vehicle cabin that displays various information to the vehicle's driver. Display 2 includes a first display, MID60 (Multi-Information Display), and a second display, the Center Display 70.

[0016] The MID60 is a display that forms part of the instrument panel within the driver's field of vision and is designed to quickly provide driving-related information. Specifically, the MID60 is integrated into the vehicle's meter panel and displayed alongside other driving information such as the speedometer and fuel gauge. This allows the driver to instantly check the necessary information without significantly shifting their gaze.

[0017] The center display 70 is a second display arranged in the central portion of the vehicle, and is used for providing detailed information and explanations. The center display 70 is arranged at a position that can be easily visually recognized by a driver and a fellow passenger, and has a wider display area for displaying more information compared to the MID 60.

[0018] The speaker 3 is a device for outputting voice guidance and warning sounds to a driver. As one example, the speaker 3 is provided inside a vehicle door or on a back side of a meter panel.

[0019] Next, the functional configuration of the ECU 10 will be described. As shown in FIG. 1, the ECU 10 includes a traffic sign recognition unit 11 and a display control unit 12.

[0020] The traffic sign recognition unit 11 analyzes images acquired from the external camera 1 and recognizes traffic signs including road signs. The traffic sign recognition unit 11 executes, for example, various algorithms for recognizing traffic signs with high accuracy. Specifically, the traffic sign recognition unit 11 extracts feature quantities from an image using image processing technology, and collates the feature quantities with a pre-registered sign database to specify the type and content of the traffic sign. For example, based on features such as shape, color, and character information, stop signs, speed limit signs, no-entry signs, and the like are recognized.

[0021] Here, FIG. 2(a) is a diagram showing an example of a time-conditioned road sign. A time-conditioned road sign is a road sign whose applicable content such as the rules indicated on the road sign changes depending on the time period.

[0022] The traffic sign recognition unit 11 recognizes the time-conditioned road sign 50 shown in FIG. 2(a) based on the image captured by the external camera 1. The time-conditioned road sign 50 is attached with a time condition display 51 on which the time condition "7-10" is described. The time-conditioned road sign 50 shown in FIG. 2(a) indicates that left turning is prohibited in a specific time period (from 7 o'clock to 10 o'clock).

[0023] Furthermore, the traffic sign recognition unit 11 may use machine learning algorithms to improve recognition accuracy. For example, an image recognition model using deep learning may be trained to improve sign recognition accuracy under various conditions. In addition, the traffic sign recognition unit 11 may further improve recognition accuracy by complementing the location information of traffic signs by linking with GPS data and map information.

[0024] The display control unit 12 displays various information corresponding to the traffic display recognized by the traffic display recognition unit 11 on the display 2. The display control unit 12 receives the recognition result sent from the traffic display recognition unit 11 and generates the display content based on it.

[0025] The display control unit 12 displays, for example, traffic display icons and driving response methods pre-associated with traffic displays on the display 2. Traffic display icons are icons corresponding to traffic displays such as road signs. Driving response methods are the ways in which the driver should take action to comply with the rules and regulations shown in the traffic displays. The traffic display icons and driving response methods are stored in the vehicle's memory unit or the like, pre-associated with the traffic displays. The traffic display icons may be created by cutting out traffic displays from actual captured images.

[0026] Furthermore, the display control unit 12 may display a detailed explanation corresponding to the traffic display on the display 2. The detailed explanation corresponding to the traffic display may be stored in association with the traffic display, or it may be generated by a machine learning model. The detailed explanation will be a longer explanatory text than the driving instructions.

[0027] Here, Figure 2(b) shows an example of how driving instructions are displayed in the MID60. When the traffic display recognition unit 11 recognizes the time-conditional road sign 50 and time-condition display 51 shown in Figure 2(a), the display control unit 12 determines whether the current time satisfies the time condition. If the display control unit 12 determines that the current time satisfies the time condition, it displays the driving instructions stored in association with the time-conditional road sign 50 that apply when the time condition is met in the text window 61 of the MID60. For example, the display control unit 12 displays the driving instruction "You cannot turn left now" in the MID60.

[0028] On the other hand, if the display control unit 12 determines that the current time does not meet the time condition, it displays a driving action method stored in association with the time-conditional road sign 50 for when the time condition is not met in the text window 61 of the MID 60. For example, the display control unit 12 displays the driving action method "You can turn left now" in the MID 60.

[0029] Figure 2(c) shows an example of the display of road sign icons and detailed descriptions on the center display 70. The display control unit 12 displays the road sign icon 80 stored in association with the time-conditional road sign 50 recognized by the traffic sign recognition unit 11 and the time-condition display icon 81 stored in association with the time-condition display 51 on the center display 70. The display control unit 12 may also display the road sign icon 80 on the MID 60, or it may display the road sign icon 80 on both the MID 60 and the center display 70.

[0030] Furthermore, the display control unit 12 obtains a detailed explanation of the time-conditional road sign 50 from the storage unit or generates it using a machine learning model. For example, as shown in Figure 2(c), the display control unit 12 displays the detailed explanation on the center display 70, which reads, "This sign indicates that the road is one-way only during the time indicated on the sign. If it is outside of this time, the one-way restriction will be lifted." The display control unit 12 may change the content of the detailed explanation depending on whether the current time satisfies the time condition.

[0031] Next, we will explain the case of road signs with vehicle type conditions. Road signs with vehicle type conditions are road signs in which the applicable rules and regulations indicated on the road sign change depending on the vehicle type. Figure 3(a) is a diagram showing an example of road signs with vehicle type conditions. Figure 3(a) shows the first road sign with vehicle type conditions 101 and the first vehicle type condition indication 102, and the second road sign with vehicle type conditions 103 and the second vehicle type condition indication 104.

[0032] The first vehicle-type conditional road sign 101 indicates a speed limit of "80," and the first vehicle-type condition indication below it 102 indicates the vehicle type condition "large trucks and other three-wheeled towing vehicles." This indicates that a speed limit of 80 km / h applies to large trucks, three-wheeled vehicles, and towing vehicles. Similarly, the second vehicle-type conditional road sign 103 indicates a speed limit of "120," and the second vehicle-type condition indication below it 104 indicates the vehicle type condition "excluding large trucks and other three-wheeled towing vehicles." This indicates that a speed limit of 120 km / h applies to all vehicles except large trucks, three-wheeled vehicles, and towing vehicles.

[0033] The traffic sign recognition unit 11 recognizes the first vehicle type conditional road sign 101, the first vehicle type conditional display 102, the second vehicle type conditional road sign 103, and the second vehicle type conditional display 104 shown in Figure 3(a) based on the image captured by the external camera 1.

[0034] Based on the recognition result of the traffic display recognition unit 11, the display control unit 12 determines from the pre-stored vehicle type (e.g., passenger car) that the vehicle falls under the second vehicle type condition display 104, "excluding large trucks and other three-wheeled towing vehicles," and that the speed limit of 120 km / h of the second vehicle type conditional road sign 103 applies.

[0035] Based on the above determination result, the display control unit 12 displays the driving response method corresponding to the second vehicle-type conditional road sign 103 on the MID 60. Figure 3(b) is a diagram showing an example of the display of the driving response method on the MID 60. For example, as shown in Figure 3(b), the display control unit 12 displays the message "From here, the speed limit is 120 km / h" on the MID 60. The display control unit 12 may also display the road sign icon and detailed explanation corresponding to the second vehicle-type conditional road sign 103 on the center display 70.

[0036] On the other hand, the display control unit 12 does not display information regarding the first vehicle-type conditional road sign 101 that does not meet the vehicle type requirements. However, the display control unit 12 may also enable the display of the road sign icon and detailed description of the first vehicle-type conditional road sign 101 based on the driver's operation on the center display 70 which displays the road sign icon and detailed description of the second vehicle-type conditional road sign 103.

[0037] Next, we will explain the cases of road signs that are easily misinterpreted. Figure 4(a) shows an example of a road no-vehicle sign and a display of how to react while driving. Figure 4(b) shows an example of a road no-vehicle sign and a display of how to react while driving. Figure 4(c) shows an example of a road no-entry sign and a display of how to react while driving.

[0038] The road no-vehicle sign 90 shown in Figure 4(a) is a road sign that prohibits the passage of all vehicles except pedestrians. It is installed on pedestrian-only roads and in places where vehicle traffic is dangerous due to construction or other reasons. The road no-passing sign 91 shown in Figure 4(b) is a road sign that prohibits the passage of both people and vehicles. It is installed in places where physical passage is impossible. The no-entry road sign 92 shown in Figure 4(c) is a road sign that prohibits vehicles from entering that location. It is installed at the exit of a one-way street and at busy intersections.

[0039] When the traffic sign recognition unit 11 recognizes a road no-entry sign 90, the display control unit 12 displays the driving instructions corresponding to the road no-entry sign 90 on the MID 60. For example, as shown in Figure 4(a), the display control unit 12 displays the message "Road no-entry from here onwards" in the text window 61 of the MID 60.

[0040] Furthermore, the display control unit 12 displays a road sign icon and detailed explanation corresponding to the road no entry road sign 90 on the center display 70. Figure 5 is a diagram showing an example of the display of the road no entry road sign icon 95 and detailed explanation on the center display 70. As shown in Figure 5, the display control unit 12 displays the road no entry road sign icon 95 and detailed explanation on the center display 70. As an example of the detailed explanation, it displays "All vehicles (automobiles, motorcycles, and light vehicles) except pedestrians are prohibited from passing."

[0041] When the traffic sign recognition unit 11 recognizes a road closure sign 91, the display control unit 12 displays the appropriate driving procedure for the road closure sign 91 on the MID 60. For example, as shown in Figure 4(b), the display control unit 12 displays the message "Road closure from here onwards" in the text window 61 of the MID 60. The display control unit 12 may also display a road sign icon and a detailed explanation on the center display 70.

[0042] Similarly, when the traffic sign recognition unit 11 recognizes a no-entry road sign 92, the display control unit 12 displays the driving instructions corresponding to the no-entry road sign 92 on the MID 60. For example, as shown in Figure 4(c), the display control unit 12 displays the message "Entry is prohibited from here onwards" in the text window 61 of the MID 60. The display control unit 12 may also display a road sign icon and a detailed explanation on the center display 70.

[0043] In this way, the vehicle display device 100 can display driving instructions and other information on the display 2 regarding road signs 90-92 that are easily misinterpreted, thereby reducing the likelihood of drivers becoming confused or anxious when they see road signs 90-92 that are easily misinterpreted.

[0044] Next, the processing of the vehicle display device 100 according to this embodiment will be described with reference to the drawings. Figure 6 is a flowchart showing an example of the display processing of traffic display icons and driving response methods in the vehicle display device 100. This flowchart shows a series of processes that recognize traffic displays and display appropriate information on the display 2 based on them.

[0045] As shown in Figure 6, in step S10, the ECU 10 of the vehicle display device 100 recognizes the traffic display in front of the vehicle using the traffic display recognition unit 11. If the ECU 10 does not recognize the traffic display (step S10: NO), it terminates the process. If the ECU 10 recognizes the traffic display (step S10: YES), it proceeds to step S11.

[0046] In step S11, the ECU 10 displays traffic display icons and detailed descriptions on the center display 70 via the display control unit 12. Specifically, the display control unit 12 retrieves traffic display icons and their associated detailed information from the storage unit based on the recognition results of the traffic display recognition unit 11 and displays them on the center display 70. The detailed information may be generated by a machine learning model based on the image of the traffic display.

[0047] Next, in step S12, the ECU 10 displays a driving response method to the MID 60 via the display control unit 12. Specifically, the display control unit 12 retrieves a driving response method corresponding to the traffic display recognized by the traffic display recognition unit 11 from the storage unit and displays it on the MID 60. The driving response method may be generated by a machine learning model from the image of the traffic display and the vehicle's driving environment (driving lane, vehicle speed, etc.). After that, the ECU 10 terminates this process.

[0048] Figure 7 is a flowchart showing an example of the display process for conditional traffic displays in the vehicle display device 100. This flowchart shows a series of processes that recognize conditional traffic displays and display appropriate information on the display 2 based on them. Conditional traffic displays are a concept that includes the time-conditional road signs and vehicle-conditional road signs mentioned above. It is not limited to road signs, but also includes road markings, etc.

[0049] As shown in Figure 7, in step S20, the ECU 10 of the vehicle display device 100 recognizes the conditional traffic display in front of the vehicle using the traffic display recognition unit 11. If the conditional traffic display is not recognized (step S20: NO), the ECU 10 terminates the process. If the conditional traffic display is recognized (step S20: YES), the ECU 10 proceeds to step S21.

[0050] In step S21, the ECU 10 determines whether the conditions for applying the conditional traffic display are met by the display control unit 12. Specifically, in the case of a time-conditional traffic display, the current time is checked, and in the case of a vehicle-conditional traffic display, the vehicle type is checked. If the conditions for applying the condition are not met (step S21: NO), the ECU 10 terminates the current process. If the conditions for applying the condition are met (step S21: YES), the ECU 10 proceeds to step S22.

[0051] In step S22, the ECU 10 displays the applicable traffic display icon and detailed description on the center display 70 via the display control unit 12. Specifically, the display control unit 12 retrieves the applicable traffic display icon and its associated detailed information from the storage unit and displays it on the center display 70. The detailed information may be generated by a machine learning model based on the traffic display image.

[0052] Next, in step S23, the ECU 10 displays the appropriate driving response method for when the conditions are met to the MID 60 via the display control unit 12. Specifically, the display control unit 12 retrieves the appropriate driving response method for when the conditions are met, corresponding to the traffic display with applicable conditions recognized by the traffic display recognition unit 11, from the storage unit and displays it on the MID 60. The driving response method may be generated by a machine learning model from the image of the traffic display and the vehicle's driving environment (driving lane, vehicle speed, etc.). After that, the ECU 10 terminates this process.

[0053] According to the vehicle display device 100 of this embodiment described above, by displaying on the display 2 the traffic display icon corresponding to the traffic display captured by the external camera 1 and the driving response method associated with the traffic display, the driver can easily grasp the traffic display in front of the vehicle and the driving response method.

[0054] Furthermore, if the vehicle's external camera 1 recognizes a traffic display that is subject to specific conditions, the vehicle display device 100 may determine whether the vehicle's condition meets those conditions, and only if it determines that the conditions are met, it may display the traffic display icon and driving instructions corresponding to the traffic display on the display 2. This reduces the visual burden on the driver by not displaying unnecessary information. Also, since information is only displayed when the conditions are met, the driver can take appropriate driving actions and comply with traffic regulations.

[0055] Furthermore, if the vehicle's external camera 1 recognizes a traffic display that is time-conditional, the vehicle display device 100 determines whether the current time meets the time condition. If the time condition is met, it displays the traffic display icon corresponding to the traffic display and the driving response method when the time condition is met on the display 2. If the time condition is not met, it does not need to display the traffic display icon and the driving response method when the time condition is not met on the display 2. This allows drivers to easily understand the meaning of time-conditional traffic displays and take appropriate driving actions. In particular, it makes it easier to respond flexibly and with ample time to traffic regulations that differ depending on the time of day.

[0056] Furthermore, the vehicle display device 100 has a configuration that includes a MID60 as the display 2 and a center display 70 with a larger screen display than the MID60. When a traffic display icon is displayed on the MID60 or the center display 70, the MID60 displays the driving response method that is pre-associated with the traffic display, and the center display 70 displays a detailed explanation of the traffic display icon. With this configuration, the driver can quickly grasp the traffic display icon and the driving response method, and can also check the detailed explanation. In addition, by displaying the detailed explanation on the center display 70, it is possible to avoid the text becoming too small compared to when it is displayed on the MID60.

[0057] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. The present invention can be implemented in various forms, starting with the embodiments described above, by making various changes and improvements based on the knowledge of those skilled in the art.

[0058] For example, the first display is not limited to the MID60, and the second display is not limited to the center display 70. The first or second display may be an additional display installed in the vehicle cabin, or it may be a head-up display. Display 2 may consist of one display, or it may consist of three or more displays. If Display 2 consists of one display, traffic display icons, driving instructions, and detailed explanations will be displayed on the same display. In this case, the detailed explanations do not have to be displayed. [Explanation of Symbols]

[0059] 1...External camera, 2...Display, 3...Speaker, 10...ECU, 11...Traffic display recognition unit, 12...Display control unit, 60...MID, 70...Center display, 100...Vehicle display device.

Claims

1. A vehicle display device that displays traffic display icons on an in-cabin display that correspond to traffic displays in front of the vehicle recognized by an external camera of the vehicle, A vehicle display device that, when displaying the aforementioned traffic display icon on the display, displays a driving response method pre-associated with the traffic display on the display.

2. When the external camera of the vehicle recognizes a conditional traffic sign, it is determined whether the vehicle's condition satisfies the conditions for applying the conditional traffic sign. The vehicle display device according to claim 1, wherein, when it is determined that the status of the vehicle satisfies the application conditions, the traffic display icon corresponding to the conditional traffic display and the driving response method pre-associated with the traffic display are displayed on the display, and when it is determined that the status of the vehicle does not satisfy the application conditions, the traffic display icon and the driving response method are not displayed on the display.

3. When the vehicle's external camera recognizes a time-conditional traffic display, it determines whether the current time satisfies the time conditions of the time-conditional traffic display. The vehicle display device according to claim 1, wherein, when it is determined that the current time satisfies the aforementioned time conditions, the display shows the traffic display icon corresponding to the traffic display and a pre-associated driving method for when the time conditions are met, and when it is determined that the current time does not satisfy the aforementioned time conditions, the display shows the traffic display icon and a pre-associated driving method for when the time conditions are not met,

4. The aforementioned display includes a first display and a second display with a larger screen display than the first display. The vehicle display device according to claim 1, wherein when the traffic display icon is displayed on the first display or the second display, a driving method pre-associated with the traffic display is displayed on the first display, and a detailed explanation of the traffic display is displayed on the second display.

Images