Display device for vehicle

Abstract

A display device for a vehicle displays a traffic sign icon corresponding to a traffic sign ahead of the vehicle recognized by an external camera of the vehicle on a display in the vehicle cabin, wherein a driving response method associated with the traffic sign in advance is displayed on the display when the traffic sign icon is displayed on the display.

Description

Technical Field

[0001] This invention relates to a display device for vehicles. Background Technology

[0002] Previously, Japanese Patent Application Publication No. 2014-120113 was known as a technical document related to display devices for vehicles. This publication disclosed a technology that displays road signs identified by a vehicle's camera on a monitor inside the vehicle. Summary of the Invention

[0003] However, there is a problem: simply displaying road signs on the monitor inside the vehicle makes it difficult for inexperienced drivers to associate them with appropriate driving behavior.

[0004] One aspect of the present invention is a display device for a vehicle that displays traffic sign icons corresponding to traffic signs in front of the vehicle as identified by an external camera on a display screen inside the vehicle.

[0005] When a traffic sign icon is displayed on the screen, the driving response method pre-associated with that traffic sign is also displayed on the screen.

[0006] In a vehicle display device according to one aspect of the present invention, it can be as follows:

[0007] If the vehicle's external camera identifies a traffic sign with applicable conditions, determine whether the vehicle's condition meets the applicable conditions of the traffic sign.

[0008] If the vehicle's condition is determined to meet the applicable conditions, the traffic sign icon corresponding to the traffic sign with applicable conditions and the driving response method pre-associated with the traffic sign will be displayed on the screen. If the vehicle's condition is determined not to meet the applicable conditions, the traffic sign icon and driving response method will not be displayed on the screen.

[0009] In a vehicle display device according to one aspect of the present invention, it can be as follows:

[0010] If the vehicle's external camera identifies a traffic sign with time information, determine whether the current time meets the time requirements of the traffic sign.

[0011] If the current time condition is met, the traffic sign icon corresponding to the traffic sign and the driving response method when the time condition associated with the traffic sign icon is sufficient will be displayed on the screen. If the current time condition is not met, the traffic sign icon and the driving response method when the time condition associated with the traffic sign icon is insufficient will be displayed on the screen.

[0012] In a vehicle display device according to one aspect of the present invention, it can be as follows:

[0013] The display includes a first display and a second display with a larger image size than the first display.

[0014] When a traffic sign icon is displayed on a first display or a second display, a driving response method pre-associated with the traffic sign icon is displayed on the first display, and a detailed description of the traffic sign icon is displayed on the second display.

[0015] According to various embodiments of the present invention, traffic sign icons corresponding to traffic signs identified by external cameras and driving response methods pre-associated with the traffic signs are displayed on a monitor inside the vehicle. At this time, the driver can easily grasp the traffic signs and driving response methods. Therefore, the present invention has the effect of reducing the driver's cognitive load. Attached Figure Description

[0016] Hereinafter, with reference to the accompanying drawings, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, in which the same reference numerals denote the same elements, and wherein:

[0017] Figure 1 This is a block diagram illustrating a vehicle display device according to one embodiment.

[0018] Figure 2A This is an example of a road sign with time-related information.

[0019] Figure 2B This is a diagram illustrating an example of how driving responses are displayed in a MID (Multi-Input Device).

[0020] Figure 2C This is an example of a road sign icon and its detailed description displayed on a central display screen.

[0021] Figure 3A This is an example of a road sign indicating vehicle type requirements.

[0022] Figure 3B This is a diagram illustrating an example of how driving responses are displayed in a MID (Multi-Input Device).

[0023] Figure 4A This is an example of a road sign indicating that vehicles are prohibited from passing and the driving instructions.

[0024] Figure 4B This is an example of a road sign indicating no-entry and the driving instructions.

[0025] Figure 4C This is an example of a road sign indicating no entry and the driving instructions.

[0026] Figure 5 This is an example of a road sign icon and detailed description displayed on the central display screen indicating that vehicles are prohibited from passing.

[0027] Figure 6 This is a flowchart illustrating an example of the display processing of traffic sign icons and driving response methods on a vehicle's display device.

[0028] Figure 7 This is a flowchart illustrating an example of display processing for traffic signs with applicable conditions in a vehicle display device. Detailed Implementation

[0029] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0030] Figure 1 This is a block diagram illustrating a vehicle display device 100 according to one embodiment. Figure 1 The vehicle display device 100 shown is mounted on a car, truck, or other vehicle (hereinafter referred to as a vehicle), and displays information related to traffic signs identified by an onboard external camera on a display screen inside the vehicle. Traffic signs refer to signs and the like that indicating traffic-related information. Specifically, traffic signs include at least one of road signs, road markings, traffic signs, etc.

[0031] The vehicle display device 100 includes an Electronic Control Unit (ECU) 10. The ECU 10 is an electronic control unit with a Central Processing Unit (CPU) and a storage unit. The storage unit may consist of, for example, Read Only Memory (ROM), Random Access Memory (RAM), or Electrically Erasable Programmable Read-Only Memory (EEPROM). In the ECU 10, various functions are implemented, for example, by the CPU executing programs stored in the storage unit. The ECU 10 may be composed of multiple electronic units.

[0032] ECU 10 is connected to external camera 1, display 2, and speaker 3. External camera 1 is a camera device that captures images of the vehicle's external environment. External camera 1 may include, for example, a front camera mounted behind the vehicle's windshield and capturing images of the front of the vehicle. External camera 1 may also include a rear camera capturing images of the rear of the vehicle and side cameras capturing images of the left and right sides of the vehicle. External camera 1 sends images related to the vehicle's external environment to ECU 10.

[0033] Display 2 is a device installed inside the vehicle interior to display various information to the driver. Display 2 includes MID60 (Multi-Information Display) which serves as the first display and a central display 70 which serves as the second display.

[0034] The MID60 is a display that forms part of the dashboard within the driver's field of vision, designed to quickly provide driving-related information. Specifically, the MID60 is integrated into the vehicle's instrument panel and displays information along with other driving data such as the speedometer and fuel gauge. This allows the driver to immediately access the necessary information without significantly shifting their gaze.

[0035] The central display 70 is a second display located in the center of the vehicle, used to provide detailed information or instructions. The central display 70 is positioned in a location easily visible to the driver and passengers, and has a wider display area for showing more information compared to the MID 60.

[0036] Speaker 3 is a device used to output audible guidance or warning sounds to the driver. As an example, speaker 3 is located on the inside of the vehicle's door or on the back of the dashboard.

[0037] Next, the functional structure of ECU10 will be explained. For example... Figure 1 As shown, the ECU10 has a traffic sign recognition unit 11 and a display control unit 12.

[0038] The traffic sign recognition unit 11 analyzes images acquired from the external camera 1 to identify traffic signs, including road signs. The traffic sign recognition unit 11 executes various algorithms for high-precision recognition of traffic signs. Specifically, the traffic sign recognition unit 11 uses image processing techniques to extract feature quantities from the image and compares them with a pre-registered sign database to determine the type or content of the traffic sign. For example, it identifies stop signs, speed limit signs, no-entry signs, etc., based on features such as shape, color, and character information.

[0039] Here, Figure 2A This is an example of a road sign with time-based conditions. A road sign with time-based conditions is a road sign that changes the applicable rules, etc., shown on the road sign according to a time period.

[0040] Traffic sign recognition unit 11 identifies traffic signs based on images captured by external camera 1. Figure 2A The road sign 50 with time conditions is shown. The road sign 50 with time conditions is accompanied by a time condition mark 51 that indicates the time condition as "7-10". Figure 2A The road sign 50 with time information indicates that left turns are prohibited during a specific time period (from 7:00 to 10:00).

[0041] Furthermore, to improve recognition accuracy, the traffic sign recognition unit 11 can use machine learning algorithms. For example, it can train an image recognition model using deep learning to improve sign recognition accuracy in various environments. Additionally, the traffic sign recognition unit 11 can further improve recognition accuracy by collaborating with GPS data or map information to supplement the location information of traffic signs.

[0042] The display control unit 12 displays various information corresponding to the traffic signs identified by the traffic sign recognition unit 11 on the display 2. The display control unit 12 receives the recognition results sent from the traffic sign recognition unit 11 and generates display content based on the recognition results.

[0043] The display control unit 12 displays, for example, traffic sign icons and driving response methods pre-associated with traffic signs on the display 2. Traffic sign icons are icons corresponding to road signs and other traffic signs. Driving response methods are methods that the driver should use to comply with the rules indicated by the traffic signs. The traffic sign icons and driving response methods are stored in the vehicle's storage unit or similar pre-associated with the traffic signs. Traffic sign icons can also be created by cropping traffic signs from actual camera images.

[0044] Furthermore, the display control unit 12 can display detailed descriptions corresponding to traffic signs on the display 2. These detailed descriptions can be stored in association with the traffic signs, or they can be generated by a machine learning model. The detailed descriptions become explanatory text with more characters than the driving response methods.

[0045] Here, Figure 2B This diagram illustrates an example of a driving response method displayed in the MID60. It shows the method identified by the traffic sign recognition unit 11. Figure 2AIn the case of the road sign 50 with time conditions and the time condition indicator 51 shown, the display control unit 12 determines whether the current time meets the time conditions. If it is determined that the current time meets the time conditions, the display control unit 12 displays the driving response method when the time conditions are sufficient, which is stored in association with the road sign 50 with time conditions, on the text window 61 of the MID 60. For example, the display control unit 12 displays the driving response method "Currently, left turn is not allowed" on the MID 60.

[0046] On the other hand, if it is determined that the current time does not meet the time conditions, the display control unit 12 displays a driving response method stored in association with the road sign 50 that indicates insufficient time conditions. This driving response method is displayed on the text window 61 of the MID 60. For example, the display control unit 12 displays the driving response method "Currently, a left turn is permitted" on the MID 60.

[0047] Figure 2C This diagram illustrates an example of the display of road sign icons and detailed descriptions on the central display 70. The display control unit 12 displays a road sign icon 80, which is associated with a time-condition road sign 50 recognized by the traffic sign recognition unit 11, and a time condition identifier icon 81, which is associated with a time condition identifier 51, on the central display 70. Alternatively, the display control unit 12 may display the road sign icon 80 on the MID 60, or on both the MID 60 and the central display 70.

[0048] Furthermore, the display control unit 12 retrieves detailed information about the road sign 50 with time-related conditions from the storage unit or generates it using a machine learning model. For example, the display control unit 12... Figure 2C As shown, the detailed explanation that "One-way traffic is only permitted during the time indicated by the sign. This means that if it is not the designated time, one-way traffic will be cancelled" is displayed on the central display 70. The display control unit 12 can change the content of the detailed explanation according to whether the time condition is met at the current time.

[0049] Next, we will explain road signs that specify vehicle type requirements. Road signs that specify vehicle type requirements refer to road signs that change the applicable rules and other information based on the vehicle type. Figure 3A This is an example of a road sign indicating vehicle type requirements. In Figure 3A The image shows a road sign 101 with first vehicle type conditions and a first vehicle type condition identifier 102, a road sign 103 with second vehicle type conditions and a second vehicle type condition identifier 104.

[0050] The first road sign with vehicle type conditions, 101, indicates a speed limit of "80 km / h," and the second sign below it, 102, indicates a speed limit of "large trucks and other three-wheeled tractors." This means that a speed limit of 80 km / h applies to large trucks, three-wheeled vehicles, and tractor-trailers. Similarly, the second road sign with vehicle type conditions, 103, indicates a speed limit of "120 km / h," and the second sign below it, 104, indicates a speed limit of "excluding large trucks and other three-wheeled tractors." This means that a speed limit of 120 km / h applies to vehicles other than large trucks, three-wheeled vehicles, and tractor-trailers.

[0051] Traffic sign recognition unit 11 identifies traffic signs based on images captured by external camera 1. Figure 3A The road sign 101 with first vehicle type condition and the vehicle type condition identifier 102 are shown. Furthermore, the road sign 103 with second vehicle type condition and the vehicle type condition identifier 104 are also identified.

[0052] Based on the recognition result of the traffic sign recognition unit 11, the display control unit 12 determines that the vehicle type (e.g., sedan) stored in the pre-stored vehicle type meets the second vehicle type condition label 104, which states "excluding large trucks and other three-wheeled tractors". Furthermore, the display control unit 12 determines that the speed limit of 120 km / h applies to the road sign 103 with the second vehicle type condition.

[0053] Based on the above determination result, the display control unit 12 displays the driving response method corresponding to the road sign 103 with the second vehicle type condition on the MID60. Figure 3B This diagram illustrates an example of a display showing the driving response method in the MID60. The display control unit 12, for example... Figure 3B As shown, the message "From here onwards, the speed limit is 120 km / h" is displayed on the MID60. The display control unit 12 can display the road sign icon or detailed description corresponding to the road sign 103 with the second vehicle type condition on the central display 70.

[0054] On the other hand, the display control unit 12 does not display information related to the road sign 101 that does not meet the first vehicle type condition. Alternatively, the display control unit 12 can display the road sign icon and detailed description of the road sign 101 that meets the first vehicle type condition through the driver's operation of the central display 70 indicating the road sign icon and detailed description of the road sign 103 that meets the second vehicle type condition. In this case, the display control unit 12 displays information about the road sign 101 that meets the first vehicle type condition based on the driver's operation.

[0055] Next, we will explain the situations where road signs are easily misidentified. Figure 4A This is an example of a road sign indicating that vehicles are prohibited from passing and the driving instructions. Figure 4BThis is an example of a road sign indicating no-entry and the driving instructions. Figure 4C This is an example of a road sign indicating no entry and the driving instructions.

[0056] Figure 4A The road sign 90, which prohibits vehicles from passing, means that all vehicles except pedestrians are prohibited from passing. It is placed on pedestrian-only roads or in locations where vehicle traffic is dangerous due to construction or other reasons. Figure 4B The road sign 91 shown is a road sign that prohibits the passage of people or vehicles. It is placed in locations where passage is physically impossible. Figure 4C The "No Entry" sign 92 shown indicates that vehicles are prohibited from entering this location. It is typically placed at one-way exits or high-traffic intersections.

[0057] When the traffic sign recognition unit 11 recognizes a road sign 90 prohibiting vehicles from passing, the display control unit 12 displays the driving response method corresponding to the road sign 90 on the MID 60. For example, Figure 4A As shown, the display control unit 12 displays the message "Vehicles are prohibited from passing from here on" in the text window 61 of the MID 60.

[0058] Furthermore, the display control unit 12 displays the road sign icon and detailed description corresponding to the road sign 90 prohibiting vehicle passage on the central display 70. Figure 5 This is an example of a road sign icon 95 indicating no-passage for vehicles displayed on the central display 70, along with a detailed description. (See figure.) Figure 5 As shown, the display control unit 12 displays the road sign icon 95 prohibiting vehicles from passing and its detailed description on the central display 70. For example, the detailed description displays the message: "All vehicles (cars, motorized bicycles, light vehicles) except pedestrians are prohibited from passing."

[0059] When the traffic sign recognition unit 11 recognizes a no-entry road sign 91, the display control unit 12 displays the driving response method corresponding to the no-entry road sign 91 on the MID 60. For example, Figure 4B As shown, the display control unit 12 displays the message "No passage from here" in the text window 61 of the MID 60. The display control unit 12 can also display road sign icons and detailed descriptions on the central display 70.

[0060] Similarly, when the traffic sign recognition unit 11 recognizes a no-entry road sign 92, the display control unit 12 displays the driving response method corresponding to the no-entry road sign 92 on the MID 60. For example, as Figure 4CAs shown, the display control unit 12 displays the message "Entry prohibited from here" in the text window 61 of the MID 60. The display control unit 12 can also display road sign icons and detailed descriptions on the central display 70.

[0061] Thus, in the vehicle display device 100, driving methods for easily misidentified road signs 90 to 92 can be displayed on the display screen 2. Therefore, it is possible to reduce the confusion or anxiety experienced by the driver when seeing easily misidentified road signs 90 to 92.

[0062] Next, the processing of the vehicle display device 100 according to this embodiment will be described with reference to the accompanying drawings. Figure 6 This is a flowchart illustrating an example of the display processing of traffic sign icons and driving response methods in a vehicle display device 100. The flowchart illustrates a series of processes that identify traffic signs and, based on the identification results, display appropriate information on the display 2.

[0063] like Figure 6 As shown, in S10, the ECU10 of the vehicle display device 100 identifies traffic signs in front of the vehicle by the traffic sign recognition unit 11. If no traffic sign is identified (S10: No), the ECU10 terminates the current processing. If a traffic sign is identified (S10: Yes), the ECU10 proceeds to S11.

[0064] In step S11, the ECU 10 displays traffic sign icons and detailed descriptions on the central display 70 via the display control unit 12. Specifically, the display control unit 12 retrieves detailed information associated with the traffic sign icon from the storage unit based on the recognition result of the traffic sign recognition unit 11 and displays it on the central display 70. The detailed information can be generated by a machine learning model based on the image of the traffic sign.

[0065] Next, in S12, ECU10 displays the driving response method on MID60 via display control unit 12. Specifically, display control unit 12 retrieves the driving response method corresponding to the traffic sign recognized by traffic sign recognition unit 11 from storage unit and displays it on MID60. The driving response method can be generated by a machine learning model based on the image of the traffic sign and the vehicle's driving environment (driving lane, vehicle speed, etc.). Then, ECU10 ends this processing.

[0066] Figure 7This is a flowchart illustrating an example of the display processing in the vehicle display device 100 when a traffic sign with applicable conditions is displayed. The flowchart shows a series of processes that identify a traffic sign with applicable conditions and display appropriate information on the display 2 based on the identification result. Traffic signs with applicable conditions refer to concepts including road signs with time conditions or road signs with vehicle type conditions, as described above. This is not limited to road signs but also includes road surface markings, etc.

[0067] like Figure 7 As shown, in S20, the ECU10 of the vehicle display device 100 identifies traffic signs with applicable conditions in front of the vehicle by the traffic sign recognition unit 11. If no traffic sign with applicable conditions is identified (S20: No), the ECU10 ends the current processing. If a traffic sign with applicable conditions is identified (S20: Yes), the ECU10 proceeds to S21.

[0068] In S21, ECU10 determines, via display control unit 12, whether the applicable conditions of the traffic sign with applicable conditions are met. Specifically, in the case of a traffic sign with time conditions, the current time is confirmed; in the case of a traffic sign with vehicle type conditions, the vehicle type is confirmed. If the applicable conditions are not met (S21: No), ECU10 ends the current processing. If the applicable conditions are met (S21: Yes), ECU10 proceeds to S22.

[0069] In step S22, the ECU10 displays the applicable traffic sign icon and its detailed description on the central display 70 via the display control unit 12. Specifically, the display control unit 12 retrieves the applicable traffic sign icon and the detailed information associated with that traffic sign icon from the storage unit and displays it on the central display 70. The detailed information can be generated by a machine learning model based on the image of the traffic sign.

[0070] Next, in S23, ECU10 displays the driving response method under sufficient conditions on MID60 via display control unit 12. Specifically, display control unit 12 retrieves the driving response method under sufficient conditions corresponding to the traffic sign with applicable conditions identified by traffic sign recognition unit 11 from storage unit and displays it on MID60. The driving response method can be generated by a machine learning model based on the image of the traffic sign and the vehicle's driving environment (driving lane, vehicle speed, etc.). Then, ECU10 ends this processing.

[0071] According to the vehicle display device 100 described above in this embodiment, the traffic sign icon corresponding to the traffic sign captured by the external camera 1 and the driving response method associated with the traffic sign are displayed on the display screen 2. Thus, the driver can easily grasp the traffic signs ahead of the vehicle and their corresponding driving response methods.

[0072] If the traffic sign identified by the vehicle's external camera 1 is a traffic sign with applicable conditions, the vehicle display device 100 determines whether the vehicle's condition meets those conditions. Then, the vehicle display device 100 can only display the traffic sign icon and driving response method corresponding to that traffic sign on the display 2 if the applicable conditions are determined to be met. This avoids displaying unnecessary information to the driver, reducing visual burden. Furthermore, since information is displayed only when the applicable conditions are met, the driver can take appropriate driving actions and comply with traffic regulations.

[0073] When the traffic sign identified by the vehicle's external camera 1 is a time-sensitive traffic sign, the vehicle display device 100 determines whether the current time meets the time condition. If the time condition is met, the vehicle display device 100 displays the traffic sign icon corresponding to the traffic sign and the driving response method when the time condition is sufficient on the display 2. Furthermore, if the time condition is not met, the vehicle display device 100 may not display the traffic sign icon or the driving response method when the time condition is insufficient on the display 2. Therefore, the driver can easily understand the meaning of traffic signs with time conditions and take appropriate driving actions. In particular, it is also easier to calmly and flexibly respond to traffic controls that vary depending on the time of day.

[0074] Furthermore, the vehicle display device 100 has a structure that includes a MID 60 and a central display 70 with a larger screen size than the MID 60 as a display 2. When a traffic sign icon is displayed on the MID 60 or the central display 70, a driving response method pre-associated with the traffic sign is displayed on the MID 60, and a detailed description of the traffic sign icon is displayed on the central display 70. With this structure, the driver can quickly grasp the traffic sign icon and the driving response method, and can confirm the detailed description. Moreover, by displaying the detailed description on the central display 70, compared to displaying it on the MID 60, the characters are prevented from becoming too small.

[0075] The embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments. The present invention can be implemented in various ways, including the above embodiments and various modifications and improvements made by those skilled in the art based on their knowledge.

[0076] For example, the first display is not limited to MID60, and the second display is not limited to the central display 70. The first or second display can be an additional display installed inside the vehicle interior, or it can be a head-up display. Display 2 can consist of one display or more than three displays. When display 2 consists of one display, traffic sign icons, driving instructions, and detailed explanations are displayed on the same display. In this case, the detailed explanations may not be displayed.

Claims

1. A display device for a vehicle that displays a traffic sign icon corresponding to a traffic sign in front of the vehicle as identified by an external camera on a display screen inside the vehicle, characterized in that, When the traffic sign icon is displayed on the display, the driving response method pre-associated with the traffic sign is also displayed on the display.

2. The vehicle display device according to claim 1, characterized in that, If the vehicle's external camera identifies a traffic sign with applicable conditions, it is determined whether the vehicle's condition meets the applicable conditions of the traffic sign. If the vehicle's condition is determined to meet the applicable conditions, the traffic sign icon corresponding to the traffic sign with applicable conditions and the driving response method pre-associated with the traffic sign will be displayed on the display. If the vehicle's condition is determined not to meet the applicable conditions, the traffic sign icon and the driving response method will not be displayed on the display.

3. The vehicle display device according to claim 1, characterized in that, If the vehicle's external camera identifies a traffic sign with time-related conditions, it determines whether the current time meets the time conditions of the traffic sign. If the current time condition is met, the traffic sign icon corresponding to the traffic sign and the driving response method when the time condition pre-established with the traffic sign icon is sufficient are displayed on the display. If the current time condition is not met, the traffic sign icon and the driving response method when the time condition pre-established with the traffic sign icon is insufficient are displayed on the display.

4. The vehicle display device according to claim 1, characterized in that, The display includes a first display and a second display with a larger screen size than the first display. When the traffic sign icon is displayed on the first display or the second display, a driving response method pre-associated with the traffic sign is displayed on the first display, and a detailed description of the traffic sign is displayed on the second display.

Images