Vehicle translation information projection system and translation information projection method for the system

Abstract

A vehicle translation information projection system mounted on a vehicle, comprising a language recognition unit that recognizes a language displayed on a traffic sign object within an image captured by a front camera of the vehicle, a translation processing unit that translates language content different from the user language on the traffic sign object into user language content when the language displayed on the traffic sign object is different from a preset user language, and a projection display unit that projects information onto the front glass of the vehicle, wherein the projection display unit projects user language content onto the front glass so that it is positioned above or near the traffic sign object when viewed from the driver of the vehicle.

Description

Technology Field

[0001] The present invention relates to a vehicle translation information projection system and a method for projecting translation information of a vehicle translation information projection system. Background Technology

[0002] Conventionally, Japanese Patent Publication No. 2013-068620 is known as a technical document regarding a vehicle translation information projection system. This publication discloses a system that translates a language included in an acquired image of a sign, road sign, or other object recognized by a vehicle driver into a second language and displays it on a display unit. Prior art literature

[0003] Japanese Patent Publication No. 2013-068620 The problem to be solved

[0004] In conventional systems, the translation result of a sign being viewed by the driver is displayed on a display device such as a center display. In such conventional systems, the driver needs to shift their gaze to the center display to check the translation result of the sign being viewed, which may cause the driver's attention to be distracted. means of solving the problem

[0005] One aspect of the present invention is a vehicle translation information projection system mounted on a vehicle. The vehicle translation information projection system comprises a language recognition unit that recognizes a language displayed on a traffic sign object within an image captured by a front camera of the vehicle, a translation processing unit that translates language content different from the user language on the traffic sign object into user language content when the language displayed on the traffic sign object differs from a preset user language, and a projection display unit that projects information onto the front glass of the vehicle. The projection display unit projects user language content onto the front glass so that it is positioned above or near the traffic sign object when viewed from the driver of the vehicle.

[0006] According to a vehicle translation information projection system according to one aspect of the present invention, when the language displayed on a traffic sign object is different from the user language, translated user language content is projected onto the front glass above or near the traffic sign object when viewed by the driver. By doing so, the driver can understand the content of the traffic sign object displayed in a different language without significantly diverting their gaze from the front of the vehicle.

[0007] In a vehicle translation information projection system according to one embodiment of the present invention, the traffic sign object may include at least one of a road sign, a road construction sign, a traffic guide board, and a road surface marking.

[0008] In a vehicle translation information projection system according to one embodiment of the present invention, the projection display unit may project user language content onto the front glass by overlapping the language portion so as to mask the language portion different from the user language in the traffic sign object when viewed from the driver.

[0009] According to this vehicle translation information projection system, the projection display projects user language content onto the front glass by overlapping it with a language portion that is different from the user's set language, so that the projection display masks the language portion that is different from the user's set language. As a result, the driver can see only the translated information without the original language being mixed in. Consequently, confusion of information is prevented, making it possible for the driver to quickly and clearly understand the necessary information.

[0010] In a vehicle translation information projection system according to one embodiment of the present invention, a voice recognition unit is further provided to recognize a voice spoken to the driver from outside the vehicle while the vehicle is stopped, and the projection display unit may project the voice recognized by the voice recognition unit as text in the user language onto the front glass.

[0011] According to this vehicle translation information projection system, a voice recognition unit recognizes voice spoken to the driver from outside the vehicle while the vehicle is stopped, and projects it onto the front glass as text in the user's language. As a result, the driver can quickly understand the content even if spoken to in a different language, and can communicate smoothly.

[0012] Another aspect of the present invention is a method for projecting translation information of a vehicle translation information projection system mounted on a vehicle. The method for projecting translation information of a vehicle translation information projection system recognizes a language displayed on a traffic sign object within an image captured by a front camera of a vehicle, and if the language displayed on the traffic sign object is different from a preset user language, translates the language content different from the user language on the traffic sign object into user language content, and projects the user language content onto the front glass of the vehicle so that it is positioned above or near the traffic sign object when viewed from the driver of the vehicle.

[0013] According to a translation information projection method of a vehicle translation information projection system according to another aspect of the present invention, when the language displayed on a traffic sign object is different from the user language, translated user language content is projected onto the front glass above or near the traffic sign object when viewed by the driver. By doing so, the driver can understand the content of the traffic sign object displayed in a different language without significantly diverting their gaze from the front of the vehicle. Effects of the invention

[0014] According to each embodiment of the present invention, a driver can understand the content of a traffic sign object displayed in another language in the user's language without significantly diverting their gaze from the front of the vehicle. Brief explanation of the drawing

[0015] FIG. 1 is a block diagram showing a vehicle translation information projection system according to one embodiment. Figure 2 is a diagram showing an example of a translation display on a traffic sign when the user language is Japanese. Figure 3 is a diagram showing an example of a translation display on a traffic sign when the user language is English. Figure 4 is a drawing showing an example of masking a translation display on a traffic sign. Figure 5 is a diagram illustrating an example of a voice translation display at a toll booth. Figure 6 is a flowchart illustrating an example of translation information projection processing of a vehicle translation information projection system. Figure 7 is a flowchart illustrating an example of voice translation information projection processing of a vehicle translation information projection system. Specific details for implementing the invention

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0017] The vehicle translation information projection system (100) illustrated in FIG. 1 is a system that is mounted on a vehicle and supports the driver in understanding the meaning of traffic sign objects displayed in a language other than the user language. The user language is a language that the driver has pre-set for the system. The user language is set to a language that the driver can understand, such as the driver's native language, for example.

[0018] The traffic sign object is an object captured by the front camera (1) of the vehicle and includes at least one of a road sign, a road construction sign, a traffic guide sign, a road surface marking, etc. These traffic sign objects provide traffic information to the driver and are preferably understood by the driver beyond the language barrier.

[0019] Road signs are signs installed to provide drivers with information regarding road traffic regulations, instructions, warnings, and guidance. Specific examples include "Maximum Speed" signs indicating speed limits, "Stop" signs indicating a temporary stop, "No Entry" signs indicating a ban on entry, "No Traffic" signs indicating a ban on vehicle traffic, and "Right Turn Allowed" or "Left Turn Allowed" signs indicating whether right or left turns are permitted. These signs are distinguished by their shapes, colors, and symbols, and are designed to be understood by drivers at a glance.

[0020] Road construction signs are installed to inform drivers of temporary changes in traffic conditions, such as road construction or maintenance work. Specific examples include "Under Construction" signs indicating ongoing construction, "One-Way Traffic" signs indicating alternating traffic, "Lane Reduction" signs indicating a reduction in lanes, "Detour" signs guiding drivers to alternative routes, and "Slow Down" signs urging drivers to reduce their speed. These signs enhance visibility by using vivid colors or flashing lights to draw attention.

[0021] Traffic information boards are electronic display boards installed on highways or major roads that provide real-time traffic information to drivers. Specific information displayed includes "Traffic Congestion 5km Ahead," indicating congestion; "Accident Occurred," informing of accidents; "Dense Fog Warning" or "Ice Warning," conveying weather conditions; and "No Traffic" or "Lane Restriction" as road regulation information. Traffic information boards sometimes convey information intuitively using symbols or pictograms in addition to text.

[0022] Road markings are markings drawn directly on the pavement of a road to provide drivers with information such as the direction of travel and traffic regulations. Specific examples include "arrow markings" indicating the direction of travel, "stop lines" indicating stopping locations, "crosswalks" indicating pedestrian crossings, "lane dividers" or "no lane change lines" indicating lane boundaries, and "speed markings" indicating speed restrictions. Additionally, there are markings such as "Bus Only" or "Bicycle Only" indicating lanes where only specific vehicles are permitted to travel. Reflective materials or fluorescent paints are often used for road markings to enhance visibility at night or in inclement weather.

[0023] <Configuration of the Vehicle Translation Information Projection System>

[0024] The vehicle translation information projection system (100) is equipped with a translation ECU (Electronic Control Unit) (10). The translation ECU (10) is an electronic control unit having a CPU (Central Processing Unit) and a memory unit. The memory unit is composed of, for example, ROM (Read Only Memory), RAM (Random Access Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), etc. In the translation ECU (10), various functions are realized by, for example, executing a program stored in the memory unit with the CPU. The translation ECU (10) may be composed of multiple electronic units.

[0025] As illustrated in FIG. 1, the translation ECU (10) is connected to a front camera (1), a microphone (2), a driver monitor camera (3), a vehicle speed sensor (4), a HUD (5), and a speaker (6).

[0026] The front camera (1) is an imaging device that captures the front of the vehicle. The front camera (1) is provided, for example, behind the front glass of the vehicle. The front camera (1) transmits the captured image regarding the situation in front of the vehicle to the translation ECU (10).

[0027] The microphone (2) is a voice input device provided to collect voice from outside the vehicle. The microphone (2) is positioned, for example, outside the body of the vehicle, such as near the door mirror on the driver's side or on the door panel, and is configured to accurately receive voice spoken from outside the vehicle while it is stopped. The microphone (2) may also be provided inside the vehicle cabin.

[0028] The microphone (2) is equipped with a noise canceling function to reduce the influence of ambient noise or wind noise. In addition, it has waterproof and dustproof performance and is designed to withstand the external environment of the vehicle. As a result, it is possible to stably acquire voice even under various weather conditions.

[0029] The driver monitor camera (3) is a camera for capturing images of the driver. The driver monitor camera (3) may be a digital camera having an imaging element such as a CCD [Charge Coupled Device] or a CIS [CMOS Image Sensor]. The driver monitor camera (3) is, for example, provided on the cover of the steering column in front of the driver and captures the head portion including the driver's face at a predetermined frame rate.

[0030] The placement location of the driver monitor camera (3) is not limited to the cover of the steering column, but can be placed in a position where the driver's face or eyes can be photographed from the front, such as near the rearview mirror or on the top of the dashboard inside the vehicle. The driver monitor camera (3) transmits the captured image of the driver to the translation ECU (10). In addition, the driver monitor camera (3) has the function of detecting the driver's gaze, facial expression, and head movements. Accordingly, the projection display unit (14) described later can adjust the display position or angle of the projected user language content in real time according to the driver's gaze position or posture.

[0031] The vehicle speed sensor (4) is a sensor for detecting the driving speed of a vehicle. The vehicle speed sensor (4) is composed of a wheel speed sensor or a speed detection device built into the transmission, and transmits vehicle speed information to the translation ECU (10).

[0032] The vehicle speed information obtained from the vehicle speed sensor (4) is utilized by the voice recognition unit (12) or the projection display unit (14) described later. For example, when it is detected that the vehicle is stopped, the voice recognition unit (12) receives voice input from the outside. Meanwhile, when the vehicle is in motion, the projection display unit (14) may optimize the timing or position of the display content according to the vehicle speed and control it so that the driver can properly check the information.

[0033] A HUD (5) (Head Up Display) is a display device that projects information onto the front glass of a vehicle and allows the driver to obtain necessary information without moving their gaze significantly. The HUD (5) consists of a projection unit built into the dashboard and a reflective surface provided on the front glass.

[0034] The HUD (5) is equipped with, for example, a high-brightness projection system and can display a clear image regardless of day or night or weather conditions. It also has a function to adjust the projection position and focal length, so that information is displayed in a position that is easy to see according to the driver's seat position and viewing angle. In addition, it performs distortion correction and color correction of the displayed content and provides accurate information.

[0035] The speaker (6) is an audio device for outputting voice guidance or warning sounds. The speaker (6) is placed, for example, on the dashboard or door panel inside the vehicle, and a high-quality sound system is employed to provide appropriate voice to the driver.

[0036] The speaker (6) may be capable of playing voice output in the user language from the translation processing unit (13). The speaker (6) may be capable of providing voice guidance for alarms or navigation information from the system.

[0037] Next, the functional configuration of the translation ECU (10) will be described. As illustrated in FIG. 1, the translation ECU (10) has a language recognition unit (11), a speech recognition unit (12), a translation processing unit (13), and a projection display unit (14).

[0038] The language recognition unit (11) has the function of recognizing language displayed on traffic sign objects within an image acquired from the front camera (1). This language includes user language and languages ​​other than user language. Specifically, high-precision character recognition is achieved by combining image analysis technology and optical character recognition (OCR) technology. First, as an image preprocessing step, noise removal, contrast correction, and brightness adjustment are performed to make the image suitable for character information extraction. Next, character edges or line segments are detected by a feature extraction algorithm, and character regions are specified.

[0039] After the character area is specified, each character is converted into digital text data by an OCR engine. At this time, to support multiple languages ​​or fonts, a comparison is performed with a large-scale language database. A machine learning model is utilized for the comparison, and it may be designed to adaptively maintain recognition accuracy even when new languages ​​or fonts are added. The language recognition unit (11) is not limited to a machine learning model and may recognize language on a traffic sign object using a well-known image processing method. The language recognition unit (11) determines whether the traffic sign object contains a language other than the user's language. Additionally, the function of the language recognition unit (11) may be executed on an external server.

[0040] FIG. 2 is a diagram showing an example of a translation display of a traffic sign when the user language is Japanese. FIG. 2 shows a first traffic sign SB1 and a second traffic sign SB2 in front of a vehicle. The first traffic sign SB1 and the second traffic sign SB2 are traffic signs (traffic sign objects) installed on the road. FIG. 2 also shows the front glass W of a vehicle, the first user language content display TR1, and the second user language content display TR2, which will be used in the following description.

[0041] The language recognition unit (11) recognizes the language of the first traffic sign SB1 and the language of the second traffic sign SB2 from the captured image shown in FIG. 2. The language recognition unit (11) recognizes language content such as “NORTH,” “San Francisco,” and “EXIT” written on the first traffic sign SB1. Likewise, the language recognition unit (11) recognizes language content such as “San Jose,” “SF Intl Airport,” and “EXIT” written on the second traffic sign SB2. These languages ​​are English and are different from Japanese, which is set as the user language. Additionally, the language recognition unit (11) may recognize language content in combination with numerical values ​​such as “101” or symbols such as arrows.

[0042] The voice recognition unit (12) has the function of interpreting a voice signal acquired through the microphone (2) while the vehicle is stopped and converting the content into text data. Whether the vehicle is stopped is determined based on the vehicle speed information from the vehicle speed sensor (4). It is not limited to cases where the vehicle is completely stopped, and it may be determined that the vehicle is stopped when the vehicle speed is below a certain speed. It may be possible to recognize voice guidance from a police officer or traffic guide outside the vehicle while the vehicle is moving slowly.

[0043] The voice recognition unit (12) interprets the voice signal using an acoustic model and a speech language model. First, the acquired analog voice signal is converted into a digital signal, and frequency analysis is performed to extract voice features. These features are compared with the acoustic model, and recognition is performed at the level of phonemes or syllables.

[0044] Next, based on the speech language model, recognized phonemes or syllables are combined to form words or sentences. The speech recognition unit (12) is equipped with signal processing techniques, such as background noise removal or echo cancellation, and is designed to maintain high recognition accuracy even in a noisy environment around the vehicle.

[0045] The translation processing unit (13) has the function of translating text data obtained from the language recognition unit (11) and the voice recognition unit (12) into the user language. The translation processing unit (13) translates language content other than the user language of the traffic sign object recognized by the language recognition unit (11) into user language content. Language content is character information displayed on the traffic sign object. Language content includes, for example, text of a road sign, guidance sentence of a road construction sign, display content of a traffic sign, character information of a road surface marking, etc. User language content is language content translated into the user language.

[0046] The translation processing unit (13) is equipped with a machine translation model using, for example, a neural network, and realizes natural translation that takes into account context and expression. Specifically, it uses a deep learning model having an encoder-decoder structure to convert input text into an internal expression, and then generates translation text for the user language.

[0047] The translation processing unit (13) may refer to a dictionary database or a parallel corpus for the translation of technical terms or proper nouns. Additionally, it may be in a form that continuously acquires the latest translation data set from an external network and performs updates. Furthermore, the function of the translation processing unit (13) may be in a form that is executed on an external server.

[0048] The translation processing unit (13) translates the language content of the first traffic sign SB1 and the language content of the second traffic sign SB2, which are recognized by the language recognition unit (11) in FIG. 2, for example. The translation processing unit (13) translates the language content (English) such as “NORTH”, “San Francisco”, “EXIT” in the first traffic sign SB1, for example, into user language content (Japanese) such as “North”, “San Francisco”, “Exit”.

[0049] The projection display unit (14) has the function of transmitting user language content obtained from the translation processing unit (13) to the HUD (5) and projecting information onto the front glass W. The projection display unit (14) is responsible for controlling the display position or display format of the translation information so that the driver can see the necessary information without moving their gaze too much.

[0050] The projection display unit (14) projects user language content so that it overlaps on or near the traffic sign object when viewed from the driver, based on the position of the driver's face or eyes captured by the driver monitor camera (3) and the position of the location of the traffic sign object other than the user language recognized by the language recognition unit (11).

[0051] The projection display unit (14) adjusts the projection position of user language content based on the time change in the position of the traffic sign object viewed by the driver, in that the positional relationship between the vehicle and the traffic sign object while driving changes over time. The projection display unit (14) predicts the position of the traffic sign object after a predetermined time based on the vehicle speed information of the vehicle speed sensor (4) and adjusts the projection position of the user language content.

[0052] The projection display unit (14) may be in a form that predicts the position of a traffic sign object after a predetermined time based on the steering angle detected by the steering sensor or the yaw rate detected by the yaw rate sensor. The projection display unit (14) may also be in a form that re-detects the position of the driver's face or eyes captured by the driver monitor camera (3) at regular intervals or whenever the driver's seat's front-to-back position is adjusted or the seat belt is attached or detached, and reflects this in the adjustment of the projection position of the user language content.

[0053] Specifically, the projection display unit (14) interprets the location information of the traffic sign object obtained from the front camera (1) in real time and controls the projection position so that the user language content overlaps on or near the traffic sign object. "On the traffic sign object" means a position where at least a part of the user language content overlaps with the traffic sign object. "Near the traffic sign object" means a position where the user language content enters the driver's field of vision when viewing the traffic sign object. The user language content can be placed at a proximity where the relationship between the traffic sign object and the user language content can be understood. By doing so, the driver can intuitively and quickly recognize the translated information without shifting their gaze.

[0054] The projection display unit (14) projects a first user language content display TR1 directly below the first traffic sign SB1, which is a traffic sign object, as shown in FIG. 2, as an example. The first user language content display TR1 includes content such as "North," "San Francisco," and "Exit" translated by the translation processing unit (13). Likewise, the projection display unit (14) projects a second user language content display TR2 directly below the second traffic sign SB2. The second user language content display TR2 includes content such as "South," "San Jose SF International Airport," and "Exit" translated by the translation processing unit (13).

[0055] FIG. 3 is a diagram showing an example of a translation display on a traffic sign when the user language is English. FIG. 3 shows a third traffic sign SB3 with Japanese text and a third user language content display TR3 corresponding to the third traffic sign SB3. In the situation illustrated in FIG. 3, the language recognition unit (11) recognizes the language of the third traffic sign SB3 from the captured image illustrated in FIG. 3. The language recognition unit (11) recognizes language content such as "Nagoya," "Matsumoto," and "Shinshu Shinmachi" written on the third traffic sign SB3. The translation processing unit (13) translates the language content recognized by the language recognition unit (11) into user language content (English) such as "Nagoya," "Matsumoto," and "Shinshushi Nachi." The projection display unit (14) projects user language content (English), such as “Nagoya”, “Matsumoto”, and “Shinshushi nmachi” translated by the translation processing unit (13), onto the front glass W as a third user language content display TR3.

[0056] The display mode of user language content by the projection display unit (14) is not limited to the modes shown in FIG. 2 and FIG. 3. The projection display unit (14) may project user language content onto the front glass W by overlapping the language portion so as to mask the language portion different from the user language in the traffic display object when viewed from the driver.

[0057] Specifically, the projection display unit (14) adjusts the color tone or contrast of the user language content being displayed and effectively masks the original language portion. For example, by placing a background with adjusted opacity on the back of the user language content, the original language portion is covered and hidden. As a result, the driver can obtain only the necessary information without being distracted by unnecessary information. In addition, masking and the display of the user language content may be performed as independent processes.

[0058] Here, FIG. 4 is a drawing showing an example of masking a translation display of a traffic sign. In FIG. 4, a first user language content display TR1 is projected onto the front glass W to cover and hide the language portion of a first traffic sign SB1 when viewed from the driver. Likewise, a second user language content display TR2 is projected onto the front glass W to cover and hide the language portion of a second traffic sign SB2 when viewed from the driver.

[0059] The projection display unit (14) may have a function to adjust the position, size, and brightness of the projected user language content in real time by taking into account information such as the movement of the vehicle, the driver's gaze position, and the seat position. To realize this function, the projection display unit (14) acquires position information of the language portion of the traffic sign object provided by the language recognition unit (11). Based on this information, it controls the projection of the user language content to a position that overlaps with the original language portion. Similarly, the projection display unit (14) may acquire size information of the language portion of the traffic sign object from the language recognition unit (11) and project the user language content at the same size.

[0060] Additionally, when a voice of a language different from the user language is detected by the voice recognition unit (12), the translation ECU (10) translates the voice into user language content by the translation processing unit (13). The projection display unit (14) projects user language content corresponding to a voice spoken from outside onto, for example, the driver's front position on the front glass W or the edge of the driver's seat side of the front glass W.

[0061] FIG. 5 is a diagram illustrating an example of a voice translation display at a toll booth. FIG. 5 shows a vehicle M, the driver D of vehicle M, a toll booth TG, and a toll booth staff TC of the toll booth TG. In FIG. 5, vehicle M is assumed to be stopped. In the situation illustrated in FIG. 5, the voice recognition unit (12) recognizes a call voice for the driver D of vehicle M from the toll booth staff TC of the toll booth TG through the microphone (2). The voice recognition unit (12) converts the content of the voice from the toll booth staff TC into text data. At this timing, the voice recognition unit (12) detects whether the voice is in a language other than the user language.

[0062] The translation processing unit (13) translates the voice of the toll booth staff TC into user language content (e.g., Japanese) when the voice recognition unit (12) detects a voice of a language different from the user language. The projection display unit (14) projects the user language content translated by the translation processing unit (13) onto the front glass W in front of the driver.

[0063] Translation Information Projection Method of Vehicle Translation Information Projection System

[0064] Next, a method for projecting translation information of a vehicle translation information projection system (100) according to the present embodiment will be explained with reference to the drawings. FIG. 6 is a flowchart showing an example of the translation information projection processing of a vehicle translation information projection system (100).

[0065] As illustrated in FIG. 6, the translation ECU (10) of the vehicle translation information projection system (100) detects the language of a traffic sign object by the language recognition unit (11) in step S10. If no language is detected (step S10: No), the translation ECU (10) terminates the current processing. If a language is detected (step S10: Yes), the translation ECU (10) proceeds to step S11.

[0066] In step S11, the translation ECU (10) determines whether the language of the traffic sign object includes a language different from the user language by means of the language recognition unit (11). If a language different from the user language is included (Step S11: Yes), the translation ECU (10) proceeds to step S12. If a language different from the user language is not included (Step S11: No), the translation ECU (10) terminates the current processing.

[0067] In step S12, the translation ECU (10) translates the language content of the traffic sign object into user language content by the translation processing unit (13). Then, in step S13, the translation ECU (10) projects the user language content onto the front glass W of the vehicle by the projection display unit (14). The projection display unit (14) projects the user language content so that it is positioned above or near the traffic sign object when viewed from the driver by controlling the HUD (5). Then, the translation ECU (10) finishes the current processing.

[0068] FIG. 7 is a flowchart illustrating an example of voice translation information projection processing of a vehicle translation information projection system. As shown in FIG. 7, in step S20, the translation ECU (10) determines whether the vehicle is stopped. The translation ECU (10) makes a determination based on the vehicle speed information from the vehicle speed sensor (4). If the vehicle is not stopped (step S20: No), the translation ECU (10) terminates the current processing. If the vehicle is stopped (step S20: Yes), the translation ECU (10) proceeds to step S21.

[0069] In step S21, the translation ECU (10) determines whether a voice of a language different from the user language is detected by the voice recognition unit (12) (microphone). The voice recognition unit (12) acquires a voice through the microphone (2) by speaking to the driver from outside, and determines whether the acquired voice is a language different from the user language. If a voice of a different language is detected (step S21: yes), the translation ECU (10) proceeds to step S22. If a voice of a different language is not detected (step S21: no), the translation ECU (10) terminates the current processing.

[0070] In step S22, the translation ECU (10) translates voice data by the translation processing unit (13). The translation processing unit (13) translates voice in a language different from the user language into user language content as text data. The translation processing unit (13) may also be in the form of translating into voice in the user language. After that, the translation ECU (10) proceeds to step S23.

[0071] In step S23, the translation ECU (10) projects user language content onto the front glass W by means of a projection display unit (14). The projection display unit (14) projects user language content corresponding to a voice speaking from outside onto, for example, the driver's front position on the front glass W or the edge of the front glass W on the driver's side. By doing so, even when a voice is spoken from outside the vehicle while the vehicle is stopped, the driver can understand the content.

[0072] According to the vehicle translation information projection system (100) according to the embodiment described above, when the language displayed on the traffic sign object is different from the user language, the translated user language content is projected onto the front glass W above or near the traffic sign object when viewed by the driver, thereby enabling the driver to understand the content of the traffic sign object displayed in a different language without turning their gaze far away from the front of the vehicle.

[0073] In addition, according to the vehicle translation information projection system, the projection display unit (14) projects user language content onto the front glass W by overlapping it with the language portion so that the user's set language portion is masked, thereby allowing the driver to see only the translated information without the original language being mixed in. This prevents confusion of information and enables the driver to quickly and clearly understand the necessary information. In particular, it reduces visual strain and allows the driver to maintain attention while driving.

[0074] In addition, according to the vehicle translation information projection system (100), voice spoken to the driver from outside the vehicle while the vehicle is stopped is recognized by the voice recognition unit (12) and projected onto the front glass W as text in the user language, so that the driver can quickly understand the content even if the driver speaks to him in another language, and can perform smooth communication.

[0075] As described above, according to the present invention, a vehicle translation information projection system can be provided that enables a driver to reliably recognize traffic sign objects or voice information while stopped in the user's language.

[0076] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. The present invention may be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art, including the embodiments described above.

[0077] The vehicle translation information projection system (100) does not necessarily need to respond to the translation of voice spoken to the driver from outside the vehicle. The translation ECU (10) does not need to have a voice recognition unit (12). In this case, a microphone (2) is also unnecessary.

[0078] The vehicle translation information projection system (100) may be capable of corresponding to a traffic sign object that displays a symbol or pictogram. The vehicle translation information projection system (100) may project user language content onto the front glass that displays the meaning of the traffic sign object as text in the user language, based on image data of the traffic sign object containing the symbol or pictogram and the symbol or pictogram of the traffic sign object recognized from the image captured by the front camera (1). The traffic sign object containing the symbol or pictogram may include at least one of the following: a "No Pedestrian Crossing" sign, a "No Entry" sign combining a vehicle silhouette and a red diagonal line, a "One-Way" sign using an arrow, a "Construction" sign depicting a construction worker silhouette, a "One-Way Alternating Traffic" sign combining a vehicle silhouette and an arrow, a "No Entry" sign depicting a barricade silhouette, a "Bicycle Lane" road surface marking depicting a bicycle silhouette, and a "Bus Lane" road surface marking drawing a bus silhouette. Explanation of the symbols

[0079] 1: Front camera 2: Microphone 3: Driver monitor camera 4: Vehicle speed sensor 5: HUD 6: Speaker 10: Translation ECU 11: Language recognition unit 12: Voice recognition unit 13: Translation Processing Unit 14: Projection display 100: Vehicle Translation Information Projection System

Claims

Claim 1 A vehicle translation information projection system that provides translation information to a driver of a vehicle, comprising: a language recognition unit that recognizes a language displayed on a traffic sign object within an image captured by a front camera of the vehicle; a translation processing unit that translates content in a language different from the user language on the traffic sign object into user language content when the language displayed on the traffic sign object is different from a preset user language; and a projection display unit that projects information onto the front glass of the vehicle, wherein the projection display unit projects the user language content onto the front glass so that it is positioned above the traffic sign object or near the traffic sign object when viewed from the driver of the vehicle. Claim 2 A vehicle translation information projection system according to claim 1, wherein the traffic sign object includes at least one of a road sign, a road construction sign, a traffic guide board, and a road surface marking. Claim 3 A vehicle translation information projection system according to claim 1, wherein the projection display unit projects the user language content onto the front glass by overlapping the language portion to mask the language portion different from the user language in the traffic display object when viewed by the driver. Claim 4 A vehicle translation information projection system according to claim 1, further comprising a voice recognition unit that recognizes a voice spoken toward the driver from outside the vehicle while the vehicle is stopped, and a projection display unit that projects the voice recognized by the voice recognition unit as text in the user language onto the front glass. Claim 5 A method for projecting translation information of a vehicle translation information projection system mounted on a vehicle, wherein the method recognizes a language displayed on a traffic sign object within an image captured by a front camera of the vehicle, and when the language displayed on the traffic sign object is different from a preset user language, translates the language content different from the user language above the traffic sign object into user language content, and projects the user language content onto the front glass of the vehicle so as to be located above or near the traffic sign object when viewed by the driver of the vehicle.

Images