Smart Glasses

Abstract

We provide smart glasses that enable smooth two-way communication, such as conversation, between people who use different means of information transmission, such as language. [Solution] The smart glasses (1) include a lens section (2) that allows the wearer to see their forward field of view, an inner display section (38, 84) that displays information toward the wearer, and a transparent outer display section (40) that displays information toward the person the wearer is talking to on the lens section (2).

Description

Technical Field

[0001] The present invention relates to smart glasses.

Background Art

[0002] Conventionally, there has been known a glasses-type display device that translates the words of a speaker into a foreign language and displays them as text, in other words, smart glasses (see Patent Documents 1 to 4). Conventional smart glasses as disclosed in Patent Documents 1 and 2, for example, when people who speak different languages converse, the smart glasses recognize the words spoken by the interlocutor (such as English) and translate them into the words spoken by the wearer of the glasses (such as Japanese), and display the translated text (characters) on a display formed on the lens of the glasses, thereby conveying what the interlocutor wants to say to the wearer of the glasses.

[0003] Also, the smart glasses disclosed in Patent Documents 3 and 4 are provided with a projection-type display device that projects and displays an image to the wearer of the glasses instead of the display on the lens. This display device is provided on the rim that constitutes the frame of the smart glasses, and by directly projecting the translated text (characters) from the display device within the wearer's field of vision, what the interlocutor wants to say can be conveyed to the wearer of the glasses.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, conventional smart glasses do not allow the wearer's spoken language (such as Japanese) to be translated into the conversation partner's language (such as English). In other words, conventional smart glasses only have the function of displaying external information to the wearer through the glasses, and are only capable of one-way information transmission.

[0006] This invention has been made in view of these problems, and aims to provide smart glasses that enable smooth two-way communication, such as conversation, between people with different means of information transmission, such as language. [Means for solving the problem]

[0007] To achieve the above objective, the smart glasses of the present invention are smart glasses equipped with a lens portion that allows the wearer to view their forward field of vision, and include an inner display portion that displays information toward the wearer, and a transparent outer display portion in the lens portion that displays information toward the person the wearer is talking to. [Effects of the Invention]

[0008] The smart glasses of the present invention enable smooth two-way communication, such as conversation, between people who have different means of information transmission, such as language. [Brief explanation of the drawing]

[0009] [Figure 1] This is a perspective view of smart glasses according to the first embodiment. [Figure 2] This is a schematic diagram showing the lens portion located between the wearer and the person they are talking to, viewed from the side, in the first embodiment. [Figure 3] This is a hardware configuration diagram for smart glasses. [Figure 4] This is a software module configuration diagram for a translation program. [Figure 5] This is a network configuration diagram of the smart glasses and server involved in translation. [Figure 6]It is a front view of smart glasses showing the display mode of text in the outer display unit of the first embodiment. [Figure 7] It is a front view of smart glasses showing another display mode of text in the outer display unit of the first embodiment. [Figure 8] It is a perspective view of smart glasses according to the second embodiment. [Figure 9] It is a front view of smart glasses showing the display mode of text in the outer display unit of the second embodiment. [Figure 10] It is a perspective view of smart glasses according to the third embodiment. [Figure 11] It is a schematic view of the lens unit located between the wearer and the interlocutor in the third embodiment as seen from the side. [Figure 12] It is a front view of smart glasses showing the display mode of text in the outer display unit of the third embodiment. [Figure 13] It is a front view of smart glasses showing another display mode of text in the outer display unit of the third embodiment. [Figure 14] It is a perspective view of smart glasses according to the fourth embodiment. [Figure 15] It is a front view of smart glasses showing the display mode of text in the outer display unit of the fourth embodiment. [Figure 16] It is a side view when the wearer wears the smart glasses according to the fifth embodiment. [Figure 17] It is a partial perspective view of the smart glasses according to the modification as seen from the inside (wearer side). [Figure 18] It is a schematic view of the lens unit and the display located between the wearer and the interlocutor in the case of FIG. 17 as seen from the side.

Mode for Carrying Out the Invention

[0010] Hereinafter, the smart glasses 1 according to each embodiment of the present invention will be described with reference to the drawings. <First Embodiment> FIG. 1 shows a perspective view of the smart glasses 1 according to the first embodiment. The smart glasses 1 (hereinafter also simply referred to as glasses 1) include left and right lens parts 2 through which the wearer can view the front field of view, and a frame 4 that supports the lens parts 2. The frame 4 includes rims 6 that fix each lens part 2, a bridge 8 that connects the left and right lens parts 2, left and right temple pieces 10 that contact the upper ear parts of the wearer, left and right tip cells 12 formed at the tips of each temple piece 10, left and right armors 14 that connect the rims 6 and each temple piece 10, etc. Further, the glasses 1 also include left and right pads 16 that are nose pads, left and right hinges 18 that support the temple pieces 10 so as to be openable and closable with respect to the armors 14, etc.

[0011] In the case of this embodiment, a camera 20 is installed in the bridge 8, and forward microphones (second microphones) 22 and forward speakers (speakers) 24 are installed in the left and right armors 14, respectively. The camera 20 images the front of the wearer, the forward microphone 22 acquires the voice emitted by the interlocutor, and the forward speaker 24 transmits the voice converted by the voice conversion module 64 described later to the interlocutor. Also, in the left temple piece 10, a light source module 26 is built in in order from the hinge 18 side, an operation unit 28 is installed further below it, and a wearer speaker (voice transmission unit) 30 is installed further below that.

[0012] On the other hand, in the right temple piece 10, a light source module 26 (not shown) similar to the above is built in in order from the hinge 18 side (not shown), a wearer microphone (first microphone) 32 is installed further below it, and an operation unit 34 and a wearer speaker (voice transmission unit) 30 similar to the above are installed further below that. The wearer microphone 32 acquires the voice emitted by the wearer, and each wearer speaker 30 transmits the voice converted by the voice conversion module 64 described later to the wearer.

[0013] Figure 2 shows a schematic side view of the lens portion 2 located between the wearer and the person speaking to them in the first embodiment. The lens portion 2 is composed of a composite lens 42 having a transmissive inner display portion 38 that can display information only to the wearer and a transmissive outer display portion 40 that can display information only to the person speaking to them. In this embodiment, the composite lens 42 is formed in the left and right lens portions 2, which are separated by a bridge 8 that constitutes the frame 4 of the eyeglasses 1.

[0014] The inner display unit 38 is, for example, a light guide plate type transparent resin lens, and is installed on the wearer's eyepiece side (inside) in the composite lens 42. On the inner surface (wearer-side surface) 38a of the inner display unit 38, information such as text is displayed towards the wearer via the aforementioned light source module 26 in the direction indicated by the dashed line. However, when this information is displayed, the outer surface (interactive-side surface) 38b of the inner display unit 38 does not display the same information.

[0015] The outer display unit 40 is, for example, a single-sided emitting transparent OLED (organic EL display), and in the composite lens 42, its light-emitting surface is positioned facing the interlocutor (outside). On the outer surface (interlocutor-facing surface) 40a of the outer display unit 40, information such as text is displayed in the direction indicated by the dashed line toward the interlocutor. However, when this information is displayed, the inner surface (wearer-facing surface) 40b of the outer display unit 40 does not display the same information. Since the composite lens 42 as a whole is transparent, the wearer and the interlocutor can see each other from both sides through the lens unit 2, as indicated by the dashed line.

[0016] Figure 3 shows the hardware configuration diagram of the smart glasses 1. In this embodiment, the glasses 1 have a translation program installed as application software. This translation program executes the conversation translation mode, sign language translation mode, and image translation mode, which will be described later. In conversation translation mode, when two people who speak different languages ​​converse, the glasses 1 recognize the words spoken by the interlocutor (such as English), translate them into the words spoken by the wearer of the glasses 1 (such as Japanese), and display the translated text on the inner surface 38a of the inner display unit 38, thereby conveying what the interlocutor wants to say to the wearer of the glasses 1. In addition, this translation program also allows the glasses 1 to recognize the words spoken by the wearer of the glasses 1 (such as Japanese), translate them into the words spoken by the interlocutor (such as English), and display the translated text on the outer surface 40a of the outer display unit 40, thereby conveying what the wearer wants to say to the interlocutor.

[0017] More specifically, in addition to the configuration shown in Figure 1, the glasses 1 include a control unit 44, a memory unit 46, an inner display drive unit 36, an outer display drive unit 37, a wireless communication unit 48, a baseband processing unit 50, a first sound input / output unit 52, a second sound input / output unit 54, a power supply unit 56, and I / O ports 58, etc. The control unit 44 has an MPU (Micro Processing Unit) and executes a predetermined basic OS (Operating System) and middleware programs. By executing these programs, the control unit 44 controls each part of the glasses 1, builds a native platform environment and application execution environment in the software configuration, and performs various processing necessary for the glasses 1.

[0018] The memory unit 46 includes RAM (Random Access Memory) as volatile memory and internal memory storage, also known as ROM (Read Only Memory), as non-volatile memory. Dynamic RAM is used for the RAM in this memory unit 46, and flash memory is used for the ROM. The memory unit 46 stores in the ROM an operating system program for controlling the glasses 1, driver programs used for processing in each part of the glasses 1, and application programs related to various translations.

[0019] From this point forward, the ROM corresponds to a storage medium that stores the translation program. In particular, the storage unit 46 stores a basic OS as an operating system program, such as a general-purpose OS like Android® or a proprietary OS developed by a manufacturer. The storage unit 46 also stores various text data, audio data, image data, etc., and temporarily stores calculation result data generated in the process of calculations performed by predetermined processing carried out by the control unit 44 in RAM.

[0020] The inner display drive unit 36 ​​includes an optical component consisting of the inner display unit 38 described above (for example, a light guide plate type transparent resin lens) and a light source module 26 that emits and supplies image light from an RGB light source to its incident part, and further includes various components for image display such as a power supply circuit, a video signal processing circuit, and a drive circuit dedicated to the light source module. The outer display drive unit 37 includes the outer display unit 40 described above (for example, a display panel body made of a single-sided emitting transparent OLED) on its output side, and further includes various components for image display such as a dedicated power supply circuit, a video signal processing circuit, and a light emission drive circuit.

[0021] The wireless communication unit 48 functions as a wireless LAN communication means that communicates via a wireless LAN access point, and as a mobile communication means that communicates via a cellular communication base station, and is composed of, for example, a synthesizer, frequency converter, high-frequency amplifier, antenna, etc. The wireless communication unit 48 performs high-frequency signal processing for wireless communication with the access point using a predetermined wireless LAN communication method such as IEEE802.11. The wireless communication unit 48 also performs high-frequency signal processing for wireless communication with the base station using a predetermined cellular communication method such as 3G, LTE (Long Term Evolution), or 5G (5th generation mobile phone system). Furthermore, the wireless communication unit 48 performs high-frequency signal processing for wireless communication with an external communication terminal (such as a smartphone) that supports wireless communication compliant with the Bluetooth® standard, using a predetermined Bluetooth communication method.

[0022] The baseband processing unit 50 performs digital signal processing for voice and data communication with external communication terminals and various servers. The baseband processing unit 50 and the wireless communication unit 48 are connected via a D / A converter and an A / D converter. The first sound input / output unit 52 consists of the aforementioned wearer microphone 32, forward-facing speaker 24, signal processing circuit, etc. The first sound input / output unit 52 converts the analog audio signal input from the wearer microphone 32 into a digital signal using the signal processing circuit and outputs it to the control unit 44, baseband processing unit 50, etc. The first sound input / output unit 52 also converts the digital signals input from the control unit 44, baseband processing unit 50, etc. into analog signals using the signal processing circuit and outputs them to the forward-facing speaker 24. The forward-facing speaker 24 outputs audio based on this analog signal input toward the person speaking.

[0023] The second sound input / output unit 54 consists of the aforementioned forward-facing microphone 22, individual wearer speakers 30, signal processing circuits, etc. The second sound input / output unit 54 converts the analog audio signal input from the forward-facing microphone 22 into a digital signal using the signal processing circuit and outputs it to the control unit 44, baseband processing unit 50, etc. The second sound input / output unit 54 also converts the digital signals input from the control unit 44, baseband processing unit 50, etc. into analog signals using the signal processing circuit and outputs them to each wearer speaker 30. Each wearer speaker 30 outputs audio based on this analog signal input to the wearer.

[0024] The control units 28 and 34 consist of various operation buttons provided on the glasses 1, as well as a power switch for turning the power on and off. These control units 28 and 34 are used by the wearer to turn the power of the glasses 1 on and off, to select menus and modes related to the operation of applications, to switch the display image on the display screen, to change settings necessary for the operation of various software, and to input text information.

[0025] The camera 20 consists of a lens, an imaging device, etc., and is used to photograph people performing sign language actions, landscapes, etc. The imaging device uses a CCD (Charge Coupled Device) image sensor or a CMOS image sensor. The power supply unit 56 includes a rechargeable battery, a power supply circuit that converts the battery output into power of a predetermined voltage and supplies it to each part of the glasses 1, and a charging circuit for charging the battery.

[0026] The I / O port 58 is primarily for connecting cables equipped with connectors such as USB (Universal Serial Bus) Type-C (trademark) or Lightning (trademark) standards. The control unit 44 can send and receive data with external devices via the cable connected to this I / O port 58 using the said connector. In addition, the battery of the glasses 1 can be charged by supplying power from an external source via the cable connected to this I / O port 58. In this case, the power supplied from an external source via the cable is sent to the charging circuit of the power supply unit 56. The charging circuit uses that power to charge the battery.

[0027] Figure 4 shows a diagram of the software module configuration of the translation program. The translation program of this embodiment consists of a speech recognition module (first speech recognition unit, second speech recognition unit) 60, a translation module (first translation unit, second translation unit, third translation unit) 62, a speech conversion module (first speech conversion unit, second speech conversion unit, third speech conversion unit) 64, and an image recognition module (first image recognition unit, second image recognition unit) 66, etc. By appropriately using these modules, the glasses 1 can execute each of the modes described below, such as the conversation translation mode, sign language translation mode, and image translation mode.

[0028] <Conversation Translation Mode> The speech recognition module 60 is a software module that recognizes the words spoken by the wearer from the digital signal (voice data) of the wearer's voice, which is input from the wearer microphone 32 and converted by the first sound input / output unit 52, and outputs the recognition result as text data. In addition, the speech recognition module 60 recognizes the words spoken by the interlocutor from the digital signal of the interlocutor's voice, which is input from the front microphone 22 and converted by the second sound input / output unit 54, and outputs the recognition result as text data.

[0029] The translation module 62 is a software module that translates the text data of the speech recognition result output from the speech recognition module 60 into another language and outputs the translated text data as the translation result. Specifically, the translation module 62 translates the text converted by the speech recognition module 60 from the speech acquired by the wearer microphone 32 into a language that the conversationalist can understand, and also translates the text converted by the speech recognition module 60 from the speech acquired by the front microphone 22 into a language that the wearer can understand.

[0030] The voice conversion module 64 is a software module that converts the translated text data output from the translation module 62 into speech and outputs the resulting speech data (digital signal). Specifically, the voice conversion module 64 converts the text based on the wearer's utterance translated by the translation module 62 into speech and transmits this speech to the person speaking by sending it toward the forward-facing speaker 24. The voice conversion module 64 also converts the text based on the person speaking translated by the translation module 62 into speech and transmits this speech to each wearer speaker 30.

[0031] The text based on the speaker's utterance translated by the translation module 62 can be transmitted to the wearer by displaying it on the inner surface 38a of the inner display unit 38 in addition to the audio output to each wearer speaker 30, or it can be transmitted to the wearer by displaying it only on the inner surface 38a of the inner display unit 38 without any audio output from each wearer speaker 30. Similarly, the text based on the speaker's utterance translated by the translation module 62 can be transmitted to the wearer by displaying it on the outer surface 40a of the outer display unit 40 in addition to the audio output from the front speaker 24, or it can be transmitted to the wearer by displaying it only on the outer surface 40a of the outer display unit 40 without any audio output from the front speaker 24.

[0032] <Sign language translation mode> The image recognition module 66 is a software module that recognizes the sign language movements of an interlocutor from images (including videos) acquired by the camera 20 and converts the content of the conversation indicated by those sign language movements into text data. The text converted by the image recognition module 66 may be displayed on the inner surface 38a of the inner display unit 38, or it may be converted into speech by the speech conversion module 64 and output as sound from each wearer's speaker 30. Alternatively, both text display and speech output may be performed.

[0033] <Image Translation Mode> The image recognition module 66 can also recognize and extract text from images (including videos) acquired by the camera 20. Specifically, the image recognition module 66 extracts text from images such as explanatory texts about works of art exhibited in museums and art galleries, or menus in restaurants, captured by the camera 20. This text is then translated into a language understandable to the wearer by the translation module 62. The text translated by the translation module 62 may be displayed on the inner surface 38a of the inner display unit 38, or it may be converted into speech by the speech conversion module 64 and output from each wearer's speaker 30. Alternatively, both text display and speech output may be performed.

[0034] Figure 5 shows a network configuration diagram of the smart glasses 1 and server involved in translation. The processing for each mode described above is preferably performed by the control unit 44 executing the translation program installed in the glasses 1, so that the processing is completed within the glasses 1 itself. However, considering the processing load of the glasses 1 and the power consumption associated with the processing, the processing for each mode may be performed by the glasses 1 directly using a cloud service 70 that provides such processing (speech recognition, translation, speech conversion, image recognition, etc.) as a service connected to the Internet 68 (the embodiment (a) shown in Figure 5), or by the glasses 1 requesting the processing from a terminal 72 that realizes external computing such as a smartphone and having the terminal 72 perform the processing (the embodiment (b) shown in Figure 5).

[0035] The processing requested by terminal 72 may be performed by local processing on terminal 72, or terminal 72 may perform it using the cloud service 70 (as shown in the embodiment (c) of Figure 5). Specifically, for example, it is possible to use one of the speech conversion or recognition functions provided as a cloud service by Microsoft, Google, Amazon, etc., using their TTS (Text To Speech) or STT (Speech To Text) APIs (Application Programming Interface), or to use language translation functions provided by Microsoft, Google, Amazon, DeepL, etc. When glasses 1 directly uses the cloud service 70, processing is performed via cellular wireless communication such as 4G LTE via base station 74 (Figure 5(a)). When processing is requested from terminal 72, terminal 72 and glasses 1 are connected by a wired connection using USB, etc. (solid line in Figure 5(b)) or by a wireless connection using Bluetooth, Wi-Fi, etc. (dashed line in Figure 5(b)).

[0036] Figure 6 is a front view of the eyeglasses 1 showing a text display mode on the outer display section 40 of the first embodiment, and Figure 7 is a front view of the eyeglasses 1 showing another text display mode on the outer display section 40 of the first embodiment. In the first embodiment, the composite lens 42 is formed on the left and right lens sections 2 separated by a bridge 8 that constitutes the frame of the eyeglasses 1. Therefore, as shown in Figure 6, a series of texts 76 may be displayed only on the outer surface 40a of the outer display section 40 of one lens section 2, or a series of texts 76 may be displayed on the outer surface 40a of the outer display section 40 of one lens section 2, and the next series of texts 76 may be displayed on the outer surface 40a of the outer display section 40 of the other lens section 2, and so on, in sequence. Alternatively, as shown in Figure 7, a series of texts 76 may be displayed continuously across the outer surface 40a of the outer display section 40 of both lens sections 2.

[0037] As described above, the smart glasses 1 of this embodiment are composed of a composite lens 42 which has a transparent inner display unit 38 that can display information only to the wearer and a transparent outer display unit 40 that can display information only to the person it is talking to. This makes it possible to smoothly conduct two-way communication, such as conversation, between people who have different means of information transmission, such as language.

[0038] Specifically, the glasses 1 are equipped with a wearer microphone 32 that acquires the wearer's voice. In conversation translation mode, the voice recognition module 60 recognizes the voice acquired by the wearer microphone 32 and converts it into text. The translation module 62 then translates the text converted by the voice recognition module 60 into a language that the conversationalist can understand, and displays the translated text on the external display unit 40. Alternatively, along with this display, the glasses 1 can also convert the text translated by the translation module 62 into speech using the speech conversion module 64, and transmit the converted speech to the conversationalist through the forward-facing speaker 24. This allows the wearer's message to be smoothly conveyed to the conversationalist even when speaking different languages.

[0039] Furthermore, the glasses 1 are equipped with a forward-facing microphone 22 that acquires the voice of the person speaking. The voice recognition module 60 recognizes the voice acquired by the forward-facing microphone 22 and converts it into text. The translation module 62 translates the text converted by the voice recognition module 60 into a language that the wearer can understand, and displays the translated text on the inner display unit 38. Alternatively, along with this display, the glasses 1 can also convert the text translated by the translation module 62 into speech using the speech conversion module 64, and transmit the converted speech to the wearer through the wearer speaker 30. This allows for smooth communication of what the person speaking is trying to say to the wearer, even between people who speak different languages.

[0040] Furthermore, the glasses 1 are equipped with a camera 20 that captures images in front of the wearer. In sign language translation mode, the image recognition module 66 recognizes the sign language movements of the person speaking from the image acquired by the camera 20, converts the content of the conversation indicated by those sign language movements into text, and displays the text converted by the image recognition module 66 on the inner display unit 38. Alternatively, along with this display, the glasses 1 can also convert the text converted by the image recognition module 66 into speech using the speech conversion module 64, and transmit the speech converted by the speech conversion module 64 to the wearer through the wearer speaker 30. This allows for smooth communication of what the person speaking wants to say to the wearer, even if the wearer does not understand sign language, when communicating with someone who needs to communicate in sign language.

[0041] In addition, in image translation mode, the image recognition module 66 recognizes and extracts text from images acquired by the camera 20, the translation module 62 translates the text extracted by the image recognition module 66 into a language that the wearer can understand, and the translated text is displayed on the inner display unit 38. Alternatively, along with the display, the text translated by the translation module 62 can be converted into speech by the speech conversion module 64, and the speech can be transmitted to the wearer through the wearer speaker 30. This allows the wearer to smoothly obtain the information they need in areas with different languages.

[0042] <Second Embodiment> Figure 8 shows a perspective view of the smart glasses 1 according to the second embodiment, and Figure 9 is a front view of the smart glasses 1 showing the text display mode in the outer display section 40 of the second embodiment. In the following descriptions of each embodiment, configurations that differ from the first embodiment will be mainly described, and configurations similar to the first embodiment may be omitted from illustration or description. The composite lens 42 of this embodiment is formed in a left and right integrated lens section 2 fixed to the rim 6 that constitutes the frame 4 of the glasses 1. In this case, a wide area for displaying text 76 can be secured on the outer surface 40a of the outer display section 40 of the seamless lens section 2. As a result, as shown in Figure 9, a series of texts 76 can be displayed simultaneously in multiple rows on the outer surface 40a. Therefore, two-way communication between the wearer and the person they are talking to, and the wearer's acquisition of information can be made even smoother.

[0043] <Third Embodiment> Figure 10 shows a perspective view of the smart glasses 1 according to the third embodiment, and Figure 11 shows a schematic side view of the lens section 2 located between the wearer and the person speaking to the wearer in the third embodiment. The composite lens 42 of this embodiment, as in the first embodiment, consists of left and right lens sections 2 separated by a bridge 8 that constitutes the frame 4 of the glasses 1. The outer display sections 40 of each of the left and right lens sections 2 in this embodiment are connected to the rims 6 that constitute the frame 4 of the glasses 1 via hinges 78. Each outer display section 40 is flipped up in the direction of the arrows shown in Figure 11 via the hinges 78 relative to the inner display section 38, and displays to the person speaking to the wearer. By using such flip-up glasses 1, the text display to the person speaking to the wearer can be emphasized.

[0044] Figure 12 shows a front view of smart glasses 1 showing a text display mode on the outer display unit 40 of the third embodiment, and Figure 13 shows a front view of smart glasses showing another text display mode on the outer display unit 40 of the third embodiment. In this embodiment, as shown in Figure 12, a series of texts 76 may be displayed only on the outer surface 40a of the outer display unit 40 when one lens unit 2 is flipped up, or a series of texts 76 may be displayed on the outer surface 40a of the outer display unit 40 when one lens unit 2 is flipped up, and the next series of texts 76 may be displayed on the outer surface 40a of the outer display unit 40 when the other lens unit 2 is flipped up, and so on, in sequence.

[0045] Furthermore, as shown in Figure 13, it is also possible to continuously display a series of texts 76 across the outer surface 40a of the outer display section 40 when both lens sections 2 are flipped up. Therefore, by using flip-up type glasses 1, two-way communication between the wearer and the person they are talking to, and the wearer's acquisition of information, can be made even smoother. Note that the outer display section 40 of the lens section 2 in the third embodiment may be detachable from the glasses 1, and in such a configuration, the weight of the glasses 1 can be reduced compared to when they are worn by removing the outer display section 40 when it is not used for conversation.

[0046] <Fourth Embodiment> Figure 14 shows a perspective view of the smart glasses 1 according to the fourth embodiment, and Figure 15 shows a front view of the smart glasses 1 showing the text display mode in the outer display unit 40 of the fourth embodiment. The composite lens 42 of this embodiment, as in the second embodiment, constitutes a left and right integrated lens unit 2 fixed to the rim 6 that constitutes the frame 4 of the glasses 1. Also, the outer display unit 40 of the lens unit 2 of this embodiment is connected to the rim 6 that constitutes the frame 4 of the glasses 1 via a hinge 78, as in the third embodiment.

[0047] In this embodiment, a wide area for displaying text 76 can be secured on the outer surface 40a of the seamless outer display section 40 of the lens section 2, and as shown in Figure 15, a series of texts 76 can be displayed simultaneously in multiple rows on the outer surface 40a. Furthermore, by using flip-up type glasses 1, the text display can be emphasized for the person speaking. Therefore, two-way communication between the wearer and the person speaking, and the wearer's acquisition of information can be made even smoother. Note that the outer display section 40 of the lens section 2 in the fourth embodiment may be detachable from the glasses 1, and in such a configuration, the weight of the glasses 1 can be reduced compared to when it is worn by removing the outer display section 40 when it is not used for conversation.

[0048] <Fifth Embodiment> Figure 16 shows a side view of the smart glasses 1 according to the fifth embodiment when worn by a wearer. The glasses 1 comprises the aforementioned pads 16 and end cells 12, and the aforementioned power supply unit 56 is built into at least one of the end cells 12. This balances the weight of the power supply unit 56 with the weight of the composite lens 42 side of the glasses 1, using the wearer's ear 80 as a fulcrum, thereby reducing the load of the glasses 1 on each pad 16. Consequently, the wearer's comfort of the glasses 1 can be improved. Furthermore, by adjusting the above balance, the battery capacity of the power supply unit 56 can be increased, enabling longer operation of the glasses 1.

[0049] Furthermore, as shown in Figure 16, the aforementioned operating parts 28 and 34 for the wearer to operate may be placed on at least one of the upper and lower parts of the power supply unit 56. By adjusting the balance, the wobbling of the glasses 1 when the operating parts 28 and 34 are pressed can be suppressed, thereby improving the operability of the glasses 1. In addition, a bone conduction unit (sound transmission unit) 82 may be built into at least one of each end cell 12 instead of the aforementioned wearer speaker 30. This allows sound to be transmitted to the wearer by transmitting vibrations to the bone near the wearer's ear 80. Therefore, even in environments surrounded by noise, the sound transmitted from the person speaking can be reliably transmitted to the wearer.

[0050] This concludes the description of embodiments of the present invention. However, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the invention. For example, the translation program described above may be installed on the glasses 1 itself, or on a terminal 72 such as a smartphone. Needless to say, this terminal 72 may be any other general computer such as a tablet, notebook, or desktop (tower) PC.

[0051] Furthermore, the camera 20, front-facing microphone 22, front-facing speaker 24, individual wearer speakers 30, individual wearer microphones 32, and individual control units 28 and 34 in the glasses 1 are not limited to the aforementioned installation positions, and various installation configurations are conceivable.

[0052] In addition to the configuration of the embodiments described above, this specification also includes a configuration in which the outer display unit 40 of the lens unit 2 is separated from the main body of the smart glasses 1 and made into a separate unit. In this configuration, the outer display unit 40 and the inner display unit 38 are separated and made into separate units. Therefore, by attaching the outer display unit 40 to the smart glasses 1 so as to cover or place over the front surface of the smart glasses 1 (i.e., the outer surface 38b of the inner display unit 38), it becomes possible to provide the smart glasses 1 with the same information display function for the listener as the smart glasses with the configuration described above.

[0053] In this configuration, when a separate external display unit 40 is attached to the front of the smart glasses 1, an electrical connection is established between them by connecting the connectors / sockets provided at predetermined locations on both units. Therefore, it is preferable to supply power and video signals from the smart glasses 1 unit to the external display unit 40 via this connection.

[0054] Furthermore, as a technology for displaying information to the wearer of the smart glasses 1, technologies that project images directly onto the wearer's eyes, such as the projection technology disclosed in Patent Documents 3 and 4, and other retinal projection technologies, can be applied to the smart glasses 1 of each embodiment described above. In particular, when using the projection technology disclosed in Patent Documents 3 and 4, as shown in Figure 17, the smart glasses 1 are equipped with a projection-type display unit (internal display unit) 84 that projects and displays images (including text).

[0055] The display unit 84 is located on the upper part of the rim 6 that constitutes the frame 4 of the smart glasses 1, and has an opening 84a that projects image light directly toward the wearer, displaying information toward the wearer through the opening 84a. In this case, as shown in Figure 18, the lens unit 2 consists only of the outer display unit 40 and is not a composite lens 42. Therefore, in this embodiment, the lens unit 2 can have a simple configuration. When this technology is applied to the smart glasses 1, as in the embodiments described above, two-way communication such as conversation can be smoothly performed between people with different means of information transmission, such as language.

[0056] Furthermore, the present invention can be applied to information display technologies, including the various video projection and display methods described above, as well as technologies comprising video projectors and combiners, for displaying information to both the wearer of the smart glasses 1 and the person interacting with them. Such information display technologies are used in HUDs (Head-Up Displays) mounted in the driver's seat of automobiles and the like, and such technologies can be particularly applied to the outer display unit 40 of the smart glasses 1. [Explanation of symbols]

[0057] 1. Smart Glasses 2 Lens section 4 frames 6 rims 8 Bridge 10 Temples 12 First Cell 16 pads 20 cameras 22. Front-facing microphone (second microphone) 24. Forward-facing speaker (speaker) 28 Control section 30. Wearer's speaker (voice transmission unit) 32. Wearer's microphone (1st microphone) 34 Control section 38 Inner display 40 Outer display 42 Compound lenses 56 Power supply section 60. Speech recognition module (first speech recognition unit, second speech recognition unit) 62 Translation Modules (Translation Section 1, Translation Section 2, Translation Section 3) 66 Image recognition module (first layer recognition unit, second image recognition unit) 76 Text 78 Hinge 80 The wearer's ear 82 Bone conduction section (sound transmission section) 84 Display (inner display)

Claims

1. Smart glasses equipped with a lens portion that allows the wearer to see their forward field of vision, An internal display unit that displays information to the wearer, The lens portion includes a transmissive external display unit that displays information toward the person the wearer is interacting with. Smart glasses equipped with these features.

2. The smart glasses according to claim 1, wherein the inner display unit is provided on the upper part of the rim constituting the frame of the smart glasses, has an opening that projects image light directly toward the wearer, and displays information toward the wearer from the opening.

3. The smart glasses according to claim 1, characterized in that the lens portion is composed of a composite lens comprising a transparent inner display portion that displays information only toward the wearer and a transparent outer display portion that displays information only toward the person being spoken to.

4. The smart glasses according to claim 3, wherein the composite lenses are formed on the left and right lens portions separated by a bridge that constitutes the frame of the smart glasses.

5. The smart glasses according to claim 3, wherein the composite lens is formed on the left and right integrated lens portion fixed to the rim constituting the frame of the smart glasses.

6. The external display unit is connected to the rim that constitutes the frame of the smart glasses via a hinge. The smart glasses according to claim 3, wherein the outer display unit is flipped up relative to the inner display unit via the hinge and displays information toward the person speaking.

7. A first microphone for acquiring the voice emitted by the wearer, A first speech recognition unit that recognizes the voice acquired by the first microphone and converts it into text, A first translation unit that translates the text converted by the first speech recognition unit into a language that the interlocutor can understand. Equipped with, The smart glasses according to claim 2 or 3, wherein the text translated by the first translation unit is displayed via the external display unit.

8. A first microphone for acquiring the voice emitted by the wearer, A first speech recognition unit that recognizes the voice acquired by the first microphone and converts it into text, A first translation unit that translates the text converted by the first speech recognition unit into a language that the interlocutor can understand, A first speech conversion unit that converts the text translated by the first translation unit into speech, A speaker that transmits the voice converted by the first voice conversion unit to the person speaking. Smart glasses according to claim 2 or 3, comprising:

9. A second microphone for acquiring the voice spoken by the aforementioned interlocutor, A second speech recognition unit that recognizes the voice acquired by the second microphone and converts it into text, A second translation unit translates the text converted by the second speech recognition unit into a language that the wearer can understand. Equipped with, The smart glasses according to claim 7 or 8, wherein the text translated by the second translation unit is displayed via the internal display unit.

10. A second microphone for acquiring the voice spoken by the aforementioned interlocutor, A second speech recognition unit that recognizes the voice acquired by the second microphone and converts it into text, A second translation unit translates the text converted by the second speech recognition unit into a language that the wearer can understand, A second speech conversion unit converts the text translated by the second translation unit into speech, A voice transmission unit that transmits the voice converted by the second voice conversion unit to the wearer. The smart glasses according to claim 7 or 8, comprising:

11. A camera that captures images in front of the wearer, A first image recognition unit recognizes the sign language movements of the interlocutor from the image acquired by the camera and converts the content of the conversation indicated by the sign language movements into text. Equipped with, The smart glasses according to claim 7 or 8, wherein the text converted by the first image recognition unit is displayed via the inner display unit.

12. A camera that captures images in front of the wearer, A first image recognition unit recognizes the sign language movements of the interlocutor from the image acquired by the camera and converts the content of the conversation indicated by the sign language movements into text. A third speech conversion unit converts the text converted by the first image recognition unit into speech, A voice transmission unit that transmits the voice converted by the third voice conversion unit to the wearer. The smart glasses according to claim 7 or 8, comprising:

13. The aforementioned smart glasses have a nose pad, and The tip of each of the pair of temples of the aforementioned smart glasses is formed at the end of each temple Equipped with, The smart glasses according to claim 2 or 3, wherein at least one of the aforementioned front cells has a built-in power supply unit for supplying power to the smart glasses.

14. The smart glasses according to claim 13, wherein an operating unit for the wearer is disposed on at least one of the upper and lower parts of the tip cell in which the power supply unit is built.

Images