Multimedia Display System

Abstract

To provide a multimedia display system which enables communication and smooth exchange between people who speak different languages. [Solution] A multimedia display system 3 includes a multimedia display device 1 and a cloud device, and the multimedia display device includes a transparent display. The transparent display includes a calculation module, an audio signal processing module, and a network module. The calculation module includes a voice recognition unit and a voice synthesis unit to convert a user's voice into a language command, and the audio signal processing module includes a speaker and a microphone. The voice recognition unit converts the voice into text, and the voice synthesis unit converts the text into voice. The calculation module transmits the language command to a machine translation module of the cloud device via the network module, and the machine translation module translates the language command into a target language and transmits it to the calculation module.

Description

[Technical field]

[0001] The present invention relates to a device and system, and more particularly to a double-sided transparent multimedia display device and multimedia display system. [Background technology]

[0002] In today's diversified society, effective means of communication are especially important for building relationships and mutual understanding. When people communicate, they rely on speech and visual cues (e.g. facial expressions and hand movements) to understand what the other person is trying to express.

[0003] However, when interacting with people who speak different languages, many people face communication difficulties and difficulties in understanding. Therefore, if there is a tool that can provide real-time translation, people of different languages ​​can easily communicate effectively and build relationships with each other. Summary of the Invention [Problem to be solved by the invention]

[0004] Solve communication problems and difficulties in understanding that arise when interacting with people who speak different languages. [Means for solving the problem]

[0005] The objective of the present invention is to provide a multimedia display device and a multimedia display system with a real-time translation function, which allows people of different languages ​​to communicate and exchange information smoothly.

[0006] The present invention provides a multimedia display device and a multimedia display system that have the function of real-time response and communication.

[0007] In order to achieve the above objectives, the multimedia display device of the present invention is applied in conjunction with a cloud device, and the multimedia display device includes a transparent display. The transparent display includes a calculation module, a voice signal processing module, and a network module. The calculation module is electrically connected to the voice signal processing module and the network module, respectively, and the calculation module includes a voice recognition unit and a voice synthesis unit, so as to allow a user to use voice and convert it into a language command; the voice signal processing module includes a speaker and a microphone. The voice recognition unit converts the voice into text, and the voice synthesis unit converts the text into voice. The calculation module sends the language command to a machine translation module of the cloud device through the network module, and the machine translation module translates the language command into a target language and sends it to the calculation module.

[0008] In order to achieve the above object, the multimedia display system of the present invention includes a transparent display and a cloud device. The transparent display includes a calculation module, an audio signal processing module, and a network module. The calculation module is electrically connected to the audio signal processing module and the network module, respectively. The calculation module includes a voice recognition unit and a voice synthesis unit, so that a user can use voice and convert it into a language command, and the audio signal processing module includes a speaker and a microphone. The voice recognition unit converts the voice into text, and the voice synthesis unit converts the text into voice. The cloud device and the transparent display are connected to communicate with each other, and the cloud device includes a data bank and a machine translation module using the data bank. The calculation module transmits the language command to the machine translation module of the cloud device via the network module, and the machine translation module translates the language command into a target language and transmits it to the calculation module.

[0009] In some embodiments, the computation module further comprises a natural language processing unit, which is used to understand the linguistic content.

[0010] In some embodiments, the cloud device further includes a natural language processing unit, which is used to understand the linguistic content.

[0011] In some embodiments, the transparent display captures the user's voice through a microphone.

[0012] In some embodiments, a speech synthesis unit converts the target language into a target speech, which is then presented via a speaker.

[0013] In some embodiments, a virtual person is displayed on the transparent display.

[0014] In some embodiments, the transparent display displays the target language.

[0015] In some embodiments, the virtual person is interconnected with electronic devices in an internal or external network.

[0016] In some embodiments, the language command is sent to a response module of the cloud device, and the response module generates a response language according to the language command and sends it to the calculation module. The voice synthesis unit converts the response language into a response voice, and the response voice is then presented through a speaker.

[0017] In some embodiments, the data bank includes a vocabulary bank, a grammar bank, a terminology bank, a domain knowledge bank, or a private cloud knowledge bank, or a combination thereof.

[0018] As described above, in the multimedia display device and multimedia display system of the present invention, the multimedia display device includes a transparent display, which includes a calculation module, a voice signal processing module and a network module. The calculation module includes a voice recognition unit and a voice synthesis unit, allowing the user to use voice and convert it into language commands, and the voice signal processing module includes a speaker and a microphone. The voice recognition unit converts voice into text, and the voice synthesis unit converts text into voice. The calculation module transmits the language commands to a machine translation module of a cloud device through a network module, and the machine translation module translates the language commands into a target language and transmits it to the calculation module. Thus, the multimedia display device and multimedia display system of the present invention have a real-time translation function, and can achieve the purpose of enabling people who use different languages ​​to communicate and smoothly communicate with each other. The multimedia display device and multimedia display system of the present invention also have a real-time reply and communication function. Effect of the Invention

[0019] The multimedia display device and multimedia display system provided by the present invention have the functions of real-time reply and communication, allowing people of different languages ​​to communicate and exchange smoothly. [Brief description of the drawings]

[0020] [Figure 1] 1 is a diagram showing a multimedia display system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a flow chart illustrating the functions of the multimedia display system according to one embodiment of the present invention; [Diagram 3] FIG. 2 is another diagram showing a multimedia display device according to an embodiment of the present invention; [Figure 4] FIG. 4 is a flow chart illustrating the function of the multimedia display system according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] The present invention will be described below with reference to the accompanying drawings, in which like reference numerals denote like elements having the same functions.

[0022] Please note first that the "sound" appearing below is voice, and the "language" is "characters", which can be characters that humans can recognize (human language), or characters that machines can recognize (machine language).

[0023] Fig. 1 is a diagram showing a multimedia display system according to an embodiment of the present invention, Fig. 2 is a functional flow chart of the multimedia display system according to an embodiment of the present invention, Fig. 3 is another diagram showing a multimedia display device according to an embodiment of the present invention.

[0024] First, please refer to Figure 1 and Figure 2. The multimedia display system 3 of the present invention includes a multimedia display device 1 and a cloud device 2 that is applied together with the multimedia display device 1.

[0025] The multimedia display device 1 includes a transparent display 11, which is a double-sided transparent display and has a light transmittance of 80 degrees or more. In one embodiment, the multimedia display device 1 is a portable display, a tabletop display or a stand display. In one embodiment, the transparent display 11 can be a liquid crystal display, a light emitting diode display, an organic light emitting diode display or a micro light emitting diode display, and is not limited in the present invention.

[0026] As shown in FIG. 2, the transparent display 11 includes a calculation module 111, a voice signal processing module 112, and a network module 113. The calculation module 111 is electrically connected to the voice signal processing module 112 and the network module 113, respectively. The calculation module 111 includes a voice recognition unit 1111, a natural language processing unit 1112, and a voice synthesis unit 1113, so as to allow a user to use voice (Voice or Speech) and convert it into language instructions. Thus, the natural language processing unit 1112 can be selected not to be included in the calculation module 111 (for example, when the cloud device 2 adopts the natural language processing unit 1112). In one embodiment, the voice recognition unit 1111 converts voice into text; in one embodiment, the voice synthesis unit 1113 converts text into voice; in one embodiment, the natural language processing unit 1112 is used to understand the content of language and understand the user's intention. In one embodiment, the voice recognition unit 1111, the natural language processing unit 1112 and the voice synthesis unit 1113 are realized by software, hardware or firmware. In addition, the voice signal processing module 112 includes a speaker 1121 and a microphone 1122, the speaker 1121 is for outputting voice, and the microphone 1122 is for capturing voice. Therefore, the transparent display 11 captures the user's speaking voice through the microphone 1122, and then outputs voice through the speaker 1121. In addition, the network module 113 includes a wired network module and / or a wireless network module. The wireless network module includes a WIFI module, a Bluetooth module, or a mobile network module such as 3G, 4G, 4GLTE, 5G or 6G, and the multimedia display device 1 (transparent display 11) connects to and communicates with an external device (e.g., a cloud device 2) through the network module 113.

[0027] The cloud device 2 is connected to and communicates with the multimedia display device 1 (transparent display 11). Here, the cloud device 2 has another network module 23 corresponding to the network module 113, and this network module 23 is connected to and communicates with the network module 113 of the transparent display 11. The cloud device 2 of this embodiment includes a data bank 21 and a machine translation module 22 using the data bank 21, and the calculation module 111 transmits a language command to the machine translation module 22 of the cloud device 2 via the network module 113 (and the network module 23), and the machine translation module 22 translates the language command into a target language and transmits it to the calculation module 111. As a result, the content of the words in the target language and the content of the words in the user's language command may have the same or similar meaning because they are different languages, and the accuracy of the answer is determined by the degree of machine learning, so it is a different category from the present invention.

[0028] In one embodiment, the data bank 21 includes a vocabulary bank, a grammar bank, a terminology bank, a domain knowledge bank, a private cloud knowledge bank, or a combination thereof. The machine translation module 22 can use the vocabulary bank, the grammar bank, the terminology bank, the domain knowledge bank, and / or the private cloud knowledge bank stored in the data bank 21, and translates the language command into the target language using big data and machine learning (ML), for example, a deep learning model, and the cloud device 2 returns the target language to the computation module 111 of the transparent display 11 via the network module 23 and the network module 113. The language command and the target language are both text files, but are different languages. For example, the machine translation module 22 can translate Japanese (language command) into other languages ​​(target languages), such as Chinese, English, Spanish, etc. Or, the machine translation module 22 can translate Chinese (language command) into other languages ​​(target languages), such as Russian, Japanese, English, etc. The present invention does not limit which language is translated into, but rather selects in advance which language is translated into in the user interface (UI) of the transparent display 11.

[0029] In one embodiment, after the target language translated by the machine translation module 22 is received by the calculation module 111, the speech synthesis unit 1113 can convert the target language into a target voice (sound), and the target voice is then presented via the speaker 1121 of the voice signal processing module 112. In one embodiment, after the target language translated by the machine translation module 22 is received by the calculation module 111, it is first decomposed into the target language by the natural language processing unit 1112, and the decomposed target language is converted into the target voice by the speech synthesis unit 1113, and the target voice is then presented via the speaker 1121 of the voice signal processing module 112.

[0030] In one embodiment, in addition to presenting the target voice through the speaker 1121, the multimedia display device 1 can also display a virtual person 114 (e.g., a virtual customer service staff) on the transparent display 11 as shown in FIG. 3, that is, the transparent display 11 can display the virtual person 114 and the target voice can be emitted through the virtual person 114. The transparent display 11 displays the virtual person 114 and emits the target voice, while also displaying the target language (characters). In one embodiment, the virtual person 114 can be customized according to the user, the user's voice, or requirements. In one embodiment, the transparent display is electrically connected to electronic devices of an internal network or an external network, or is interconnected with electronic devices of an internal network or an external network. In one embodiment, the virtual person is interconnected with electronic devices of an internal network or an external network.

[0031] The multimedia display device 1 of the present invention can be applied to, but is not limited to, a service counter or reception desk. When combined with the machine translation module 22 of the cloud device 2, it has a real-time translation function, so that people of different languages ​​can communicate with each other and communicate smoothly.

[0032] Thus, the multimedia display device 1 (transparent display 11) of this embodiment is exemplified as a relatively large-sized display, but is not limited thereto. In different embodiments, for example, when applied to the needs of personal use, a small-sized multimedia display device is designed to be easy to carry. For example, tour guides often explain the attractions of tourist spots to people who speak different languages, so the multimedia display device can be made into a relatively small-sized mobile device and carried by the tour guide. The tour guide can translate in real time through the portable multimedia display device, so that tour members will not miss the attractions of the tourist spots due to language problems.

[0033] FIG. 4 is a flow chart illustrating the function of the multimedia display according to another embodiment of the present invention.

[0034] Please refer to FIG. 4. The combination of elements and the relationship between each element of the multimedia display system 3a of this embodiment are almost the same as those of the multimedia display system 3 of the above embodiment. The difference is that in the multimedia display system 3a of this embodiment, the cloud device 2a includes an answer module 24, and the above language command is sent to the answer module 24, and the answer module 24 generates an answer language according to the language command and sends it to the calculation module 111. The voice synthesis unit 1113 of the calculation module 111 can convert the answer language into an answer voice, and the answer voice is presented to the user through the speaker 1121. In one embodiment, the answer module 24 generates an answer word corresponding to the language command according to the cloud information or other information. In one embodiment, the answer module 24 performs machine learning while using big data, for example, a deep learning model generates an answer language corresponding to the language command. The cloud device 2a further returns the answer language to the calculation module 111 of the transparent display 11 through the network module 23 and the network module 113, and the calculation module 111 presents the answer voice to the user through the speaker 1121. Thus, the semantic content of the answer language and the semantic content of the user's language command may adopt the same language that interacts with or is related to each other, and the semantic content of the answer language is to reply to the semantic content of the user's language command, and the accuracy of the reply is determined by the degree of machine learning, so it is a different category from the present invention. As described above, the answer voice is issued through the virtual person 114 of the multimedia display device 1, and at the same time, the characters of the answer language are also displayed, so that the user can hear and see the content of the reply. Thus, the answer voice of this embodiment and the voice used by the user are in the same language, for example, both are Japanese. For example, when a customer asks whether the person in charge is in the office, the answer module 24 can check the cloud schedule or attendance information of the person in charge, so that it can check whether the person in charge is in a meeting; or the answer module 24 can call the person in charge to check whether he or she is in the office, so that a corresponding answer voice can be generated to reply to the customer.

[0035] In one embodiment, the multimedia display device of the present invention can call someone (electronic device) (landline or mobile phone) through a voice call module (Callout). In one embodiment, the multimedia display device can connect to and communicate with an external network through a network module (which can be a wired or wireless network). In one embodiment, the multimedia display device can customize the image (virtual person) of a customer service staff member according to the needs of a client. In one embodiment, the multimedia display device triggers an application program pre-downloaded, for example, triggers a certain APP to perform a certain function. That is, the multimedia display device (transparent display) of the present invention is interconnected with a portable or non-portable electronic device of a specific person or a non-specific person on an internal network or an external network. In addition, the virtual person itself has a corresponding setting or image according to the identity of a specific person or a non-specific person, and is interconnected with an internal network or an external network and an electronic device related to the specific person or the non-specific person.

[0036] To sum up, in the multimedia display device and multimedia display system of the present invention, the multimedia display device includes a transparent display, which includes a calculation module, a voice signal processing module and a network module. The calculation module includes a voice recognition unit and a voice synthesis unit, allowing the user to use voice and convert it into language commands, and the voice signal processing module includes a speaker and a microphone. In addition, the voice recognition unit converts voice into text, and the voice synthesis unit converts text into voice. The calculation module transmits the language commands to a machine translation module of a cloud device through a network module, and the machine translation module translates the language commands into a target language and transmits it to the calculation module. In this way, the multimedia display device and multimedia display system of the present invention have a real-time translation function, and can achieve the purpose of enabling people who use different languages ​​to communicate and facilitate communication. In addition, the multimedia display device and multimedia display system of the present invention also have the function of real-time reply and communication.

[0037] The above are only some of the embodiments of the present invention, and are not intended to limit the present invention. Any modifications or changes made without departing from the spirit and scope of the present invention should fall within the scope of the claims of the present invention. [Industrial Applicability]

[0038] This invention has a real-time translation function, allowing people of different languages ​​to communicate and exchange information more smoothly. [Explanation of symbols]

[0039] 1. Multimedia display devices 11 Transparent Display 111 Calculation Module 112 Audio signal processing module 113 Network Module 114 Virtual Person 1111 Voice Recognition Unit 1112 Natural Language Processing Unit 1113 Speech synthesis unit 1121 Speaker 1122 Mike 2. 2a Cloud device 21 Data Bank 22 Machine Translation Module 23 Network Module 24 Answer Module 3. 3a Multimedia Display System

Claims

1. A multimedia display system including a transparent display and a cloud device, The transparent display includes a calculation module, a voice signal processing module, and a network module, the calculation module is electrically connected to the voice signal processing module and the network module respectively, the calculation module includes a voice recognition unit and a voice synthesis unit, so as to allow a user to use voice and convert it into a linguistic command, the voice signal processing module includes a speaker and a microphone, the voice recognition unit converts voice into characters, and the voice synthesis unit converts characters into voice; The cloud device and the transparent display are connected to each other to communicate with each other, and the cloud device includes a data bank and a machine translation module using the data bank; The multimedia display system, characterized in that the calculation module transmits language commands to the machine translation module of the cloud device via the network module, and the machine translation module translates the language commands into a target language and transmits the translated commands to the calculation module.

2. 2. The multimedia display system of claim 1, wherein the data bank comprises a vocabulary bank, a grammar bank, a terminology bank, a domain knowledge bank, a private cloud knowledge bank, or a combination thereof.

3. The multimedia display system of claim 1 , wherein the computing module further comprises a natural language processing unit, the natural language processing unit being used to understand linguistic content.

4. The multimedia display system of claim 1 , wherein the cloud device further includes a natural language processing unit, the natural language processing unit being used to understand language content.

5. The multimedia display system according to claim 1 , wherein the transparent display captures the user's voice through the microphone.

6. 2. The multimedia display system of claim 1, wherein said speech synthesis unit converts said target language into a target speech, and further wherein said target speech is presented via said speaker.

7. The multimedia display system of claim 6, wherein the transparent display displays a virtual person.

8. 7. The multimedia display system according to claim 5, wherein the target language is displayed on the transparent display.

9. The multimedia display system according to claim 8, wherein a virtual person is displayed on the transparent display, and the virtual person on the transparent display is interconnected with electronic devices on an internal network or an external network.

10. 2. The multimedia display system of claim 1, wherein the transparent display is interconnected with electronic devices in an internal or external network.

11. The multimedia display system of claim 1, characterized in that the cloud device includes a rotation module, the linguistic command is sent to the answer module, the answer module generates an answer language based on the linguistic command and sends it to the calculation module, the voice synthesis unit converts the answer language into an answer voice, and the answer voice is presented through the speaker.

Images