Communication system, information terminal device, server, translation method, and translation program

Abstract

To provide a communication system, an information terminal device, a server, a translation method, and a translation program that enable people who speak different languages to communicate naturally.SOLUTION: When a short-range communication unit 130 of a first information terminal device 100 becomes able to communicate with a second information terminal device 200 via short-range communication, the short-range communication unit transmits a first language which is a language type spoken by an operator of the first information terminal device and is pre-stored in the first information terminal device 100, and a short-range communication unit 230 of the second information terminal device 200 receives the first language transmitted from the first information terminal device 100 and stores the received first language in a memory unit 220.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a communication system, an information terminal device, a server, a translation method, and a translation program. [Background technology]

[0002] In recent years, domestic inbound demand has expanded, with people from various countries visiting Japan for tourism. Many translation devices and translation systems have been provided to enable smooth communication with such visitors. One of these systems is a two-way speech translation system that executes a process of synthesizing speech translated from a first language into a second language, and a process of synthesizing speech translated from the second language into the first language (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2023-022150 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 requires travelers to first operate a two-way speech translation system and speak in their own language, which means that it is difficult for Japanese speakers to naturally initiate a conversation with travelers.

[0005] The present invention has been made in consideration of the above, and has an object to provide a communication system, an information terminal device, a server, a translation method, and a translation program that enable people who speak different languages ​​to communicate naturally. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, when a first information terminal device becomes able to communicate with a second information terminal device via short-range communication, it transmits a first language, which is a language type spoken by an operator of the first information terminal device and is pre-stored in the first information terminal device, and the second information terminal device receives the first language transmitted from the first information terminal device and stores the received one language in a memory unit. Effect of the Invention

[0007] According to the present invention configured as described above, it is possible to provide an effect that people who speak different languages ​​can communicate naturally. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of a communication system 10 according to a first embodiment. [Diagram 2] 13 is a flowchart showing a translation process executed by information terminal device 100 and information terminal device 200. [Diagram 3] FIG. 2 is an explanatory diagram showing an example of a flow of a translation process in the first embodiment. [Figure 4] 11 is a flowchart showing a translation process procedure executed by the information terminal device 100 and the information terminal device 200 in low-power wireless communication or the like. [Diagram 5] FIG. 11 is a block diagram showing a configuration of a communication system 20 according to a second embodiment. [Figure 6] 13 is a flowchart showing a translation process procedure executed by the beacon device 300, the information terminal device 400, the server 500, and the information terminal device 600. [Figure 7] FIG. 11 is an explanatory diagram showing an example of a flow of a translation process according to the second embodiment. [Figure 8] 13 is a flowchart showing a translation process procedure executed by the beacon device 300, the information terminal device 400, the server 500, and the information terminal device 600. [Figure 9] FIG. 11 is an explanatory diagram illustrating an example of a flow of a translation process. [Figure 10]FIG. 11 is a block diagram showing a configuration of a communication system 30 according to a third embodiment. [Figure 11] 13 is a flowchart showing a translation process executed by an information terminal device 700, a server 800, and an information terminal device 900. [Figure 12] FIG. 13 is an explanatory diagram showing an example of a flow of a translation process according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Below, examples of the communication system, information terminal device, server, translation method, and translation program according to the present application will be described with reference to the accompanying drawings. The following explanations are examples of the embodiments of the present application, and the communication system, information terminal device, server, translation method, and translation program according to the present application are not limited to these examples.

[0010] <Example 1> 1 is a block diagram showing a configuration of a communication system 10 according to Example 1. The communication system 10 includes an information terminal device 100 and an information terminal device 200.

[0011] The information terminal device 100 is a computer operated by a foreign language speaker (hereinafter also referred to as the first speaker) such as a foreign tourist, and may be a wearable terminal, smartphone, tablet terminal, personal computer (hereinafter also referred to as a PC), etc. The information terminal device 200 is a computer operated by a Japanese speaker (hereinafter also referred to as the second speaker) who converses with the first speaker, and may be a wearable terminal, smartphone, tablet terminal, PC, etc., like the information terminal device 100. As an example, the wearable terminal is a vest worn by the first speaker or second speaker, and a display unit is placed in a position that is easily visible to the conversation partner, such as the chest part of the vest.

[0012] 1, information terminal device 100 includes a control unit 110, a storage unit 120, a short-range communication unit 130, an audio input / output unit 140, an operation unit 150, and a display unit 160. Control unit 110 reads out programs and the like stored in storage unit 120 and executes the read out programs to control each unit of information terminal device 100 and transmit data to each unit.

[0013] The storage unit 120 stores a first language, which is a language type spoken by a first speaker, and a second language, which is a language type spoken by a second speaker who is a conversation partner. The short-distance communication unit 130 receives the second language transmitted from the information terminal device 200, and stores it in the storage unit 120. The storage unit 120 stores operation programs for controlling each unit, application programs, various control information, intermediate files, and the like.

[0014] The short-range communication unit 130 transmits a first language to the information terminal device 200 and receives a second language from the information terminal device 200. The short-range communication may be infrared communication (IrDA: Infrared Data Association), BLE (Bluetooth Low Energy) communication, low-power wireless communication (half-duplex communication method in the same frequency band), or communication based on WPAN (Wireless Personal Area Network) technology such as Z-Wave (registered trademark) or ZigBee (registered trademark).

[0015] The voice input / output unit 140 receives input of voice uttered by a first speaker obtained from a microphone or the like. The voice input / output unit 140 outputs voice from a speaker, earphones, headphones, or the like. The operation unit 150 receives operations on the information terminal device 100. The display unit 160 is a display or the like that displays characters, images, and the like. The operation unit 150 and the display unit 160 may be a touch panel or the like configured by superimposing a liquid crystal display and a touch sensor.

[0016] The control unit 110 will be described. The control unit 110 includes a character conversion unit 111 and a translation unit 112. The character conversion unit 111 converts the voice of a first speaker received from the voice input / output unit 140 into characters in a first language. The translation unit 112 translates the characters in the first language converted by the character conversion unit 111 into characters in a second language.

[0017] Next, the configuration of the information terminal device 200 operated by the second speaker will be described. The information terminal device 200 includes a control unit 210, a storage unit 220, a short-distance communication unit 230, an audio input / output unit 240, an operation unit 250, and a display unit 260. The information terminal device 200 is almost identical to the information terminal device 100, and has almost the same functions. Therefore, the description of each unit of the information terminal device 100 will be referred to, and the following will describe the parts that differ from the information terminal device 100.

[0018] The storage unit 220 stores the first language transmitted from the information terminal device 100, and stores the second language which is the language type spoken by the second speaker. The short-distance communication unit 230 receives the first language transmitted from the information terminal device 100, and stores it in the storage unit 220. The voice input / output unit 240 accepts input of the voice uttered by the second speaker acquired from a microphone or the like.

[0019] The character conversion unit 211 converts the voice of the second speaker received from the voice input / output unit 240 into characters in the second language. The translation unit 212 translates the characters in the second language converted by the character conversion unit 211 into characters in the first language.

[0020] The translation process executed in the communication system 10 configured as described above will be described. Fig. 2 is a flowchart showing the procedure of the translation process executed by the information terminal device 100 and the information terminal device 200. In this flowchart, processing in infrared communication, BLE communication, etc. will be described.

[0021] As advance preparations for information terminal device 100 and information terminal device 200, operation unit 150 of information terminal device 100 accepts input of a first language (step S201) and stores the accepted first language in memory unit 120. Operation unit 250 of information terminal device 200 accepts input of a second language (step S202) and stores the accepted second language in memory unit 220.

[0022] When the information terminal device 100 and the information terminal device 200 enter each other's communication range for short-distance communication, the short-distance communication unit 130 and the short-distance communication unit 230 execute a connection process (step S203). The short-distance communication unit 130 transmits the first language (step S204), and the short-distance communication unit 230 receives the first language and stores the received first language in the storage unit 220 (step S205). FIG. 3 is an explanatory diagram showing an example of the flow of the translation process in the first embodiment. For example, as shown in (1) and (2) of FIG. 3, the information terminal device 100 transmits "English" as the first language, and stores "English" as the first language in the storage unit 220 of the information terminal device 200.

[0023] Returning to Fig. 2, short-range communication unit 230 transmits the second language (step S206). Short-range communication unit 130 receives the second language and stores the received second language in memory unit 120 (step S207). As shown in (3) and (4) of Fig. 3, "Japanese" is transmitted from information terminal device 200 as the second language, and "Japanese" is stored as the second language in memory unit 120 of information terminal device 100.

[0024] In the information terminal device 200, the voice input / output unit 240 determines whether or not voice input has been received (step S208). As shown in (5) of FIG. 3, the voice input of "Hello" uttered by the second speaker is received. If it is determined that voice input has been received (step S208: Yes), the character conversion unit 211 converts the voice in the second language into characters in the second language (step S209). If it is determined that there has been no voice input (step S208: No), the system waits for voice input in step S208.

[0025] The translation unit 212 translates the converted characters of the second language into characters of the first language (step S210). The display unit 260 displays the characters of the first language (step S211). As shown in (6) of FIG. 3, the display unit 260 displays the characters "Hello." obtained by translating the voice of "Hello" uttered by the second speaker. The voice input / output unit 240 judges whether the voice input has ended (step S212). If it is judged that the voice input has not ended (step S212: No), the voice input is waited for in step S208. If it is judged that the voice input has ended (step S212: Yes), "Not set" is stored in the first language of the storage unit 220, and the process is terminated. The judgment of whether the voice input is to be ended may be judged based on whether the voice input has been received within a predetermined time or whether short-distance communication with the information terminal device 100 is possible.

[0026] The voice input / output unit 140 of the information terminal device 100 determines whether or not voice input has been received (step S213). For example, as shown in (7) of FIG. 3, the voice input of "Hello" uttered by the first speaker is received. If it is determined that voice input has been received (step S213: Yes), the character conversion unit 111 converts the voice in the first language into characters in the first language (step S214). If it is determined that voice input has not been received (step S213: No), the information terminal device 100 waits for voice input in step S213.

[0027] The translation unit 112 translates the converted characters of the first language into characters of the second language (step S215). The display unit 160 displays the characters of the second language (step S216). As shown in (8) of FIG. 3, the display unit 160 displays the characters "Hello." obtained by translating the voice of "Hello" uttered by the first speaker. The voice input / output unit 140 judges whether the voice input has ended (step S217). If it is judged that the voice input has not ended (step S217: No), the voice input is awaited in step S213. If it is judged that the voice input has ended (step S217: Yes), "Not set" is stored in the second language of the storage unit 120, and the process ends.

[0028] Regarding the voice input in steps S208 and S213 described above, after the information terminal device 200 stores the first language in the memory unit 220 and the information terminal device 100 stores the second language in the memory unit 120, either the first or second speaker may speak first, and thereafter the first and second speakers may speak alternately or one may speak consecutively.

[0029] In this way, simply by bringing information terminal device 100 operated by a first speaker who is a foreign language speaker close to information terminal device 200, the words spoken by a second speaker who is a Japanese speaker are translated into a first language that the first speaker understands and displayed, without the first speaker having to perform any operation himself.

[0030] This allows visitors to hear the language of the country they are visiting while seeing text translated into a language they can understand, so they can easily communicate with Japanese speakers while enjoying the Japanese language, just like subtitles are displayed in a TV drama. Japanese speakers can also easily communicate with others by speaking to them in Japanese, without having to worry about what language the other person understands.

[0031] In the above-described embodiment, the processing in infrared communication, BLE communication, etc. has been described, but next, the processing in low-power wireless communication will be described. Fig. 4 is a flowchart showing a translation processing procedure executed by the information terminal device 100 and the information terminal device 200 in low-power wireless communication, etc.

[0032] As advance preparations for information terminal device 100 and information terminal device 200, operation unit 150 of information terminal device 100 accepts input of a first language (step S401) and stores the accepted first language in memory unit 120. Operation unit 250 of information terminal device 200 accepts input of a second language (step S402) and stores the accepted second language in memory unit 220.

[0033] The short-distance communication unit 130 of the information terminal device 100 transmits the first language (step S403). The short-distance communication unit 130 transmits the first language, for example, in Morse code or another method indicating the first language. The short-distance communication unit 230 of the information terminal device 200 transmits the second language (step S404). Like the short-distance communication unit 130, the communication unit 230 also transmits the second language in Morse code or another method indicating the second language.

[0034] The voice input / output unit 240 of the information terminal device 200 judges whether or not voice input has been received (step S405). If it is judged that voice input has been received (step S405: Yes), the short-distance communication unit 230 stops transmitting the second language (step S406). The short-distance communication unit 230 receives the first language transmitted from the information terminal device 100 (step S407) and stores the first language in the storage unit 220 (step S408). Since the low-power wireless communication is a half-duplex communication method in the same frequency band, radio waves from one device can be received by stopping transmission from the other device. If it is judged that voice input has not been received (step S405: No), the voice input is awaited in step S405.

[0035] The character conversion unit 211 converts the received voice in the second language into characters in the second language (step S409). The translation unit 212 translates the converted characters in the second language into the first language (step S410). The display unit 260 displays the characters in the first language (step S411). The voice input / output unit 240 judges whether the voice input has ended (step S412). If it is judged that the voice input has not ended (step S412: No), the voice input is awaited in step S405. After the first language is received, the processes of steps S406 to S408 are not executed, and step S409 and onward are executed. If it is judged that the voice input has ended (step S412: Yes), "Not set" is stored in the first language of the storage unit 220, and the process is terminated. The judgment of whether the voice input is to end may be judged based on whether the voice input has been received within a predetermined time.

[0036] In the information terminal device 100, the voice input / output unit 140 judges whether or not voice input has been received (step S413). If it is judged that voice input has been received (step S413: Yes), the short-distance communication unit 130 stops transmitting the first language (step S414). The short-distance communication unit 130 receives the second language transmitted from the information terminal device 200 (step S415) and stores the second language in the storage unit 120 (step S416). If it is judged that voice input has not been received (step S413: No), the short-distance communication unit 130 waits for voice input in step S413.

[0037] The character conversion unit 111 converts the received voice in the first language into characters in the first language (step S417). The translation unit 112 translates the converted characters in the first language into the second language (step S418). The display unit 160 displays the characters in the second language (step S419). The voice input / output unit 140 judges whether the voice input has ended (step S420). If it is judged that the voice input has not ended (step S420: No), the voice input is waited for in step S413. After the second language is received, the processes of steps S414 to S416 are not executed, and steps S417 and after are executed. If it is judged that the voice input has ended (step S420: Yes), the process is terminated. The judgment of whether the voice input is to end may be judged based on whether the voice input has been received within a predetermined time.

[0038] In this way, by using low-power wireless communication between information terminal device 100 and information terminal device 200 to transmit the first language and the second language, which are the language types of the other party, to each other, foreign language speakers and Japanese speakers of various languages ​​can converse freely.

[0039] As Modification 1, an embodiment will be described in which the short-range communication unit 130 of the information terminal device 100 only has a function of transmitting the first language, and the short-range communication unit 230 of the information terminal device 200 only has a function of receiving the first language. In this case, the storage unit 120 of the information terminal device 100 stores the first language and the second language in advance, and the storage unit 220 of the information terminal device 200 stores the second language in advance.

[0040] In variant example 1, when the short-range communication unit 130 and the short-range communication unit 230 perform infrared communication, the first language transmitted from the information terminal device 100 is received by the short-range communication unit 230 of the information terminal device 200 and stored in the memory unit 220 without executing the connection process (step S203) shown in FIG. 2.

[0041] Similarly, when the communication unit is a low-output line communication unit, the first language transmitted from the information terminal device 100 is received by the short-range communication unit 230 of the information terminal device 200 and stored in the memory unit 220 without determining whether or not voice input from the speaker has been accepted.

[0042] By executing the translation process with such a configuration, it is possible to display text obtained by translating words spoken in the second language by a second speaker into the first language on information terminal device 200 with a simpler configuration than that of FIG. 1, and it is also possible to display text obtained by translating words spoken in the first language by a first speaker into the second language on information terminal device 100.

[0043] In addition, when information terminal device 100 or information terminal device 200 is a smartphone or the like and is connected to a server via a network not shown in FIG. 1, the voice or characters converted from the voice and the language type may be transmitted to the server, and the voice or characters received at the server may be translated into voice or characters of the received language type and transmitted to information terminal device 100 or information terminal device 200.

[0044] In addition to the above-mentioned processing, when an information terminal device 200 operated by a Japanese speaker receives a first language from an information terminal device 100 operated by a foreign language speaker, a notification indicating that the first language has been received may be output to the information terminal device 200 by a text message, a sound such as a chime, or the like, or by turning on a light, etc. A similar notification may be given when the information terminal device 100 receives a second language.

[0045] The second language indicating the language type understood by the Japanese speaker may be, instead of Japanese (standard language), a dialect such as Japanese (Tsugaru dialect), Japanese (Kagoshima dialect), etc. The foreign language speaker (first speaker) is not limited to a foreign language speaker, and may be, for example, a speaker of a Japanese dialect.

[0046] <Example 2> 5 is a block diagram showing a configuration of a communication system 20 according to Example 2. The communication system 20 includes a beacon device 300, an information terminal device 400, a server 500, and an information terminal device 600.

[0047] The beacon device 300 is a device that transmits light or radio waves called a beacon. The beacon device 300 transmits a beacon signal including a beacon ID that identifies the beacon at a predetermined period. The beacon device 300 is arranged in association with the information terminal device 600. For example, the beacon device 300 may be held by the operator of the information terminal device 600, or may be integrated with the information terminal device 600 or arranged in the vicinity of the information terminal device 600.

[0048] The information terminal device 400 is a computer operated by a foreign language speaker, similar to the information terminal device 100 of the first embodiment, and may be a wearable terminal, a smartphone, a tablet terminal, a PC, or the like. The information terminal device 400 includes a control unit 110, a storage unit 120, a short-range communication unit 431, a communication unit 432, a voice input / output unit 140, an operation unit 150, and a display unit 160. Since the information terminal device 400 has a part of the configuration in common with the information terminal device 100, the description of the functions of the same reference numerals is omitted here and reference is made to the description of each unit of the information terminal device 100.

[0049] When the information terminal device 400 enters the communication range of the beacon device 300 and becomes communicable, the short-distance communication unit 431 receives a beacon signal transmitted by the beacon device 300. The short-distance communication unit 431 acquires a beacon ID from the beacon signal.

[0050] The communication unit 432 transmits the first language stored in the storage unit 120, the beacon ID acquired from the beacon signal of the beacon device 300, and the terminal ID for identifying the information terminal device 400 to the server 500 via the network N. The network N is any communication network such as the Internet, an intranet, a LAN (Local Area Network), a VPN (Virtual Private Network), a mobile communication network, or a combination thereof, and a part or all of them may be wired or wireless.

[0051] The server 500 transmits and receives data to and from the information terminal device 400 and the information terminal device 600 via the network N. The server 500 includes a control unit 510, a storage unit 520, and a communication unit 530. The storage unit 520 stores a beacon ID and a terminal ID for identifying the information terminal device 600 in association with each other. The communication unit 530 receives a first language, a beacon ID, and the terminal ID of the information terminal device 400 transmitted from the information terminal device 400. The control unit 510 acquires the terminal ID for identifying the information terminal device 600 associated with the beacon ID from the storage unit 520, and the communication unit 530 transmits the first language to the information terminal device 600 associated with the terminal ID.

[0052] The information terminal device 600 is a computer operated by a Japanese speaker, which has the same functions as the information terminal device 200 and the information terminal device 400 described in the first embodiment, and is a wearable terminal, a smartphone, a tablet terminal, a PC, or the like. The information terminal device 600 includes a control unit 210, a storage unit 220, a short-distance communication unit 631, a communication unit 632, a voice input / output unit 240, an operation unit 250, and a display unit 260. The information terminal device 600 has almost the same functions as the information terminal device 200 with the same reference numerals, and therefore the description here is omitted. The short-distance communication unit 631 and the communication unit 632 have the same functions as the short-distance communication unit 431 and the communication unit 432 of the information terminal device 400. Therefore, the description of the short-distance communication unit 431 and the communication unit 432 is referred to, and the description here is omitted.

[0053] The translation process executed in the communication system 20 configured as described above will be explained. Fig. 6 is a flowchart showing the translation process procedure executed by the beacon device 300, the information terminal device 400, the server 500, and the information terminal device 600.

[0054] As advance preparations for information terminal device 400 and information terminal device 600, operation unit 150 of information terminal device 400 accepts input of a first language and a second language (step S601) and stores the accepted first language and second language in memory unit 120. Operation unit 250 of information terminal device 600 accepts input of the second language (step S602) and stores the accepted second language in memory unit 220.

[0055] The short-distance communication unit 431 of the information terminal device 400 receives a beacon signal transmitted from the beacon device 300 (step S603). FIG. 7 is an explanatory diagram showing an example of the flow of the translation process of the second embodiment. As shown in FIG. 7(1), when the information terminal device 400 enters the communication range of the beacon device 300, it receives a beacon signal. The short-distance communication unit 431 acquires a beacon ID from the beacon signal (step S604). Instead of acquiring the beacon ID from the beacon signal, the beacon ID (or the terminal ID of the information terminal device 600) may be acquired by capturing an image of a QR code (registered trademark) with a camera (not shown). The communication unit 432 transmits the first language, the beacon ID, and the terminal ID of the information terminal device 400 to the server 500 (step S605). As shown in FIG. 7(2), the first language "English", the beacon ID, and the terminal ID of the information terminal device 400 are transmitted to the server 500.

[0056] The communication unit 530 of the server 500 receives the beacon ID and acquires the terminal ID associated with the received beacon ID from the storage unit 520 (step S606). The communication unit 530 transmits the first language to the information terminal device 600 associated with the terminal ID (step S607). As shown in FIG. 7(3), the communication unit 530 transmits the first language "English" to the information terminal device 600. The communication unit 530 associates the terminal ID of the information terminal device 400 with the terminal ID corresponding to the beacon ID and stores them in the storage unit 520 (step S608). As a result, the information terminal device 400 operated by the first speaker and the information terminal device 600 operated by the second speaker are identified and are in a paired state.

[0057] The communication unit 632 of the information terminal device 600 stores the first language transmitted from the server 500 in the storage unit 220 (step S609). As shown in FIG. 7(4), the first language "English" is stored in the storage unit 220. Since the processes of steps S610 to S613 for translating the speech are the same as the processes of steps S208 to S211 described in FIG. 2, refer to the description in FIG. 2 and the description here will be omitted.

[0058] The information terminal device 400 executes a process of translating the voice uttered by the first speaker into characters in the second language. The process of translating the voice from step S614 to step S617 is the same as the process of step S213 to step S216 described in FIG. 2, so refer to the description in FIG. 2 and omit the description here. The short-distance communication unit 431 judges whether or not a beacon signal has been received within a certain time (step S618). If it is judged that a beacon signal has been received within a certain time (step S618: Yes), that is, if the information terminal device 400 has not left the beacon device 300 and is within a communication range where a beacon signal can be received, it waits for voice input in step S614.

[0059] When it is determined that the beacon signal has not been received within a certain period of time (step S618: No), the communication unit 432 transmits to the server 500 a notification that the beacon signal has not been received (step S619), and ends the process. When the server 500 receives a notification that the beacon signal has not been received from the information terminal device 400, the server 500 deletes the terminal ID of the information terminal device 400 associated with the terminal ID of the information terminal device 600 from the storage unit 520 (step S620). This cancels the pairing between the information terminal device 400 and the information terminal device 600, and the information terminal device 600 can be paired with a new information terminal device 400 that has received the beacon signal.

[0060] In this way, a foreign language speaker can simply approach a Japanese speaker and have the translated text of the Japanese speaker's speech displayed on the Japanese speaker's device. In addition, a foreign language speaker can also speak in their own language and have the text translated into Japanese displayed on the display. This allows both foreign language speakers and Japanese speakers to communicate in their own language without worrying about whether they can understand the language of the person they want to talk to. For example, Japanese speakers working in souvenir shops or convenience stores can easily talk to travelers even if they do not understand the language they understand. A foreign language speaker can enjoy the atmosphere of traveling abroad by listening to the local language, while checking the translated text in their own language without having to launch an application program or operate a smartphone or the like to translate it themselves, allowing them to enjoy their trip without stress.

[0061] As a first modification, a connection request may be transmitted and received between the information terminal device 400 and the beacon device 300, and a connection process may be executed between the information terminal device 400 and the beacon device 300. After the connection process is completed, the beacon device 300 transmits the beacon ID to the information terminal device 400, so that the information terminal device 400 can reliably obtain the beacon ID.

[0062] A second modification will be described. Instead of displaying the characters obtained by translating the speech uttered by the foreign language speaker on the display unit 160 such as a display in step S617, the translated characters may be transmitted to the information terminal device 600 via the server 500, read out by a reading unit (not shown), and output from earphones, headphones, or the like of the speech input / output unit 240.

[0063] In the above-described embodiment, the translation process is performed by the information terminal device 400 and the information terminal device 600, but instead of performing the translation by the information terminal device 400 and the information terminal device 600, characters translated into the first language or the second language by the server 500 may be transmitted to the information terminal device 400 and / or the information terminal device 600.

[0064] Modification 3 will now be described. Information terminal device 600 of modification 3 is not a device operated by a Japanese speaker, but a digital signage device (so-called electronic signboard) installed on the walls or pillars of buildings, sidewalks or roads, squares, parks, etc., and outputs various contents. Information terminal device 600 has output unit 660 instead of display unit 260. Output unit 660 outputs various contents such as images, videos, VR (Virtual Reality), audio, music, SNS (Social Networking Service) messages, chat, etc., instead of or in addition to characters.

[0065] FIG. 8 is a flowchart showing a translation process procedure executed by the beacon device 300, the information terminal device 400, the server 500, and the information terminal device 600. In FIG.

[0066] As a preparation for the information terminal device 400 and the information terminal device 600, the operation unit 150 of the information terminal device 400 accepts an input of a first language (step S801) and stores the accepted first language in the storage unit 120. The operation unit 250 of the information terminal device 600 accepts an input of a second language (step S802) and stores the accepted second language in the storage unit 220. At this point, the first language in the storage unit 220 is stored as "not set". The output unit 660 of the information terminal device 600 outputs the content (step S803). For example, when the first language is set as "not set" and the second language is set as "Japanese", the output language is determined to be "Japanese" and the content including Japanese characters and sounds is output. The output language is the language type of the content when the content is output by the information terminal device 600.

[0067] The short-distance communication unit 431 of the information terminal device 400 receives a beacon signal transmitted from the beacon device 300 (step S804). FIG. 9 is an explanatory diagram showing an example of the flow of the translation process. When a plurality of information terminal devices 400-1 to 3 enter the communication range of the beacon device 300 shown in FIG. 9 (1), each of the information terminal devices 400-1 to 3 receives a beacon signal. The short-distance communication unit 431 of the information terminal device 400 acquires a beacon ID from the beacon signal (step S805). The communication unit 432 transmits the first language and the beacon ID to the server 500 (step S806). As shown in FIG. 9 (2), each of the information terminal devices 400-1 to 3 transmits the first language and the beacon ID to the server 500.

[0068] The communication unit 530 of the server 500 transmits the retransmission time of the first language and the beacon ID to the information terminal device 400 (step S807). If the retransmission time is preset in the information terminal device 400, the server 500 does not need to transmit the retransmission time. The communication unit 432 of the information terminal device 400 judges whether the retransmission time has elapsed (step S808). If it is judged that the retransmission time has not elapsed (step S808: No), the communication unit 432 waits for the retransmission time to elapse in step S808. If it is judged that the retransmission time has elapsed (step S808: Yes), the communication unit 432 judges whether a beacon signal has been received (step S809). If it is judged that a beacon signal has been received (step S809: Yes), the communication unit 432 transmits the first language and the beacon ID in step S806. If it is judged that a beacon signal has not been received (step S809: No), the processing ends. In this way, when the retransmission time has elapsed and a beacon signal has been received, by retransmitting the first language and beacon ID to the server, it can be determined that the foreign language speaker who is the operator of information terminal device 400 continues to be present in the vicinity of information terminal device 600.

[0069] The communication unit 530 of the server 500 receives the first language and the beacon ID, and acquires the terminal ID of the information terminal device 600 associated with the beacon ID from the storage unit 520 (step S810). The communication unit 530 transmits the first language to the information terminal device 600 corresponding to the beacon ID (step S811). As shown in FIG. 9(3), the communication unit 530 transmits the first language received from each of the information terminal devices 400-1 to 3 to the information terminal device 600.

[0070] The communication unit 632 of the information terminal device 600 receives the first language, and the control unit 210 counts the language type (step S812). The control unit 210 judges whether a predetermined time has elapsed by a timer (step S813). If it is judged that the predetermined time has not elapsed (step S813: No), the control unit 210 receives the first language from the server 100. If it is judged that the predetermined time has elapsed (step S813: Yes), the control unit 210 stores the first language (step S814). More specifically, the control unit 210 stores the most common language type among the language types of the first language received within the predetermined time in the first language. If the language type of the first language is not received within the predetermined time, the control unit 210 stores "not set" in the first language. In FIG. 9, as shown in (4), "English", which is the most common language type of the information terminal devices 400-1 to 400-3, in the first language.

[0071] The control unit 210 judges whether or not to change the language type of the content (step S815). More specifically, it judges whether or not the current output language and the language type set in the first language match. If the first language is set to "not set", it judges whether or not the current output language and the language type set in the second language match. If it is judged that the language type of the content is to be changed (step S815: Yes), the translation unit 212 translates the content into the output language (step S816). The output unit 660 outputs the content for a predetermined time (step S817). For example, as shown in FIG. 9(5), the content translated into "English" is displayed. If it is judged that the language type of the content is not to be changed (step S815: No), the content is not translated and the content is output as it is. Note that the content output in step S817 does not wait for the predetermined time to elapse, and returns to step S811 to receive the first language.

[0072] In this way, when the owner of information terminal device 400 who understands a language other than Japanese approaches information terminal device 600 displaying guidance using Japanese characters, audio, video, etc., the content output from information terminal device 600 is translated into content in a language understood by the owner of information terminal device 400 and output. Also, when multiple people who own information terminal device 400 approach information terminal device 600, the content is translated and output in the language type that is most commonly understood by the owners of information terminal device 400.

[0073] A description will now be given of a variation of the above-mentioned variation 3. Instead of translating the content only into the most popular language type as in the above-mentioned embodiment, the content may be translated into all languages ​​in descending order of the number of languages ​​set for foreign language speakers, and each translated content may be output at a predetermined time interval.

[0074] As another modification, in step S805 of FIG. 8, the short-distance communication unit 431 acquires the radio wave strength of the beacon signal in addition to the beacon ID. In step S806, the communication unit 432 transmits the radio wave strength to the server 500 in addition to the first language and the beacon ID, and in step S811, transmits the first language and the radio wave strength to the information terminal device 600. In step S814, the language type of the information terminal device 400 that transmitted the strongest radio wave strength instead of the most common language type is set as the first language. This allows the content to be translated in a language type that the holder of the information terminal device 400 closest to the beacon device 300 understands.

[0075] In the above-described modified example, the language type is counted in the information terminal device 600, and the language type of the first language is set, but this may be executed in the server 500. Furthermore, instead of translating the content in the information terminal device 600, the server 500 may translate the content and output the translated content in the information terminal device 600. Also, instead of translating the content, the content may be replaced with content created in the output language and output. Also, in this modified example, instead of the beacon signal, a QR code may be captured by a camera to obtain a beacon ID or a terminal ID of the information terminal device 600.

[0076] In this way, by possessing an information terminal device 400 that receives beacon signals, a foreign language speaker can, instead of or in addition to being able to converse freely with Japanese speakers, have the content output by the digital signage device output in a language that the foreign language speaker himself or herself understands.

[0077] <Example 3> 10 is a block diagram showing a configuration of a communication system 30 according to a third embodiment. The communication system 30 includes an information terminal device 700, a server 800, and an information terminal device 900. The information terminal device 700, the server 800, and the information terminal device 900 are connected via a network N.

[0078] The information terminal device 700 is a computer operated by a foreign language speaker, and may be a wearable terminal, a smartphone, a tablet terminal, a PC, or the like. The information terminal device 700 includes a control unit 110, a storage unit 120, a communication unit 432, an audio input / output unit 140, an operation unit 150, a display unit 160, and a camera 770. The information terminal device 700 shares some configuration with the information terminal device 100 and the information terminal device 400. Since the same reference numerals have almost the same functions, the description of each unit of the information terminal device 100 and the information terminal device 400 will be referred to, and the following description will focus on the differences.

[0079] The camera 770 captures an image of a QR code displayed on the display unit 260 of the information terminal device 900 or a printed QR code. The QR code includes a terminal ID for identifying the information terminal device 900 and a URL (Uniform Resource Locator) for accessing a web page.

[0080] The communication unit 732 transmits to the server 800 a request for a web page specified by the URL read from the QR code and the terminal ID for identifying the information terminal device 900. The communication unit 732 receives the web page. The communication unit 732 transmits to the server 800 the voice uttered by the foreign language speaker received by the voice input / output unit 140.

[0081] Server 800 includes a control unit 810, a storage unit 820, and a communication unit 830. Communication unit 830 receives a web page request and a terminal ID for identifying information terminal device 900 transmitted from information terminal device 700, and transmits the web page. Control unit 810 acquires the language type included in the web page request. Communication unit 830 transmits the language type to information terminal device 900 corresponding to the terminal ID, and transmits the voice transmitted from information terminal device 700 to information terminal device 900.

[0082] The information terminal device 900 is a computer operated by a Japanese speaker, such as a wearable device, a smartphone, a tablet device, or a PC. The information terminal device 900 includes a control unit 210, a storage unit 220, a communication unit 632, a voice input / output unit 240, an operation unit 250, and a display unit 260. The information terminal device 900 has almost the same functions as the information terminal device 200 and the information terminal device 600 of the first and second embodiments with the same reference numerals, and therefore the description thereof will be omitted here.

[0083] The translation process executed by the information terminal device 700, the server 800, and the information terminal device 900 of the communication system 30 configured as described above will be described below.

[0084] The camera 770 of the information terminal device 700 captures an image of the QR code (step S1101). FIG. 12 is an explanatory diagram showing an example of the flow of the translation process of the third embodiment. As shown in FIG. 12(1), the camera 770 of the information terminal device 700 captures an image of the QR code displayed on the display unit 260 of the information terminal device 900. Returning to FIG. 11, the camera 770 acquires information (the URL of the web page and the terminal ID of the information terminal device 900) from the QR code (step S1102). The communication unit 732 transmits a request for the web page specified by the acquired URL, the terminal ID of the information terminal device 900, and the terminal ID of the information terminal device 700 to the server 800 (step S1103). The request for the web page includes the language type (first language) set in the web browser that displays the web page. As shown in FIG. 12(2), a web page request, the terminal ID of the information terminal device 900 (second speaker) and the terminal ID of the information terminal device 700 (first speaker) are transmitted to the server 800.

[0085] The communication unit 830 of the server 800 receives the web page request, the terminal ID of the information terminal device 900, and the terminal ID of the information terminal device 700, and stores the received terminal ID of the information terminal device 900 in association with the terminal ID of the information terminal device 700 in the storage unit 820 (step S1104). This results in pairing of the information terminal device 900 and the information terminal device 700. The communication unit 830 transmits the first language transmitted from the information terminal device 700 to the information terminal device 900 corresponding to the terminal ID (step S1105). As shown in FIG. 12(3), the communication unit 830 transmits the first language: English to the information terminal device 900. The communication unit 632 of the information terminal device 900 receives the language type, and stores the received language type in the storage unit 220 (step S1106).

[0086] The communication unit 830 of the server 800 transmits a web page for accepting voice input to the information terminal device 700 (step S1107). As shown in FIG. 12(4), the web page is transmitted to the information terminal device 700. The communication unit 732 of the information terminal device 700 receives the web page, and the display unit 160 displays the received web page in a web browser (step S1108). The operation unit 150 accepts voice in the first language input from a microphone or the like in response to an instruction from a menu displayed on the web page (step S1109). The communication unit 732 transmits the voice in the first language and the terminal ID of the information terminal device 700 to the server 800 (step S1110). As shown in FIG. 12(5), the English voice and the terminal ID received from the web page are transmitted to the server 800.

[0087] The communication unit 830 of the server 800 receives the voice in the first language and the terminal ID of the information terminal device 700, and acquires the terminal ID corresponding to the received terminal ID of the information terminal device 700 from the storage unit 820 (step S1111). The communication unit 830 transmits the voice in the first language to the information terminal device 900 associated with the terminal ID (step S1112). As shown in FIG. 12(6), the voice in English is transmitted to the information terminal device 900.

[0088] The communication unit 632 of the information terminal device 900 receives the speech in the first language, and the translation unit 212 translates the received speech in the first language into speech in the second language stored in the storage unit 220 (step S1113). The speech input / output unit 240 outputs the translated speech in the second language from a headset or the like (step S1114). As shown in FIG. 12 (7), Japanese speech is output. If speech is input from a Japanese speaker, the speech input / output unit 240 executes speech translation processing (step S1115). The speech translation processing is similar to the processing in steps S208 to S212 in FIG. 2, so please refer to the flowchart and explanation of steps S208 to S212 and will not be explained here.

[0089] Operation unit 150 of information terminal device 700 determines whether or not an instruction to end has been given on the web page (step S1116). If it is determined that an instruction to end has not been given on the web page (step S1116: No), voice input is accepted in step S1109. If it is determined that an instruction to end has been given on the web page (step S1116: Yes), communication unit 732 transmits to server 800 a message indicating that voice input has ended and the terminal ID of information terminal device 700 (step S1117). Then, the process ends.

[0090] The communication unit 830 of the server 800 receives the notification that the voice input has ended and the terminal ID of the information terminal device 700, and deletes the terminal ID of the information terminal device 700 corresponding to the terminal ID of the information terminal device 900 stored in the storage unit 820 (step S1118). As a result, the information terminal device 900 operated by the Japanese speaker has no interlocutor, and can be paired with another foreign language speaker.

[0091] In this way, a foreign language speaker can obtain the language type set in the web browser of the information terminal device 700 simply by capturing an image of the QR code, so that the words spoken by the foreign language speaker are translated into Japanese and output as Japanese voice from the Japanese speaker's information terminal device 900 without the foreign language speaker setting the language type himself. In addition, voice uttered by the Japanese speaker is translated into characters in a language that the foreign language speaker understands and displayed. This allows the foreign language speaker to easily communicate with the Japanese speaker without having to download an application program in advance or set the language type himself.

[0092] In a modified example, steps S1110 to S1112 may be performed without using the server 800, by transmitting and receiving audio between the information terminal device 700 and the information terminal device 900 in a peer-to-peer manner using the WebRTC (Web Real-Time Communication) mechanism.

[0093] The hardware configuration of the information terminal device 100, information terminal device 200, information terminal device 400, server 500, information terminal device 600, information terminal device 700, server 800, and information terminal device 900 according to the above-mentioned embodiments is a normal computer including one or more processors such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), or a GPU (Graphics Processing Unit), and equipped with an external storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), a flash memory, or an SSD (Solid State Drive), a communication control device, etc., and the above-mentioned configuration and functions are realized by the CPU or the like reading and running a program stored in the ROM, RAM, HDD, etc. The control unit may be an electronic circuit such as an ASIC (Application Specific Integrated Circuit) or a PLD (Programmable Logic Device).

[0094] The programs that operate on the information terminal device 100, information terminal device 200, information terminal device 400, server 500, information terminal device 600, information terminal device 700, server 800, and information terminal device 900 may be stored on a computer connected to a network such as the Internet and provided by downloading via the network, or may be recorded in an installable or executable format on a computer-readable recording medium such as a CD-ROM, DVD, USB memory, or SD card and provided. The programs that realize the above-mentioned functions and processes may be provided in the form of an API (Application Programming Interface), SaaS (Software as a Service), or cloud computing.

[0095] The present invention is not limited to the above-mentioned embodiments, and does not necessarily have to be physically configured as shown in the drawings. In addition, the present invention can be configured by functionally or physically dividing, integrating, replacing, modifying, or deleting all or part of the components described in each of the first to third embodiments and the modified examples in any unit according to various loads, usage conditions, etc. [Explanation of symbols]

[0096] 100...information terminal device, 110...control unit, 111...character conversion unit, 112...translation unit, 120...storage unit, 130...short-distance communication unit, 140...voice input / output unit, 150...operation unit, 160...display unit, 200...information terminal device, 210...control unit, 211...character conversion unit, 212...translation unit, 220...storage unit, 230...short-distance communication unit, 240...voice input / output unit, 250...operation unit, 260...display unit, 300 ...Beacon device, 400...information terminal device, 431...short-distance communication unit, 432...communication unit, 500...server, 510...control unit, 520...storage unit, 530...communication unit, 600...information terminal device, 631...short-distance communication unit, 632...communication unit, 700...information terminal device, 732...communication unit, 770...camera, 800...server, 810...control unit, 820...storage unit, 830...communication unit, 900...information terminal device

Claims

1. A communication system including a first information terminal device and a second information terminal device, The first information terminal device is a transmission means for transmitting a first language, which is a language type spoken by an operator of the first information terminal device and which is stored in advance in the first information terminal device, when communication with the second information terminal device is possible through short-distance communication; The second information terminal device is a receiving means for receiving the first language transmitted from the first information terminal device; a storage means for storing the one language received by the receiving means in a memory unit.

2. a receiving means for receiving a first language, which is a language type spoken by a first speaker, from a first information terminal device, which is a terminal operated by a first speaker, when the first information terminal device and the first language terminal device are capable of communicating with each other through short-distance communication; a storage unit for storing the one language received by the receiving unit in a storage unit.

3. A voice receiving means for receiving a voice input; A character conversion means for converting the voice into characters; a translation means for translating the characters converted by the character conversion means into characters of the first language; 3. The information terminal device according to claim 2, further comprising: display means for displaying the characters translated by the translation means.

4. A voice receiving means for receiving a voice input; a transmission means for transmitting the voice and the first language received by the voice reception means to a server connected via a network; a character receiving means for receiving characters obtained by translating the voice into the first language from the server; 3. The information terminal device according to claim 2, further comprising: display means for displaying the characters received by said character receiving means.

5. An information terminal device according to any one of claims 2 to 4, wherein, instead of receiving the first language from the first information terminal device by the receiving means, the first language is acquired by capturing an image of a QR code including the first language by an imaging means.

6. A communication system comprising a first information terminal device, a server, a second information terminal device, and a beacon device arranged in association with the second information terminal device, The first information terminal device is A first receiving means for receiving a beacon ID for identifying the beacon device when communication with the beacon device is possible; a first transmission means for transmitting the beacon ID received by the first reception means and a first language, which is a language type pre-stored in the first information terminal device, to the server; The server, a language transmitting means for transmitting the first language to the second information terminal device associated with the beacon ID received from the first information terminal device, The second information terminal device is a second receiving means for receiving the first language from the server; a storage means for storing the one language received by the second receiving means in a storage unit; A communication system comprising:

7. When communication with a beacon device is possible, a server that connects to a first information terminal device via a network, the server receiving the beacon ID that identifies the beacon device and transmitting the received beacon ID and a first language that is a language type to be stored in advance, a language transmitting means for transmitting the first language to a second information terminal device associated with the beacon ID received from the first information terminal device.

8. When communication with a beacon device is possible, a first information terminal device that receives a beacon ID that identifies the beacon device and transmits the received beacon ID and a first language that is a language type to be stored in advance, and a server that transmits the first language to an information terminal device associated with the beacon ID received from the first information terminal device is connected via a network, receiving means for receiving the first language from the server; a storage means for storing the first language received by the receiving means in a storage unit; An information terminal device comprising:

9. A voice receiving means for receiving a voice input; A character conversion means for converting the voice into characters; a translation means for translating the characters in the first language received from the server; 9. The information terminal device according to claim 8, further comprising: display means for displaying the characters translated by the translation means.

10. An information terminal device connected to a server via a network, A receiving means for receiving a beacon ID for identifying the beacon device when communication with the beacon device is possible; a transmission means for transmitting the beacon ID received by the receiving means and a pre-stored language type to the server; An information terminal device comprising:

11. The information terminal device according to claim 10, further comprising: a QR code acquisition means for acquiring the beacon ID by capturing an image of a QR code including the beacon ID with an imaging means, instead of receiving the beacon ID from the beacon device with the receiving means.

12. When communication with a beacon device is possible, a server that connects to a first information terminal device via a network and a second information terminal device that receives a beacon ID that identifies the beacon device and transmits the received beacon ID and a first language that is a language type to be stored in advance, The server, a language transmitting means for transmitting the first language to the second information terminal device associated with the beacon ID received from the first information terminal device, The second information terminal device is a language setting means for setting the one language received from the server as an output language; A communication system comprising:

13. The second information terminal device is The communication system according to claim 12 , wherein the language setting means, when receiving a plurality of the first languages ​​within a predetermined time, sets the most common language type among the first languages ​​as the output language.

14. The second information terminal device is The communication system according to claim 12, wherein when the language setting means receives a plurality of the first languages ​​and radio wave intensities within a predetermined time, the language setting means sets the first language associated with the strongest radio wave intensity among the first language and radio wave intensities as the output language.

15. The second information terminal device is a translation means for translating content into the output language; 15. The communication system according to claim 12, further comprising: an output unit that outputs the content translated by the translation unit.

16. A communication system comprising a first information terminal device, a server, and a second information terminal device, The first information terminal device is An imaging means for imaging a QR code; a first transmission means for acquiring a URL of a web page included in the QR code and a terminal ID for identifying the second information terminal device, and transmitting a request for a web page designated by the acquired URL and the terminal ID to the server; The server, a language transmission means for transmitting a language type included in the request for the web page received from the first information terminal device to the second information terminal device associated with the terminal ID, The second information terminal device is a storage means for receiving the language type from the server and storing the received language type in a storage unit; A communication system comprising:

17. A server connected to a first information terminal device via a network, the server capturing an image of a QR code, acquiring a URL of a web page included in the QR code and a terminal ID for identifying an information terminal device, and transmitting a request for a web page specified by the acquired URL, a language transmission means for transmitting a language type included in the request for the web page received from the first information terminal device to a second information terminal device associated with the terminal ID.

18. An information terminal device connected via a network to a first information terminal device which captures a QR code, acquires a URL of a web page included in the QR code and a terminal ID for identifying an information terminal device, and transmits a request for a web page specified by the acquired URL and the terminal ID, and a server which transmits a language type included in the request for the web page received from the first information terminal device to the information terminal device associated with the terminal ID, A voice receiving means for receiving a voice input; A character conversion means for converting the voice into characters; a translation means for translating the characters into the language type received from the server; a display means for displaying the characters translated by the translation means; An information terminal device comprising:

19. 1. A computer-implemented method comprising: a receiving step of receiving a first language, which is a language type spoken by a first speaker, from a first information terminal device, which is a terminal operated by a first speaker, when the first information terminal device is capable of communicating with the first speaker through short-distance communication; a storage step of storing the one language received in the receiving step in a storage unit.

20. When communication with a beacon device is possible, a beacon ID that identifies the beacon device is received, and the received beacon ID and a first language that is a language type that is stored in advance are transmitted to a first information terminal device that is connected via a network. A translation method comprising: a language transmitting step of transmitting the first language to a second information terminal device associated with the beacon ID received from the first information terminal device.

21. A method executed by a computer connected to a first information terminal device via a network, the method comprising: capturing an image of a QR code; acquiring a URL of a web page included in the QR code and a terminal ID for identifying an information terminal device; and transmitting a request for a web page specified by the acquired URL, the method comprising: A translation method comprising: a language transmitting step of transmitting a language type included in the request for the web page received from the first information terminal device to a second information terminal device associated with the terminal ID.

22. A translation program for causing a computer to execute the translation method according to any one of claims 19 to 21.

Images