Information display system, display control method, and program

Abstract

To support the accurate communication between a first-language speaker and a second-language speaker in their conversation. [Solution] The information display system 1 comprises a display unit 2, an automatic translation unit 24, and a display control unit 26. The display unit 2 is positioned between a first person who speaks the first language and a second person who speaks the second language, and is visible to both the first and second people. The automatic translation unit 24 translates the statements made by the first person to generate second language text, and also translates the statements made by the second person to generate first language text. The display control unit 26 displays the first language text and the second language text on the display unit 2. If the first language text or the second language text contains a word that points to a specific direction, the display control unit 26 replaces that word with a word that points to the opposite direction and displays it on the display unit 2.

Description

Technical Field

[0001] The present invention relates to an information display system, a display control method, and a program. In particular, the present invention relates to a technology for supporting conversations conducted by a plurality of facing speakers in different languages.

Background Art

[0002] In order to deal with visitors who visit a facility, it has been proposed to use a transparent display. For example, Patent Document 1 discloses that a transparent display is installed between a visitor and a responder, and the work of responding to the visitor is remotely supported by displaying an image on the transparent display.

[0003] At the front desk of a hotel or the like, people who speak various languages come to visit. The front desk staff must deal with those people who speak various languages. If the front desk staff cannot understand the language spoken by the visitor, they cannot provide sufficient services to the visitor. Therefore, in recent years, attempts have been made to utilize the above-mentioned transparent display.

[0004] For example, a system in which a computer or a tablet terminal installed with a real-time automatic translation application is connected to a transparent display has been proposed. This system translates in real time what the visitor or the front desk staff says, and displays the translation result in text form on the transparent display. Since the visitor and the front desk staff talk with the transparent display in between, they can confirm the content spoken by the other party in the text displayed on the transparent display. In addition, the visitor and the front desk staff can proceed with the conversation while seeing the expression of the other party. Therefore, such a system is expected to greatly contribute to the realization of smooth communication between people who speak different languages.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] The system described above can also display images on a transparent display. For example, in a hotel or other service counter, a map image can be displayed on the transparent display, allowing the counter staff to guide visitors while they refer to the map image. However, when a map image is displayed on a transparent display positioned between the visitor and the counter staff, the map image will appear horizontally reversed to both the visitor and the counter staff. Therefore, if the counter staff gives directions based on the map image displayed on the transparent display, the visitor may not receive accurate directions. This is because if the counter staff utters a word indicating left while looking at the map image, the text displayed as a result of real-time translation will indicate left, even though the map image the visitor is looking at actually indicates right. The same applies if the visitor utters a word related to a specific direction such as left or right in relation to the map image. Therefore, the system described above may not be able to achieve smooth communication.

[0007] This invention was made to solve the above problems. Specifically, the present invention aims to provide an information display system, a display control method, and a program that can help ensure that the content of a conversation between a first person who speaks a first language and a second person who speaks a second language is accurately conveyed to the other party. [Means for solving the problem]

[0008] To achieve the above objective, the invention according to claim 1 is an information display system comprising: a display unit positioned between a first person who speaks a first language and a second person who speaks a second language, and which is visible to both the first and second persons; an automatic translation unit that translates a statement made by the first person to generate a second language text and translates a statement made by the second person to generate a first language text; and a display control unit that displays the first language text and the second language text on the display unit, and, if the first language text or the second language text contains a word that points to a specific direction, replaces the word with a word that points to the opposite direction and displays it on the display unit.

[0009] The invention according to claim 2 is characterized in that, in the information display system of claim 2, the specific direction includes the left direction and the right direction.

[0010] The invention according to claim 3 is an information display system according to claim 2, wherein the display unit is a transparent display, and a first person and a second person are each made to view different display surfaces of the transparent display.

[0011] The invention according to claim 4 is characterized in that, in the information display system of claim 3, the display control unit displays either the first language text or the second language text inverted horizontally.

[0012] The invention according to claim 5 is characterized in that, in the information display system of claim 1, the automatic translation unit is capable of selecting a first language and a second language to be translated from among a plurality of languages.

[0013] The invention according to claim 6 is characterized in that the information display system according to claim 1 further comprises an audio output unit that reads aloud and outputs the first language text or the second language text displayed by the display control unit.

[0014] The invention according to claim 7 is characterized in that, in the information display system of claim 6, the voice output unit reads aloud and outputs the replaced word when the word indicating a specific direction is replaced with a word indicating the opposite direction.

[0015] The invention according to claim 8 is characterized in that the information display system according to claim 1 further comprises: an audio input unit that inputs statements made by a first person and a second person as audio; and a person identification unit that identifies whether the person who made the statement is the first person or the second person by analyzing the audio input to the audio input unit.

[0016] The invention according to claim 9 is characterized in that, in the information display system of claim 8, the display area of ​​the first language text and the display area of ​​the second language text are displayed separately based on the results of the person identification by the person identification unit.

[0017] The invention according to claim 10 is characterized in that, in the information display system of claim 1, when the display control unit replaces a word indicating a specific direction with a word indicating a direction opposite to the specific direction, it highlights the replaced word in the first language text or the second language text.

[0018] The invention according to claim 11 is characterized in that, in the information display system of claim 1, the specific direction includes the left direction, the right direction, the up direction, and the down direction.

[0019] The invention according to claim 12 is an information display system according to claim 11, characterized in that the display unit is a non-transparent display, and the same display surface is made visible to a first person and a second person, respectively.

[0020] The invention according to claim 13 is the information display system according to claim 2 or 11, wherein when the display control unit displays a map image in a partial display area of the display unit, it switches to a word replacement mode for detecting whether the first language text or the second language text contains a word indicating the specific direction.

[0021] The invention according to claim 14 is the information display system according to claim 13, wherein when the display control unit switches to the word replacement mode, it displays that it is in the word replacement mode in a part of the display unit.

[0022] The invention according to claim 15 is the information display system according to claim 1, further comprising a storage unit for storing word information associating a word indicating the specific direction with a word indicating a direction opposite to the specific direction, and the display control unit determines whether the first language text or the second language text contains a word indicating the specific direction based on the word information.

[0023] The invention according to claim 16 is the information display system according to claim 15, wherein the words registered in the word information can be added or deleted.

[0024] The invention according to claim 17 is a display control method for controlling a display unit that is arranged between a first person who speaks a first language and a second person who speaks a second language and is commonly visible to the first person and the second person, the method including an automatic translation step of translating a statement by the first person to generate second language text and translating a statement by the second person to generate first language text, and a display control step of displaying the first language text and the second language text on the display unit and, when the first language text or the second language text contains a word indicating a specific direction, replacing the word with a word indicating a direction opposite to the specific direction and displaying it on the display unit.

[0025] The invention according to claim 18 is a program executed in an information display system disposed between a first person speaking a first language and a second person speaking a second language, and including a display unit that is commonly visible to the first person and the second person, the information display system being caused to execute: an automatic translation step of translating a statement by the first person to generate a second language text and translating a statement by the second person to generate a first language text; and a display control step of displaying the first language text and the second language text on the display unit, and when the first language text or the second language text includes a word indicating a specific direction, replacing the word with a word indicating a direction opposite to the specific direction and displaying the word on the display unit.

Effect of the Invention

[0026] According to the present invention, the conversation between two people can be supported by displaying the automatically translated text on a display unit disposed between a first person speaking a first language and a second person speaking a second language. Further, in a situation where the conversation is being supported, when a statement regarding a specific direction is made, the content of the statement regarding the specific direction can be accurately conveyed to the other party.

Brief Description of the Drawings

[0027] [Figure 1] It is a diagram showing an example of a configuration of an information display system. [Figure 2] It is a block diagram showing a functional configuration of an information display system. [Figure 3] It is a diagram showing text generated by a voice recognition unit and an automatic translation unit. [Figure 4] It is a diagram showing an example of a screen displayed on a display unit by a display control unit. [Figure 5] It is a diagram showing an example of a screen of a display unit displaying a map image. [Figure 6] It is a diagram showing an example of word information. [Figure 7] It is a flowchart showing an example of a processing procedure performed by a computer. [Figure 8] This flowchart shows an example of a detailed processing procedure for word replacement. [Figure 9] This flowchart shows an example of a detailed processing procedure for display. [Figure 10] This figure shows an example of a screen displayed on the display unit by the display control unit. [Figure 11] This figure shows another example configuration of the information display system. [Figure 12] This figure shows an example of a screen displayed on the display unit by the display control unit. [Figure 13] This figure shows other examples of word information. [Figure 14] This figure shows an example of a screen displayed on the display unit by the display control unit. [Modes for carrying out the invention]

[0028] Preferred embodiments of the present invention will be described in detail below with reference to the drawings. In the embodiments described below, elements common to all are denoted by the same reference numerals, and redundant explanations of these elements will be omitted.

[0029] (First Embodiment) Figure 1 shows an example configuration of an information display system 1 in a first embodiment of the present invention. This information display system 1 is a system that supports smooth face-to-face conversation between a first person P1 who speaks a first language and a second person P2 who speaks a second language different from the first language. In this embodiment, the first language is English and the second language is Japanese. However, the first and second languages ​​are not limited to English or Japanese.

[0030] The information display system 1 comprises a display unit 2, a computer 4, a microphone 7, and a speaker 8.

[0031] The display unit 2 is installed between the first person P1 and the second person P2, and is visible to both the first person P1 and the second person P2. In this embodiment, the display unit 2 is configured as a transparent display equipped with a transparent screen 3. The display unit 2 displays information on the transparent screen 3 that is visible from both sides. The transparent display is erected between the first person P1 and the second person P2. The first person P1 views one side of the transparent screen 3. The second person P2 views the other side of the transparent screen 3. The transparent screen 3 transmits light. Therefore, the first person P1 and the second person P2 can see each other's facial expressions through the transparent screen 3 and confirm the image displayed on the transparent screen 3.

[0032] Computer 4 is comprised of, for example, a tablet device or a personal computer (PC). Computer 4 includes a CPU 5 and a memory unit 6. The CPU 5 is a hardware processor that reads and executes computer-readable programs stored in the memory unit 6. By executing programs, the CPU 5 launches various applications in Computer 4. One of these applications is a conversation support application that assists in a conversation between a first person P1 and a second person P2. By launching this conversation support application, Computer 4 assists in a conversation between the first person P1 and the second person P2 in different languages. The memory unit 6 is a non-volatile storage device, for example, a hard disk drive (HDD) or a solid-state drive (SSD).

[0033] Computer 4 is connected to the display unit 2 and controls the content displayed on the display unit 2. A microphone 7 and a speaker 8 are also connected to Computer 4. Computer 4 acquires the voices of the first person P1 and the second person P2 via the microphone 7. Computer 4 also outputs the voices via the speaker 8. Furthermore, Computer 4 is equipped with other user interfaces, such as a keyboard, mouse, and touch keys, which are not shown in the diagram.

[0034] In the following, we will describe in detail the specific functions and operations of the information display system 1 in this embodiment, assuming a scenario in which a person P1 who speaks a first language and a person P2 who speaks a second language converse in information services at a hotel or similar establishment.

[0035] Figure 2 is a block diagram showing the functional configuration of the information display system 1. The CPU 5 of the computer 4 reads and executes the program 10 stored in the memory unit 6. This program 10 is a program that starts a conversation support application in the computer 4. In addition to the program 10, the memory unit 6 stores a map image 11 and word information 12. Details of the word information 12 will be described later.

[0036] For example, when person P1 or person P2 instructs the user interface to start the conversation support application, CPU 5 reads program 10 from memory unit 6 and executes it. By executing program 10, CPU 5 functions as an operation input unit 20, a language setting unit 21, a voice input unit 22, a voice recognition unit 23, an automatic translation unit 24, a person identification unit 25, a display control unit 26, and a voice output unit 29. The conversation support application functions as each of these units operates within CPU 5. This conversation support application has a real-time automatic translation function.

[0037] The operation input unit 20 receives user input for the user interface. For example, the operation input unit 20 receives language setting operations from person P1 or person P2. The operation input unit 20 also receives instructions to display or hide the map image 11 on the display unit 2.

[0038] The language setting unit 21 sets two languages, specified by person P1 or person P2, as the target languages ​​for translation from among multiple languages. That is, the language setting unit 21 sets the first language and the second language as the target languages ​​for translation and sets the system to a state where automatic translation from the first language to the second language and automatic translation from the second language to the first language can be performed. Once the target languages ​​for translation are set by the language setting unit 21, the preparatory operations for conversation support are completed, and the voice input unit 22 starts voice input.

[0039] The audio input unit 22 activates the microphone 7 and acquires the sound detected by the microphone 7 as an audio signal. The audio input unit 22 outputs this audio signal to the voice recognition unit 23 and the person identification unit 25.

[0040] The speech recognition unit 23 analyzes the audio signal output from the audio input unit 22 to recognize the speech. First, the speech recognition unit 23 identifies whether the speech detected by the microphone 7 is in the first language or the second language through speech analysis. Once it has identified whether it is the first or second language, the speech recognition unit 23 performs speech recognition based on the identified language and converts the audio signal into text data. Then, the speech recognition unit 23 outputs the generated text data to the automatic translation unit 24 and the display control unit 26.

[0041] The automatic translation unit 24 is equipped with a translation engine and automatically translates the text data output from the speech recognition unit 23 in real time. The automatic translation unit 24 is configured with a first language and a second language as the target languages ​​for translation by the language setting unit 21. When the automatic translation unit 24 acquires text data, it identifies whether it is the first language or the second language. If it is the first language, the automatic translation unit 24 translates the first language into the second language. If it is the second language, the automatic translation unit 24 translates the second language into the first language. The automatic translation unit 24 then generates text data showing the result of the automatic translation. The automatic translation unit 24 then outputs this text data to the display control unit 26. Note that the automatic translation unit 24 does not necessarily have its own translation engine and may perform automatic translation using a translation engine installed on an external server or the like.

[0042] Figures 3(a) and 3(b) show the text generated by the speech recognition unit 23 and the automatic translation unit 24. For example, as shown in Figure 3(a), when person P1 speaks in the first language, the speech recognition unit 23 outputs the first language text 41, which is the text data of the first language. This first language text 41 is supplied to the automatic translation unit 24. When the automatic translation unit 24 receives the first language text 41 as input, it performs automatic translation from the first language to the second language and generates the second language text 42, which is the text data of the second language. Therefore, when person P1, who speaks the first language, speaks, the first language text 41 generated by the speech recognition unit 23 and the second language text 42 generated by the automatic translation unit 24 are output to the display control unit 26.

[0043] Furthermore, as shown in Figure 3(b), for example, when person P2 speaks in the second language, the speech recognition unit 23 outputs the second language text 42, which is the text data of the second language. This second language text 42 is supplied to the automatic translation unit 24. When the automatic translation unit 24 receives the second language text 42 as input, it performs automatic translation from the second language to the first language and generates the first language text 41, which is the text data of the first language. Therefore, when person P2, who speaks the second language, speaks, the second language text 42 generated by the speech recognition unit 23 and the first language text 41 generated by the automatic translation unit 24 are output to the display control unit 26.

[0044] Returning to Figure 2, the person identification unit 25 performs frequency analysis of the audio signal output from the audio input unit 22 to determine whether the person who spoke is person P1 or person P2. However, the method used by the person identification unit 25 to identify a person is not necessarily limited to frequency analysis of the audio signal. For example, the person identification unit 25 may identify the person who spoke based on whether the language identified by the speech recognition unit 23 is the first language or the second language. Once the person identification unit 25 has identified the person who spoke, it notifies the display control unit 26 of the measured person.

[0045] When the display control unit 26 acquires the text output from the speech recognition unit 23 and the automatic translation unit 24, it outputs that text to the display unit 2 for display. At this time, the display control unit 26 switches the area in which the text is displayed according to the person who spoke.

[0046] Figure 4 shows an example of a screen displayed on the display unit 2 by the display control unit 26. In Figure 4, person P1 asks person P2 in the first language (English), "Where is the nearest station?", and person P2 replies in the second language (Japanese), "Please go straight past the hotel." In this case, the display control unit 26 displays the content of person P1's statement in the first display area 31 of the display unit 2, and displays the content of person P2's statement in the second display area 32 of the display unit 2. Specifically, the display control unit 26 displays the first language text 41 indicating the content of what person P1 said, and the second language text 42 which is a translation of the first language text 41, in the first display area 31. The display control unit 26 also displays the second language text 42 indicating the content of what person P2 said, and the first language text 41 which is a translation of the second language text 42, in the second display area 32.

[0047] At this time, the display control unit 26 displays the first language text 41 facing person P1 and the second language text 42 facing person P2. That is, since person P1 and person P2 are looking at the transparent screen 3 from opposite sides, the display control unit 26 displays one of the first language text 41 and the second language text 42 with the image reversed horizontally. As a result, person P1, who is looking at one side of the transparent screen 3, can see the first language text 41 normally, and person P2, who is looking at the other side of the transparent screen 3, can also see the second language text 42 normally.

[0048] Person P1 can determine whether their statements are being accurately recognized by the information display system 1 by viewing the first language text 41 displayed in the first display area 31. Person P2 can also understand the content of Person P1's statements by reading the second language text 42 displayed in the first display area 31, even if they do not understand the first language.

[0049] Furthermore, person P2 can determine whether their statements are being accurately recognized by the information display system 1 by viewing the second language text 42 displayed in the second display area 32. Also, person P1 can understand the content of person P2's statements by reading the first language text 41 displayed in the second display area 32, even if they do not understand the second language.

[0050] When the display control unit 26 outputs the first language text 41 and the second language text 42 to the display unit 2, it can also output these texts to the audio output unit 29 and output them as sound from the speaker 8. For example, the display control unit 26 outputs text data generated by automatic translation by the automatic translation unit 24 to the audio output unit 29. In this case, the audio output unit 29 reads aloud the text generated by the automatic translation unit 24 and outputs it as sound from the speaker 8.

[0051] Therefore, this information display system 1 supports smooth communication between two people, P1 and P2, who speak different languages.

[0052] Incidentally, when person P2 is giving directions to person P1, they sometimes think it would be easier to explain the route while referring to a map. In such cases, person P2 operates the user interface and instructs it to display a map image. When the operation input unit 20 receives the instruction from person P2 to display a map image, it instructs the display control unit 26 to display the map image 11. Based on that instruction, the display control unit 26 reads the map image 11 from the storage unit 6 and displays it on the display unit 2. If multiple map images 11 are stored in the storage unit 6, the display control unit 26 reads the map image 11 specified by the user and displays it on the display unit 2.

[0053] Figure 5 shows an example of the screen of the display unit 2 displaying the map image 11. For example, as shown in Figure 5, the display control unit 26 displays the map image 11 in a third display area 33 located in the center between the first display area 31 and the second display area 32. When the map image 11 is displayed on the display unit 2, people P1 and P2 can converse while looking at the same map image 11, allowing for smooth conversation, such as giving directions.

[0054] On the other hand, when the map image 11 is displayed on the transparent screen 3 between person P1 and person P2, the map image 11 seen by person P1 and the map image 11 seen by person P2 are reversed horizontally. Therefore, for example, if person P2 gives directions by saying "Turn right" while looking at the map image 11, those directions will be incorrect on the map image 11 seen by person P1. For example, as shown in Figure 5, even if person P2 tells person P1 "Go straight, and when you get to the hospital, turn right," the map image 11 seen by person P1 is reversed horizontally, so person P1 will be given the wrong directions.

[0055] Therefore, when the display control unit 26 of this embodiment is supporting a conversation between person P1 and person P2, it turns on the word replacement mode when a predetermined condition is met. The word replacement mode is an operation mode in which, if the first language text 41 or the second language text 42 contains a word that points to a specific direction, that word is automatically replaced with another word. The predetermined condition is, for example, in this embodiment, when the map image 11 is displayed on the display unit 2.

[0056] When the word replacement mode is turned on and the operating mode is switched, the display control unit 26 activates the word detection unit 27 and the replacement unit 28. The word detection unit 27 detects words that point to a specific direction contained in the first language text 41 or second language text 42 that are to be displayed. The word detection unit 27 reads word information 12 from the storage unit 6 and detects words registered in that word information 12.

[0057] Figure 6 shows an example of word information 12. As shown in Figures 6(a) and 6(b), word information 12 has a table for the first language and a table for the second language. Figure 6(a) shows the table for the first language. For example, when the word detection unit 27 detects a word from the first language text 41, it refers to the table for the first language. In the table for the first language, words that indicate a specific direction are pre-registered as specific words. For example, in Figure 6(a), "left," "right," "east," and "west" are registered as specific words. These specific words are words that indicate the left or right direction in the map image 11 in the first language. By referring to this table, the word detection unit 27 detects words that indicate the left or right direction from the first language text 41 as words that indicate a specific direction.

[0058] Figure 6(b) shows a table related to the second language. For example, when the word detection unit 27 detects words from the second language text 42, it refers to the table related to the second language. In the table related to the second language, words that indicate a specific direction are pre-registered as specific words. For example, in Figure 6(b), "right," "left," "west," and "east" are registered as specific words. These specific words are words that indicate the left or right direction in the map image 11 in the second language. By referring to this table, the word detection unit 27 detects words that indicate the left or right direction from the second language text 42 as words that indicate a specific direction.

[0059] Specific words registered in word information 12 can be added as needed by person P2 or others performing operations on computer 4. For example, person P2 can register a word indicating a specific direction in word information 12 by specifying a language and performing a registration operation for a specific word. Specific words registered in word information 12 can also be deleted.

[0060] When the word detection unit 27 detects a word from the first language text 41 or the second language text 42, it activates the replacement unit 28. However, even if the word detection unit 27 performs word detection processing based on the word information 12, it may not detect a word that points in a specific direction. In that case, the replacement unit 28 does not function in the display control unit 26.

[0061] The replacement unit 28 replaces a specific word that indicates a particular direction with another word when detected by the word detection unit 27. Specifically, the replacement unit 28 replaces the specific word with a word that indicates the opposite direction. For example, if the word detection unit 27 detects the word "right," the replacement unit 28 replaces the word "right" with the word "left." This allows person P1 and person P2 to appropriately convey the direction they are pointing to each other when they are having a conversation while looking at a map image 11 that is horizontally flipped in half.

[0062] When replacing a word, the replacement unit 28 refers to the word information 12 to identify the word to be replaced. The tables shown in Figures 6(a) and 6(b) register word numbers with the opposite meaning for a given word. By referring to these word numbers, the replacement unit 28 identifies a word with the opposite meaning to the given word. For example, for the given word "left" shown in Figure 6(a), the word number "2" is registered as the word with the opposite meaning. Therefore, the replacement unit 28 can identify the word "right" as the word with the opposite meaning to the given word "left". The replacement unit 28 then replaces the word "left" in the first language text 41 with the word "right" and updates the first language text 41. Also, if a given word is detected in the second language text 42, the replacement unit 28 replaces the word based on the table in Figure 6(b) and updates the second language text 42.

[0063] In word replacement mode, when a word indicating a specific direction is replaced with a word indicating the opposite direction, the display control unit 26 outputs the first language text 41 or second language text 42 with the replaced word to the display unit 2. As a result, the display unit 2 displays the first language text 41 or second language text 42 with the replaced word.

[0064] For example, as shown in Figure 5, if person P2 says to person P1, "Go straight, and when you reach the hospital, turn right," the word detection unit 27 detects the word "right" as a word indicating a specific direction. The replacement unit 28 replaces the word "right" with the word "left." As a result, the first language text 41 displayed in the second display area 32 of the display unit 2 becomes the text shown in Figure 5. That is, in the second display area 32, the text displayed as part of person P2's statement instructs person P1 to turn left. Therefore, when people P1 and P2 are conversing while looking at a map image 11 that is horizontally inverted, if either person makes a statement about left or right directions, the direction included in that statement can be horizontally inverted and conveyed to the other person. Thus, people P1 and P2 can communicate smoothly.

[0065] Next, we will explain the specific processing procedure in computer 4. Figures 7 to 9 are flowcharts illustrating an example of a processing procedure performed by computer 4. This procedure is carried out by the execution of program 10 on CPU 5. In other words, this procedure is carried out by the launch of the conversation support application on computer 4.

[0066] When the conversation support application is launched, computer 4 accepts language selection (step S10). For example, person P2 inputs their own language and the language spoken by person P1 as the first and second languages ​​to computer 4. Computer 4 accepts the input operation from person P2 and sets the first and second languages ​​(step S11). Note that person P1 may also perform the language input operation instead of person P2. After setting the languages, computer 4 initially turns off the word replacement mode (step S12). Then computer 4 starts real-time translation of the conversation between person P1 and person P2 (step S13).

[0067] Computer 4 determines whether or not it has detected voice input from person P1 or person P2 (step S14). If voice input is detected (YES in step S14), computer 4 activates the person identification unit 25 to identify the person who made the statement (step S15). Computer 4 also activates the speech recognition unit 23 to perform speech recognition (step S16). This speech recognition generates a first language text 41 or a second language text indicating the content of what person P1 or person P2 said.

[0068] Computer 4 automatically translates the first language text 41 or the second language text 42 generated by speech recognition (step S17). If the first language text 41 is generated by speech recognition, the second language text 42 is generated by automatic translation. Also, if the second language text 42 is generated by speech recognition, the first language text 41 is generated by automatic translation.

[0069] When performing automatic translation, computer 4 determines whether word replacement mode is on or off (step S18). If word replacement mode is on (YES in step S18), computer 4 performs word replacement (step S19). Word replacement is the process of replacing a word that points to a specific direction with a word that points to the opposite direction. Details of the word replacement process will be described later. On the other hand, if word replacement mode is off (NO in step S18), computer 4 does not perform word replacement.

[0070] Next, computer 4 performs display processing (step S20). Display processing is the process of displaying the first language text 41 or the second language text 42 on the display unit 2. Details of the display processing will be described later.

[0071] When the computer 4 displays the first language text 41 or the second language text 42 on the display unit 2, it determines whether the conversation between person P1 and person P2 has ended (step S21). For example, if person P2 instructs the conversation support application to end, the computer 4 determines that the conversation between person P1 and person P2 has ended. If the conversation has not ended (NO in step S21), the computer 4 returns to step S14. Therefore, each time it detects voice input from person P1 or person P2, the computer 4 repeatedly executes the processes in steps S15 to S20.

[0072] If computer 4 does not detect voice input from person P1 or person P2 (NO in step S14), it determines whether it has detected an operation by person P1 or person P2 (step S22). If it has detected an operation by person P1 or person P2 (YES in step S22), computer 4 determines whether it is a command to display the map image 11 (step S23). If it is a command to display the map image 11 (YES in step S23), computer 4 reads the map image 11 specified by person P1 or person P2 and displays the map image 11 in the third display area 33 of the display unit 2 (step S24). This allows person P1 and person P2 to continue their conversation while looking at the map image 11 displayed on the display unit 2. Once computer 4 has displayed the map image 11 on the display unit 2, it sets the word replacement mode to ON (step S25). Furthermore, if the instruction given by person P1 or person P2 is not an instruction to display map image 11 (NO in step S23), the processes in steps S24 and S25 are not performed.

[0073] Next, the computer 4 determines whether the instruction from person P1 or person P2 is an instruction to hide the map image 11 (step S26). If the instruction to hide is received while the map image 11 is being displayed on the display unit 2 (YES in step S26), the computer 4 terminates the display of the map image 11 on the display unit 2 (step S27). As a result, the display unit 2 returns to a state where the map image 11 is not being displayed. As the map image 11 on the display unit 2 is switched to hidden, the computer 4 switches the word replacement mode to off (step S28). If the instruction from person P1 or person P2 is not an instruction to hide the map image 11 (NO in step S26), the processes in steps S27 and S28 are not performed.

[0074] When the conversation support application is running on computer 4, the processes described in steps S14 to S28 are repeatedly executed.

[0075] Figure 8 is a flowchart showing an example of a detailed processing procedure for the word replacement process (step S19). When computer 4 starts the word replacement process, it acquires the text generated by speech recognition (step S16) and translation process (step S17) (step S30). Computer 4 reads the word information 12 (step S31) and performs a specific word detection process to detect a specific word from the text acquired in step S30 (step S32). Then computer 4 determines whether or not it has detected a specific word (step S33). If the specific word is not detected from the first language text 41 or the second language text 42 (NO in step S33), the word replacement process by computer 4 ends. In this case, the words contained in the first language text 41 and the second language text 42 are not replaced.

[0076] On the other hand, if a specific word is detected (YES in step S33), the computer 4 refers to the word information 12 and finds a word with the opposite meaning to the detected word (step S34). The computer 4 then replaces the word indicating a specific direction detected in the first language text 41 or the second language text 42 with a word indicating the opposite direction (step S35). The computer 4 overwrites and updates the first language text 41 or the second language text 42 with the replaced word (step S36). Through this process, if the content of a statement by person P1 or person P2 contains a word indicating a specific direction, that word is replaced with a word indicating the opposite direction.

[0077] Figure 9 is a flowchart showing an example of a detailed processing procedure for the display process (step S20). When the computer 4 starts the display process, it acquires the text to be displayed (step S40). That is, the computer 4 acquires the first language text 41 or the second language text 42 to be displayed.

[0078] When computer 4 acquires text, it identifies the display area for the acquired text based on the person identified in step S15 (step S41). For example, if the text was generated based on a statement by person P1, computer 4 identifies the first display area 31 as the display area for the text. If the text was generated based on a statement by person P2, computer 4 identifies the second display area 32 as the display area for the text.

[0079] Next, the computer 4 determines whether or not it is necessary to horizontally flip the text when displaying it (step S42). For example, if the text to be displayed is the first language text 41, the computer 4 determines that horizontal flipping is not necessary. On the other hand, if the text to be displayed is the second language text 42, the computer 4 determines that horizontal flipping is necessary. If it is determined that horizontal flipping is necessary (YES in step S42), the computer 4 performs horizontal flipping of the text (step S43). For example, when the second language text 42 is displayed in the first display area 31 or the second display area 32, horizontal flipping of the text is performed. As a result, the second language text 42 is displayed on the display unit 2 in a manner that is easy for person P2 to read. If horizontal flipping of the text is not necessary (NO in step S42), horizontal flipping of the text is not performed.

[0080] Next, computer 4 determines whether a word in the text to be displayed has been replaced by a word replacement process (step S19) (step S44). If a word has been replaced (YES in step S44), computer 4 identifies the replaced word from the text to be displayed (step S45). Then computer 4 performs a word emphasis process on the identified word (step S46). The word emphasis process applies different character decoration to the replaced word in the text to be displayed compared to other words. Such word emphasis processes include making the word bold, highlighting, underlining, and enclosing it in a rectangular frame. If a word in the text to be displayed has not been replaced (NO in step S44), steps S45 and S46 are not performed.

[0081] The computer 4 then outputs the text to be displayed to the display unit 2 and displays it (step S47). Figure 10 shows an example of a screen displayed on the display unit 2 by the display processing (step S20). For example, if the statement of person P2 contains a word that points in a specific direction, the word contained in the first language text 41 and the second language text 42 displayed in the second display area 32 of the display unit 2 is replaced with another word. At this time, as shown in Figure 10, the replaced word 49 is highlighted.

[0082] Person P1 looks at the first language text 41 displayed in the second display area 32 and understands what Person P2 said. Person P2 also looks at the second language text 42 displayed in the second display area 32 and understands whether what he said has been accurately reflected. At this time, the word 49 that has been replaced by the computer 4 is highlighted in the second display area 32. Therefore, Person P2 can understand that the word "right" that he uttered has been replaced with "left," and can understand that the replaced word 49 is not a misrecognition by the computer 4.

[0083] Furthermore, the computer 4 may output the first language text 41 or the second language text 42 displayed on the display unit 2 by the display control unit 26 to the audio output unit 29. In this case, the audio output unit 29 reads the first language text 41 or the second language text 42 aloud and outputs the sound from the speaker 8. Even in this case, the audio output unit 29 reads the text with the words replaced. Therefore, person P1 or person P2 can correctly understand what the other person is saying by listening to the sound output from the speaker 8.

[0084] Furthermore, when the computer 4 turns on the word replacement mode, it is preferable to display that it is in word replacement mode on a part of the display unit 2. By displaying that the word replacement mode is on, person P1 or person P2 can recognize that a word pointing in a specific direction has been automatically replaced with another word.

[0085] As described above, the information display system 1 is positioned between a first person P1 who speaks the first language and a second person P2 who speaks the second language, and includes a display unit 2 that can be seen by both the first and second people. The information display system 1 also translates the statements made by the first person P1 to generate second language text 42, and translates the statements made by the second person P2 to generate first language text 41. The information display system 1 then displays the first language text 41 and the second language text 42 on the display unit 2, thereby supporting smooth conversation between person P1 and person P2. In particular, if the first language text 41 or the second language text 42 contains a word that points to a specific direction, the information display system 1 replaces that word with a word that points to the opposite direction and displays it on the display unit 2. Therefore, the information display system 1 can prevent a specific direction from being misunderstood by person P1 and person P2 during their conversation. In other words, this information display system 1 can help ensure that the content of statements regarding a specific direction is accurately conveyed to the other party in a conversation between a first person P1 who speaks a first language and a second person P2 who speaks a second language.

[0086] (Second Embodiment) Next, a second embodiment of the present invention will be described. In the first embodiment described above, the display unit 2 is configured as a transparent display equipped with a transparent screen 3, and an example was described in which person P1 and person P2 converse while viewing different display surfaces of the transparent display. In contrast, in this embodiment, an example will be described in which the display unit 2 is configured as a non-transparent display, and person P1 and person P2 converse while viewing the same display surface of the display unit 2.

[0087] Figure 11 shows an example of the configuration of the information display system 1 in the second embodiment. Similar to the first embodiment, this information display system 1 includes a display unit 2 positioned between person P1 and person P2. However, the display unit 2 in this embodiment is configured as a non-transparent display. The display unit 2 in this embodiment is composed of a display device such as a tablet terminal, and has a display surface 2a on one side of the display device. Persons P1 and P2 will converse while viewing the same display surface 2a of the display unit 2. However, person P1 and person P2 view the display unit 2 from different directions. Therefore, when an image is displayed on the display unit 2, the image viewed by person P2 is inverted vertically and horizontally compared to the image viewed by person P1.

[0088] The computer 4 of this embodiment has the same configuration as in the first embodiment. When the display control unit 26 displays the first language text 41 or the second language text 42 on the display unit 2, it displays one of them upside down.

[0089] Figure 12 shows an example of a screen displayed on the display unit 2 by the display control unit 26. In Figure 12, person P1 speaks to person P2 in the first language (English) and asks, "Where is the nearest station?", to which person P2 replies in the second language (Japanese) and says, "Please go straight past the hotel." The display control unit 26 displays the content of person P1's statement in the first display area 31 of the display unit 2 and the content of person P2's statement in the second display area 32 of the display unit 2. At this time, the display control unit 26 displays the first language text 41 indicating the content of what person P1 said and the second language text 42 which is a translation of the first language text 41 in the first display area 31. The display control unit 26 also displays the second language text 42 indicating the content of what person P2 said and the first language text 41 which is a translation of the second language text 42 in the second display area 32.

[0090] Furthermore, the display control unit 26 displays the first language text 41 facing person P1 and the second language text 42 facing person P2. That is, since person P1 and person P2 are looking at the display unit 2 from opposite sides, the display control unit 26 displays one of the first language text 41 or the second language text 42 upside down. In the example in Figure 12, the second language text 42 is displayed upside down. As a result, person P1 can read the first language text 41 normally, and person P2 can also read the second language text 42 normally.

[0091] When the display control unit 26 displays the map image 11 on the display unit 2 based on instructions from person P1 or person P2, it turns on the word replacement mode. The display control unit 26 then activates the word detection unit 27 and the replacement unit 28. In the word replacement mode, the word detection unit 27 and the replacement unit 28 replace a word that points to a specific direction with a word that points to the opposite direction.

[0092] Figure 13 shows an example of word information 12 in this embodiment. Figure 13(a) shows a table relating to the first language. For example, when the word detection unit 27 detects words from the first language text 41, it refers to the table relating to the first language. In the table relating to the first language, words that indicate a specific direction are pre-registered as specific words. For example, in Figure 13(a), "left," "right," "up," and "down" are registered as specific words. These specific words are words that indicate the left, right, up, and down directions of the map image 11 in the first language. By referring to this table, the word detection unit 27 detects words that indicate the left, right, up, or down directions from the first language text 41 as words that indicate a specific direction.

[0093] Figure 13(b) shows a table related to the second language. For example, when the word detection unit 27 detects words from the second language text 42, it refers to the table related to the second language. In the table related to the second language, words that indicate specific directions are pre-registered as specific words. For example, in Figure 13(b), "right," "left," "up," and "down" are registered as specific words. These specific words are words that indicate the right, left, up, and down directions in the map image 11 in the second language. By referring to this table, the word detection unit 27 detects words that indicate the right, left, up, or down directions from the second language text 42 as words that indicate specific directions.

[0094] When the word detection unit 27 detects a specific word that points to a particular direction, the replacement unit 28 replaces that specific word with another word. That is, the replacement unit 28 refers to the word information 12 and replaces the specific word with a word that points to the opposite direction. For example, if the word detection unit 27 detects the word "down", the replacement unit 28 replaces the word "down" with the word "up". Also, for example, if the word detection unit 27 detects the word "right", the replacement unit 28 replaces the word "right" with the word "left". This makes it possible to appropriately convey the direction that person P1 and person P2 are pointing to each other when they are having a conversation while looking at a map image 11 that is inverted vertically and horizontally.

[0095] In word replacement mode, when a word indicating a specific direction is replaced with a word indicating the opposite direction, the display control unit 26 outputs the first language text 41 or second language text 42 with the replaced word to the display unit 2. As a result, the display unit 2 displays the first language text 41 or second language text 42 with the replaced word.

[0096] Figure 14 shows an example of a screen displayed on the display unit 2 by the display control unit 26. For example, if the statement of person P2 includes a word that points in a specific direction, the word in the first language text 41 and the second language text 42 displayed in the second display area 32 is replaced with another word 49. At this time, as shown in Figure 14, the replaced word 49 is highlighted.

[0097] Person P1 looks at the first language text 41 displayed in the second display area 32 and understands what Person P2 said. Person P2 also looks at the second language text 42 displayed in the second display area 32 and understands whether what he said has been accurately reflected. At this time, the word 49 that has been replaced by the computer 4 is highlighted in the second display area 32. Therefore, Person P2 can understand that the word "down" that he uttered has been replaced with "up," and can understand that the replaced word 49 is not a misrecognition by the computer 4.

[0098] Therefore, the information display system 1 of this embodiment can appropriately convey the direction that each person P1 and person P2 is pointing to when they are having a conversation while looking at a map image 11 displayed on a single display unit 2 from opposite directions.

[0099] Furthermore, the other configurations and operations in this embodiment are the same as those described in the first embodiment.

[0100] (modified version) Preferred embodiments of the present invention have been described above. However, the present invention is not limited to those described in the above embodiments, and various modifications are applicable.

[0101] For example, in the above embodiment, when the map image 11 is displayed on the display unit 2, the word replacement mode is turned on, and the computer 4 replaces a word indicating a specific direction with a word indicating the opposite direction. However, the timing at which the word replacement mode is turned on in the information display system 1 is not necessarily limited to when the map image 11 is displayed on the display unit 2. For example, the information display system 1 may turn on the word replacement mode when displaying a specific image on the display unit 2. In this case, the specific image is not limited to the map image 11. Also, the information display system 1 may turn on the word replacement mode when there is an instruction from person P1 or person P2. Furthermore, the information display system 1 may always have the word replacement mode turned on when the conversation support application is running.

[0102] Furthermore, the above embodiment illustrates a case where the program 10 executed by the CPU 5 is pre-stored in the storage unit 6. However, the program 10 is not limited to being pre-stored in the storage unit 6. That is, the program 10 can be traded on its own. In that case, the program 10 may be provided in a form that can be downloaded via a network such as the Internet. Alternatively, the program 10 may be provided in a form that is recorded on a computer-readable recording medium such as a USB memory stick or CD-ROM. [Explanation of Symbols]

[0103] 1. Information display system 2 Display section 5 CPU 6 Memory section 10 Programs 11 Map image 12 Word Information 20 Operation Input Section 21 Language Settings Section 22 Voice input section 23. Voice Recognition Unit 24. Machine Translation Section 25 Person identification part 26 Display Control Unit 27 Word detection unit 28 Replacement part 29 Audio output section

Claims

1. A display unit is placed between a first person who speaks the first language and a second person who speaks the second language, and is visible to both the first and second people. An automatic translation unit that translates the statements of the first person to generate a second language text, and translates the statements of the second person to generate a first language text, A display control unit that displays the first language text and the second language text on the display unit, and, if the first language text or the second language text contains a word that points to a specific direction, replaces that word with a word that points to the opposite direction and displays it on the display unit, An information display system characterized by comprising the following features.

2. The information display system according to claim 1, characterized in that the aforementioned specific direction includes the left direction and the right direction.

3. The information display system according to claim 2, characterized in that the display unit is a transparent display, and a first person and a second person are each made to view different display surfaces of the transparent display.

4. The information display system according to claim 3, characterized in that the display control unit displays either the first language text or the second language text by horizontally reversing it.

5. The information display system according to claim 1, characterized in that the automatic translation unit can select a first language and a second language to be translated from among multiple languages.

6. A voice output unit that reads aloud and outputs the first language text or the second language text displayed by the display control unit. The information display system according to claim 1, further comprising the following:

7. The information display system according to claim 6, characterized in that the voice output unit reads out and outputs the replaced word when the word indicating a specific direction is replaced with a word indicating the opposite direction.

8. A voice input unit that inputs speech from the first person and the second person as audio, A person identification unit analyzes the audio input to the aforementioned audio input unit to identify whether the person who spoke is the first person or the second person, The information display system according to claim 1, further comprising the following:

9. The information display system according to claim 8, characterized in that, based on the results of person identification by the person identification unit, the display area of ​​the first language text and the display area of ​​the second language text are displayed separately.

10. The information display system according to claim 1, characterized in that when the display control unit replaces a word indicating a specific direction with a word indicating the opposite direction, it highlights the replaced word in the first language text or the second language text.

11. The information display system according to claim 1, characterized in that the aforementioned specific directions include the left direction, the right direction, the up direction, and the down direction.

12. The information display system according to claim 11, characterized in that the display unit is a non-transparent display, and the same display surface is made visible to a first person and a second person, respectively.

13. The information display system according to claim 2 or 11, characterized in that when the display control unit displays a map image in a part of the display area of ​​the display unit, it switches to a word replacement mode that detects whether or not the first language text or the second language text contains a word indicating a specific direction.

14. The information display system according to claim 13, characterized in that when the display control unit switches to the word replacement mode, it displays on a part of the display unit that it is in the word replacement mode.

15. A storage unit that stores word information relating a word that points to the aforementioned specific direction and a word that points to the opposite direction to the aforementioned specific direction. Furthermore, The information display system according to claim 1, characterized in that the display control unit determines, based on the word information, whether or not the first language text or the second language text contains a word indicating a specific direction.

16. The information display system according to claim 15, characterized in that the words registered in the word information can be added or deleted.

17. A display control method for controlling a display unit that is positioned between a first person who speaks a first language and a second person who speaks a second language, and which is commonly visible to both the first and second people, An automatic translation step that translates the statement of the first person to generate a second language text, and translates the statement of the second person to generate a first language text, A display control step which involves displaying the first language text and the second language text on the display unit, and if the first language text or the second language text contains a word that points to a specific direction, replacing that word with a word that points to the opposite direction and displaying it on the display unit; A display control method characterized by having the following features.

18. A program to be executed in an information display system which is positioned between a first person who speaks a first language and a second person who speaks a second language, and which has a display unit that can be seen in common by both the first person and the second person, wherein the information display system An automatic translation step that translates the statement of the first person to generate a second language text, and translates the statement of the second person to generate a first language text, A display control step which involves displaying the first language text and the second language text on the display unit, and if the first language text or the second language text contains a word that points to a specific direction, replacing that word with a word that points to the opposite direction and displaying it on the display unit; A program characterized by causing the execution of a specific action.

Images