Translation and Communication System

Abstract

To provide a technique that enables the output of voice of an appropriate language out of an original language voice and a translated voice without requiring the trouble of manually switching time slots.SOLUTION: A translation communication system 1 includes: a management device 12 including a translation processing unit for generating a translated voice of an original language voice, a slot control unit for generating slot language information indicating a language type assigned to each time slot, and a management device communication unit for transmitting the original language voice and the translated voice in the time slots assigned to each and also transmitting the slot language information, on the downlink; and a terminal device 16 including a storage unit for storing information on a language used preset in the terminal device 16, a terminal device communication unit for receiving the original language voice, translated voice, and slot language information on the downlink, a selection unit for selecting a time slot based on the slot language information, and a voice output unit for outputting a voice signal of the selected time slot as voice.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a translation communication system and a terminal device. [Background technology]

[0002] A translation communication system has been developed in which a translation device translates voices transmitted in different languages ​​from each terminal device in group communication between multiple terminal devices and transmits the translated voice to the other terminal devices. For example, Patent Document 1 describes a translation communication system that can switch between an original language and a translated language in a voice call. In the translation communication system disclosed in Patent Document 1, the called mobile phone receives voice data in the original language in a first time slot and receives translated voice data in a second time slot. Either the original language voice data or the translated voice data is selected by a user operation, and the selected voice is output from a speaker. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-79852 A Summary of the Invention [Problem to be solved by the invention]

[0004] In a translation communication system, when a calling mobile terminal is switched to another mobile terminal, the language may be switched to a language different from the one that was being spoken up until then. In this case, in the conventional translation communication system as described above, in order to output the voice of the desired language from the speaker of the called mobile terminal, the user must manually and selectively switch the time slot in accordance with the language switch, which is a time-consuming task.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide a technology that enables output of speech in the appropriate language, either the original language speech or the translated speech, without the need to manually switch time slots. [Means for solving the problem]

[0006] A translation communication system of one embodiment of the present invention comprises a management device including a translation processing unit that generates a translated speech of an original speech received via an uplink from a terminal device; a slot control unit that assigns the original speech and the translated speech to different time slots that are time-division multiplexed and generates slot language information indicating the language type assigned to each time slot; and a management device communication unit that transmits the original speech and the translated speech in their assigned time slots and transmits the slot language information via a downlink of a different frequency than the uplink; and a terminal device including a memory unit that stores information on a language used that has been pre-set in the terminal device; a terminal device communication unit that receives the original speech, the translated speech, and the slot language information via the downlink; a selection unit that selects the time slot that corresponds to the information on the set language used based on the slot language information; and an audio output unit that outputs the audio signal of the selected time slot as audio.

[0007] Another aspect of a terminal device of the present invention comprises a memory unit that stores information on a predetermined language used, a communication unit that receives original language audio and translated audio of the original language audio in different time slots that are time-division multiplexed, and receives slot language information indicating the language type of each time slot from a management device, a selection unit that selects the time slot that corresponds to the set language used information based on the slot language information, and an audio output unit that outputs the audio signal of the selected time slot as audio.

[0008] Any combination of the above components or mutual substitution of the components or expressions of the present invention between methods, devices, systems, etc. are also effective as aspects of the present invention. Effect of the Invention

[0009] According to the present invention, it is possible to output speech in an appropriate language from the original speech and the translated speech without the need for the effort of manually switching time slots. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating a translation and communication system according to an embodiment. [Figure 2A] FIG. 2 is a diagram showing an outline of speech translation communication in the translation communication system of FIG. 1. [Figure 2B] FIG. 2 is a diagram showing an outline of speech translation communication in the translation communication system of FIG. 1. [Diagram 3] 2 is a diagram showing a hardware configuration and functional blocks of the mobile terminal of FIG. 1. [Figure 4] 2 is a diagram illustrating a hardware configuration and functional blocks of a management device of FIG. 1. [Diagram 5] FIG. 5 is a diagram showing an example of a terminal management table of FIG. [Figure 6] FIG. 11 is a sequence diagram illustrating a registration process in the translation communication system. [Figure 7] 10 is a sequence diagram illustrating a communication process between mobile terminals when a PTT-on operation is performed. FIG. [Figure 8] 11 is a diagram illustrating an example of frame data transmitted on the downlink after transmission of header information. FIG. [Figure 9] 11 is a sequence diagram illustrating a communication process between mobile terminals when a PTT-off operation is performed. FIG. [Figure 10] 13 is a diagram illustrating an example of frame data transmitted on the downlink when a PTT-off operation is performed. FIG. [Figure 11] 10 is a sequence diagram illustrating a communication process between mobile terminals when a PTT-on operation is performed in a state in which a speech channel is acquired. FIG. [Figure 12]13 is a diagram illustrating an example of frame data transmitted on the downlink when a PTT-on operation is performed while the speech channel is released. FIG. [Figure 13] FIG. 11 is a sequence diagram illustrating a communication process between mobile terminals in the second embodiment. [Figure 14] FIG. 13 is a sequence diagram illustrating a communication process between mobile terminals in the third embodiment. [Figure 15] FIG. 2 is a diagram illustrating an example of frame data transmitted on the downlink when transmitting a text message. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this specification and the drawings, elements having substantially the same functions and configurations are denoted by the same reference numerals to avoid repetitive explanations, and elements not directly related to the present invention are not shown.

[0012] (First embodiment) FIG. 1 is a schematic diagram of a translation communication system 1 according to the present embodiment. The translation communication system 1 includes a management device 12, a relay station 14, and three mobile terminals 16A to 16C, which may be collectively referred to as a mobile terminal 16. The management device 12 and the mobile terminals 16 are connected to each other via the relay station 14 and a communication network NW. The communication network NW is, for example, an IP (Internet Protocol) network. Note that, for the sake of simplicity, only one relay station 14 is shown in the example of FIG. 1, but two or more relay stations may be provided. Similarly, although three mobile terminals 16A to 16C are shown in the example of FIG. 1, two or more mobile terminals 16 may be provided. The mobile terminal 16 in the present embodiment is an example of a terminal device.

[0013] The translation communication system 1 has one or more call channels. Communication in the translation communication system 1 is performed using a time division multiple access (TDMA) system. In particular, the translation communication system 1 performs downlink and uplink at different frequencies. The uplink and downlink each have a number of time slots that are time division multiplexed. In this embodiment, the uplink and downlink each have two time slots, a first time slot and a second time slot.

[0014] The mobile terminal 16 is also called an IP transceiver or a PoC (Push-to-Talk over Cellular) transceiver, and performs voice communication. The voice communication is a push-to-talk type, and individual calls, group calls, and mass calls are also possible. In order to perform such voice communication, the mobile terminal 16 is connected to a relay station 14. In this embodiment, half-duplex communication is used as a communication method between the relay station 14 and the mobile terminal 16. The management device 12 performs sequence processing when a call is made between the mobile terminals 16. In addition, the management device 12 in this embodiment has a voice translation function.

[0015] In such a configuration, a user of a mobile terminal 16 presses a talk button on the mobile terminal 16 to perform a voice call with another user using another mobile terminal 16. In half-duplex communication, while any user of multiple mobile terminals 16 involved in a certain voice call has pressed the talk button and is speaking, the other users cannot speak even if they press the talk button.

[0016] 2A and 2B show an overview of speech translation communication in the translation communication system 1. Here, communication is assumed between three mobile terminals 16, namely, first to third mobile terminals 16A to 16C, but communication may be between three or more mobile terminals 16, or one-to-many group communication. The first mobile terminal 16A is used by a first user 18A, the second mobile terminal 16B is used by a second user 18B, and the third mobile terminal 16C is used by a third user 18C. The first to third users 18A to 18C may be collectively referred to as users 18. FIG. 2A shows an example of communication when the first user 18A presses the talk button of the first mobile terminal 16A. FIG. 2B shows an example of communication when the third user 18C presses the talk button of the third mobile terminal 16C. Although communication between each mobile terminal 16 and management device 12 is performed via relay station 14, relay station 14 is omitted in FIGS. 2A and 2B for the sake of simplicity.

[0017] As shown in FIG. 2A, when the first user 18A utters an English voice such as "Hello", the original voice is transmitted from the first mobile terminal 16A to the management device 12 using the uplink frequency. The management device 12 has information on the language used by each mobile terminal 16 registered in advance. For example, the management device 12 registers the language used by the first and second mobile terminals 16A and 16B as English, and the language used by the third mobile terminal 16C as Japanese. The management device 12 executes a translation process for translating the received original English voice into Japanese, which is the language used by the third mobile terminal 16C. The management device 12 transmits the original English voice and the translated Japanese voice to the second and third mobile terminals 16B and 16C using the downlink frequency in different time slots. The second mobile terminal 16B selects a time slot in which the original English voice, which is the language used by the second mobile terminal 16B, is used, from among the downlink time slots, and outputs the voice of "Hello" by playing back the original English voice. The third mobile terminal 16C selects a time slot for the translation speech in Japanese, which is the language used by the third mobile terminal 16C, from among the downlink time slots, and outputs the speech of "Hello" translated into Japanese, "Konjō," by playing back the translated speech in Japanese. In this case, because translation takes time, the timings at which the third mobile terminal 16C starts and ends transmission of the translated speech translated into Japanese are delayed compared to the timings at which the second mobile terminal 16B starts and ends transmission of the original language speech.

[0018] As shown in FIG. 2B, when the third user 18C utters a Japanese voice such as "Hello", the voice is transmitted from the third mobile terminal 16C to the management device 12 using the uplink frequency. The management device 12 executes a translation process for translating the received voice into English, which is the working language of the first and second mobile terminals 16A and 16B. The management device 12 transmits the original Japanese voice and the translated English voice to the first and second mobile terminals 16A and 16B using the downlink frequency in different time slots. The first and second mobile terminals 16A and 16B select a time slot in which the translated voice in English, which is their working language, is used from among the downlink time slots, and output the voice of "Hello", which is the English translation of "Hello", by playing back the translated English voice.

[0019] 3 shows a hardware configuration and functional blocks of the mobile terminal 16. The mobile terminal 16 includes a CPU (Central Processing Unit) 110, a user IF (Interface) 120, a communication IF 130, a voice IF 140, and a storage unit 150. The CPU 110, the user IF 120, the communication IF 130, the voice IF 140, and the storage unit 150 are connected by a bus 160.

[0020] The user IF 120 is an interface for the user 18. The user IF 120 accepts information from the user 18 and presents information to the user 18. The communication IF 130 is connected to the communication network NW and communicates with the relay station 14. Furthermore, the communication IF 130 communicates with the management device 12 via the relay station 14. The storage unit 150 is a medium for storing information, and is, for example, a hard disk or an SSD (Solid State Drive).

[0021] The CPU 110 includes a control unit 111 and a selection unit 112. The user IF 120 includes an input unit 121 and a display unit 122. Although omitted in FIG. 3, the user IF 120 may include a light-emitting device (light-emitting unit) such as an LED. The communication IF 130 includes a communication unit 131. The audio IF 140 is an interface for inputting and outputting call audio, and includes an audio input unit 141 and an audio output unit 142. The audio input unit 141 is composed of, for example, a microphone, and the audio output unit 142 is composed of, for example, a speaker.

[0022] The control unit 111 executes various controls in the mobile terminal 16. The selection unit 112 selects whether to use the received original language speech or the translated speech by selecting a time slot that uses a language previously set in the mobile terminal 16, based on slot language information described below.

[0023] The input unit 121 includes various buttons operated by the user 18 of the mobile terminal 16, an input device such as a touch panel, and a call button that is pressed when transmitting voice. For example, when the call button is pressed in a push-to-talk (PTT) off state, which is a receiving state, the state becomes a PTT on state, which is a transmitting state, while the button is kept pressed, and when the call button is released, the state returns to a PTT off state, which is a receiving state. Hereinafter, the operation of pressing the call button to turn the PTT on state is sometimes referred to as a PTT on operation, and the operation of releasing the call button to turn the PTT off state is sometimes referred to as a PTT off operation. The pressed state of the call button is detected by the input unit 121 and input to the control unit 111. The display unit 122 is an interface that displays the settings and call state of the mobile terminal 16, and is, for example, a display.

[0024] The communication unit 131 can communicate with other mobile terminals 16 using a mobile phone system, a business wireless system, a wireless LAN (Local Area Network) connected to the Internet, etc. The communication unit 131 transmits and receives audio data to and from other mobile terminals 16 in a half-duplex communication system. The communication unit 131 of this embodiment is an example of a terminal device communication unit.

[0025] The voice input unit 141 picks up voice based on the speech of the user 18 of the mobile terminal 16 to make a voice call. The voice output unit 142 outputs the call voice. The voice output unit 142 outputs the voice signal of the time slot selected by the selection unit 112 as voice.

[0026] Storage unit 150 stores various programs for executing various controls in control unit 111. Storage unit 150 stores the mobile terminal name, an individual ID for identifying mobile terminal 16, a group ID for identifying a group to which mobile terminal 16 belongs, and language information including information on the language (Japanese, English, Spanish, etc.) used in mobile terminal 16. The mobile terminal name, individual ID, group ID, and language information are set in advance.

[0027] 4 shows a hardware configuration and functional blocks of the management device 12. The management device 12 includes a CPU 200, a communication IF 210, and a storage unit 220. The CPU 200, the communication IF 210, and the storage unit 220 are connected by a bus 230.

[0028] The CPU 200 executes the process in the management device 12. The CPU 200 includes a control unit 201 and a translation processing unit 202. The control unit 201 includes a communication control unit 203, a channel control unit 204, and a slot control unit 205. The translation processing unit 202 generates a translated voice by translating an original voice received from one of the multiple mobile terminals 16 into a language different from the language used in the original voice. The communication control unit 203 executes various controls required for communication with the mobile terminal 16 in a half-duplex communication method. The channel control unit 204 executes a control process related to the use of a channel (relay station 14) for calls. The slot control unit 205 assigns the original voice and the translated voice to different time slots that are time-division multiplexed. The slot control unit 205 of this embodiment assigns the original voice to a first time slot and the translated voice to a second time slot. In addition, the slot control unit 205 generates slot language information indicating the language type assigned to each time slot.

[0029] The communication IF 210 is connected to the communication network NW and communicates with the relay station 14. Furthermore, the communication IF 210 communicates with the mobile terminal 16 via the relay station 14. The communication IF 210 includes a communication unit 211 that transmits and receives various data. For example, the communication unit 211 transmits original language audio on the uplink. Furthermore, the communication unit 211 transmits original language audio and translated audio in their assigned time slots on the downlink, and also transmits slot language information. The communication unit 211 in this embodiment is an example of a management device communication unit.

[0030] The storage unit 220 stores various programs for executing various controls and processes in the control unit 201 and the translation processing unit 202. The storage unit 220 is a medium for storing information, such as a hard disk or SSD. The storage unit 220 includes a terminal management table 221. In this terminal management table 221, various information about the mobile terminal 16 is recorded. FIG. 5 shows an example of the terminal management table 221. For example, in the terminal management table 151, the mobile terminal name, individual ID, group ID, and used language information are stored for each mobile terminal 16.

[0031] Next, a sequence process performed between mobile terminal 16 and management device 12 in this embodiment will be described. First, registration process S10 in translation communication system 1 will be described with reference to Fig. 6. In this example, for simplicity of explanation, the registration process related to first mobile terminal 16A will be described.

[0032] When the communication unit 131 of the first mobile terminal 16A receives a report signal from the relay station 14 (step S11), the control unit 111 recognizes that the terminal itself is located within the coverage of the relay station 14 and starts location registration processing. The communication unit 131 transmits a location registration request message including the individual ID, group ID, and language information of the terminal itself to the relay station 14 (step S12). The relay station 14 that has received the location registration request message transfers the received location registration request message to the management device 12 in order to determine whether or not to permit location registration (step S13).

[0033] The communication control unit 203 of the management device 12 judges whether or not to permit the location registration of the first mobile terminal 16A, i.e., whether or not to permit communication in the local system by the first mobile terminal 16A (step S14). If the location registration is permitted, the communication unit 211 transmits a location registration permission response message to the relay station 14 (step S15). At this time, the communication control unit 203 additionally registers the individual ID, group ID and used language information of the first mobile terminal 16A in the storage unit 220. The relay station 14 that has received the location registration permission response message transfers the location registration permission response message to the first mobile terminal 16A (step S16).

[0034] The control unit 111 of the first mobile terminal 16A that has received the location registration permission response message completes the location registration (step S17). This enables calls within the coverage (site) of the wireless communication system. Thereafter, the registration process S10 ends. The second and third mobile terminals 16B and 16C also perform the same operation, enabling calls at the same site.

[0035] Next, the communication process S100 between the mobile terminals when the PTT-on operation is performed will be described with reference to FIG. 7. FIG. 7 shows an example in which the first mobile terminal 16A transmits a speech. In this example, it is assumed that the location registration of the first to third mobile terminals 16A to 16C has been completed, and the languages ​​used by the first to third mobile terminals 16A to 16C are set to English, English, and Japanese in the terminal management table 221, respectively. In this example, the downlink includes a first time slot TSa and a second time slot TSb. Note that the communication between each mobile terminal 16 and the management device 12 is performed via a relay station 14, but the relay station 14 is omitted in FIG. 7 for the sake of simplicity. Similarly, the relay station 14 is omitted in the following FIG. 9, FIG. 11, FIG. 13, and FIG. 14.

[0036] When the first mobile terminal 16A detects a PTT-on operation from the input unit 121 (step S101), it starts a voice call (group call). Next, the communication unit 131 transmits a voice transmission start request message including the individual ID and group ID of the first mobile terminal 16A to the management device 12, and requests acquisition of a call channel (step S102).

[0037] The communication control unit 203 of the management device 12 that has received the voice transmission start request message determines whether or not the communication is possible (step S103). If the channel control unit 204 permits the communication (acquisition of a speech channel), it starts the transmission using the speech channel (relay station 14) (step S104A). At the same time, the communication unit 211 transmits a transmission start permission response message to the first mobile terminal 16A and the other mobile terminals 16 (the second and third mobile terminals 16B and 16C) waiting with the group ID designated by the first mobile terminal 16A (step S104B). This transmission start permission response message includes an OK message to the voice transmission start request message, the individual ID and group ID of the mobile terminal 16 (the first mobile terminal 16A) that is the sender, channel information indicating the frequency of the speech channel to be used, uplink slot designation information designating the uplink time slot for transmitting voice in the uplink, and language information for each mobile terminal 16.

[0038] When the control unit 111 of the first mobile terminal 16A receives the transmission start permission response message, it determines an uplink time slot based on the uplink slot designation information (step S105). When the other mobile terminals 16 present in the same site that are waiting with the group ID designated by the first mobile terminal 16A receive the transmission start permission response message, they recognize that a call request has been sent to their own terminal.

[0039] When the control unit 111 of each mobile terminal 16 receives the transmission start permission response message, it starts a receiving (frequency switching) operation of the designated call channel based on the channel information (step S106). Since the call button is kept pressed with the designated call channel set, the communication unit 131 of the first mobile terminal 16A starts voice transmission on the designated call channel and time slot, and transmits header information (step S107). This header information indicates that it is a voice transmission start message, and includes the individual ID and group ID of the first mobile terminal 16A that is the sender.

[0040] The translation processing unit 202 of the management device 12 determines the type of language after translation by referring to the information of the language set in the other mobile terminal 16 using the terminal management table 221 based on the group ID of the header information (step S108). Here, the translation processing unit 202 determines to translate into Japanese, which is the language used by the third mobile terminal 16C. Based on this determination result, the slot control unit 205 assigns the original language voice and the translated voice to different time slots and generates slot language information indicating the language type assigned to each time slot (step S109). Here, the slot control unit 205 designates the first downlink time slot TSa as the time slot for the original language voice in English and the second time slot TSb as the time slot for the translated voice in Japanese. The communication unit 211 adds the slot language information to the received header information and transmits this header information to each mobile terminal 16 in the first downlink time slot TSa, which is the time slot for the original language voice (step S110). In this example, the slot language information is added to the header information transmitted at the beginning of the transmission, but the present invention is not limited to this. For example, it is possible to add the slot language information to an IDLE message that does not contain voice data, which will be described later, and transmit the message. Furthermore, the time slot for transmitting the slot language information may be a time slot other than that for the original voice.

[0041] When the selection unit 112 of the second and third mobile terminals 16B and 16C receives the header information of the first downlink time slot TSa, which is a time slot for original language voice, the selection unit 112 compares the language used for each time slot indicated in the slot language information of the header information with the language used set in the terminal. The selection unit 112 of the second and third mobile terminals 16B and 16C selects a time slot using a language that matches the set language used as a receiving time slot, and starts a receiving operation in the receiving time slot (step S111). The communication unit 131 of the first mobile terminal 16A transmits the original language voice in English based on the speech of the first user 18A in FIG. 2A to the management device 12 in the uplink time slot (step S112).

[0042] The translation processing unit 202 of the management device 12 translates the received original speech into the language determined in step S108 (step S113). In this embodiment, the original speech in English is translated into a translated speech in Japanese.

[0043] The communication unit 211 transmits the voice of each language to the second and third mobile terminals 16B and 16C in a different time slot for each language (step S114). The communication unit 211 repeats (loops back) and transmits the original English voice as is using the first time slot TSa (step S114A). The voice output unit 142 of the second mobile terminal 16B outputs the English voice by playing back the original voice received from the first time slot TSa determined as the receiving time slot. On the other hand, the third mobile terminal 16C discards the original voice received from the first time slot TSa which is different from the second time slot TSb determined as the receiving time slot.

[0044] Since the above-mentioned translation process takes time, there are cases where the translated speech has not yet been generated immediately after the start of the translation process. In this case, the communication unit 211 transmits an IDLE message that does not include speech data in the second time slot TSb immediately after the start of the translation process (step S114B). Meanwhile, the communication unit 211 transmits the original speech in English that continues from step S114A in the first time slot TSa (step S114C). Next, when the translation process is completed and the translated speech has been generated, the communication unit 211 transmits the translated speech in the second time slot TSb (step S114D). The speech output unit 142 of the third mobile terminal 16C outputs the Japanese speech by reproducing the translated speech received from the second time slot TSb. Meanwhile, the second mobile terminal 16B discards the translated speech received from the second time slot TSb. Thereafter, the communication unit 211 repeats transmission of the original English voice and the translated voice using the first and second time slots TSa and TSb, respectively (steps S114E, S114F, etc.) until the first user 18A finishes speaking and turns off the PTT. This makes it possible to carry out group communication using multiple languages ​​on the speech channel.

[0045] Fig. 8 illustrates an example of frame data transmitted in the downlink after the header information is transmitted in step S110. The frame data in Fig. 8 includes a first time slot TSa and a second time slot TSb. The frame data in Fig. 8 includes, from the beginning of the time slot, header information, slot language information Slot, IDLE message, original speech, IDLE message, original speech, and translated speech (hereinafter, the original speech and the translated speech are repeated). In the example of Fig. 8, the original speech is placed in the first time slot TSa, which is the third from the beginning, but the translated speech is placed in the second time slot TSb, which is the sixth from the beginning, due to a delay in the translation process. Until the translated speech is generated, an IDLE message is placed in the second time slot TSb, just like the second and fourth second time slots TSb from the beginning.

[0046] Next, the communication process S200 between mobile terminals when the PTT is turned off will be described with reference to Fig. 9. Fig. 9 is assumed to be a continuation from step S114 of voice transmission in Fig. 7.

[0047] When a PTT-off operation of the first mobile terminal 16A is detected during group communication using multiple languages ​​on a speech channel (step S201), the communication unit 131 transmits a transmission end signal to the management device 12 (step S202). The communication unit 211 of the management device 12 first transmits a call-end message for notifying the end of the call using the first time slot TSa for the original language voice for which voice data transmission has been completed (step S203). The control unit 111 of the second mobile terminal 16B that receives the call-end message determines the end of reception and mutes the speaker. At this time, if the translation process into another language or the transmission of the translated voice is not completed, the communication unit 211 continues the transmission of the translated voice in the second time slot TSb of another language (step S114 following step S203). While the transmission of the translated speech is continuing, the communication unit 211 sets an IDLE message in the first time slot TSa and transmits it to the first and second mobile terminals 16A and 16B (step S204).

[0048] When the transmission of the translated voice is completed, the communication unit 211 transmits a call-end message in the second time slot TSb for the translated voice (step S205). The control unit 111 of the third mobile terminal 16C that receives the call-end message determines the end of reception and mutes the speaker. This ends the calls in all time slots. After this, the communication channel acquisition state continues for a certain period of time in order to continue waiting for transmission by PTT-on operation from each mobile terminal 16 using the same frequency. During this time, the communication unit 211 transmits an IDLE message to each mobile terminal 16 (step S206). The IDLE message reporting state in which calls in all time slots have ended is a state in which all mobile terminals 16 are able to start voice transmission (communication channel acquisition state).

[0049] An example of frame data transmitted in the downlink when the PTT-off operation is performed in step S201 will be described with reference to FIG. 10. When the PTT-off operation is performed in the first mobile terminal 16A, the first mobile terminal 16A transmits a call-end message (call-end MSG) in the uplink. When the management device 12 receives the call-end message, it transmits the call-end message in the first time slot TSa of the downlink that appears immediately before. Thereafter, an IDLE message is transmitted in the first time slot TSa. In the second time slot TSb of the downlink, translated speech A is transmitted until the translation process of the original speech A from the first mobile terminal 16A is completed, and when the translation process is completed, the call-end message is transmitted. Thereafter, an IDLE message is transmitted in the second time slot TSb.

[0050] Next, the communication process S300 between mobile terminals when a PTT-on operation is performed in the speech channel acquisition state will be described with reference to Fig. 11. Fig. 11 continues from step S206 of IDLE message transmission in Fig. 9. Therefore, Fig. 11 starts from a state in which each mobile terminal 16 has acquired a speech channel. Steps S302 to S309 are basically the same as steps S107 to S114 in Fig. 7 except where specifically mentioned, and therefore the description thereof may be omitted.

[0051] When the third mobile terminal 16C performs a PTT-on operation (step S301) during the IDLE message broadcast (step S206), the communication unit 131 of the third mobile terminal 16C starts voice transmission as a response to the speech of the first mobile terminal 16A (step S302). Here, the third mobile terminal 16C uses Japanese, which is a language different from the language used by the first mobile terminal 16A. The voice transmission of the third mobile terminal 16C has header information at the beginning of the transmission data, similar to the voice transmission of the first mobile terminal 16A.

[0052] The management device 12 executes a process of determining a translation language (step S303), a process of specifying a time slot for each language (step S304), and a process of transmitting header information to which language information for each time slot is added (step S305). In step S304, the slot control unit 205 specifies the first time slot TSa of the downlink as a time slot for original Japanese speech, and specifies the second time slot TSb as a time slot for translated English speech.

[0053] The control units 111 of the first and second mobile terminals 16A and 16B determine the second time slot TSb as the receiving time slot based on the slot language information (step S306). The communication unit 131 of the third mobile terminal 16C transmits the original Japanese voice based on the speech of the third user 18C in Fig. 2B to the management device 12 in the uplink time slot (step S307).

[0054] The translation processing unit 202 of the management device 12 executes a translation process to translate the original Japanese speech into the translated English speech (step S308). The communication unit 211 executes a transmission process of the original speech and the translated speech (step S309). The communication unit 211 repeats (loops back) and transmits the original speech as it is using the first time slot TSa (steps 309A, C). At this time, both the first and second mobile terminals 16A and 16B select the second time slot TSb as the reception time slot. Therefore, the first and second mobile terminals 16A and 16B discard the original speech. When the translation process is not completed, the communication unit 211 transmits an IDLE message in the second time slot TSb (step S309B). When the translated speech is generated, the communication unit 211 transmits the translated speech in the second time slot TSb (step 309D). The audio output unit 142 of the first and second mobile terminals 16A and 16B outputs English audio by playing back the translated audio received from the second time slot TSb. Thereafter, the communication unit 211 repeats the transmission of the original audio and the translated audio. After that, when the third user 18C finishes speaking and turns off the PTT of the third mobile terminal 16C, the same process as that described above in FIG. 9 is executed with the PTT off operation (S201) performed on the third mobile terminal 16C.

[0055] Here, in the voice transmission of FIG. 7, Japanese and English are assigned to the first and second time slots TSa and TSb, respectively. On the other hand, in the voice transmission of FIG. 11, English and Japanese are assigned to the first and second time slots TSa and TSb, respectively. In this way, every time the mobile terminal 16 that turns on the PTT and speaks is switched, the languages ​​assigned to the first and second time slots TSa and TSb of the downlink may be different. However, slot language information is transmitted to each mobile terminal 16 in steps S110 and S305. This makes it possible to know the language used in each time slot based on the slot language information. Therefore, each mobile terminal 16 can determine an appropriate reception time slot without having to manually switch the reception time slot.

[0056] Referring again to FIG. 10, an example of frame data transmitted in the downlink when the PTT-on operation is performed in step S301 will be described. When the PTT-on operation is performed in the third mobile terminal 16C in the speech channel acquisition state, the third mobile terminal 16C transmits its own header information (header information C) in the uplink. The management device 12 that has received the header information C determines the time slot to be used for each language. In the state in which the speech channel is acquired, the management device 12 continues to use the first time slot TSa that was used for the original language speech A until then for the original language speech, and determines the first time slot TSa as the time slot for the language speech (original language speech C) of the third mobile terminal 16C. The management device 12 transmits the header information C and the slot language information Slot using the first time slot TSa in the downlink. After that, the management device 12 transmits the original language speech C in the first time slot TSa and transmits the translated speech (translated speech C) in the second time slot TSb, as described above in FIG. 8.

[0057] With reference to FIG. 12, an example of frame data transmitted in the downlink when a PTT-on operation is performed while the speech channel is released will be described. If a call-off message is transmitted to each mobile terminal 16 and no call (transmission) is made by each mobile terminal 16 through a PTT-on operation even after a preset certain time has elapsed, the speech channel is disconnected and released (channel release). If a PTT-on operation is then performed by, for example, the third mobile terminal 16C, the speech channel is acquired again. In this case, depending on the system situation, the second time slot TSb that has been used for the translated voice may be switched to the time slot for the original voice as shown in FIG. 12. Even in such a case, each mobile terminal 16 can check and select the time slot that it should receive by referring to the slot language information. Note that even when a speech channel is acquired again, the time slots for the original voice and the translated voice may not be switched, and the same time slot allocation as that of the previous communication may continue to be used. Even in this case, it is possible to select an appropriate time slot by referring to the slot language information.

[0058] The operation and effect of this embodiment will be described. In a group call in which multiple users who use different languages ​​receive original language speech and translated speech in separate time slots in a communication system equipped with a translation function, the language transmitted from the mobile terminal 16 may be switched by the caller. Accordingly, the time slots for the original language speech and the translated speech may also be switched, and it is necessary to manually switch the time slots in the mobile terminal 16 on the called side to output speech in the desired language. This has led to a problem in that the switching operation is time-consuming.

[0059] In this embodiment, the management device 12 transmits slot language information indicating the language type assigned to each time slot to the mobile terminal 16. Therefore, the mobile terminal 16 can automatically select a receiving time slot corresponding to the information on the language used that is preset in the mobile terminal 16 based on the slot language information, and output the desired audio from the selected time slot. With this configuration, it is possible to output audio in the appropriate language from the original audio and the translated audio, without the need to manually switch between time slots.

[0060] In this embodiment, the communication unit 131 of the mobile terminal 16 transmits used language information including information on the used language set in the mobile terminal 16 to the management device 12, and the translation processing unit 202 determines the type of language after translation based on the used language information transmitted from the terminal device 16. With this configuration, the type of language after translation is predetermined in the translation processing unit 202, so it becomes possible to easily determine the type of language after translation.

[0061] Second embodiment A second embodiment of the present invention will be described below. In the drawings and description of the second embodiment, the same or equivalent components and members as those of the first embodiment are denoted by the same reference numerals. Explanations that overlap with the first embodiment will be omitted as appropriate, and the description will focus on the configurations that differ from the first embodiment.

[0062] Because it takes time to translate voice data, the timing of sending and receiving the translated voice is basically delayed with respect to the timing of sending and receiving the original voice. As a result, for example, the following disadvantages (1) and (2) are expected to occur. (1) Such delayed transmission of the translated voice will increase the usage time of the speech channel. (2) There is a possibility that the timing between the completion of reception and the response will differ between the mobile terminal 16 receiving the translated voice and the mobile terminal 16 receiving the original voice.

[0063] In order to eliminate or reduce the above disadvantages, in this embodiment, the translation processor 202 increases the playback speed of the speech data in the translation language after the transmission of the original speech is completed. This makes it possible to reduce the delay until the reception of the translated speech is completed.

[0064] The communication process S400 between mobile terminals in this embodiment will be described with reference to Fig. 13. Fig. 13 is assumed to continue from step S114 of voice transmission in Fig. 7. Steps S401 to S403 and S405 to S407 are basically the same as steps S201 to S203 and S204 to S206 in Fig. 9 described above, except where specifically noted, and therefore the description thereof may be omitted.

[0065] After steps S401 to S403, the translation processing unit 202 of the management device 12 makes the playback speed of the translated speech generated in the translation process faster than the playback speed of the translated speech transmitted before the transmission of the original language speech was completed. For example, the translation processing unit 202 makes the playback speed of the translated speech faster than normal as long as the call recipient can hear it. For example, the translation processing unit 202 makes the playback speed of the translated speech faster at the point in time when the end of transmission of the original language speech is confirmed by receiving the transmission end signal in step S402. To prevent the translation processing unit 202 from becoming difficult to hear due to an increase in the playback speed, the translation processing unit 202 may detect the beginning and end of sentences and words and adjust the timing at which the playback speed of the translated speech is changed.

[0066] Third embodiment Hereinafter, a third embodiment of the present invention will be described. In the drawings and description of the third embodiment, the same or equivalent components and members as those of the first embodiment are denoted by the same reference numerals. Explanations that overlap with the first embodiment will be omitted as appropriate, and the description will focus on the configurations that differ from the first embodiment.

[0067] As described above, the timing of transmitting and receiving the translated voice is basically delayed with respect to the timing of transmitting and receiving the original voice. Furthermore, after the management device 12 finishes transmitting the original voice, the time slot for the original voice is no longer used, and an IDLE message is transmitted in the time slot for the original voice. In this embodiment, the time slot for the original voice that is no longer used after the transmission of the original voice is finished is effectively utilized, and while the delayed translated voice is being transmitted, a text message is transmitted in place of the IDLE message using this time slot for the original voice. By viewing the text message in addition to the translated voice, it is possible to improve the recognition and understanding of the translated language by the call recipient.

[0068] Regarding the registration process S10 in the translation communication system 1 in this embodiment, differences from the first embodiment will be described. In this embodiment, a text reception request for requesting reception of text data after the transmission end signal is received and the end of transmission of the original language speech is confirmed is added to the location registration request message in step S12 in Fig. 6. When management device 12 receives this text reception request, it further registers the text reception request when notifying relay station 14 of a location registration permission response message in step S15.

[0069] The communication process S500 between mobile terminals in this embodiment will be described with reference to Fig. 14. Fig. 14 is assumed to be continued from step S114 of voice transmission in Fig. 7. Steps S501 to S503 are basically the same as steps S201 to S203 in Fig. 9 described above, except where specifically mentioned, and therefore the description thereof may be omitted.

[0070] After steps S501 to S503, the translation processing unit 202 of the management device 12 converts the translation result into text data (step S504). Here, before the end of call message is transmitted in step S503, the first and second time slots TSa and TSb are in use for transmitting the original language voice and the translated voice, respectively. On the other hand, after the end of call message is transmitted in step S503, only the second time slot TSb for the translation language is in use, and the first time slot TSa is in an IDLE state. In this embodiment, the management device 12 transmits the text data in the translation language using this free first time slot TSa.

[0071] The slot control unit 205 changes the corresponding language of the first time slot TSa by making both the first and second time slots TSa and TSb time slots for Japanese (step S505). The slot control unit 205 also designates the first time slot TSa as a time slot for Japanese text data, and designates the second time slot TSb as a time slot for translated voice (Japanese voice). The slot control unit 205 regenerates slot language information indicating that both the first and second time slots TSa and TSb are time slots for Japanese. Although it is necessary to consider the continuity of the voice data, in step S505, the slot control unit 205 may designate the first time slot TSa as a time slot for translated voice (Japanese voice), and designate the second time slot TSb as a time slot for Japanese text data. The communication unit 211 transmits this regenerated slot language information in the first time slot TSa (step S506). The control unit 111 of the third mobile terminal 16C, which has received this slot language information, recognizes that transmission of text data of the translated speech will start in the first time slot TSa.

[0072] The selection unit 112 selects both the first and second time slots TSa and TSb as receiving time slots based on the received slot language information (step S507). That is, the third mobile terminal 16C, which has previously received only translated speech in one time slot, switches to receiving operation in two time slots. The third mobile terminal 16C continues to receive translated speech in the second time slot TSb, while newly receiving text data in the first time slot TSa.

[0073] The communication unit 211 transmits the text data in the first time slot TSa (steps S508A and S508B). When the transmission of the translated voice and text data is completed, the communication unit 211 transmits an end call message (step S509). The display unit 122 of the third mobile terminal 16C displays the text data when the reception of all the text data is completed (step S510).

[0074] With reference to FIG. 15, an example of frame data transmitted in the downlink when transmitting a text message will be described. When the first mobile terminal 16A performs a PTT-off operation and transmits an end-of-call message in the first time slot TSa, slot language information Slot2 is transmitted in the next first time slot TSa. This slot language information Slot2 indicates that the first time slot TSa is a Japanese text (Japanese Text) and the second time slot TSb is a Japanese voice (Japanese) time slot. Thereafter, a translation language is transmitted in the second time slot TSb, and text data TEXT is transmitted in the first time slot TSa. When transmission of the text data TEXT in the first time slot TSa is completed, an end-of-call message is transmitted in the second time slot TSb. Thereafter, both the first and second time slots TSa and TSb are in the IDLE state.

[0075] In this manner, in this embodiment, after the transmission of the original language voice is completed, the following processing is executed. That is, the translation processing unit 202 of the management device 12 generates text data of the translated voice. The slot control unit 205 regenerates slot language information in which the time slot used for the original language voice is set as the time slot for the text data. The communication unit 211 transmits the text data and the translated voice in different time slots, and also transmits the regenerated slot language information. The selection unit 112 of the mobile terminal 16 selects the time slot for the text data and the translated voice based on the regenerated slot language information. The display unit 112 displays the received text data. With this configuration, it is possible to improve the recognition and understanding of the translation language by the user 18 by referring to the text data.

[0076] (Modification) In the embodiment, the downlink has two time slots, a first time slot and a second time slot, but is not limited thereto, for example, three or more time slots may be used in the downlink.

[0077] In the embodiment, audio in two languages, English and Japanese, is communicated in two downlink time slots, but the present invention is not limited to this. For example, audio in three or more languages ​​may be communicated in three or more downlink time slots.

[0078] In the embodiment, the slot language information is communicated in the first time slot TSa, but is not limited to this, and may be communicated in, for example, the second time slot TSb.

[0079] The present invention has been described above with reference to the above-mentioned embodiment, but the present invention is not limited to the above-mentioned embodiment, and appropriate combinations or substitutions of each configuration are also included in the present invention. [Explanation of symbols]

[0080] 1...translation communication system, 12...management device, 14...relay station, 16...mobile terminal, 18...user, 110...CPU, 111...control unit, 112...selection unit, 120...user IF, 121...input unit, 122...display unit, 130...communication IF, 131...communication unit, 140...voice IF, 141...voice input unit, 142...voice output unit, 150...memory unit, 160...bus, 200...CPU, 201...control unit, 202...translation processing unit, 203...communication control unit, 204...channel control unit, 205...slot control unit, 210...communication IF, 211...communication unit, 220...memory unit, 221...terminal management table, 230...bus.

Claims

1. a translation processing unit that generates a translated speech of an original language speech received from a terminal device via an uplink; a slot control unit that allocates the original speech and the translated speech to different time slots that are time-division multiplexed, and generates slot language information that indicates the language type allocated to each time slot; a management device communication unit that transmits the original speech and the translated speech in the assigned time slots and also transmits the slot language information via a downlink having a frequency different from that of the uplink; A management device including: A storage unit that stores information on a language that is preset in the terminal device; a terminal device communication unit for receiving the original speech, the translated speech, and the slot language information via the downlink; a selection unit that selects the time slot corresponding to the set information of the used language based on the slot language information; an audio output unit that outputs the audio signal of the selected time slot as audio; A display unit that displays various information; and a terminal device including: After the management device has finished transmitting the original language audio, The translation processing unit of the management device generates text data of the translated speech, the slot control unit regenerates the slot language information so that the time slot used for the original voice is set as the time slot for the text data; the management device communication unit of the management device transmits the text data and the translated speech in different time slots, and transmits the regenerated slot language information; the selection unit of the terminal device selects a time slot for the text data and the translated speech based on the regenerated slot language information; The display unit displays the received text data. Translation communication system.

2. the terminal device communication unit of the terminal device transmits used language information including information on the set used language to the management device; 2. The translation and communication system according to claim 1, wherein the translation processing unit of the management device determines a type of language after translation based on the used language information transmitted from the terminal device.

3. After completing the transmission of the original language speech, the translation processing unit of the management device 3. The translation and communication system according to claim 1, wherein the playback speed of the translated speech is made faster than the playback speed before the transmission of the original speech was terminated.

Images