Caption telephone system, method, and program

Abstract

To provide a technique capable of realizing a caption telephone in a telephone communication between two users using a telephone number recorded to a SIM of an own telephone machine and a telephone number recorded to a SIM of a mating destination.SOLUTION: A user communication device 1 comprises: a sound information reception part 31 that transmits sound information of a mating destination in a telephone communication with a telephone number recorded to a SIM of a user telephone communication device 1 to a display device 3 in a radio communication, and in which the display device 3 receives the sound information transmitted in the radio communication; and a sound information transmission part 32 that transmits the sound information to be received to a sound recognition result acquisition device 4 via the user communication device 1. The sound recognition result acquisition device 4 acquires a sound recognition result of the sound information generated by an operator 5 or a sound recognition device 6, and transmits the sound recognition result to the display device 3 via the user communication device 1. The display device 3 further comprises: a sound recognition result reception part 33 that receives the sound recognition result; and the display part 34 that displays the sound recognition result to be received.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The disclosed technology relates to closed caption telephone technology. [Background technology]

[0002] As a conventional closed caption telephone technique, the technique described in Non-Patent Document 1 is known.

[0003] In the technology of Non-Patent Document 1, a closed-captioned telephone call was realized through a three-way call using the telephone number recorded in the SIM of one's own telephone, the telephone number recorded in the SIM of the other party's telephone, and a telephone number starting with 050. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] AIESEC Japan, "Subtitle telephone service", [online], [searched on August 8, 2023], Internet<URL:http: / / www.iscecj.co.jp / parts / pages / jimaku.html> Summary of the Invention [Problem to be solved by the invention]

[0005] The technology in Non-Patent Document 1 did not allow for closed caption telephone calls to be realized in two-way calls using a telephone number recorded in the SIM of one's own telephone and a telephone number recorded in the SIM of the other party's telephone.

[0006] This is because the voice of the other party in a call to a phone number recorded in the SIM of one's phone cannot be used for voice recognition within one's phone.

[0007] The disclosed technology aims to provide a subtitled telephone system, method, and program for realizing a subtitled telephone call in a two-way conversation using a telephone number recorded in the SIM of your telephone and a telephone number recorded in the SIM of the other party's telephone. [Means for solving the problem]

[0008] One aspect of the disclosed technology is a subtitled telephone system comprising a user communication device, a display device which is a single device, and a voice recognition result acquisition device, wherein the user communication device transmits voice information of the other party in a call using a telephone number recorded in the SIM of the user communication device to the display device via wireless communication, the display device includes a voice information receiving unit which receives the voice information transmitted via wireless communication and a voice information transmitting unit which transmits the received voice information to the voice recognition result acquisition device via the user communication device, the voice recognition result acquisition device acquires a voice recognition result of the voice information generated by an operator or a voice recognition device and transmits the voice recognition result to the display device via the user communication device, and the display device further includes a voice recognition result receiving unit which receives the voice recognition result and a display unit which displays the received voice recognition result. Effect of the Invention

[0009] According to the disclosed technology, a subtitled telephone call can be realized in a two-way call using a telephone number recorded in the SIM of one's own telephone and a telephone number recorded in the SIM of the other party's telephone. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional configuration of a closed caption telephone system. [Diagram 2] FIG. 2 is a diagram showing an example of a processing procedure of the closed caption telephone method. [Diagram 3] FIG. 3 is a diagram showing an example of a display on the display device 3. As shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of a display on the display device 3. As shown in FIG. [Diagram 5] FIG. 5 is a diagram illustrating an example of a functional configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the disclosed technology will be described with reference to the drawings. In the drawings, components having the same functions are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0012] [Subtitled telephone system and method] As shown in FIG. 1, the closed-caption telephone system includes a user communication device 1, a destination communication device 2, a display device 3, a voice recognition result acquisition device 4, and a voice recognition device 6, for example.

[0013] The closed-caption telephone method is realized, for example, by each component of the closed-caption telephone system performing the processes of steps S1 to S6 exemplified below.

[0014] <User communication device 1 and destination communication device 2> The user communication device 1 and the destination communication device 2 are communication devices that can make calls via a telephone network using telephone numbers recorded in SIMs. Examples of the user communication device 1 and the destination communication device 2 are communication devices such as landlines, mobile phones, smartphones, and tablet terminals.

[0015] The SIMs of the user communication device 1 and the counterparty communication device 2 may be SIM cards or eSIMs (embedded subscriber identity modules).

[0016] The user communication device 1 is a communication device used by a user of the closed caption telephone system.

[0017] The destination communication device 2 is a communication device used by a destination party to make a telephone call with the user.

[0018] It is assumed that the user communication device 1 is capable of wirelessly transmitting and receiving data to and from the display device 3. Examples of wireless communication include Bluetooth (registered trademark), Wi-Fi (registered trademark), Z-Wave (registered trademark), ZigBee, Thread, and Matter.

[0019] The user communication device 1 transmits voice information of the other party in a call using a telephone number recorded in the SIM of the user communication device 1 to the display device 3 by wireless communication (step S1).

[0020] A call made using a telephone number recorded in a SIM is a call made from a telephone number recorded in the SIM of the user communication device 1 to the destination communication device 2, or a call made from the destination communication device 2 to a telephone number recorded in the SIM of the user communication device 1.

[0021] <Display device 3> 1, the display device 3 is a communication device including a voice information receiving unit 31, a voice information transmitting unit 32, a voice recognition result receiving unit 33, and a display unit 34. Examples of the display device 3 are a smartphone and a tablet terminal. The display device 3 may be a device of a similar size to the user communication device 1.

[0022] The display device 3 may be removably stored in a case that stores the user communication device 1. For example, the user communication device 1 may be stored in one storage space of a double-sided case having two storage spaces, and the display device 3 may be removably stored in the other storage space. In this case, for example, when starting an operation using the display device 3, the user removes the display device 3 from the case and places the display device 3 in a position where the display on the display device 3 is easily viewable.

[0023] It is assumed that the voice information transmission unit 32 can connect to the Internet via the user communication device 1. More specifically, it is assumed that the voice information transmission unit 32 can connect to the Internet via a wireless communication connection with the user communication device 1.

[0024] The voice information receiving unit 31 receives the voice information transmitted from the user communication device 1 by wireless communication (step S2).

[0025] The voice information sending unit 32 sends the received voice information to the voice recognition result acquisition device 4 via the user communication device 1 (step S3). More specifically, the voice information sending unit 32 sends the voice information to the voice recognition result acquisition device 4 by connecting to the Internet via the user communication device 1.

[0026] For example, in a communication device that runs on Android OS version Android 6.0 or later, not all applications are given the authority to process voice information of a caller using the communication device. For this reason, some applications cannot process voice information of a caller using the communication device.

[0027] However, by transmitting voice information of the other party in a call using a telephone number recorded in the SIM of the user communication device 1 to the display device 3 via wireless communication, the display device 3 can use that voice information.

[0028] The display device 3 transmits voice information to the voice recognition result acquisition device 4 in order to obtain a highly accurate voice recognition result.

[0029] After the process of step S4 described later, the voice recognition result receiving unit 33 of the display device 3 receives the voice recognition result by connecting to the Internet via the user communication device 1 (step S5).

[0030] The display unit 34 of the display device 3 displays the voice recognition result received by the voice recognition result receiving unit 33 (step S6). The display unit 34 is a display such as a liquid crystal display or an organic EL display.

[0031] The display unit 34 of the display device 3 may display a display area in which the user can select the operator 5 or the voice recognition device 6. This display area may be one in which the user can select the operator 5 or the voice recognition device 6 during a call. An example of this display area will be described later. When the user selects the operator 5 or the voice recognition device 6 by operating the display area, the selection information is transmitted to the voice recognition result acquisition device 4 via the user communication device 1.

[0032] The voice information transmitting unit 32 may transmit the voice information to the voice recognition result acquisition device 4 via the user communication device 1 after the user selects the operator 5 or the voice recognition device 6 .

[0033] <Voice recognition result acquisition device 4> The voice recognition result acquisition device 4 receives voice information from the display device 3 .

[0034] The voice information is provided to an operator 5 or to a voice recognition device 6 .

[0035] The operator 5 is a human being who inputs characters corresponding to the voice information he or she has heard into the voice recognition result acquisition device 4 using input means such as a keyboard.

[0036] The voice recognition device 6 is a device that generates characters corresponding to input voice information by voice recognition.

[0037] The voice recognition result acquisition device 4 acquires the voice recognition result of the voice information generated by the operator 5 or the voice recognition device 6, and transmits the acquired voice recognition result to the display device 3 via the user communication device 1 (step S4).

[0038] The voice recognition result acquisition device 4 is a device composed of at least one communication device, server, etc. that has the function of acquiring the voice recognition result of voice information generated by the operator 5 or the voice recognition device 6, and transmitting the acquired voice recognition result to the display device 3 via the user communication device 1.

[0039] When the voice recognition result acquisition device 4 receives the selection information from the display device 3, the voice recognition result acquisition device 4 acquires the voice recognition result of the voice information generated by the operator 5 or the voice recognition device 6 selected by the user as determined by the received selection information. For this purpose, the voice recognition result acquisition device 4 may provide the voice information only to the operator 5 or the voice recognition device 6 selected by the user as determined by the received selection information. The acquired voice recognition result is transmitted to the display device 3 via the user communication device 1.

[0040] In this way, the subtitled telephone system and method make it possible to realize a subtitled telephone call in a two-way conversation using the telephone number recorded in the SIM of your telephone and the telephone number recorded in the SIM of the other party's telephone.

[0041] The closed-caption telephone system and method are effective for telephone conversations between elderly people and hearing-impaired people who can speak, for example, and therefore the user communication device 1 may be a communication device used by elderly people or hearing-impaired people who can speak.

[0042] [Example of display on display device 3] An example of the display on the display device 3 is shown in FIG.

[0043] 3, three buttons are displayed: a button labeled "Start Voice Recognition," a button labeled "End Voice Recognition," and a button labeled "Operator." These three buttons are an example of a display area in which the user can select the operator 5 or the voice recognition device 6.

[0044] [[If speech recognition device 6 is selected]] When a user presses a button labeled "Start voice recognition" before or during a call with the other party, the display device 3 transmits selection information indicating that the voice recognition device 6 has been selected to the voice recognition result acquisition device 4 via the user communication device 1.

[0045] The voice recognition result acquisition device 4 receives the voice information and the selection information. In this case, the selection information indicates that the voice recognition device 6 is selected, so the voice recognition result acquisition device 4 acquires the voice recognition result of the voice information generated by the voice recognition device 6, and transmits the acquired voice recognition result to the display device 3 via the user communication device 1.

[0046] The display unit 34 of the display device 3 displays the received voice recognition result in the voice recognition result area E1. As a result, as shown in FIG. 4, text corresponding to the voice recognition result is displayed in the voice recognition result area E1.

[0047] In addition, in order to indicate that the voice recognition device 6 has been selected, when the user presses the button labeled "Start Voice Recognition," the color of the button labeled "Start Voice Recognition" may change.

[0048] It should be noted that even when Operator 5 is selected, the user may be able to press a button labeled "Start voice recognition."

[0049] Conversely, when Operator 5 is selected, the user may not be able to press the button marked "Start voice recognition." In this case, when the user presses the button marked "End voice recognition," the user may be able to press the button marked "Start voice recognition." This can prevent erroneous operation by the user.

[0050] [[If Operator 5 is selected]] When a user presses a button labeled "operator" before or during a call with the other party, the display device 3 transmits selection information indicating that an operator 5 has been selected to the voice recognition result acquisition device 4 via the user communication device 1.

[0051] The voice recognition result acquisition device 4 receives the voice information and the selection information. In this case, the selection information indicates that the operator 5 has been selected, so the voice recognition result acquisition device 4 acquires the voice recognition result of the voice information generated by the operator 5, and transmits the acquired voice recognition result to the display device 3 via the user communication device 1.

[0052] The display unit 34 of the display device 3 displays the received voice recognition result in the voice recognition result area E1. As a result, as shown in FIG. 4, text corresponding to the voice recognition result is displayed in the voice recognition result area E1.

[0053] In addition, in order to indicate that the operator 5 has been selected, when the user presses the button labeled "operator," the color of the button labeled "operator" may change.

[0054] It should be noted that even when the voice recognition device 6 is selected, the user may be able to press a button labeled "operator."

[0055] Conversely, when the voice recognition device 6 is selected, the user may not be able to press the button marked "operator." In this case, the user may be able to press the button marked "operator" when the user presses the button marked "end voice recognition." This can prevent erroneous operation by the user.

[0056] For example, in this manner, the user may be able to change the selection of the voice recognition device 6 to the selection of the operator 5 before or during a call with the other party.

[0057] Since the number of operators 5 is limited, even if an operator 5 is selected, there are cases where the voice recognition result by the operator 5 cannot be obtained. Therefore, when the user selects an operator 5 or a voice recognition device 6, the display device 3 or the voice recognition result acquisition device 4 may determine whether the voice recognition result by the operator 5 can be obtained. Then, when it is determined that the voice recognition result by the operator 5 cannot be obtained, the display unit 34 of the display device 3 may display that effect.

[0058] The display device 3 or the voice recognition result acquisition device 4 may determine at a predetermined time interval whether a voice recognition result by the operator 5 can be acquired, and the user may be able to select the operator 5 only when it is determined that a voice recognition result by the operator 5 can be acquired. For example, the user may be able to press a button labeled "operator" only when it is determined that a voice recognition result by the operator 5 can be acquired.

[0059] In general, voice recognition by the voice recognition device 6 has the advantage of being faster than voice recognition by the operator 5, but has the disadvantage of more recognition errors. Conversely, voice recognition by the operator 5 has the disadvantage of being slower than voice recognition by the voice recognition device 6, but has the advantage of fewer recognition errors.

[0060] As explained in [[When the voice recognition device 6 is selected]] and [[When the operator 5 is selected]], the user may be able to change between the selection of the operator 5 and the selection of the voice recognition device 6 before or during a call with the other party. This has the advantage that the user can selectively use voice recognition by the voice recognition device 6 and voice recognition by the operator 5 depending on the importance of the call. For example, when the importance of the call is high, such as a call to consult a doctor, the user may select voice recognition by the operator 5, which is less likely to result in erroneous recognition.

[0061] [Variations] The specific configurations of the embodiments of the disclosed technology are not limited to the configurations described above. The specific configurations of the embodiments of the disclosed technology can be appropriately modified in design without departing from the spirit of the embodiments of the disclosed technology.

[0062] For example, the display unit 34 of the display device 3 may differentiate the text corresponding to the voice recognition result generated by the operator 5 from the text corresponding to the voice recognition result generated by the voice recognition device 6 in the voice recognition result area E1. The display unit 34 of the display device 3 may, for example, differentiate the font, color, background, etc. of these texts. This allows the user to easily distinguish between the text corresponding to the voice recognition result generated by the operator 5 and the text corresponding to the voice recognition result generated by the voice recognition device 6.

[0063] The display unit 34 of the display device 3 may further display the voice recognition result of the user's voice information in addition to the voice recognition result of the other party's voice information. This allows the user to visually confirm not only the voice recognition result of the other party's voice information but also the voice recognition result of the user's voice information, thereby increasing the convenience of the closed-caption telephone system and method.

[0064] For this purpose, for example, the user communication device 1 transmits the user's voice information together with the voice information of the other party in a call using a telephone number recorded in the SIM of the user communication device 1 to the display device 3 by wireless communication. In this case, the voice information receiving unit 31 of the display device 3 receives the voice information of the other party and the voice information of the user transmitted by wireless communication from the user communication device 1. In addition, the voice information transmitting unit 32 of the display device 3 transmits the voice information of the other party and the voice information of the user to the voice recognition result acquiring device 4 via the user communication device 1. The voice recognition result acquiring device 4 acquires the voice recognition result of the voice information of the other party and the voice recognition result of the user's voice information generated by the operator 5 or the voice recognition device 6, and transmits the voice recognition result of the voice information of the other party and the voice recognition result of the user's voice information to the display device 3 via the user communication device 1. In addition, the voice recognition result receiving unit 33 of the display device 3 receives the voice recognition result of the voice information of the other party and the voice recognition result of the user's voice information by an Internet connection via the user communication device 1. The display unit 34 of the display device 3 displays the voice recognition result of the other party's voice information and the voice recognition result of the user's voice information received by the voice recognition result receiving unit 33.

[0065] The display unit 34 of the display device 3 may make the text corresponding to the voice recognition result of the other party's voice information and the text corresponding to the voice recognition result of the user's voice information different from each other in the voice recognition result area E1. For example, the display unit 34 of the display device 3 may make the font, color, background, etc. of these texts different from each other.

[0066] In addition, if the display device 3 is capable of connecting to the Internet without going through the user communication device 1, the display device 3 may send and receive data such as voice information, voice recognition results, etc. to the voice recognition result acquisition device 4 via an Internet connection that does not go through the user communication device 1.

[0067] If the voice recognition result generated by the operator 5 or the voice recognition device 6 contains a predetermined keyword, the display unit 34 of the display device 3 may perform a predetermined display. An example of the predetermined keyword is a word related to so-called "it's my son" scams. In this case, an example of the predetermined display is a warning display saying "beware of it's my son" scams. The determination of whether the voice recognition result generated by the operator 5 or the voice recognition device 6 contains a predetermined keyword is made by, for example, any of the voice recognition result acquisition device 4, the operator 5, the voice recognition device 6, or the display device 3.

[0068] If silence continues for a predetermined length or longer during a call using a phone number recorded in the SIM of the user communication device 1, the voice recognition result acquisition device 4 may stop acquiring the voice recognition result. For example, if the volume of the voice information of the user and the other party during a call using a phone number recorded in the SIM of the user communication device 1 is equal to or lower than a predetermined threshold, it can be determined that there is silence. The determination of whether silence has continued for a predetermined length or longer during a call using a phone number recorded in the SIM of the user communication device 1 is made by any one of the voice recognition result acquisition device 4, the operator 5, the voice recognition device 6, or the display device 3.

[0069] The various processes described in the embodiments of the disclosed technology may not only be performed chronologically in the order described, but may also be performed in parallel or individually depending on the processing capabilities of the device performing the processes or as necessary.

[0070] For example, data exchange between the devices in the closed caption telephone system may be performed directly or via a device not shown.

[0071] The subtitled telephone system and method may be capable of displaying the voice recognition results of the voice information in an IP telephone call between the user communication device 1 and the destination communication device 2. Examples of IP telephone calls are LINE (registered trademark) calls, Facebook (registered trademark) Messenger calls, and Skype (registered trademark) calls.

[0072] To achieve this, the subtitled telephone system and method simply performs processing similar to that described above, using audio information in an IP telephone call between the user communication device 1 and the recipient communication device 2, instead of the audio information of the recipient in a call using a telephone number recorded in the SIM of the user communication device 1.

[0073] [Programs, recording media] The processing of each unit of the display device 3 may be realized by a computer. In this case, the processing contents of the functions to be possessed by the display device 3 are described by a program. Then, this program is loaded into the storage unit 1020 of the computer 1000 shown in Fig. 5 and operated in the arithmetic processing unit 1010, the input unit 1030, the output unit 1040, the display unit 1060, etc., whereby the processing functions of each unit in the display device 3 are realized on the computer.

[0074] The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a non-transitory recording medium, specifically, a magnetic recording device, an optical disk, etc.

[0075] The program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0076] A computer that executes such a program, for example, first stores a program recorded on a portable recording medium or a program transferred from a server computer in the auxiliary recording unit 1050, which is a non-transient storage device of the computer. Then, when executing a process, the computer reads the program stored in the auxiliary recording unit 1050, which is a non-transient storage device of the computer, into the storage unit 1020, and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium into the storage unit 1020 and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the above-mentioned process by a so-called ASP (Application Service Provider) type service that realizes a processing function only by issuing an execution instruction and obtaining a result. Note that the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (data that is not a direct instruction to a computer but has a nature that specifies the processing of the computer, etc.).

[0077] In this embodiment, the device is configured by executing a predetermined program on a computer, but at least a part of the processing may be realized by hardware. For example, the voice information receiving unit 31, the voice information transmitting unit 32, the voice recognition result receiving unit 33, and the display unit 34 may be configured by a processing circuit.

[0078] Needless to say, other modifications can be made without departing from the spirit of the present invention.

[0079] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A user communication device; a display device that is a single device; A voice recognition result acquisition device; A closed caption telephone system comprising: The user communication device transmits voice information of a call destination using a telephone number recorded in a SIM of the user communication device to the display device via wireless communication; the display device includes a voice information receiving unit that receives voice information transmitted by the wireless communication, and a voice information transmitting unit that transmits the received voice information to the voice recognition result acquisition device via the user communication device, the voice recognition result acquisition device acquires a voice recognition result of the voice information generated by an operator or a voice recognition device, and transmits the voice recognition result to the display device via the user communication device; The display device further includes a voice recognition result receiving unit that receives the voice recognition result, and a display unit that displays the received voice recognition result. Closed captioning telephone system.

2. The closed caption telephone system of claim 1, the display unit displays a display area in which the user can select the operator or the voice recognition device during the call; When the user selects the operator or the voice recognition device by operating the display area, the voice recognition result acquisition device acquires a voice recognition result of the voice information generated by the operator or the voice recognition device selected by the user. Closed captioning telephone system.

3. The closed caption telephone system of claim 1, The display device is detachably housed in a case for housing the user communication device. Closed captioning telephone system.

4. A caption telephone method performed by a user communication device, a display device which is a single device, and a speech recognition result acquisition device, comprising: A step of transmitting voice information of a caller in a call using a telephone number recorded in a SIM of the user communication device to the display device by wireless communication, by the user communication device; receiving, by the display device, audio information transmitted via the wireless communication; a step of transmitting the received voice information to the voice recognition result acquisition device via the user communication device by the display device; the voice recognition result acquisition device acquires a voice recognition result of the voice information generated by an operator or a voice recognition device, and transmits the voice recognition result to the display device via the user communication device; receiving the speech recognition result by the display device; a display device displaying the received speech recognition result; Includes subtitling phone methods.

5. A program for causing a computer to function as each part of a display device of the closed caption telephone system according to any one of claims 1 to 3.

Images