Computer programs for terminal devices, computer programs for speech recognition servers, and communication systems

Abstract

To provide a technology that can display appropriate text strings in situations where both the voice of a user with a speech impediment and the voice of a user without a speech impediment may be input. [Solution] The computer program causes the computer of the terminal device to function as follows: a supply unit that supplies voice data to a speech recognition engine; an acquisition unit that obtains a string set including first string data and second string data from the speech recognition engine; and a display control unit that causes the display unit to display a string screen containing a string corresponding to at least one of the first string data and the second string data. The first string data is data generated based on a first speech recognition model for a user with a speech disorder, and the second string data is data generated based on a second speech recognition model for a user without a speech disorder.

Description

Technical Field

[0004] , ,

[0005] , , ,

[0001] This specification discloses a technology related to speech recognition.

Background Art

[0002] Patent Document 1 discloses an application that recognizes speech and displays a character string.

Prior Art Documents

[0006] According to the above configuration, the terminal device obtains from the speech recognition engine a first string data generated based on a first speech recognition model for users with speech impairments, and a second string data generated based on a second speech recognition model for users without speech impairments. The terminal device then displays a string screen containing at least one of the first string and the second string. To this end, the terminal device can display an appropriate string in situations where both the voice of a user with a speech impairment and the voice of a user without a speech impairment may be input.

[0007] A computer-readable recording medium for storing the above-mentioned computer program is also novel and useful. Furthermore, the terminal device itself and the method executed by the terminal device are also novel and useful. In addition, a communication system comprising the terminal device and a voice recognition server is also novel and useful.

[0008] This specification further discloses a computer program for a speech recognition server. The speech recognition server may include a computer and a memory that stores a first speech recognition model for a user with a speech disorder and a second speech recognition model for a user without a speech disorder. The computer program may cause the computer to function as follows: a generation unit that, when speech data is received from a terminal device, generates first string data corresponding to the speech data based on the first speech recognition model and generates second string data corresponding to the speech data based on the second speech recognition model; a proofreading unit that performs proofreading on a first string corresponding to the first string data and a second string corresponding to the second string data; and a transmission unit that transmits string data corresponding to one of the strings determined to be an accurate string by the proofreading to the terminal device.

[0009] According to the above configuration, the speech recognition server generates first string data based on a first speech recognition model for users with speech disorders and second string data based on a second speech recognition model for users without speech disorders. The speech recognition server then performs proofreading on the first and second strings and transmits the string data corresponding to the string that is determined to be accurate through the proofreading to the terminal device. As a result, the terminal device can display the appropriate string in situations where both the voice of a user with a speech disorder and the voice of a user without a speech disorder may be input.

[0010] A computer-readable recording medium for storing the above-mentioned computer program is also novel and useful. Furthermore, the above-mentioned speech recognition server itself and the method executed by the above-mentioned speech recognition server are also novel and useful. In addition, a communication system comprising the above-mentioned speech recognition server and terminal equipment is also novel and useful. [Brief explanation of the drawing]

[0011] [Figure 1]This shows the configuration of the communication system. [Figure 2] The sequence diagrams for each process in the first embodiment are shown. [Figure 3] The sequence diagrams for each process in the second embodiment are shown. [Figure 4] The sequence diagrams for each process in the third embodiment are shown. [Figure 5] The sequence diagrams for each process in the fourth embodiment are shown. [Figure 6] The sequence diagrams for each process in the fifth embodiment are shown. [Modes for carrying out the invention]

[0012] (First embodiment) (Configuration of communication system 2: Figure 1) As shown in Figure 1, the communication system 2 comprises a mobile terminal 10 and a speech recognition server 100. In this embodiment, a technique is disclosed for the mobile terminal 10 to display strings corresponding to the voices of a user without articulation impairment and a user with articulation impairment.

[0013] (Configuration of mobile device 10) The mobile terminal 10 is a portable user terminal device such as a smartphone, tablet PC, or mobile phone. In a modified example, a stationary user terminal device may be used instead of the mobile terminal 10. The mobile terminal 10 comprises an operation unit 12, a display unit 14, a microphone 16, a communication interface 18, and a control unit 30.

[0014] The operation unit 12 is a user interface for the user to input various information, and includes, for example, hardware keys. The display unit 14 is a display for displaying various information. The display unit 14 functions as a touch panel. That is, the display unit 14 also functions as the operation unit 12. The microphone 16 is a device for inputting the user's voice. The communication interface 18 is a wireless interface in this embodiment, but in modified versions it may be a wired interface. The communication interface 18 may be an interface for connecting to a wireless Local Area Network (LAN) 4, or it may be an interface for performing wireless communication such as 4G or 5G. The mobile terminal 10 can communicate with the voice recognition server 100 via the communication interface 18.

[0015] The control unit 30 comprises a CPU 32 and a memory 34. The memory 34 comprises volatile memory and non-volatile memory. The volatile memory includes RAM and cache memory. The non-volatile memory may be ROM, flash memory, Solid State Drive (SSD), Hard Disk Drive (HDD), or a combination thereof. The non-volatile memory stores programs 40 and 42. Programs 40 and 42 stored in the non-volatile memory are loaded into the volatile memory, and various processes are realized by the CPU 32 executing programs 40 and 42.

[0016] The OS program 40 is a program that enables the basic operation of the mobile terminal 10. The voice display application 42 is installed on the mobile terminal 10 from, for example, a server on the internet (not shown). Hereinafter, the voice display application 42 will be simply referred to as "app 42". When app 42 detects user voice input to the microphone 16, it sends voice data corresponding to the voice to the voice recognition server 100 and receives string data corresponding to the voice data from the voice recognition server 100. Then, app 42 displays a screen containing the string corresponding to the string data on the display unit 14.

[0017] (Configuration of the Speech Recognition Server 100) The speech recognition server 100 is installed on the Internet 6. The speech recognition server 100 functions as a speech recognition engine. The speech recognition server 100 receives speech data from an external device (e.g., the mobile terminal 10), generates string data corresponding to the speech data, and transmits the string data to the external device. The speech recognition server 100 may be a single server or a collection of multiple servers. Hereinafter, the speech recognition server 100 will be simply referred to as the "server 100".

[0018] The server 100 includes a communication interface 118 and a control unit 130. The communication interface 118 is an interface for communicating with an external device (e.g., the mobile terminal 10). The control unit 130 includes a CPU 132 and a memory 134. The CPU 132 executes various processes according to the program 140 stored in the memory 134. The memory 134 includes a volatile memory and a non-volatile memory. The program 140 stored in the non-volatile memory is loaded into the volatile memory, and various processes are realized by the CPU 132 executing the program 140. The memory 134 further stores speech recognition model data 150.

[0019] The speech recognition model data 150 is data for converting speech data into string data. The speech recognition model data 150 includes, for each of a plurality of users with speech disorders, a user ID for identifying the user and a speech recognition model for the user. For example, the user ID "001" is associated with the speech recognition model 152, and the user ID "002" is associated with the speech recognition model 154. The speech recognition model data 150 further includes a general model for users without speech disorders, the non-disorder speech recognition model 160. Hereinafter, the speech recognition model 152 (or 154) will be simply referred to as the "model 152 (or 154)". Also, the non-disorder speech recognition model 160 will be simply referred to as the "model 160".

[0020] A user with speech disorder, for example, the owner of the mobile terminal 10, accesses the server 100 in advance using the application 42 and registers the user ID "001" with the server 100. The user further inputs various voices of the user himself / herself into the mobile terminal 10 and provides voice data corresponding to the voices to the server 100. Thereby, the server 100 generates a model 152 for the user based on the voice data and stores the generated model 152 in association with the user ID "001".

[0021] (Processes executed by each device 10, 100: FIG. 2) Subsequently, referring to FIG. 2, the processes executed by each device 10, 100 will be described. In the following description, for the sake of easy understanding, the operations realized by the CPUs 32, 132 of each device are described mainly with the mobile terminal 10 and the server 100 instead of mainly with the CPUs.

[0022] The mobile terminal 10 is owned by a user with speech disorder (hereinafter referred to as "disabled user"). The disabled user is with a human without speech disorder (hereinafter referred to as "non-disabled user") and hopes to communicate with the non-disabled user. In this case, the disabled user activates the application 42 at T10. As a result, the mobile terminal 10 executes the following respective processes according to the application 42.

[0023] The mobile terminal 10 causes a login screen to be displayed on the display unit 14 at T12. The mobile terminal 10 receives an input operation of login information including the user ID "001" from the disabled user at T14. The login information may further include a password. When the mobile terminal 10 receives the input operation of the login information at T14, it transmits the login information including the user ID "001" to the server 100 at T16.

[0024] When server 100 receives login information from mobile terminal 10 in T16, it identifies model 152 associated with user ID "001" included in the login information. Based on this, server 100 can perform speech recognition on voice data received from mobile terminal 10 thereafter, using model 152, and generate string data corresponding to the voice data. Note that "based on" here can be rephrased as "using" or "consequently."

[0025] Subsequently, the disabled user speaks. In this case, the mobile terminal 10 detects the voice input via the microphone 16 at T20. After the disabled user finishes speaking, if a predetermined period of silence (e.g., 3 seconds) has elapsed, the mobile terminal 10 determines that the speaking has ended and, at T22, sends voice data corresponding to the voice detected at T20 to the server 100.

[0026] When server 100 receives voice data from mobile terminal 10 at T22, it performs speech recognition on the voice data at T24. Specifically, server 100 performs speech recognition based on the identified model 152 to generate string data corresponding to the voice data, and also performs speech recognition based on model 160 to generate string data corresponding to the voice data. Hereinafter, the former string data will be referred to as "model 152 string data," and the latter string data will be referred to as "model 160 string data." Then, at T30, server 100 sends a string set containing both the model 152 string data and the model 160 string data to mobile terminal 10.

[0027] When the mobile terminal 10 receives a string set from the server 100 at T30, at T40, it displays an updated talk screen SC11 on the display unit 14, which contains two strings corresponding to the two string data included in the string set. The updated talk screen SC11 includes one box. This box contains the string "How are you" corresponding to the model 152 string data and the string "Ho are yo" corresponding to the model 160 string data. In this way, in response to a single utterance, one box is displayed that contains both the speech recognition result based on model 152 for the disabled user (i.e., "How are you") and the speech recognition result based on model 160 for the non-disabled user (i.e., "Ho are yo"). Therefore, the user can recognize the correct string from the two strings contained in the box. At T20, since the voice of the disabled user was input, the speech recognition result "How are you" based on model 152 for the disabled user is an accurate string (i.e., a string with recognizable meaning). On the other hand, the speech recognition result "Ho are yo" based on Model 160 for non-disabled users is an inaccurate string (i.e., a string whose meaning cannot be recognized). Therefore, non-disabled users can appropriately recognize what a disabled user has said by looking at the speech recognition result based on Model 152.

[0028] In particular, within a single box corresponding to a single utterance, the string corresponding to the Model 152 string data and the string corresponding to the Model 160 string data are displayed vertically side by side, with a line break between the former and the latter strings. Therefore, compared to a configuration where the former and the latter strings are displayed without a line break, non-disabled users can easily recognize the former string. In this embodiment, within a single box, the speech recognition result of Model 152 for disabled users is displayed at the top, and the speech recognition result of Model 160 for non-disabled users is displayed at the bottom. However, in a modified version, the speech recognition result of Model 160 for non-disabled users may be displayed at the top and the result of Model 160 for non-disabled users may be displayed at the bottom.

[0029] Next, a non-faulty user speaks. In this case, the mobile terminal 10 detects the voice input via the microphone 16 at T50. After a predetermined period of silence has elapsed since the non-faulty user finished speaking, the mobile terminal 10 determines that the speaking has ended and sends voice data corresponding to the voice detected at T50 to the server 100 at T52.

[0030] When server 100 receives voice data from mobile terminal 10 at T52, at T54, similar to T24, it performs voice recognition based on two models 152 and 160 to generate model 152 string data and model 160 string data. Then, at T60, server 100 sends a string set containing model 152 string data and model 160 string data to mobile terminal 10.

[0031] When the mobile terminal 10 receives a string set from the server 100 at T60, at T70, it displays an updated talk screen SC12 on the display unit 14, which contains two strings corresponding to the two string data included in the string set. The updated talk screen SC12 has one box added below the one box displayed at T40. The box added at T70 contains the string "I fin thank you" corresponding to the model 152 string data and the string "I'm fine thank you" corresponding to the model 160 string data. At T50, since the voice of a non-disabled user was input, the speech recognition result "I'm fine thank you" based on model 160 for the non-disabled user is an accurate string. On the other hand, the speech recognition result "I fin thank you" based on model 152 for the disabled user is an inaccurate string. Therefore, the disabled user can appropriately recognize what the non-disabled user said by looking at the speech recognition result based on model 160.

[0032] Subsequently, each time a disabled or non-disabled user speaks, a box containing the speech recognition results from each of the two models, 152 and 160, is displayed. This allows both disabled and non-disabled users to properly understand what the other person has said.

[0033] (Effects of this embodiment) In this embodiment, the mobile terminal 10 receives from the speech recognition server 100 string data generated based on model 152 for a disabled user and string data generated based on model 160 for a non-disabled user (T30, T60). The mobile terminal 10 then displays updated talk screens SC11 and SC12 containing two strings corresponding to the two string data (T40, T70). In this way, the mobile terminal 10 can display the appropriate string in situations where both the voice of a disabled user and the voice of a non-disabled user may be input.

[0034] (Correspondence) Mobile terminal 10 and server 100 are examples of "terminal device" and "speech recognition engine," respectively. Model 152 and Model 160 are examples of "first speech recognition model" and "second speech recognition model," respectively. Model 152 string data and Model 160 string data are examples of "first string data" and "second string data," respectively. Screens SC11 and SC12 are examples of "string screens." Furthermore, a state of silence continuing for a predetermined period of time after voice input is detected is an example of "a predetermined condition being met."

[0035] T22 and T52 are examples of processes performed by the "supply unit" and "terminal-side transmission unit" of the "terminal device". T30 and T60 are examples of processes performed by the "acquisition unit" and "receiving unit" of the "terminal device". T40 and T70 are examples of processes performed by the "display control unit" of the "terminal device". T24 and T54 are examples of processes performed by the "generation unit" of the "speech recognition server". T30 and T60 are examples of processes performed by the "server-side transmission unit" of the "speech recognition server".

[0036] (Second embodiment: Figure 3) Next, with reference to Figure 3, a second embodiment will be described. In the first embodiment, two strings corresponding to a single utterance are displayed in one box. In this embodiment, one of the two strings corresponding to a single utterance is displayed inside the box, and the other string is displayed outside the box. That is, the two strings are displayed in different display modes.

[0037] First, the same process as T10 to T16 in Figure 2 is executed. T120 to T130 are the same as T20 to T30 in Figure 2. When the mobile terminal 10 receives a string set from the server 100 at T130, at T140, it displays the updated talk screen SC21 on the display unit 14, which contains two strings corresponding to the two string data included in the string set. The updated talk screen SC21 includes one box containing a string corresponding to the model 152 string data. In addition, below this box, the updated talk screen SC21 has a string corresponding to the model 160 string data. The latter string is displayed outside the box and has a smaller font size than the former string. In this way, the two strings are displayed in different ways, so the user can easily understand that the two strings are the result of speech recognition based on different speech recognition models. T150 to T160 are the same as T50 to T60 in Figure 2. T170 is the same as T140 except that the strings are different. As a result, the updated chat screen SC22 will be displayed on the T170.

[0038] In the second embodiment, as an example of displaying two strings in different ways, one string is placed outside the box, and the font sizes of the two strings are different. Alternatively, any of the following embodiments may be adopted: (1) One string and the other string are placed in the same box, and the font sizes of the two strings are different. (2) One string and the other string are placed in the same box, or these strings are placed in separate boxes, and the font colors of the two strings are different. (3) One string and the other string are placed in the same box, or these strings are placed in separate boxes, and the background colors of the two strings are different.

[0039] (Third embodiment: Figure 4) Next, a third embodiment will be described with reference to Figure 4. In the first and second embodiments, two strings corresponding to a single utterance are displayed simultaneously. In this embodiment, only one of the two strings corresponding to a single utterance is displayed initially, and when a predetermined operation is performed in that state, the other string is displayed.

[0040] First, the same process as T10-T16 in Figure 2 is performed. T220-T230 is the same as T20-T30 in Figure 2. When the mobile terminal 10 receives a string set from the server 100 at T230, at T232 it performs proofreading on the two strings corresponding to the two string data contained in the string set. Specifically, the mobile terminal 10 identifies the number of characters in each of the two strings and determines that the string with the larger number of characters is the correct string. In particular, at T232, the string "How are you" corresponding to the model 152 string data has 9 characters, and the string "Ho are yo" corresponding to the model 160 string data has 7 characters. Therefore, the mobile terminal 10 determines that the string "How are you" is the correct string. When speech recognition is performed on the same audio data based on two different speech recognition models and two strings are generated, the string with the larger number of characters is usually more likely to be the correct string corresponding to the audio. In this embodiment, the mobile terminal 10 performs proofreading based on the number of characters, so it can appropriately identify the correct string.

[0041] The mobile terminal 10 displays the updated talk screen SC31 on the display unit 14, which contains boxes with only the strings that were deemed accurate in the proofreading process of T232 in T240. In this way, only one string corresponding to each utterance is displayed, so the user can easily understand the results of the speech recognition.

[0042] The mobile terminal 10 accepts an operation in T242 to select a box within the updated chat screen SC31. In this case, the mobile terminal 10 displays the updated chat screen SC32 on the display unit 14, which contains the string that was not deemed correct during the proofreading process in T232, instead of the string contained in the box within the updated chat screen SC31. In this way, the user can check the results of speech recognition based on the other model 160 by performing a predetermined operation while the results of speech recognition based on model 152 of the two models 152, 160 are displayed.

[0043] T250 to T270 are the same as T220 to T240, except that the input voice is different. In particular, in T262, the string "I fin thank you" corresponding to the model 152 string data has 12 characters, and the string "I'm fine thank you" corresponding to the model 160 string data has 14 characters. For this reason, the mobile terminal 10 determines that the string "I fin thank you" is the correct string, and in T240, it displays the updated talk screen SC33 on the display unit 14, which contains a box containing only that string.

[0044] (Fourth embodiment: Figure 5) Next, with reference to Figure 5, the fourth embodiment will be described. In the fourth embodiment, similar to the third embodiment, only one of the two strings corresponding to a single utterance is displayed. However, unlike the third embodiment, the other string is not displayed in place of the other string.

[0045] First, the same process as T10 to T16 in Figure 2 is performed. T320 to T340 is the same as T220 to T240 in Figure 4. At T340, the mobile terminal 10 displays the updated chat screen SC41 on the display unit 14, which contains boxes containing only the strings that were judged to be accurate in the proofreading at T332. In this embodiment, unlike the third embodiment, no matter what operation is performed while the updated chat screen SC41 is displayed, the strings that were not judged to be accurate in the proofreading at T332 will not be displayed instead of the strings contained in the boxes within the updated chat screen SC41.

[0046] T350-T370 are the same as T320-T340, except that the input voice is different. This displays the updated chat screen SC42. In the updated chat screen SC42, no matter what operation is performed, strings that were deemed incorrect by the T362 proofreading will not be displayed.

[0047] (Fifth embodiment: Figure 6) Next, a fifth embodiment will be described with reference to Figure 6. In the fifth embodiment, the speech recognition server 100 performs proofreading.

[0048] First, the same process as T10 to T16 in Figure 2 is performed. T420 to T424 are the same as T20 to T24 in Figure 2. At T426, the speech recognition server 100 performs proofreading on the two strings corresponding to the two string data generated in 424. The proofreading method is the same as T232 in Figure 4. That is, the speech recognition server 100 identifies the number of characters in each of the two strings and determines that the string with the larger number of characters is the correct string. Then, at T430, the speech recognition server 100 sends one string data corresponding to the string that was determined to be the correct string in the proofreading at T424 to the mobile terminal 10. Here, the speech recognition server 100 does not send one string data corresponding to the string that was not determined to be the correct string in the proofreading at T424 to the mobile terminal 10.

[0049] When the mobile terminal 10 receives string data from the speech recognition server 100 at T430, it displays the updated talk screen SC51 on the display unit 14 at T440, where a box containing the string corresponding to the string data is located. In this way, since a single string that has been judged to be accurate during proofreading is displayed, the user can properly understand the result of speech recognition. T450 to T470 are the same as T420 to T440 except that the input voice is different. This displays the updated talk screen SC52.

[0050] (Effects of this embodiment) In this embodiment, the speech recognition server 100 generates string data based on model 152 for disabled users and string data based on model 160 for non-disabled users (T424, T454). The speech recognition server 100 then performs proofreading on the two strings corresponding to the two string data (T426, T456) and sends the string data corresponding to the string that is determined to be accurate through proofreading to the mobile terminal 10 (T430, T460). As a result, the mobile terminal 10 can display the appropriate string in situations where both the voice of a user with articulation impairment and the voice of a user without articulation impairment may be input.

[0051] (Correspondence) T422 and T452 are examples of processes executed by the "terminal-side transmitting unit" of the "terminal device". T430 and T460 are examples of processes executed by the "receiving unit" of the "terminal device". T440 and T470 are examples of processes executed by the "display control unit" of the "terminal device". T424 and T454 are examples of processes executed by the "generation unit" of the "speech recognition server". T426 and T456 are examples of processes executed by the "proofreading unit" of the "speech recognition server". T430 and T460 are examples of processes executed by the "server-side transmitting unit" of the "speech recognition server".

[0052] Although specific examples of the present invention have been described in detail above, these are merely illustrative and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes to the specific examples illustrated above. Modifications of the above embodiments are listed below.

[0053] (Modification 1) Instead of providing a speech recognition server 100, the application 42 of the mobile terminal 10 may perform speech recognition. In this modification, the application 42 may use the CPU 32 as a detection module for detecting voice input, a speech recognition engine, and a display module. When the detection module detects voice input, it supplies voice data to the speech recognition engine. The display module obtains a string set from the speech recognition engine and, similar to the first embodiment described above, displays an updated talk screen containing each string corresponding to each string data included in the string set on the display unit 14. Note that by having the application 42 perform speech recognition, an updated talk screen similar to that of the second to fifth embodiments may be displayed. In this modification, the detection module is an example of a "supply unit," and the display module is an example of an "acquisition unit" and a "display control unit." In particular, in this modification, the speech recognition engine does not need to have multiple speech recognition models to correspond to multiple disabled users, and may have only one speech recognition model to correspond to one disabled user who is a user of the mobile terminal 10. Generally speaking, a "speech recognition engine" does not need to have multiple first speech recognition models to accommodate multiple users with articulation disorders.

[0054] (Modification 2) In the third embodiment described above, the proofreading of T232 and T262 in Figure 4 may be omitted. In this case, the mobile terminal 10 may first display a box containing the string corresponding to one of the predetermined string data from the Model 152 string data and Model 160 string data included in the string set, and if that box is selected, it may display a box containing the string corresponding to the other string data. Generally speaking, the "display control unit" does not need to perform proofreading.

[0055] (Modification 3) Proofreading is not limited to determining whether a string of characters is accurate. For example, proofreading may include determining whether each word in a string is included in the dictionary based on dictionary data, and determining whether a string has many words included in the dictionary. Alternatively, proofreading may include identifying the number of typographical errors in a string and determining whether a string has few typographical errors.

[0056] (Modification 4) In the first embodiment described above, one box containing two vertically aligned strings is displayed in response to one voice input. Alternatively, various display formats are possible. For example, one box containing two horizontally aligned strings may be displayed in response to one voice input. Another box containing one string corresponding to Model 152 string data and another box containing one string corresponding to Model 160 string data may be displayed in response to one voice input. That is, two boxes may be displayed in response to one voice input. Furthermore, no boxes may be displayed. For example, two vertically aligned strings may be displayed without a box in response to one voice input. The interval between these two strings is the first interval. In addition, two more vertically aligned strings may be displayed without a box in response to the next voice input. The interval between these two strings is the first interval. Here, the interval between the first two strings displayed and the next two strings displayed is a second interval that is larger than the first interval. Even in this format, users can easily understand that the first two strings of text are displayed in response to one voice input, and the next two strings of text are displayed in response to the next voice input.

[0057] (Modification 5) In each of the above embodiments, the mobile terminal 10 determines that speech has ended when voice is input and silence continues for a predetermined period of time, and sends the voice data to the voice recognition server 100. Alternatively, the mobile terminal 10 may display a screen that includes a start button to be selected before voice input begins and an end button to be selected when voice input is completed. In this case, the mobile terminal 10 detects voice input after the start button is selected, and then determines that speech has ended when the end button is selected, and sends the voice data to the voice recognition server 100. In this modification, the selection of the end button is an example of "a predetermined condition being met".

[0058] (Modification 6) In the above embodiment, each process in Figures 2 to 6 is implemented by software, but at least one of these processes may be implemented by hardware such as a logic circuit.

[0059] Furthermore, the technical elements described herein or in the drawings demonstrate technical usefulness individually or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technologies illustrated herein or in the drawings achieve multiple objectives simultaneously, and achieving even one of these objectives constitutes technical usefulness in itself.

[0060] Even if, in the claims of this patent application, each claim depends on only some of the claims, it is not limited to the claim being dependent only on those specific claims. To the extent that it is not technically contradictory, each claim may be dependent on other claims that were not dependent at the time of application. That is, the technologies of each claim can be combined in various ways as follows: (Item 1) A computer program for a terminal device, The computer of the aforementioned terminal device comprises the following parts, namely: A supply unit that, when voice is input, supplies voice data corresponding to the voice to the voice recognition engine, An acquisition unit that, when the voice data is supplied to the speech recognition engine, acquires a string set from the speech recognition engine, the string set comprising a first string data corresponding to the voice data and a second string data corresponding to the voice data, wherein the first string data is data generated by the speech recognition engine based on a first speech recognition model for a user with a speech disorder, and the second string data is data generated by the speech recognition engine based on a second speech recognition model for a user without a speech disorder. When the string set is obtained from the speech recognition engine, a display control unit causes the terminal device's display unit to display a string screen containing at least one of the strings: a first string corresponding to the first string data and a second string corresponding to the second string data. A computer program that functions as such. (Item 2) The computer program described in item 1, wherein the string screen displays both the first string and the second string. (Item 3) The computer program described in item 2, wherein the first string and the second string are displayed vertically side by side on the string screen, and a line break is inserted between the first string and the second string. (Item 4) The supply unit supplies one audio data corresponding to the audio to the speech recognition engine each time an audio input is received and predetermined conditions are met. Each time a single audio data is supplied to the speech recognition engine, the acquisition unit acquires from the speech recognition engine a set of strings that includes a first string data corresponding to the single audio data and a second string data corresponding to the single audio data. The computer program according to item 2 or 3, wherein each time a set of strings is obtained from the speech recognition engine, the display control unit places a box corresponding to the set of strings on the string screen, and the box contains both a first string corresponding to a first string data contained in the set of strings corresponding to the box, and a second string corresponding to a second string data contained in the set of strings. (Item 5) The computer program described in any one of items 2 to 4, wherein the first string and the second string are displayed in different ways on the aforementioned string screen. (Item 6) The computer program according to item 2, wherein the display control unit, when the set of strings is obtained from the speech recognition engine, places only one of the first string and the second string on the string screen, and when a predetermined operation is performed while one of the strings is placed on the string screen, places the other of the first string and the second string on the string screen in place of the one string. (Item 7) The computer program according to item 6, wherein the display control unit, when the set of strings is obtained from the speech recognition engine, performs proofreading on the first string and the second string, and places the string that is determined to be an accurate string by the proofreading on the string screen. (Item 8) In the aforementioned string screen, only one of the first string and the second string is displayed. The computer program according to item 1, wherein the display control unit, when the set of strings is obtained from the speech recognition engine, performs proofreading on the first string and the second string, and places the string that is determined to be an accurate string by the proofreading on the string screen. (Item 9) The aforementioned proofreading includes determining which of the first string and the second string has more characters to be the correct string, as described in item 7 or 8. (Item 10) It is a communication system, Terminal device and Equipped with a speech recognition server, The aforementioned terminal device is Display unit and A terminal-side transmission unit that, when voice is input, transmits voice data corresponding to the voice to the voice recognition server, When the aforementioned audio data is transmitted to the speech recognition server, the receiving unit receives a string set from the speech recognition server, which includes a first string data corresponding to the audio data and a second string data corresponding to the audio data. A display control unit causes the display unit to display a string screen containing at least one of the following strings when the string set is received from the speech recognition server: a first string corresponding to the first string data and a second string corresponding to the second string data. Equipped with, The aforementioned speech recognition server, A memory that stores a first speech recognition model for users with speech disorders and a second speech recognition model for users without speech disorders, A generation unit that, when the voice data is received from the terminal device, generates the first string data based on the first voice recognition model and generates the second string data based on the second voice recognition model, A server-side transmission unit that transmits the string set, which includes the first string data and the second string data, to the terminal device, A communication system equipped with these features. (Item 11) A computer program for a speech recognition server, The aforementioned speech recognition server, Computers and, A memory that stores a first speech recognition model for users with speech disorders and a second speech recognition model for users without speech disorders, Equipped with, The aforementioned computer program comprises the following components of the computer, namely: A generation unit that, when voice data is received from a terminal device, generates first string data corresponding to the voice data based on the first voice recognition model, and generates second string data corresponding to the voice data based on the second voice recognition model, A proofreading unit that performs proofreading on a first string corresponding to the first string data and a second string corresponding to the second string data, A transmission unit transmits string data corresponding to one of the strings that is determined to be an accurate string by the aforementioned proofreading to the terminal device. A computer program that functions as such. (Item 12) The aforementioned proofreading includes determining which of the first string and the second string has more characters to be the correct string, as described in item 11. (Item 13) It is a communication system, Terminal device and Equipped with a speech recognition server, The aforementioned terminal device is Display unit and A terminal-side transmission unit that, when voice is input, transmits voice data corresponding to the voice to the voice recognition server, When the aforementioned audio data is transmitted to the speech recognition server, the receiving unit receives string data corresponding to the audio data from the speech recognition server, A display control unit that, when the string data is received from the voice recognition server, causes the display unit to display a string screen containing the string corresponding to the string data, Equipped with, The aforementioned speech recognition server, A memory that stores a first speech recognition model for users with speech disorders and a second speech recognition model for users without speech disorders, A generation unit that, when the voice data is received from the terminal device, generates the first string data based on the first voice recognition model and generates the second string data based on the second voice recognition model, A proofreading unit that performs proofreading on a first string corresponding to the first string data and a second string corresponding to the second string data, A server-side transmission unit transmits the string data corresponding to one of the strings that is determined to be an accurate string by the aforementioned proofreading to the terminal device. A communication system equipped with these features. [Explanation of symbols]

[0061] 2: Communication system, 10: Mobile terminal, 12: Operation unit, 14: Display unit, 16: Microphone, 18: Communication interface, 30: Control unit, 32: CPU, 34: Memory, 40: OS program, 42: Voice display application, 100: Voice recognition server, 118: Communication interface, 130: Control unit, 132: CPU, 134: Memory, 140: Program, 150: Voice recognition model data, 152, 154: User-specific voice recognition model, 160: Fault-free voice recognition model

Claims

1. A computer program for a terminal device, The computer of the aforementioned terminal device comprises the following parts, namely: A supply unit that, when voice is input, supplies voice data corresponding to the voice to the voice recognition engine, An acquisition unit that, when the voice data is supplied to the voice recognition engine, acquires a string set from the voice recognition engine, the string set including a first string data corresponding to the voice data and a second string data corresponding to the voice data, wherein the first string data is data generated by the voice recognition engine based on a first voice recognition model for a user with a speech disorder, and the second string data is data generated by the voice recognition engine based on a second voice recognition model for a user without a speech disorder, When the string set is obtained from the speech recognition engine, a display control unit causes the terminal device's display unit to display a string screen containing at least one of the strings: a first string corresponding to the first string data and a second string corresponding to the second string data. A computer program that functions as such.

2. The computer program according to claim 1, wherein both the first string and the second string are displayed on the string screen.

3. The computer program according to claim 2, wherein the first string and the second string are displayed side by side vertically on the string screen, and a line break is inserted between the first string and the second string.

4. The supply unit supplies one audio data corresponding to the audio to the speech recognition engine each time an audio input is received and predetermined conditions are met. Each time a single audio data is supplied to the speech recognition engine, the acquisition unit acquires from the speech recognition engine a set of strings including a first string data corresponding to the single audio data and a second string data corresponding to the single audio data. The computer program according to claim 2, wherein each time a set of strings is obtained from the speech recognition engine, the display control unit places a box corresponding to the set of strings on the string screen, and the box contains both a first string corresponding to a first string data included in the set of strings corresponding to the box, and a second string corresponding to a second string data included in the set of strings.

5. The computer program according to claim 2, wherein the first string and the second string are displayed in different display modes on the string screen.

6. The computer program according to claim 2, wherein the display control unit, when the set of strings is obtained from the speech recognition engine, places only one of the first string and the second string on the string screen, and when a predetermined operation is performed while one of the strings is placed on the string screen, places the other of the first string and the second string on the string screen in place of the one of the strings.

7. The computer program according to claim 6, wherein the display control unit, when the set of strings is obtained from the speech recognition engine, performs proofreading on the first string and the second string, and places the string that is determined to be an accurate string by the proofreading on the string screen.

8. In the aforementioned string screen, only one of the first string and the second string is displayed. The computer program according to claim 1, wherein the display control unit, when the set of strings is obtained from the speech recognition engine, performs proofreading on the first string and the second string, and places the one of the strings that is determined to be an accurate string by the proofreading on the string screen.

9. The computer program according to claim 7 or 8, wherein the proofreading includes determining that the string with the greater number of characters among the first string and the second string is the correct string.

10. It is a communication system, Terminal device and Equipped with a speech recognition server, The aforementioned terminal device is Display unit and A terminal-side transmission unit that, when voice is input, transmits voice data corresponding to the voice to the voice recognition server, When the aforementioned audio data is transmitted to the speech recognition server, a receiving unit receives a string set from the speech recognition server, which includes a first string data corresponding to the audio data and a second string data corresponding to the audio data. A display control unit causes the display unit to display a string screen containing at least one of the following strings when the string set is received from the speech recognition server: a first string corresponding to the first string data and a second string corresponding to the second string data. Equipped with, The aforementioned speech recognition server, A memory that stores a first speech recognition model for users with speech disorders and a second speech recognition model for users without speech disorders, When the audio data is received from the terminal device, the generation unit generates the first string data based on the first speech recognition model and generates the second string data based on the second speech recognition model, A server-side transmission unit that transmits the string set, which includes the first string data and the second string data, to the terminal device, A communication system equipped with these features.

11. A computer program for a speech recognition server, The aforementioned speech recognition server, Computers and, A memory that stores a first speech recognition model for users with speech disorders and a second speech recognition model for users without speech disorders, Equipped with, The aforementioned computer program comprises the following components of the computer, namely: A generation unit that, when voice data is received from a terminal device, generates first string data corresponding to the voice data based on the first voice recognition model, and generates second string data corresponding to the voice data based on the second voice recognition model, A proofreading unit that performs proofreading on a first string corresponding to the first string data and a second string corresponding to the second string data, A transmission unit that transmits string data corresponding to one of the strings that is determined to be an accurate string by the aforementioned proofreading to the terminal device, A computer program that functions as such.

12. The computer program according to claim 11, wherein the proofreading includes determining that the string with the greater number of characters among the first string and the second string is the correct string.

13. It is a communication system, Terminal device and Equipped with a speech recognition server, The aforementioned terminal device is Display unit and A terminal-side transmission unit that, when voice is input, transmits voice data corresponding to the voice to the voice recognition server, When the aforementioned audio data is transmitted to the speech recognition server, the receiving unit receives string data corresponding to the audio data from the speech recognition server, A display control unit that, when the string data is received from the voice recognition server, causes the display unit to display a string screen containing the string corresponding to the string data, Equipped with, The aforementioned speech recognition server, A memory that stores a first speech recognition model for users with speech disorders and a second speech recognition model for users without speech disorders, When the audio data is received from the terminal device, the generation unit generates first string data based on the first speech recognition model and generates second string data based on the second speech recognition model, A proofreading unit that performs proofreading on a first string corresponding to the first string data and a second string corresponding to the second string data, A server-side transmission unit transmits the string data corresponding to one of the strings that is determined to be an accurate string by the aforementioned proofreading to the terminal device. A communication system equipped with these features.

Images