Computer program for terminal device, terminal device, and method executed by terminal device

Abstract

The purpose of the present invention is to provide a technique by which the speech of a user not having an articulation disorder and the speech of a user having an articulation disorder can be appropriately displayed as a string of text.　When first information for requesting first text string data corresponding to the speech of a user not having an articulation disorder is supplied to a speech recognition engine, the terminal device of the present invention acquires first text string data from the speech recognition engine in response to the fact that a first speech recognition model for the user not having an articulation disorder is used by the speech recognition engine, and acquires second text string data from the speech recognition engine in response to the fact that a second speech recognition model for a user having an articulation disorder is used by the speech recognition engine when supplying the speech recognition engine with second information for requesting second text string data corresponding to the speech of the user having an articulation disorder.

Description

Computer program for a terminal device, terminal device, and method executed by a terminal device

[0001] This specification discloses a technique related to speech recognition.

[0002] Patent Document 1 discloses an application that recognizes speech and displays character strings.

[0003] Japanese Patent Application Laid-Open No. 2019-016206

[0004] The above-mentioned technology does not disclose anything about recognizing speech by distinguishing between users without dysarthria and users with dysarthria. This specification provides a technology that can appropriately display the speech of users without dysarthria and users with dysarthria as character strings.

[0005] This specification discloses a computer program for a terminal device. The computer program may cause a computer of the terminal device to function as the following units: a first supply unit that supplies information indicating whether the speech is that of a user without an articulation disorder or that of a user with an articulation disorder to a speech recognition engine, a second supply unit that, when speech input is detected, supplies speech data corresponding to the speech to the speech recognition engine, an acquisition unit that acquires character string data corresponding to the speech data from the speech recognition engine, and a first display control unit that displays a first screen including character strings corresponding to the character string data on a display unit of the terminal device. The acquisition unit may acquire the first character string data from the speech recognition engine in response to a first speech recognition model for the user without an articulation disorder being used by the speech recognition engine when first information for requesting first character string data corresponding to the speech of the user without an articulation disorder is supplied to the speech recognition engine, and may acquire the second character string data from the speech recognition engine in response to a second speech recognition model for the user with an articulation disorder being used by the speech recognition engine when second information for requesting second character string data corresponding to the speech of the user with an articulation disorder is supplied to the speech recognition engine.

[0006] According to the above configuration, the terminal device supplies the speech recognition engine with either first information for requesting first character string data corresponding to the speech of a user without an articulation disorder or second information for requesting second character string data corresponding to the speech of a user with an articulation disorder, thereby enabling the terminal device to obtain an appropriate speech recognition result according to the user. Therefore, the terminal device can appropriately display the speech of each of the user without an articulation disorder and the user with an articulation disorder as a character string.

[0007] A computer-readable recording medium storing the computer program is also novel and useful. The terminal device itself and a method executed by the terminal device are also novel and useful. A communication system including the terminal device and a voice recognition engine is also novel and useful.

[0008] 1 shows the configuration of a communication system. FIG. 2 shows a sequence diagram of each process executed by a mobile terminal and a voice recognition server. FIG. 3 shows a sequence diagram of Case B. FIG. 4 shows an example of a screen displayed on a mobile terminal.

[0009] (Configuration of communication system 2: FIG. 1) As shown in FIG. 1, the communication system 2 includes a mobile terminal 10 and a speech recognition server 100. In this embodiment, a technique is disclosed in which the mobile terminal 10 displays character strings corresponding to the speech of a user without an articulation disorder and a user with an articulation disorder.

[0010] (Configuration of mobile terminal 10) The mobile terminal 10 is a portable user terminal device such as a smartphone, tablet PC, or mobile phone. In a modified example, a stationary user terminal device may be used instead of the mobile terminal 10. The mobile terminal 10 includes an operation unit 12, a display unit 14, a microphone 16, a communication interface 18, and a control unit 30.

[0011] The operation unit 12 is a user interface for the user to input various information and includes, for example, hardware keys. The display unit 14 is a display for displaying various information. The display unit 14 functions as a touch panel. That is, the display unit 14 also functions as the operation unit 12. The microphone 16 is a device for inputting the user's voice. The communication interface 18 is a wireless interface in this embodiment, but may be a wired interface in a modified example. The communication interface 18 may be an interface for connecting to a wireless local area network (LAN) 4, or may be an interface for performing wireless communication such as 4G or 5G. The mobile terminal 10 can communicate with the voice recognition server 100 via the communication interface 18.

[0012] The control unit 30 includes a CPU 32 and a memory 34. The memory 34 includes a volatile memory and a non-volatile memory. The volatile memory includes RAM and cache memory. The non-volatile memory may be a ROM, a flash memory, a solid state drive (SSD), a hard disk drive (HDD), or a combination thereof. The non-volatile memory stores programs 40 and 42. The programs 40 and 42 stored in the non-volatile memory are loaded into the volatile memory, and the programs 40 and 42 are executed by the CPU 32 to realize various processes.

[0013] The OS program 40 is a program for realizing the basic operations of the mobile terminal 10. The voice display application 42 is installed on the mobile terminal 10, for example, from a server (not shown) on the Internet. Hereinafter, the voice display application 42 will be simply referred to as the "app 42." When the app 42 detects a user's voice input to the microphone 16, it transmits voice data corresponding to the voice to the voice recognition server 100 and receives character string data corresponding to the voice data from the voice recognition server 100. The app 42 then displays a screen including a character string corresponding to the character string data on the display unit 14.

[0014] (Configuration of the speech recognition server 100) The speech recognition server 100 is installed on the Internet 6. The speech recognition server 100 functions as a speech recognition engine. The speech recognition server 100 receives speech data from an external device (e.g., the mobile terminal 10), generates character string data corresponding to the speech data, and transmits the character string data to the external device. The speech recognition server 100 may be a single server or a collection of multiple servers. Hereinafter, the speech recognition server 100 will be simply referred to as "server 100."

[0015] The server 100 includes a communication interface 118 and a control unit 130. The communication interface 118 is an interface for communicating with an external device (e.g., the mobile terminal 10). The control unit 130 includes a CPU 132 and a memory 134. The CPU 132 executes various processes in accordance with a program 140 stored in the memory 134. The memory 134 includes a volatile memory and a non-volatile memory. The program 140 stored in the non-volatile memory is loaded into the volatile memory, and the program 140 is executed by the CPU 132, thereby realizing various processes. The memory 134 also stores voice recognition model data 150.

[0016] The speech recognition model data 150 is data for converting speech data into character string data. The speech recognition model data 150 includes a disability-free speech recognition model 152, which is a general-purpose model for users without articulation disorders. The speech recognition model data 150 further includes, for each of a plurality of users with articulation disorders, a user ID that identifies the user and a speech recognition model for the user. For example, user ID "001" is associated with speech recognition model 154, and user ID "002" is associated with speech recognition model 156. Note that, hereinafter, the disability-free speech recognition model 152 will be simply referred to as "model 152," and the speech recognition model 154 (or 156) will be simply referred to as "model 154 (or 156)."

[0017] A user with an articulation disorder, for example, a person who owns the mobile terminal 10, accesses the server 100 in advance using the app 42 and registers a user ID "001" in the server 100. The user then inputs various voices of the user into the mobile terminal 10 and provides the server 100 with voice data corresponding to the voices. The server 100 then generates a model 154 for the user based on the voice data and stores the generated model 154 in association with the user ID "001."

[0018] After storing the user ID "001" and the model 154 in association with each other, the server 100 further transmits to the mobile terminal 10 model identification information for identifying the model 152, the user ID "001", and the model identification information for identifying the model 154. As a result, the app 42 of the mobile terminal 10 stores the user ID "001" and the model identification information.

[0019] (Processing Executed by Each Device 10, 100: FIG. 2) Next, the processing executed by each device 10, 100 will be described with reference to FIG. 2. In the following description, for ease of understanding, the operations realized by the CPU 32, 132 of each device will be described mainly with respect to the mobile terminal 10 and the server 100, rather than mainly with respect to the CPU.

[0020] The mobile device 10 is owned by a user with an articulation disorder (hereinafter referred to as the "disabled user"). The disabled user is with a person without an articulation disorder (hereinafter referred to as the "non-disabled user") and wishes to communicate with the non-disabled user. In this case, the disabled user launches the app 42 at T10. As a result, the mobile device 10 executes the following processes in accordance with the app 42.

[0021] At T12, the mobile terminal 10 displays a login screen on the display unit 14. At T14, the mobile terminal 10 accepts input of login information including the user ID "001" from the disabled user. The login information may further include a password. If authentication of the login information is successful, at T20 the mobile terminal 10 displays an initial talk screen SC0 (see FIG. 4) on the display unit 14. Below, the case will be divided into Case A and Case B (see FIG. 3) depending on whether the user who speaks first after the app 42 is launched is a non-disabled user or a disabled user.

[0022] (Case A) First, we will describe Case A, in which the first person to speak after the app 42 is launched is a non-disabled user. As shown in FIG. 4 , the initial talk screen SC0 includes a start icon IC1 and a microphone icon IC2. The start icon IC1 is an icon that should be selected before a non-disabled user speaks, and the microphone icon IC2 is an icon that should be selected before a disabled user speaks. In Case A, the mobile terminal 10 accepts a selection operation of the start icon IC1 at T22. This selection operation may be performed by either the disabled user or the non-disabled user.

[0023] When the mobile device 10 accepts the selection operation of the start icon IC1 at T22, the mobile device 10 transmits, at T24, to the server 100 an activation request indicating model identification information that identifies the model 152. The activation request is a command requesting the server 100 to perform speech recognition using the model 152, and will be referred to below as a "model 152 activation request." In this way, when the mobile device 10 accepts the selection operation of the start icon IC1 included in the initial talk screen SC0, the mobile device 10 can transmit, to the server 100, a model 152 activation request for requesting character string data corresponding to the speech of the non-disabled user.

[0024] When the mobile terminal 10 transmits a request to activate the model 152 to the server 100, the model 152 is activated in the server 100, and thereafter, speech recognition is performed using the model 152. Then, at T26, the mobile terminal 10 receives a response from the server 100 indicating that the model 152 has been activated.

[0025] After that, the non-disabled user speaks. In this case, the mobile terminal 10 detects voice input via the microphone 16 at T30, and transmits voice data corresponding to the voice detected at T30 to the server 100 at T32.

[0026] When the server 100 receives the voice data from the mobile terminal 10 in T32, the server 100 performs voice recognition in T34. Specifically, the server 100 generates character string data from the voice data using the model 152 in response to the request to start the model 152. Then, the server 100 transmits the character string data to the mobile terminal 10 in T36.

[0027] When the mobile device 10 receives the character string data from the server 100 at T36, at T40, the mobile device 10 displays an updated talk screen SC1 on the display unit 14, in which a character string corresponding to the character string data is described. As shown in FIG. 4 , the updated talk screen SC1 includes the character string "Hello." By looking at the character string, the disabled user can know what the non-disabled user has said. The updated talk screen SC1 also includes a microphone icon IC2. The microphone icon IC2 included in the updated talk screen SC1 is the same as the microphone icon IC2 included in the initial talk screen SC0.

[0028] At T42, the mobile device 10 accepts a selection operation of the microphone icon IC2 included in the updated talk screen SC1. This selection operation may be performed by a disabled user or a non-disabled user. In this case, at T44, the mobile device 10 transmits to the server 100 an activation request indicating model identification information that identifies the model 154. This activation request is a command requesting the server 100 to perform speech recognition using the model 154, and will be referred to below as a "model 154 activation request."

[0029] When the mobile terminal 10 transmits a request to activate the model 154 to the server 100, the model 154 is activated in the server 100, and thereafter, speech recognition is performed using the model 154. Then, at T46, the mobile terminal 10 receives a response from the server 100 indicating that the model 154 has been activated.

[0030] After that, the disabled user speaks. In this case, the mobile terminal 10 detects voice input via the microphone 16 at T50, and transmits voice data corresponding to the voice detected at T50 to the server 100 at T52.

[0031] When server 100 receives voice data from mobile terminal 10 in T52, server 100 performs voice recognition in T54. Specifically, server 100 generates character string data from the voice data using model 154 in response to the model 154 activation request. Then, server 100 transmits the character string data to mobile terminal 10 in T56. In this way, in a situation where server 100 stores multiple models 154, 156, etc. corresponding to multiple disabled users, when server 100 receives a model 154 activation request from mobile terminal 10, server 100 performs voice recognition using model 154 in response to the model 154 activation request. Therefore, server 100 can perform voice recognition using an appropriate model for the disabled user, and as a result, can provide appropriate voice recognition results to mobile terminal 10.

[0032] When the mobile device 10 receives the character string data from the server 100 at T56, at T60, the mobile device 10 displays an updated talk screen SC2 on the display unit 14, in which a character string corresponding to the character string data is described. As shown in FIG. 4 , the updated talk screen SC2 includes the character string "How are you (question mark)" below the character string "Hello." By looking at the character string, the non-disabled user can know what the disabled user has said.

[0033] Here, the color of the background region of the box containing the character string "How are you (question mark)" is different from the color of the background region of the box containing the character string "Hello." This allows each user to easily recognize which character string corresponds to which user's voice. In a variant, instead of changing the color of the background region of each user's box, the color of the frame of each user's box may be changed, or the color of each user's character string may be changed. In another variant, nothing may be changed for each user.

[0034] The updated talk screen SC2 further includes a stop icon IC3. In this way, when a new character string corresponding to the speech of a disabled user is written, the stop icon IC3 is displayed. The stop icon IC3 is an icon that should be selected before a non-disabled user speaks, and has a different shape from either of the icons IC1 or IC2 included in the initial talk screen SC0.

[0035] In T62, the mobile device 10 accepts a selection operation of the stop icon IC3 included in the updated talk screen SC2. This selection operation may be performed by either the disabled user or the non-disabled user. In this case, in T64, the mobile device 10 transmits a request to activate the model 152 to the server 100, and in T66, receives a response from the server 100 indicating that the model 152 has been activated. The non-disabled user then speaks. In this case, in T70, the mobile device 10 detects voice input via the microphone 16, and in T72, transmits voice data corresponding to the voice detected in T70 to the server 100.

[0036] When the server 100 receives the voice data from the mobile terminal 10 in T72, the server 100 performs voice recognition in T74. T74 is the same as T34. Then, the server 100 transmits character string data to the mobile terminal 10 in T76.

[0037] When the mobile device 10 receives the character string data from the server 100 at T76, at T80, the mobile device 10 displays an updated talk screen SC3 on the display unit 14, on which a character string corresponding to the character string data is described. As shown in FIG. 4 , the updated talk screen SC3 includes the character string "I'm fine thank you." below the character string "How are you (question mark)." By looking at the character string, the disabled user can know what the non-disabled user said.

[0038] The updated talk screen SC3 further includes a microphone icon IC2. In this way, when a new character string corresponding to the voice of a non-disabled user is written, the microphone icon IC2 is displayed. By repeating the same operations and utterances, the disabled user and the non-disabled user can communicate with each other.

[0039] (Case B: FIG. 3) Next, with reference to FIG. 3 , Case B will be described in which the user with a disability speaks first after the app 42 is launched. In Case B, the mobile device 10 accepts a selection operation of the microphone icon IC2 included in the initial talk screen SC0 at T122. This selection operation may be performed by either the disabled user or the non-disabled user. In this case, the mobile device 10 transmits a model 154 activation request to the server 100 at T124, and receives a response from the server 100 at T126 indicating that the model 154 has been activated. In this way, when the mobile device 10 accepts a selection operation of the microphone icon IC2 included in the initial talk screen SC0, it can transmit a model 154 activation request to the server 100 to request character string data corresponding to the disabled user's voice.

[0040] After that, the disabled user speaks. In this case, the mobile terminal 10 detects voice input via the microphone 16 at T130, and transmits voice data corresponding to the voice detected at T130 to the server 100 at T132.

[0041] In T132, the server 100 receives the voice data from the mobile terminal 10, and in T134, generates character string data from the voice data by using the model 154 in response to the request to start the model 154. Then, in T136, the server 100 transmits the character string data to the mobile terminal 10.

[0042] When the mobile device 10 receives the character string data from the server 100 at T136, at T40, the mobile device 10 displays an updated talk screen SC4 on the display unit 14, which contains a character string corresponding to the character string data. As shown in FIG. 4, the updated talk screen SC4 includes the character string "My name is Tom." By viewing the character string, a non-disabled user can know what the disabled user has said. The updated talk screen SC4 further includes a stop icon IC3.

[0043] In T142, the mobile device 10 accepts a selection operation of the stop icon IC3 included in the updated talk screen SC4. This selection operation may be performed by either the disabled user or the non-disabled user. In this case, in T144, the mobile device 10 transmits a request to activate the model 152 to the server 100, and in T146, receives a response indicating that the model 152 has been activated from the server 100. The non-disabled user then speaks. In this case, in T140, the mobile device 10 detects voice input via the microphone 16, and in T150, transmits voice data corresponding to the voice detected in T140 to the server 100.

[0044] When the server 100 receives the voice data from the mobile terminal 10 at T152, at T154, the server 100 generates character string data from the voice data by using the model 152 in response to the request to start the model 152. Then, at T156, the server 100 transmits the character string data to the mobile terminal 10.

[0045] When the mobile device 10 receives the character string data from the server 100 at T156, at T60, the mobile device 10 displays an updated talk screen SC5 on the display unit 14, in which a character string corresponding to the character string data is written. As shown in FIG. 4 , the updated talk screen SC5 includes the character string "May I help you (question mark)" below the character string "My name is Tom." By viewing the character string, the disabled user can know what the non-disabled user has said. By repeating similar operations and utterances thereafter, the disabled user and the non-disabled user can communicate with each other.

[0046] As shown in the above cases A and B, each updated talk screen SC1 to SC5 is displayed when a disabled user and a non-disabled user speak using the same mobile terminal 10. This allows the disabled user and the non-disabled user to easily recognize what the other has said and to communicate appropriately with each other.

[0047] (Effects of the Present Embodiment) According to the present embodiment, mobile terminal 10 provides server 100 with either a model 152 activation request for requesting character string data corresponding to the speech of a non-disabled user or a model 154 activation request for requesting character string data corresponding to the speech of a disabled user (T24, T44, T64 in FIG. 2 , T124, T144 in FIG. 3 ), thereby enabling acquisition of appropriate speech recognition results for each user (T36, T56, T76 in FIG. 2 , T136, T156 in FIG. 3 ). As a result, mobile terminal 10 can appropriately display the speech of each of the non-disabled user and the disabled user as character strings (T40, T60, T80 in FIG. 2 , T140, T160 in FIG. 3 ).

[0048] (Correspondence) The mobile terminal 10 and the server 100 are examples of a "terminal device" and a "voice recognition engine," respectively. Model 152 and model 154 are examples of a "first voice recognition model" and a "second voice recognition model," respectively. The updated talk screens SC1 to SC5 are examples of a "first screen." The initial talk screen SC0 is an example of a "second screen." The character string data of T36 in FIG. 2 and the character string data of T136 in FIG. 3 are examples of "first character string data" and "second character string data," respectively. The start icon IC1 and the microphone icon IC2 included in the initial talk screen SC0 are examples of a "first icon" and a "second icon," respectively. The selection operation of the start icon IC1 and the selection operation of the microphone icon IC2 are examples of a "first operation" and a "second operation," respectively. The microphone icon IC2 included in the updated talk screen SC1 is an example of a "third icon." The stop icon IC3 included in the updated talk screen SC4 is an example of a "fourth icon."

[0049] T24, T44, and T64 in Fig. 2 and T124 and T144 in Fig. 3 are examples of processing executed by a "first supply unit." T32, T52, and T72 in Fig. 2 and T132 and T152 in Fig. 3 are examples of processing executed by a "second supply unit." T36, T56, and T76 in Fig. 2 and T136 and T156 in Fig. 3 are examples of processing executed by an "acquisition unit." T40, T60, and T80 in Fig. 2 and T140 and T160 in Fig. 3 are examples of processing executed by a "first display control unit." T20 in Fig. 2 is an example of processing executed by a "second display control unit."

[0050] Although specific examples of the present invention have been described above in detail, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above. Modifications of the above-mentioned embodiments are listed below.

[0051] (Variation 1) Instead of providing the speech recognition server 100, the app 42 may perform speech recognition. In this variation, the app 42 may cause the CPU 32 to function as a detection module that detects speech input, a speech recognition engine, and a display module. The display module provides the speech recognition engine with information indicating whether the speech is from a non-disabled user or a disabled user in response to an icon selection operation on each of the above screens SC0 to SC5. When the detection module detects speech input, it provides the speech recognition engine with speech data. The display module then obtains character string data representing the speech recognition results based on the above information and speech data from the speech recognition engine and displays updated talk screens SC1 to SC5. In particular, in this variation, the speech recognition engine may not have multiple speech recognition models corresponding to multiple disabled users, but may have only one speech recognition model corresponding to a single disabled user who is the user of the mobile device 10. Generally speaking, the "speech recognition engine" may not have multiple second speech recognition models corresponding to multiple users with articulation disorders.

[0052] (Variation 2) In the above embodiment, the mobile terminal 10 transmits a model 152 activation request or a model 154 activation request to the server 100 in response to an operation on each of the icons IC1 to IC3. Alternatively, the mobile terminal 10 may transmit a model 152 activation request or a model 154 activation request to the server 100 in response to receiving a voice input indicating which activation request should be transmitted to the server 100. Generally speaking, the "first supply unit" may not supply the first information to the voice recognition engine in response to a first operation, and may not supply the second information to the voice recognition engine in response to a second operation. Furthermore, the "second display control unit" may be omitted.

[0053] (Variation 3) The shapes of the icons IC1 to IC3 can be changed as appropriate. For example, the updated talk screens SC1, SC3, and SC5 may include an icon having a different shape from the microphone icon IC2. The updated talk screens SC2 and SC4 may include an icon having a different shape from the stop icon IC3. Furthermore, the initial talk screen SC0 may include the microphone icon IC2, but the updated talk screen SC1 may include an icon having a different shape from the microphone icon IC2.

[0054] (Variation 4) For example, the mobile terminal 10 may not execute T24 and T26, and may instead simultaneously transmit the voice data and the request to activate the model 152 to the server 100 at T32. Generally speaking, the "first supply unit" and the "second supply unit" may supply the first information or the second information together with the voice data to the voice recognition engine.

[0055] (Variation 5) For example, at T44, the mobile terminal 10 may transmit the user ID "001" to the server 100 instead of transmitting a request to activate the model 154 to the server 100. In this case, the server 100 performs speech recognition using the model 154 associated with the user ID "001." In this variation, the user ID "001" is an example of "second information." In another variation, the mobile terminal 10 may transmit to the server 100 information indicating that the speech is that of a disabled user and the user ID "001."

[0056] (Variation 6) In the above embodiment, one mobile terminal 10 accepts voice inputs from both a disabled user and a non-disabled user, transmits each voice data to server 100, and receives each character string data from server 100. Alternatively, communication system 2 may be constructed by a mobile terminal 10 owned by a disabled user, another mobile terminal owned by a non-disabled user, and server 100. In this variation, mobile terminal 10 transmits a request to activate model 154 and the voice data of the disabled user to server 100, and the other mobile terminal transmits a request to activate model 152 and the voice data of the non-disabled user to server 100. Server 100 then transmits each character string data corresponding to each voice data to mobile terminal 10 and the other mobile terminal. In this case, mobile terminal 10 and the other mobile terminal can each display the same talk screen including character strings corresponding to the voice of the disabled user and character strings corresponding to the voice of the non-disabled user. Generally speaking, the “second providing unit” does not need to transmit both the first voice data and the second voice data to the server 100 .

[0057] (Variation 7) In the above embodiment, the processes in FIGS. 2 and 3 are implemented by software, but at least one of these processes may be implemented by hardware such as a logic circuit.

[0058] Furthermore, the technical elements described in this specification or drawings exhibit technical utility either alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. Furthermore, the technologies illustrated in this specification or drawings simultaneously achieve multiple objectives, and achieving one of those objectives is itself technically useful.

[0059] In the scope of the claims at the time of filing, even if each claim depends on only some of the claims, it is not limited to the fact that each claim can depend on only those some of the claims. To the extent that there is no technical contradiction, each claim can also depend on other claims that were not dependent at the time of filing. In other words, the technology of each claim can be combined in various ways as follows: (Item 1) A computer program for a terminal device, the computer of the terminal device functions as the following units: a first supply unit that supplies a speech recognition engine with information indicating whether the speech is of a user without an articulation disorder or of a user with an articulation disorder; a second supply unit that, when speech input is detected, supplies the speech recognition engine with speech data corresponding to the speech; an acquisition unit that acquires character string data corresponding to the speech data from the speech recognition engine; and a first display control unit that displays a first screen including a character string corresponding to the character string data on a display unit of the terminal device, wherein, when first information for requesting first character string data corresponding to the speech of the user without an articulation disorder is supplied to the speech recognition engine, the acquisition unit acquires the first character string data from the speech recognition engine in response to the speech recognition engine using a first speech recognition model for the user without an articulation disorder, 2. The computer program product of claim 1, wherein the first supply unit supplies the first information to the speech recognition engine when a first operation is performed on the terminal device, and the second information to the speech recognition engine when a second operation different from the first operation is performed on the terminal device.(Item 3) The computer program according to item 2, wherein the computer program further causes the computer to function as a second display control unit that displays a second screen including a first icon and a second icon on the display unit, wherein the first operation includes an operation of selecting the first icon included in the second screen, and the second operation includes an operation of selecting the second icon included in the second screen. (Item 4) The computer program according to item 3, wherein the first screen displayed in response to an operation of the first icon included in the second screen includes a third icon, and the second operation further includes an operation of selecting the third icon included in the first screen. (Item 5) The computer program according to item 4, wherein the third icon has the same shape as the second icon. (Item 6) The computer program according to any one of items 3 to 5, wherein the first screen displayed in response to an operation of the second icon included in the second screen includes a fourth icon, and the first operation further includes an operation of selecting the fourth icon included in the first screen. (Item 7) The computer program according to any one of Items 1 to 6, wherein the speech recognition engine includes a plurality of second speech recognition models corresponding to a plurality of users with the speech disorder, the second information includes information indicating a specific second speech recognition model corresponding to a specific user with the speech disorder, and when the second information is supplied to the speech recognition engine, the acquisition unit acquires the second character string data from the speech recognition engine in accordance with the specific second speech recognition model being used by the speech recognition engine.(Item 8) The computer program of any one of items 1 to 7, wherein the second supply unit supplies first speech data corresponding to a first speech of the user without the articulation disorder to the speech recognition engine, and supplies second speech data corresponding to a second speech of the user with the articulation disorder to the speech recognition engine; the acquisition unit acquires first character string data corresponding to the first speech data from the speech recognition engine, and acquires second character string data corresponding to the second speech data from the speech recognition engine; and the first display control unit causes the display unit to display the first screen including a first character string corresponding to the first character string data and a second character string corresponding to the second character string data.

[0060] 2: Communication system, 10: Mobile terminal, 12: Operation unit, 14: Display unit, 16: Microphone, 18: Communication interface, 30: Control unit, 32: CPU, 34: Memory, 40: OS program, 42: Voice display application, 100: Voice recognition server, 118: Communication interface, 130: Control unit, 132: CPU, 134: Memory, 140: Program, 150: Voice recognition model data, 152: Fault-free voice recognition model, 154, 156: User-specific voice recognition model

Claims

1. A computer program for a terminal device, which causes a computer of the terminal device to function as the following units: a first supply unit that supplies a speech recognition engine with information indicating whether the speech is of a user without an articulation disorder or of a user with an articulation disorder; a second supply unit that, when speech input is detected, supplies the speech recognition engine with speech data corresponding to the speech; an acquisition unit that acquires character string data corresponding to the speech data from the speech recognition engine; and a first display control unit that displays a first screen including a character string corresponding to the character string data on a display unit of the terminal device, wherein, when first information for requesting first character string data corresponding to the speech of the user without an articulation disorder is supplied to the speech recognition engine, the acquisition unit acquires the first character string data from the speech recognition engine in response to the speech recognition engine using a first speech recognition model for the user without an articulation disorder, a computer program product that, when second information for requesting second character string data corresponding to the speech of the user with an articulation disorder is supplied to the speech recognition engine, acquires the second character string data from the speech recognition engine in response to a second speech recognition model for the user with an articulation disorder being utilized by the speech recognition engine.

2. The computer program of claim 1, wherein the first supply unit supplies the first information to the speech recognition engine when a first operation is applied to the terminal device, and supplies the second information to the speech recognition engine when a second operation different from the first operation is applied to the terminal device.

3. The computer program of claim 2, further causing the computer to function as a second display control unit that displays a second screen including a first icon and a second icon on the display unit, wherein the first operation includes an operation of selecting the first icon included in the second screen, and the second operation includes an operation of selecting the second icon included in the second screen.

4. The computer program of claim 3, wherein the first screen displayed in response to an operation on the first icon included on the second screen includes a third icon, and the second operation further includes an operation of selecting the third icon included on the first screen.

5. The computer program of claim 4, wherein the third icon has the same shape as the second icon.

6. The computer program of claim 3, wherein the first screen displayed in response to an operation on the second icon included in the second screen includes a fourth icon, and the first operation further includes an operation of selecting the fourth icon included in the first screen.

7. The computer program of claim 1, wherein the speech recognition engine is provided with a plurality of second speech recognition models corresponding to a plurality of users with the speech disorder, the second information includes information indicating a specific second speech recognition model corresponding to a specific user with the speech disorder, and the acquisition unit acquires the second string data from the speech recognition engine in accordance with the specific second speech recognition model being used by the speech recognition engine when the second information is supplied to the speech recognition engine.

8. The computer program of claim 1, wherein the second supply unit supplies first speech data corresponding to a first speech of the user without the articulation disorder to the speech recognition engine, and supplies second speech data corresponding to a second speech of the user with the articulation disorder to the speech recognition engine; the acquisition unit acquires the first character string data corresponding to the first speech data from the speech recognition engine, and acquires the second character string data corresponding to the second speech data from the speech recognition engine; and the first display control unit causes the display unit to display the first screen including a first character string corresponding to the first character string data and a second character string corresponding to the second character string data.

9. A terminal device comprising: a first supply unit that supplies a speech recognition engine with information indicating whether the speech is of a user without an articulation disorder or of a user with an articulation disorder; a second supply unit that, when speech input is detected, supplies the speech recognition engine with speech data corresponding to the speech; an acquisition unit that acquires character string data corresponding to the speech data from the speech recognition engine; and a first display control unit that displays a first screen including a character string corresponding to the character string data on a display unit of the terminal device, wherein, when first information for requesting first character string data corresponding to the speech of the user without an articulation disorder is supplied to the speech recognition engine, the acquisition unit acquires the first character string data from the speech recognition engine in response to the speech recognition engine using a first speech recognition model for the user without an articulation disorder; a terminal device that, when second information for requesting second character string data corresponding to the voice of the user with an articulation disorder is supplied to the speech recognition engine, acquires the second character string data from the speech recognition engine in response to a second speech recognition model for the user with an articulation disorder being utilized by the speech recognition engine.

10. A method executed by a terminal device, comprising: a first supply step of supplying information indicating whether the speech is of a user without an articulation disorder or of a user with an articulation disorder to a speech recognition engine; a second supply step of supplying speech data corresponding to the speech to the speech recognition engine when speech input is detected; an acquisition step of acquiring character string data corresponding to the speech data from the speech recognition engine; and a first display control step of displaying a first screen including a character string corresponding to the character string data on a display unit of the terminal device, wherein the acquisition step, when first information for requesting first character string data corresponding to the speech of the user without an articulation disorder is supplied to the speech recognition engine, acquires the first character string data from the speech recognition engine in response to the speech recognition engine using a first speech recognition model for the user without an articulation disorder; and obtaining second string data from the speech recognition engine in response to a second speech recognition model for the user with an articulation disorder being utilized by the speech recognition engine when second information for requesting second string data corresponding to the speech of the user with an articulation disorder is supplied to the speech recognition engine.

Images