Programs, devices and methods for assisting a user in exercising his / her mouth and / or tongue

Abstract

To provide a program to assist a user in exercising his / her mouth and / or tongue. When executed by a processor of a device, the program causes the processor to at least: receive a user's voice input; recognize at least one monosyllabic word included in the user's voice input; and display content associated with the recognized at least one monosyllabic word on the device. In one embodiment, recognizing the at least one monosyllabic word may include recognizing the at least one monosyllabic word using a speech recognition model suitable for recognizing multiple different monosyllabic words uttered in rapid succession and / or the same monosyllabic word uttered multiple times in rapid succession.

Description

[Technical field]

[0001] The present invention relates to a program, device and method for assisting a user in exercising their mouth and / or tongue. [Background technology]

[0002] 2. Description of the Related Art Deterioration around the mouth (so-called oral frailty) has attracted attention for some time now (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Japan Dental Association, "Oral frailty", [online], [searched September 26, 2024], Internet<https: / / www.jda.or.jp / enlightenment / oral / about.html> Summary of the Invention [Problem to be solved by the invention]

[0004] Although various mouth and / or tongue exercises are known to improve oral frailty, most of the conventional exercises are monotonous and boring (i.e., not sustainable), and it is unclear whether elderly people are performing such exercises correctly.

[0005] The present invention aims to provide a program, device and method for assisting a user in performing mouth and / or tongue exercises, which enables the user to know from the displayed content that he or she is performing the mouth and / or tongue exercises correctly as the user performs the mouth and / or tongue exercises in a game-like manner. [Means for solving the problem]

[0006] In one aspect of the present invention, the program of the present invention is a program for assisting a user in exercising his or her mouth and / or tongue, which, when executed by a processor unit of a device, causes the processor unit to at least perform the following operations: receive a voice input of the user, recognize at least one monosyllabic word included in the voice input of the user, and display content associated with the recognized at least one monosyllabic word on the device.

[0007] In one embodiment of the invention, recognizing the at least one monosyllabic word may include recognizing the at least one monosyllabic word using a speech recognition model suitable for recognizing a plurality of different monosyllabic words uttered in rapid succession and / or a same monosyllabic word uttered multiple times in rapid succession.

[0008] In one embodiment of the present invention, the training data for the speech recognition model may be a correspondence between monosyllabic words and speech data of the monosyllabic words.

[0009] In one embodiment of the present invention, the program, when executed by the processor unit of the device, may further cause the processor unit to display on the device an indicator for adjusting a first distance between the user's face and the device.

[0010] In one embodiment of the present invention, the indicator may include a display area for displaying the user as captured by the device's camera, a bar representing the difference between the first distance and a predetermined appropriate distance, a numerical value representing the difference between the first distance and the predetermined appropriate distance, or a guide line for guiding the position of the user's face as captured by the device's camera and displayed on the device.

[0011] In one embodiment of the invention, the size of the display area may be adjusted so that the distance between the user's face and the device is within a desired range when the user's face in the display area is at a desired size.

[0012] In one embodiment of the present invention, the program, when executed by the processor unit of the device, further causes the processor unit to determine a volume of the user's voice input, determine whether the volume of the user's voice input exceeds a first threshold, and, if it is determined that the volume of the user's voice input exceeds the first threshold, increase a magnification of the user's face displayed in the display area; and / or, when executed by the processor unit of the device, the program may further cause the processor unit to determine whether the volume of the user's voice input is below a second threshold, and, if it is determined that the volume of the user's voice input is below the second threshold, decrease a magnification of the user's face displayed in the display area.

[0013] In one embodiment of the present invention, the program, when executed by the processor unit of the device, further causes the processor unit to determine a volume of the user's voice input, determine whether the volume of the user's voice input exceeds a first threshold, and, if it is determined that the volume of the user's voice input exceeds the first threshold, perform a process to cause the user to lower the volume of the user's voice input; and / or, when executed by the processor unit of the device, the program may further cause the processor unit to determine whether the volume of the user's voice input is below a second threshold, and, if it is determined that the volume of the user's voice input is below the second threshold, perform a process to cause the user to increase the volume of the user's voice input.

[0014] In one aspect of the present invention, a device of the present invention is a device for assisting a user in exercising his or her mouth and / or tongue, the device including a processor configured to at least receive a voice input of the user, recognize at least one monosyllabic word included in the voice input of the user, and display content associated with the recognized at least one monosyllabic word on the device.

[0015] In one aspect of the present invention, a method of the present invention is a method executed in a device for assisting a user in exercising their mouth and / or tongue, the device having a processor unit, the method including: the processor unit receiving a voice input of the user; the processor unit recognizing at least one monosyllabic word included in the voice input of the user; and the processor unit displaying content associated with the recognized at least one monosyllabic word on the device. [Brief description of the drawings]

[0016] [Figure 1A] FIG. 13 is a diagram showing an example of a screen displayed on a device operated by a user. [Figure 1B] FIG. 13 is a diagram showing another example of a screen displayed on a device operated by a user; [Diagram 2] FIG. 13 is a diagram showing another example of a screen displayed on a device operated by a user; [Diagram 3] FIG. 3 shows an example of a configuration of a device 300 for assisting a user in exercising their mouth and / or tongue. [Figure 4] FIG. 3 is a diagram showing an example of processing executed in the device 300. [Diagram 5] FIG. 13 is a diagram showing another example of the process executed in the device 300. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] The terms used in this specification are defined below.

[0018] A "monosyllabic word" is a word having a single syllable that is composed of a single vowel, or a word having a single syllable that is composed of a combination of a single consonant and a single vowel. Examples of "monosyllabic words" include, but are not limited to, "pa," "ta," "ka," and "ra."

[0019] "Mouth and / or tongue exercises" refers to exercises for stimulating the muscles required for moving the mouth and / or the muscles required for moving the tongue. Therefore, "mouth and / or tongue exercises" includes exercises performed with and without vocalization.

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0021] 1. Assisting the user in exercising their mouth and / or tongue The applicant considered that in order to improve oral frailty, it is necessary to integrate oral frailty testing and training. In addition, considering that elderly people have become interested in games in recent years and that the number of elderly people who can use smartphones is expected to increase in the future, the applicant considered that it would be best to try to improve oral frailty through game applications that can be run on elderly people's devices.

[0022] One of the exercises for the mouth and / or tongue that contributes to the improvement of oral frailty is to utter a specific monosyllabic word multiple times in succession in a short period of time. However, a general speech recognition model cannot recognize that a specific monosyllabic word has been uttered multiple times in succession in a short period of time, even if the user utters the specific monosyllabic word multiple times in succession in a short period of time. This is because the general speech recognition model cannot predict that the voice input of the same monosyllabic word will be received multiple times in succession in a short period of time. For example, if a user utters a specific monosyllabic word "pa" as "papapapapa" in a short period of time, the general speech recognition model cannot predict that after receiving the first voice input of "papa", "papa" will be uttered in a short period of time, and will break down and be unable to recognize what is being input. Therefore, the applicant has newly developed a speech recognition model suitable for recognizing multiple monosyllabic words uttered in succession in a short period of time and the same monosyllabic word uttered multiple times in succession in a short period of time.

[0023] The applicant has also found that stabilizing the volume of the user's voice input is also important in recognizing multiple monosyllabic words uttered in succession within a short period of time and the same monosyllabic word uttered multiple times in succession within a short period of time. If the volume of the user's voice input is not stable, even if the user utters a monosyllabic word multiple times in succession within a short period of time (e.g., "papapapapa"), the device can only recognize that the user uttered the monosyllabic word only once (e.g., "pa"). This is because, for example, if the volume of the user's voice input continues to exceed a predetermined threshold, a monosyllabic word uttered multiple times in succession within a short period of time (e.g., "papapapa") is recognized as a single monosyllabic word with a long sound (e.g., "paaaa"), and a single monosyllabic word with a long sound (e.g., "paaaa") is considered to be a single monosyllabic word with a short sound (e.g., "pa") as the monosyllabic voice input. Therefore, stabilizing the volume of the user's voice input is necessary for the device to more accurately recognize monosyllabic words spoken by the user.

[0024] 2. Screen displayed on the device operated by the user Fig. 1A shows an example of a screen displayed on a device operated by a user. Screen 110 shown in Fig. 1A is a game screen that operates in response to a user's voice input to assist the user in exercising the mouth and / or tongue. Screen 110 may be displayed on the device when a program for assisting the user in exercising the mouth and / or tongue is started on the device and a game for exercising the mouth and / or tongue is started.

[0025] In the embodiment shown in FIG. 1A, the screen 110 includes a display area 111 for displaying an object (e.g., a user) being imaged by the camera of the device. The display area 111 may display a video of the object being imaged in real time, or a still image of the object being imaged. This allows the user to check the area around his / her mouth displayed in the display area 111 (i.e., self-monitoring) and see whether the area around the mouth is moving properly. Also, the display area 111 may display an illustration or animation of a portrait of the user or a specific person. For example, the portrait animation may be an animation that moves in accordance with the movement of the object being imaged in real time by the camera (e.g., the movement of the user's mouth).

[0026] The size of the display area 111 is adjusted so that the distance between the user's face and the device is within a desired range when the user's face in the display area is a desired size (for example, when the user's forehead and chin are displayed so as to be located near the contour of the display area 111, respectively, or when the ratio of the area of ​​the user's face in the display area to the area of ​​the entire display area is within a predetermined range). By setting the distance between the user's face and the device within the desired range, the volume of the user's voice input can be stabilized and the high and low volumes between one monosyllabic word and another monosyllabic word included in the user's voice input can be easily recognized, so that the device can distinguish and recognize each monosyllabic word included in the user's voice input, and therefore, it is possible to more accurately recognize multiple monosyllabic words uttered in succession within a short period of time and the same monosyllabic word uttered multiple times in succession within a short period of time. The desired range is, for example, a range suitable for stabilizing the volume of the user's voice input. Furthermore, a short period of time may be, for example, less than 0.1 seconds, less than 0.5 seconds, less than 1 second, less than 2 seconds, less than 3 seconds, less than 4 seconds, less than 5 seconds, less than 6 seconds, less than 7 seconds, less than 8 seconds, less than 9 seconds, less than 10 seconds, etc., but is not limited to these.

[0027] Fig. 1B shows another example of a screen displayed on a device operated by a user. Screen 120 shown in Fig. 1B is a screen transitioned from screen 110 in Fig. 1A in response to input and recognition of the user's voice.

[0028] As shown on screen 120, when a monosyllabic word among one or more monosyllabic words included in the user's voice input is recognized, content 121 associated with the recognized monosyllabic word is displayed. As a result, the user can recognize that the monosyllabic word is correctly pronounced by the content 121 associated with the recognized monosyllabic word being displayed. Monosyllabic words include, for example, words that produce a plosive sound when the lips are closed and opened, words that are produced by bringing the tip of the tongue close to or in contact with the front of the palate, words that are produced by bringing the base of the tongue close to or in contact with the rear of the palate, and / or words that are produced by pressing the tongue against the palate with the tongue rolled so that the tip of the tongue faces the base of the tongue and using the entire tongue. Words that produce a plosive sound when the lips are closed and opened may include, for example, words consisting of sounds in the ma row, the ba row, the pa row, and the like. Words that are produced by bringing the tip of the tongue close to or in contact with the front of the palate may include, for example, words consisting of sounds in the ta row, the za row, the da row, and the like. Words uttered with the base of the tongue close to or in contact with the back of the palate may include, for example, words consisting of sounds in the ka row and ga row. Words uttered using the whole tongue by pressing the tongue against the palate while rolling the tongue so that the tip faces the base of the tongue may include, for example, words consisting of sounds in the ra row. Monosyllabic words may include words uttered with the mouth opened vertically, words uttered with the teeth closed, words uttered with the mouth pursed, words uttered with the mouth opened horizontally, and the like. Words uttered with the mouth opened vertically may include, for example, words consisting of sounds in the a row. Words uttered with the teeth closed may include, for example, words consisting of sounds in the i row. Words uttered with the mouth pursed may include, for example, words consisting of sounds in the u row and o row. Words uttered with the mouth opened horizontally may include, for example, words consisting of sounds in the e row. In the embodiment shown in FIG. 1B, the user's voice input is recognized as including the monosyllabic word "pa," and the (katakana) content "pa" corresponding to the recognized monosyllabic word "pa" is displayed on screen 110.

[0029] The displayed content 121 moves toward the right side of the screen and may, for example, collide with an item moving from the right side of the screen and disappear. Such a game feature allows the user to enjoy exercising the mouth and / or tongue.

[0030] Furthermore, when a specific monosyllabic word included in the user's voice input is recognized, content 121 associated with the specific monosyllabic word may be displayed on screen 110, and when a monosyllabic word other than the specific monosyllabic word is recognized, content 122 associated with the other monosyllabic word may be displayed on screen 110. This allows the user to recognize that the specific monosyllabic word is being pronounced correctly by the display of content 121 associated with the specific monosyllabic word, and to recognize that the specific monosyllabic word is not being pronounced correctly by the display of content 122 associated with the other monosyllabic word.

[0031] Also, there may be a plurality of speech modes, and the specific monosyllabic word may be different for each speech mode. For example, in a first speech mode, if the monosyllabic word "pa" is a specific monosyllabic word, content 121 is displayed when the user utters "pa", and content 122 is displayed when the user utters "ta", and in a second speech mode different from the first speech mode, if the monosyllabic word "ta" is a specific monosyllabic word, content 122 is displayed when the user utters "pa", and content 121 is displayed when the user utters "ta".

[0032] 1B, the katakana character "pa" has been described as the content 121 corresponding to the recognized specific monosyllabic word "pa", but the present invention is not limited to this. The content 121 corresponding to the recognized specific monosyllabic word "pa" may be, for example, the hiragana character "pa", the alphabet "Pa" including uppercase letters, or the alphabet "pa" consisting of only lowercase letters.

[0033] 1A and 1B, an example has been described in which the screen 110 and the screen 120 have the display area 111, but the present invention is not limited to this. Instead of the display area 111, the screen 110 and the screen 120 may have a complete illustration or animation of a portrait, or may have a character.

[0034] 1A and 1B, the display area 111 has a perfect circle shape, but the present invention is not limited thereto. The display area 111 may have any shape as long as it can display an object captured by the camera. For example, the shape of the display area 111 may be an ellipse, a polygon such as a square or a rectangle, or a complex shape such as a star or gourd shape.

[0035] Additionally, the backgrounds and items shown on screens 110 and 120 can be changed to alternative backgrounds and items depending on the situation, concept, etc., of the mouth and / or tongue exercise being performed.

[0036] Fig. 2 shows another example of a screen displayed on a device operated by a user. Screen 210 shown in Fig. 2 is a game screen that operates in response to a voice input from a user to assist the user in exercising the mouth and / or tongue. Screen 210 may be displayed on the device when a program for assisting the user in exercising the mouth and / or tongue is started on the device and a game for exercising the mouth and / or tongue is started.

[0037] In the embodiment shown in Fig. 2, the screen 210 includes a display area 211 for displaying an object being imaged by the device's camera. The display area 211 in Fig. 2 has a similar configuration to the display area 111 in Fig. 1A, and therefore will not be described in detail here.

[0038] As shown in the screen 210, when a monosyllabic word among a plurality of monosyllabic words included in the user's voice input is recognized, the content 212 associated with the recognized monosyllabic word is displayed. In the embodiment shown in FIG. 2, when the monosyllabic word "pa" is recognized as being included in the user's voice input, a numerical value associated with the recognized monosyllabic word "pa" is displayed as the content 212 associated with the recognized monosyllabic word. The more the monosyllabic word "pa" is recognized, the more the numerical value associated with the recognized monosyllabic word "pa" may change. This allows the user to recognize that the monosyllabic word has been correctly pronounced by the content 211 associated with the recognized monosyllabic word being displayed and / or changed. Note that the numerical value associated with the recognized monosyllabic word may be, for example, a count number of recognized monosyllabic words, or a score representing an accuracy indicating the degree to which the recognized monosyllabic word matches a specific monosyllabic word.

[0039] In addition, when a specific monosyllabic word included in the user's voice input is recognized, a numerical value related to the specific monosyllabic word may be reflected on the screen 210, and when a monosyllabic word other than the specific monosyllabic word is recognized, a numerical value related to the other monosyllabic word may not be reflected on the screen 210. For example, when a specific monosyllabic word included in the user's voice input is recognized, a numerical value related to the specific monosyllabic word increases on the screen 210, and when a monosyllabic word other than the specific monosyllabic word is recognized, a numerical value related to the other monosyllabic word is not reflected on the screen 210, and a numerical value related to the specific monosyllabic word does not change on the screen 210. In this way, the user can recognize that the specific monosyllabic word has been correctly pronounced by the numerical value related to the specific monosyllabic word being reflected on the screen 210, and can recognize that the specific monosyllabic word has not been correctly pronounced by the numerical value related to the specific monosyllabic word not being reflected on the screen 210.

[0040] 2, an example in which the screen 210 includes the display area 211 has been described, but the present invention is not limited to this. Instead of the display area 211, the screen 210 may include a complete illustration or animation of a portrait, or may include a character.

[0041] 2, the display area 211 has a perfect circle shape, but the present invention is not limited thereto. The display area 211 may have any shape as long as it can display an object captured by the camera. For example, the shape of the display area 211 may be an ellipse, a polygon such as a square or a rectangle, or a complex shape such as a star or gourd shape.

[0042] Additionally, the background and items shown on screen 210 can be changed to alternative backgrounds and items depending on the situation, concept, etc., of the mouth and / or tongue exercise being performed.

[0043] 3. Configuring the Device to Assist the User in Exercising the Mouth and / or Tongue FIG. 3 shows one example of a configuration of a device 300 for assisting a user in exercising their mouth and / or tongue.

[0044] The device 300 is an electronic device operated by a user. In the embodiment shown in FIG. 3, the device 300 includes an interface unit 311, a processor unit 312 including one or more CPUs (Central Processing Units), a memory unit 313, an input unit 314, a display unit 315, and a camera 316. The device 300 may be capable of communicating with a server device (not shown) for recording and managing a log of the user's mouth and / or tongue exercises via a network. This makes it possible to perform an examination of the user's mouth and / or tongue based on the log of the user's mouth and / or tongue exercises. For example, the device 300 may be a portable wireless terminal such as a mobile phone, a smartphone, or a tablet terminal, or may be a personal computer such as a laptop PC or a notebook PC.

[0045] The interface unit 311 controls, for example, communication with a server device (not shown).

[0046] The memory unit 313 stores programs required to execute processing, data required to execute the programs, and the like. Here, there is no limit to how the programs are stored in the memory unit 313. For example, the programs may be pre-installed in the memory unit 313. Alternatively, the programs may be installed in the memory unit 313 by being downloaded via a network, or may be installed in the memory unit 313 via a storage medium such as an optical disk or USB.

[0047] The processor unit 312 controls the overall operation of the device 300. The processor unit 312 reads out a program stored in the memory unit 313 and executes the program. This allows the device 300 to function as an apparatus that executes desired steps, and the processor unit 212 of the device 300 to operate as a means for achieving a desired function.

[0048] The input unit 314 is capable of receiving voice input from the user, and is also capable of receiving input in response to a selection action (eg, click, tap) of the user.

[0049] The display unit 315 is capable of displaying, for example, the screen 110 in FIG. 1A, the screen 120 in FIG. 1B, the screen 210 in FIG.

[0050] The camera 316 can capture an image of an object to be captured. The object captured by the camera 316 can be displayed in the display area 111 on the screen 110 in Fig. 1A and the screen 120 in Fig. 1B, and in the display area 211 on the screen 210 in Fig. 2.

[0051] 4. Processing performed on the device Fig. 4 shows an example of processing executed in device 300. Each step shown in Fig. 4 is executed, for example, by processor unit 312 of device 300. Note that a program for assisting the user in exercising the mouth and / or tongue is assumed to be stored in advance in memory unit 313 of device 300. Each step shown in Fig. 4 will be described below.

[0052] Step S401: An indicator for adjusting the first distance between the user's face and the device 300 is displayed on the device 300 (particularly, the display unit 315 of the device 300). This process is executed in response to, for example, a program for supporting the user's mouth and / or tongue exercises being started on the device 300. The indicator includes, for example, a display area for displaying an object imaged by the camera 316 of the device 300, a bar representing the difference between the first distance between the user's face and the device 300 and a predetermined appropriate distance, a numerical value representing the difference between the first distance between the user's face and the device 300 and a predetermined appropriate distance, or a guide line for guiding the position of the user's face imaged by the camera 316 of the device 300 and displayed on the device 300. This display area may be, for example, the display area 111 displayed on the screen 110 of FIG. 1A and the screen 120 of FIG. 1B, or the display area 211 displayed on the screen 210 of FIG. 2. This bar may be, for example, a bar with a slider indicating a first distance between the user's face and the device 300 (or a difference between the first distance between the user's face and the device 300 and a predetermined appropriate distance), or may be a progress bar. This guide line may be, for example, a line indicating a rough guideline for the width of the user's face.

[0053] The size of the display area is adjusted so that the distance between the user's face and the device is within a desired range when the user's face in the display area is a desired size (for example, when the user's forehead and chin are displayed so as to be located near the contour of the display area, or when the ratio of the area of ​​the user's face in the display area to the area of ​​the entire display area is within a predetermined range). This makes it possible to stabilize the distance between the user's face and the device 300, and therefore to stabilize the volume of the user's voice input.

[0054] Note that multiple indicators for adjusting the first distance between the user's face and the device 300 may be simultaneously displayed on the device 300. For example, a display area and a guide line may be simultaneously displayed on the device 300, or a display area and a bar may be simultaneously displayed on the device 300. Also, when an indicator is not displayed on the device 300 (for example, when a complete illustration or animation of a portrait is displayed, or when a character is displayed), step S401 is skipped.

[0055] Step S402: A user's voice input is received. The user's voice input is received, for example, via the input unit 314 of the device 300. The user's voice input includes at least one monosyllabic word. The at least one monosyllabic word included in the user's voice input may include, for example, a plurality of different monosyllabic words uttered in succession (for example, in a short time). The at least one monosyllabic word included in the user's voice input may include, for example, the same monosyllabic word uttered multiple times in succession (for example, in a short time). The at least one monosyllabic word may include a word that produces a plosive sound when the lips are closed and then opened, a sound that is produced by placing the tip of the tongue close to or in contact with the front of the palate, a sound that is produced by placing the base of the tongue close to or in contact with the back of the palate, and / or a sound that is produced by using the entire tongue by pressing the tongue against the palate with the tongue rolled so that the tip of the tongue faces the base of the tongue. At least one monosyllabic word is produced by opening the mouth vertically, closing the dentition, pursing the lips, opening the mouth horizontally, etc.

[0056] Step S403: At least one monosyllabic word included in the user's voice input received in step S402 is recognized. This process is performed, for example, using a voice recognition model suitable for recognizing a plurality of different monosyllabic words uttered in short succession and / or the same monosyllabic word uttered multiple times in short succession. The voice recognition model may be configured to be machine-learned, for example, artificial intelligence (AI). The voice recognition model may be stored, for example, in the memory unit 313 of the device 300, or may be stored in a cloud storage connected via a network. When the voice recognition model is stored in a cloud storage connected via a network, the device 300 can use the voice recognition model via the network.

[0057] The teacher data of the speech recognition model includes, for example, a correspondence between a monosyllabic word and the voice data of the monosyllabic word. The teacher data of the speech recognition model may be, for example, only a correspondence between a monosyllabic word and the voice data of the monosyllabic word. With such teacher data, the device 300 can recognize a monosyllabic word corresponding to the voice data of a monosyllabic word identical or similar to the voice data of the received monosyllabic word as a monosyllabic word uttered by the user. In this way, by limiting the teacher data of the speech recognition model to monosyllabic words or the like, it is possible to improve the recognition ability of a plurality of monosyllabic words uttered in succession in a short period of time and / or the same monosyllabic word uttered multiple times in succession in a short period of time (particularly, the same monosyllabic word uttered multiple times in succession in a short period of time).

[0058] Step S404: Content associated with the at least one monosyllabic word recognized in step S403 is displayed. The content may include, for example, the content 121 displayed on the screen 120 of Fig. 1B or the content 212 displayed on the screen 210 of Fig. 2. The content may further include, for example, the content 122 displayed on the screen 120 of Fig. 1B.

[0059] Fig. 5 shows another example of the processing executed in the device 300. Each step shown in Fig. 5 is executed, for example, by the processor unit 312 of the device 300. Each step shown in Fig. 5 is executed after receiving a voice input from the user in step S402 of Fig. 4. Each step shown in Fig. 5 will be described below.

[0060] Step S501: The volume of the user's voice input received in step S402 of Fig. 4 is determined. For example, the volume of the user's voice input is determined with reference to a volume parameter included in the user's voice input.

[0061] Step S502: It is determined whether the volume of the user's voice input exceeds a first threshold value. If the determination result is "Yes", the process proceeds to step S503, and if the determination result is "No", the process proceeds to step S504.

[0062] Step S503: A process for making the user lower the volume of the user's voice input is executed. The process for making the user lower the volume of the user's voice input includes, for example, a process for displaying a warning on the device 300 indicating that the user's voice is too loud, a process for increasing the magnification rate of the user's face displayed in the display area 111 or the display area 211 displayed on the device 300, and a process for reducing the size and area of ​​the display area 111 or the display area 211 displayed on the device 300. By displaying a warning on the device 300 indicating that the user's voice is too loud, it is possible to induce the user to speak more quietly, and this makes it possible to reduce the volume of the user's voice input received by the device 300. In addition, the process for increasing the magnification rate of the user's face displayed in the display area 111 or the display area 211 displayed on the device 300 can be achieved by, for example, zooming in using the zoom function of the camera 316 of the device 300. By increasing the magnification rate of the user's face displayed in the display area 111 or the display area 211 displayed on the device 300, the user's face becomes larger relative to the display area, so it is possible to guide the user to move the user's face away from the device 300 to adjust the user's face to fit within the display area, and thus it is possible to reduce the volume of the user's voice input received by the device 300 without making the user speak softer. Also, by reducing the size / area of ​​the display area 111 or the display area 211 displayed on the device 300, the user's face becomes larger relative to the display area, so it is possible to guide the user to move the user's face away from the device 300 to adjust the user's face to fit within the display area, and thus it is possible to reduce the volume of the user's voice input received by the device 300 without making the user speak softer.

[0063] When it is determined that the volume of the user's voice input exceeds the first threshold, the device 300 may execute a process of reducing the volume of the user's voice input in addition to or instead of the process of causing the user to lower the volume of the user's voice input, thereby making it possible to adjust the volume of the user's voice input already received by the device 300 to an appropriate volume.

[0064] Step S504: It is determined whether the volume of the user's voice input is below a second threshold. The second threshold is smaller than the first threshold. If the determination result is "Yes", the process proceeds to step S505, and if the determination result is "No", the process ends (normal end).

[0065] Step S505: A process for making the user increase the volume of the user's voice input is executed. The process for making the user increase the volume of the user's voice input includes, for example, a process for displaying a warning on the device 300 indicating that the user's voice is too quiet. By displaying a warning on the device 300 indicating that the user's voice is too quiet, it is possible to induce the user to speak louder, and thus it is possible to increase the volume of the user's voice input received by the device 300. In addition, the process for making the user increase the volume of the user's voice input may include, for example, a process for reducing the magnification of the user's face displayed in the display area 111 or the display area 211 displayed on the device 300. This process may be achieved, for example, by zooming out using a zoom function of the camera 316 of the device 300. By reducing the magnification rate of the user's face displayed in the display area 111 or the display area 211 displayed on the device 300, the user's face becomes smaller relative to the display area, so it is possible to guide the user to move the user's face closer to the device 300 in order to adjust the user's face so that it fits within the display area, and thus it is possible to increase the volume of the user's voice input received by the device 300 without making the user speak louder. In addition, the process of making the user increase the volume of the user's voice input may include, for example, a process of increasing the size and area of ​​the display area 111 or the display area 211 displayed on the device 300. By increasing the size and area of ​​the display area 111 or the display area 211 displayed on the device 300, the user's face becomes smaller relative to the display area, so it is possible to guide the user to move the user's face closer to the device 300 in order to adjust the user's face so that it fits within the display area, and thus it is possible to increase the volume of the user's voice input received by the device 300 without making the user speak louder.

[0066] When it is determined that the volume of the user's voice input is below the second threshold, the device 300 may execute a process of increasing the volume of the user's voice input in addition to or instead of the process of causing the user to increase the volume of the user's voice input, thereby making it possible to adjust the volume of the user's voice input already received by the device 300 to an appropriate volume.

[0067] In the embodiment shown in Figures 4 to 5, an example has been described in which the processor executes a program stored in the memory to realize the processing of each step shown in Figures 4 to 5, but the present invention is not limited to this. At least a part of the processing of each step shown in Figures 4 to 5 may be realized by a hardware configuration such as a control circuit.

[0068] As described above, the present invention has been illustrated using the preferred embodiment of the present invention, but the present invention should not be interpreted as being limited to this embodiment. It is understood that the scope of the present invention should be interpreted only by the claims. It is understood that a person skilled in the art can implement an equivalent scope based on the description of the present invention and technical common sense from the description of the specific preferred embodiment of the present invention. [Industrial Applicability]

[0069] The present invention is useful for providing a program, device, method, etc. for assisting a user in performing mouth and / or tongue exercises, which enables the user to know from the displayed content that they are performing the mouth and / or tongue exercises correctly as they perform the mouth and / or tongue exercises in a game-like manner. [Explanation of symbols]

[0070] 110 screens 111 Display area 120 screens 121 Contents 122 Contents 210 screens 211 Display area 212 Contents 300 devices 311 Interface Section 312 Processor Section 313 Memory section 314 Input section 315 Display section 316 Camera

Claims

1. A program for assisting a user in exercising his / her mouth and / or tongue, the program, when executed by a processor of a device, receiving a voice input of the user; Recognizing at least one monosyllabic word in the speech input of the user; displaying on the device content associated with the recognized at least one monosyllabic word; displaying on the device an indicator for adjusting a first distance between the user's face and the device; A program that causes the processor unit to perform at least the above.

2. 2. The program of claim 1, wherein recognizing the at least one monosyllabic word comprises recognizing the at least one monosyllabic word using a speech recognition model suitable for recognizing a plurality of different monosyllabic words uttered in rapid succession and / or a same monosyllabic word uttered multiple times in rapid succession.

3. 3. The program according to claim 2, wherein the training data for the speech recognition model is a correspondence between monosyllabic words and speech data of the monosyllabic words.

4. The program of claim 1, wherein the indicator includes a display area for displaying the user captured by a camera of the device, a bar representing the difference between the first distance and a predetermined appropriate distance, a numerical value representing the difference between the first distance and the predetermined appropriate distance, or a guide line for guiding the position of the user's face captured by the camera of the device and displayed on the device.

5. 5. The program of claim 4, wherein the size of the display area is adjusted so that when the user's face in the display area is at a desired size, the distance between the user's face and the device is within a desired range.

6. When the program is executed by the processor of the device, determining a volume of the voice input of the user; and determining whether the volume of the user's voice input exceeds a first threshold; increasing a magnification ratio of the user's face displayed within the display area when it is determined that the volume of the voice input of the user exceeds the first threshold value; and / or When the program is executed by the processor of the device, determining whether the volume of the user's voice input is below a second threshold; When it is determined that the volume of the voice input of the user is lower than the second threshold, reducing a magnification ratio of the face of the user displayed within the display area. The program according to claim 5 , further causing the processor unit to perform the following:

7. When the program is executed by the processor of the device, determining a volume of the voice input of the user; and determining whether the volume of the user's voice input exceeds a first threshold; executing a process for causing the user to lower the volume of the voice input of the user when it is determined that the volume of the voice input of the user exceeds the first threshold value; and / or When the program is executed by the processor of the device, determining whether the volume of the user's voice input is below a second threshold; executing a process for causing the user to increase the volume of the voice input of the user when it is determined that the volume of the voice input of the user is below the second threshold; The program according to claim 1 , further causing the processor unit to perform the following:

8. A device for assisting a user in performing mouth and / or tongue exercises, the device comprising a processor unit, the processor unit comprising: receiving a voice input of the user; Recognizing at least one monosyllabic word in the speech input of the user; displaying on the device content associated with the recognized at least one monosyllabic word; displaying on the device an indicator for adjusting a first distance between the user's face and the device; A device that is configured to at least:

9. 1. A method implemented in a device for assisting a user in mouth and / or tongue exercises, the device comprising a processor portion, the method comprising: the processor receiving a voice input from the user; the processor unit recognizing at least one monosyllabic word included in the speech input of the user; displaying, by the processor unit, content associated with the recognized at least one monosyllabic word on the device; and the processor unit displays, on the device, an indicator for adjusting a first distance between the user's face and the device; and A method comprising:

Images