Oral cavity function evaluation device, oral cavity function evaluation system, and oral cavity function evaluation method

Abstract

This oral cavity evaluation device (100) comprises: an acquisition unit (110) which acquires voice data; an extraction unit (120) which extracts feature amounts from the voice data; an S / N ratio calculation unit (115) which calculates an S / N ratio that is the ratio of a first average intensity of sound collected in a period in which the voice is not generated and a second average intensity of sound collected in which the voice is generated; a determination unit (116) which determines an estimation equation; and an evaluation unit (140) which evaluates, on the basis of the determined estimation equation and the extracted feature amounts, a drop condition of an oral cavity function by evaluating the drop condition by using an oral cavity function evaluation index, wherein the determination unit (116) determines, as the estimation equation, a first estimation equation including a feature amount pertaining to the sound pressure when the S / N ratio is greater than a first threshold, and a second estimation equation that does not include the feature amount pertaining to the sound pressure when the S / N ratio is equal to or smaller than the first threshold.

Description

[Technical field]

[0001] The present invention relates to an oral function evaluation device, an oral function evaluation system, and an oral function evaluation method capable of evaluating the oral function of a subject. [Background technology]

[0002] A method has been disclosed in which an instrument for evaluating eating and swallowing function is attached to the neck of the person being evaluated, pharyngeal movement characteristics are obtained as an index (marker) for evaluating eating and swallowing function, and the eating and swallowing function of the person being evaluated is evaluated (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-23676 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the method disclosed in the above Patent Document 1, in order to evaluate oral functions such as eating and swallowing function, it is necessary to wear an instrument on the subject, which may cause discomfort and burden on the subject. In addition, oral function can also be evaluated by visual examination, questioning, palpation, etc. by experts such as dentists, dental hygienists, speech-language-hearing therapists, or internal medicine doctors, but the decline in oral function may be overlooked as a natural symptom of old age, even though elderly people are constantly choking or spilling food due to the effects of aging. If the decline in oral function is overlooked, it can lead to malnutrition due to, for example, a decrease in the amount of food eaten, and malnutrition leads to a decrease in immunity. In addition, it is easy to aspire, which leads to a vicious cycle in which aspiration and a decrease in immunity can lead to aspiration pneumonia.

[0005] By the way, it is possible to evaluate the oral function of the person being evaluated from the voice uttered by the person being evaluated without using such a method, but there were issues with the accuracy of the evaluation because the voice cannot be properly picked up.

[0006] Therefore, an object of the present invention is to provide an oral function evaluation device that can more accurately evaluate oral function while utilizing the voice of the person being evaluated. [Means for solving the problem]

[0007] An oral function evaluation device according to one embodiment of the present invention is an oral function evaluation device that evaluates a state of decline in oral function of a person being evaluated from the voice uttered by the person being evaluated, and includes an acquisition unit that acquires voice data obtained by collecting the voice uttered by the person being evaluated, an extraction unit that extracts features from the acquired voice data, an S / N ratio calculation unit that calculates a first average intensity of the sound collected during a period when the person being evaluated is not making a sound in the acquired voice data and a second average intensity of the sound collected during a period when the person being evaluated is making a sound, and calculates an S / N ratio that is the ratio of the second average intensity to the first average intensity, and The system comprises a determination unit that determines an estimation formula to be used in the evaluation, a calculation unit that calculates an estimate of the oral function of the person being evaluated based on the determined estimation formula and the extracted features, and an evaluation unit that evaluates the decline in the oral function of the person being evaluated by judging the calculated estimate using an oral function evaluation index, wherein if the calculated S / N ratio is greater than a first threshold, the determination unit determines a first estimation formula as the estimation formula, which includes a feature related to sound pressure among the features extracted from the voice data, and if the calculated S / N ratio is equal to or less than the first threshold, the determination unit determines a second estimation formula that does not include the feature related to sound pressure as the estimation formula.

[0008] Moreover, an oral function evaluation system according to one embodiment of the present invention is an oral function evaluation system that evaluates a state of decline in oral function of a person being evaluated from a voice uttered by the person being evaluated, and includes a terminal and an oral function evaluation device connected to the terminal, the terminal having a sound collection device used to collect the voice uttered by the person being evaluated, and a presentation device for presenting the state of decline in oral function of the person being evaluated after evaluation, the oral function evaluation device including an acquisition unit that acquires voice data obtained by collecting the voice uttered by the person being evaluated, an extraction unit that extracts features from the acquired voice data, and a first average intensity of the sound collected during a period when the person being evaluated is not making a sound in the acquired voice data, and a second average intensity of the sound collected during a period when the person being evaluated is making a sound, the first average intensity to the first average intensity; a determination unit that determines an estimation formula to be used in evaluating the oral function of the person being evaluated; a calculation unit that calculates an estimate of the oral function of the person being evaluated based on the determined estimation formula and the extracted features; and an evaluation unit that evaluates the decline in the oral function of the person being evaluated by judging the calculated estimate using an oral function evaluation index, wherein if the calculated S / N ratio is greater than a first threshold, the determination unit determines a first estimation formula including a feature related to sound pressure from the features extracted from the voice data as the estimation formula, and if the calculated S / N ratio is equal to or less than the first threshold, the determination unit determines a second estimation formula not including the feature related to sound pressure as the estimation formula.

[0009] Furthermore, an oral function evaluation method according to one embodiment of the present invention is an oral function evaluation method that is executed by a terminal and an oral function evaluation device, and evaluates a state of decline in oral function of a subject from a voice uttered by the subject, wherein the terminal obtains voice data by collecting the voice uttered by the subject, and the oral function evaluation device acquires the voice data and extracts features from the acquired voice data, calculates a first average intensity of sounds collected during a period when the subject is not making a sound in the acquired voice data and a second average intensity of sounds collected during a period when the subject is making a sound, calculates an S / N ratio that is a ratio of the second average intensity to the first average intensity, and evaluates the state of decline in oral function of the subject from the voice uttered by the subject, The method includes determining an estimation formula to be used in evaluating oral function, calculating an estimate of the oral function of the person being evaluated based on the determined estimation formula and the extracted features, evaluating the decline in the oral function of the person being evaluated by judging the calculated estimate using an oral function evaluation index, and the terminal presenting the decline in the oral function of the person being evaluated.In determining the estimation formula, if the calculated S / N ratio is greater than a first threshold, a first estimation formula including a feature related to sound pressure from the features extracted from the voice data is determined as the estimation formula, and if the calculated S / N ratio is equal to or less than the first threshold, a second estimation formula not including the feature related to sound pressure is determined as the estimation formula. Effect of the Invention

[0010] According to the oral function evaluation method of the present invention, it is possible to more accurately evaluate oral function while utilizing the voice of the person being evaluated. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing a configuration of an oral cavity function evaluation system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a characteristic functional configuration of the oral cavity function evaluation system according to the embodiment. [Figure 3A] 1 is a flowchart showing the processing steps for evaluating the oral function of a subject using an oral function evaluation method in accordance with an embodiment of the present invention. [Figure 3B] 13 is a flowchart showing a processing procedure for determining an estimation equation in the oral cavity function evaluation method according to the embodiment. [Figure 3C] 11 is a diagram showing an example of information output in the oral cavity function evaluation method of the embodiment. FIG. [Figure 3D] 1 is a graph showing the relationship between the determination of an estimation formula and accuracy (estimation precision) in an oral function evaluation method according to an embodiment. [Figure 4] A diagram showing an overview of a method for acquiring the voice of a subject using an oral function evaluation method in accordance with an embodiment of the present invention. [Figure 5A] FIG. 13 is a diagram showing an example of audio data showing the speech of the person being evaluated saying, "I've decided to draw." [Figure 5B] FIG. 13 is a diagram showing an example of changes in formant frequency in a voice uttered by a person being evaluated saying, "I've decided to draw." [Figure 6] This figure shows an example of audio data showing the subject repeatedly speaking "karakara kara..." [Figure 7] A figure showing an example of audio data showing the subject speaking "What?" [Figure 8] FIG. 1 is a diagram showing an example of a Chinese phrase or fixed phrase that is similar to a Japanese phrase or fixed phrase in the degree of tongue movement or mouth opening and closing when pronounced. [Figure 9A] FIG. 1 is a diagram showing the International Phonetic Alphabet of vowels. [Figure 9B] FIG. 1 is a diagram showing the International Phonetic Alphabet of consonants. [Figure 10A] A figure showing an example of audio data showing the subject speaking "gao dao wu da ka ji ke da yi wu zhe." [Figure 10B] This figure shows an example of changes in formant frequency in the speech of the person being evaluated saying "gao dao wu da ka ji ke da yi wu zhe." [Figure 11] FIG. 1 is a diagram showing an example of an oral function evaluation index. [Figure 12]FIG. 13 is a diagram showing an example of an evaluation result for each element of oral cavity function. [Figure 13] FIG. 13 is a diagram showing an example of an evaluation result for each element of oral cavity function. [Figure 14] 11 is an example of predetermined data used when making suggestions regarding oral cavity function. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component arrangement and connection forms, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in an independent claim showing a top concept will be described as optional components.

[0013] In addition, each drawing is a schematic diagram and is not necessarily a precise illustration. In each drawing, the same reference numerals are used for substantially the same configurations, and duplicated explanations may be omitted or simplified.

[0014] (Embodiment) [Elements of oral function] The present invention relates to a method for evaluating a decline in oral function, and oral function has various elements.

[0015] For example, elements of oral function include tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function. Here, we will briefly explain tongue coating, dry mouth, bite force, tongue pressure, and chewing function.

[0016] Tongue coating indicates the amount of bacteria or food deposits on the tongue (i.e., oral hygiene). When there is no tongue coating or it is thin, it indicates that there is an environment of mechanical abrasion (e.g., food intake), that there is a cleansing effect of saliva, and that swallowing movements (tongue movement) are normal. On the other hand, when the tongue coating is thick, the tongue movement is poor and it is difficult to eat, which indicates that there is a risk of nutritional deficiency or muscle weakness. Dry mouth is the degree of dryness of the tongue, and if it is dry, the movement required for speaking is inhibited. Also, food is crushed after being taken into the mouth, but it is difficult to swallow as it is, so saliva works to hold the crushed food together to make it easier to swallow. However, if the mouth is dry, it is difficult to form a food bolus (a mass of crushed food). Biting force is the force with which hard food is chewed, and is the strength of the jaw muscles. Tongue pressure is an indicator of the force with which the tongue presses against the palate. If tongue pressure becomes weak, swallowing movements may become difficult. In addition, when tongue pressure weakens, the speed at which the tongue moves may slow down, which may result in a slower speech rate. Chewing function is a comprehensive function of the oral cavity.

[0017] In the present invention, the deterioration state of the oral function of the person being evaluated (e.g., the deterioration state of the elements of oral function) can be evaluated from the voice uttered by the person being evaluated. This is because the voice uttered by the person being evaluated whose oral function has deteriorated has specific characteristics, and by extracting these as prosodic features, the oral function of the person being evaluated can be evaluated. The present invention is realized by an oral function evaluation method, a program for causing a computer or the like to execute the method, an oral function evaluation device which is an example of the computer, and an oral function evaluation system including the oral function evaluation device. Below, the oral function evaluation method and the like will be explained while showing the oral function evaluation system.

[0018] [Configuration of oral function evaluation system] The configuration of an oral cavity function evaluation system 200 according to an embodiment will be described.

[0019] FIG. 1 is a diagram showing the configuration of an oral cavity function evaluation system 200 according to an embodiment.

[0020] The oral function evaluation system 200 is a system for evaluating the oral function of the subject U by analyzing the voice of the subject U, and as shown in Figure 1, it comprises an oral function evaluation device 100 and a mobile terminal 300 (an example of a terminal).

[0021] The oral function evaluation device 100 is a device that acquires voice data indicating the voice uttered by the person to be evaluated U using the mobile terminal 300, and evaluates the oral function of the person to be evaluated U from the acquired voice data.

[0022] The mobile terminal 300 is a sound collection device that collects the speech of the subject U who speaks a phrase or a fixed phrase, which is composed of two or more morae including a change in the first or second formant frequency, or which includes at least one of a pop, a plosive, an unvoiced sound, a geminated consonant, and a fricative, in a non-contact manner, and outputs audio data indicating the collected speech to the oral function evaluation device 100. For example, the mobile terminal 300 is a smartphone or a tablet having a microphone. Note that the mobile terminal 300 is not limited to a smartphone or a tablet, and may be, for example, a notebook PC, as long as it has a sound collection function. The oral function evaluation system 200 may also include a sound collection device (microphone) instead of the mobile terminal 300. The oral function evaluation system 200 may also include an input interface for acquiring personal information of the subject U who is evaluated. The input interface is not particularly limited as long as it has an input function such as a keyboard or a touch panel. The volume of the microphone may also be set in the oral function evaluation system 200.

[0023] The mobile terminal 300 may be a display device having a display and displaying an image based on image data output from the oral function evaluation device 100. That is, the mobile terminal 300 is an example of a presentation device for presenting information output from the oral function evaluation device 100 as an image. The display device does not have to be the mobile terminal 300, and may be a monitor device configured with a liquid crystal panel or an organic EL panel. That is, in this embodiment, the mobile terminal 300 is both a sound collection device and a display device, but the sound collection device (microphone), the input interface, and the display device may be provided separately.

[0024] The oral function evaluation device 100 and the mobile terminal 300 need only be capable of sending and receiving audio data or image data for displaying images showing the evaluation results described below, and may be connected via a wired or wireless connection.

[0025] The oral function evaluation device 100 analyzes the voice of the person to be evaluated U based on the voice data collected by the mobile terminal 300, evaluates the oral function of the person to be evaluated U from the analysis result, and outputs the evaluation result. For example, the oral function evaluation device 100 outputs to the mobile terminal 300 image data for displaying an image showing the evaluation result, or data for making suggestions regarding the oral cavity of the person to be evaluated U that are generated based on the evaluation result. In this way, the oral function evaluation device 100 can notify the person to be evaluated U of the level of oral function and suggestions for preventing the deterioration of oral function, etc., so that the person to be evaluated U can prevent or improve the deterioration of oral function, for example.

[0026] The oral function evaluation device 100 is, for example, a personal computer, but may also be a server device. The oral function evaluation device 100 may also be a mobile terminal 300. That is, the mobile terminal 300 may have the functions of the oral function evaluation device 100 described below.

[0027] 2 is a block diagram showing a characteristic functional configuration of an oral function evaluation system 200 according to an embodiment. The oral function evaluation device 100 includes an acquisition unit 110, an S / N ratio calculation unit 115, a determination unit 116, an extraction unit 120, a calculation unit 130, an evaluation unit 140, an output unit 150, a proposal unit 160, a storage unit 170, and an information output unit 180.

[0028] The acquisition unit 110 acquires voice data obtained by the mobile terminal 300 collecting the voice uttered by the person to be evaluated U in a non-contact manner. The voice is a voice in which the person to be evaluated U speaks a phrase or a fixed sentence consisting of two or more morae including a change in the first formant frequency or a change in the second formant frequency. Or, the voice is a voice in which the person to be evaluated U speaks a phrase or a fixed sentence including at least one of a pop sound, a plosive sound, a voiceless sound, a geminated consonant, and a fricative sound. However, in some situations described later, the voice may be a voice in which an arbitrary sentence is spoken. The acquisition unit 110 may further acquire personal information of the person to be evaluated U. For example, the personal information is information input to the mobile terminal 300, and is age, weight, height, sex, BMI (Body Mass Index), dental information (for example, number of teeth, presence or absence of dentures, location of occlusal support, number of functional teeth, number of remaining teeth, etc.), serum albumin value, eating rate, etc. The personal information may be acquired by a swallowing screening tool called EAT-10, the Seirei Swallowing Questionnaire, a medical interview, the Barthel Index, a basic checklist, etc. The acquisition unit 110 is, for example, a communication interface that performs wired communication or wireless communication.

[0029] The S / N ratio calculation unit 115 is a processing unit that calculates the S / N ratio in the acquired voice data. The S / N ratio in the voice data is the ratio of the first average intensity of the sound collected in the acquired voice data during the period when the assessee U is not making a sound (period of only background noise) to the second average intensity of the sound collected in the period when the assessee U is making a sound. Therefore, the S / N ratio calculation unit 115 is configured to be able to extract the sound for the period when the assessee U is not making a sound from the voice data and calculate the first average intensity, and to extract the sound for the period when the assessee U is making a sound from the voice data and calculate the second average intensity. Specifically, the S / N ratio calculation unit 115 is realized by a processor, a microcomputer, or a dedicated circuit.

[0030] The determination unit 116 is a processing unit that determines an estimation formula to be used when the calculation unit 130 calculates an estimate of the oral function of the assessee U based on the S / N ratio calculated by the S / N ratio calculation unit 115. Specifically, the determination unit 116 determines an estimation formula to be used for estimation from among several estimation formula candidates including at least a first estimation formula and a second estimation formula that are set in advance, taking into account the influence of background noise expected from the S / N ratio. The estimation formula candidates are stored in the storage unit 170 or the like in a state of being calculated in advance based on multiple learning data. The determination unit 116 then determines an estimation formula to be used for estimation from the estimation formula candidates stored in the storage unit 170, and stores the estimation formula data 171 separately in the storage unit 170. Specifically, the determination unit 116 is realized by a processor, a microcomputer, or a dedicated circuit.

[0031] The extraction unit 120 is a processing unit that analyzes the voice data of the assessee U acquired by the acquisition unit 110. Specifically, the extraction unit 120 is realized by a processor, a microcomputer, or a dedicated circuit.

[0032] The extraction unit 120 calculates prosodic features from the speech data acquired by the acquisition unit 110. The prosodic features are numerical values ​​indicating features of the speech of the assessee U, which are extracted from the speech data used by the evaluation unit 140 to evaluate the oral function of the assessee U. The prosodic features may include at least one of the speech rate, the sound pressure difference, the time change in the sound pressure difference, the first formant frequency, the second formant frequency, the amount of change in the first formant frequency, the amount of change in the second formant frequency, the time change in the first formant frequency, the time change in the second formant frequency, and the time of a plosive sound.

[0033] The calculation unit 130 calculates an estimate of the oral function of the assessee U based on the prosodic features extracted by the extraction unit 120 and the determined estimation formula. Specifically, the calculation unit 130 is realized by a processor, a microcomputer, or a dedicated circuit.

[0034] The evaluation unit 140 evaluates the deterioration state of the oral function of the subject U by determining the estimated value calculated by the calculation unit 130 using the oral function evaluation index. Index data 172 indicating the oral function evaluation index is stored in the storage unit 170. Specifically, the evaluation unit 140 is realized by a processor, a microcomputer, or a dedicated circuit.

[0035] The output unit 150 outputs the estimated value calculated by the calculation unit 130 to the proposal unit 160. The output unit 150 may also output the evaluation result of the oral cavity function of the assessee U evaluated by the evaluation unit 140 to a mobile terminal 300 or the like. Specifically, the output unit 150 is realized by a processor, a microcomputer, or a dedicated circuit, and a communication interface that performs wired or wireless communication.

[0036] The suggestion unit 160 compares the estimated value calculated by the calculation unit 130 with predetermined data to make a suggestion regarding the oral cavity function of the person to be evaluated U. The proposal data 173, which is the predetermined data, is stored in the storage unit 170. The suggestion unit 160 may also compare personal information acquired by the acquisition unit 110 with the proposal data 173 to make a suggestion regarding the oral cavity of the person to be evaluated U. The suggestion unit 160 outputs the suggestion to the mobile terminal 300. The suggestion unit 160 is realized by, for example, a processor, a microcomputer or a dedicated circuit, and a communication interface that performs wired or wireless communication.

[0037] The storage unit 170 is a storage device that stores data (not shown) of candidates for an estimated formula of oral function calculated based on multiple learning data, estimated formula data 171 indicating the estimated formula determined by the determination unit 116, index data 172 indicating an oral function evaluation index for determining an estimated value of the oral function of the person being evaluated U, proposal data 173 indicating a relationship between the estimated value of the oral function and the proposal content, and personal information data 174 indicating the above-mentioned personal information of the person being evaluated U. The estimated formula data 171 is referred to by the calculation unit 130 when the estimated value of the oral function of the person being evaluated U is calculated. The index data 172 is referred to by the evaluation unit 140 when the deterioration state of the oral function of the person being evaluated U is evaluated. The proposal data 173 is referred to by the proposal unit 160 when a proposal regarding the oral function of the person being evaluated U is made. The personal information data 174 is, for example, data acquired via the acquisition unit 110. The personal information data 174 may be stored in the storage unit 170 in advance. The storage unit 170 is realized by, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), a semiconductor memory, a Hard Disk Drive (HDD), or the like.

[0038] The information output unit 180 is a processing unit that outputs information for increasing the S / N ratio. When the calculated S / N ratio does not meet a certain standard, the information output unit 180 generates and outputs instruction information for improving the environment for collecting the voice uttered by the assessee U. Specifically, the information output unit 180 is realized by a processor, a microcomputer, or a dedicated circuit.

[0039] The storage unit 170 may also store programs executed by a computer to realize the S / N ratio calculation unit 115, the determination unit 116, the extraction unit 120, the calculation unit 130, the evaluation unit 140, the output unit 150, the proposal unit 160, and the information output unit 180, image data showing the evaluation results used when outputting the evaluation results of the oral function of the subject U, and data such as images, videos, sounds, or text showing the contents of the proposal. The storage unit 170 may also store instruction images, which will be described later.

[0040] Although not shown, the oral function evaluation device 100 may include an instruction unit for instructing the subject U to pronounce a phrase or a fixed phrase consisting of two or more moras including a change in the first or second formant frequency, or including at least one of a pop, a plosive, an unvoiced sound, a geminated consonant, and a fricative. Specifically, the instruction unit acquires image data of an instruction image for instructing the subject U to pronounce the phrase or fixed phrase, or audio data of an instruction audio, stored in the storage unit 170, and outputs the image data or the audio data to the mobile terminal 300.

[0041] [Oral function evaluation method procedure] Next, a specific process procedure of the oral cavity function evaluation method executed by the oral cavity function evaluation device 100 will be described.

[0042] Fig. 3A is a flowchart showing a process for evaluating the oral cavity function of the subject U by the oral cavity function evaluation method according to the embodiment. Fig. 4 is a diagram showing an outline of a method for acquiring the voice of the subject U by the oral cavity function evaluation method.

[0043] First, the instruction unit instructs the user to pronounce a phrase or a fixed phrase consisting of two or more moras including a change in the first or second formant frequency, or including at least one of a pop, a plosive, an unvoiced sound, a geminated consonant, and a fricative (step S101). For example, in step S101, the instruction unit acquires image data of an image for instructing the user U stored in the storage unit 170, and outputs the image data to the mobile terminal 300. Then, as shown in (a) of FIG. 4, an image for instructing the user U is displayed on the mobile terminal 300. In addition, in (a) of FIG. 4, "I decided to draw a picture" is shown as an example of a fixed phrase, but it may be instructed to utter a fixed phrase such as "The old man who made the flowers bloom and the monkey and the crab have a contest," "Draw a picture of a flower," or "The sunflowers have bloomed." Also, the instruction may be to utter phrases such as "ippai", "ittai", "ikkai", "pattan", "kappa", "shippo", "kikkari", and "kantteni". Also, the instruction may be to utter phrases such as "kara", "sara", "chara", "jara", "shara", "kyara", and "pura". Also, the instruction may be to utter phrases such as "aei", "iea", "ai", "ia", "kakeki", "kikeka", "nane ni", "chiteta", "papepi", "pipepapa", "katepi", "chipeka", "kaki", "tachi", "papi", "misa", "rari", "wani", "niwa", "eo", "io", "iu", "teko", "kiro", "teru", "peko", "memo", and "emo". The instruction to pronounce may be an instruction to repeatedly utter such phrases.

[0044] The instruction unit may also obtain audio data of the audio instructions to the person being evaluated U stored in the storage unit 170 and output the audio data to the mobile terminal 300, thereby giving the above-mentioned instructions using the audio instructions to pronounce the sound without using an instruction image to pronounce the sound. Furthermore, an evaluator (family member, doctor, etc.) who wishes to evaluate the oral function of the person being evaluated U may give the above-mentioned instructions to the person being evaluated U in his or her own voice, without using an instruction image and audio to pronounce the sound.

[0045] For example, a phrase or a set phrase to be uttered may include two or more vowels or a combination of a vowel and a consonant, which requires opening and closing of the mouth or moving the tongue back and forth to be uttered. For example, an example of such a phrase or set phrase in Japanese is "I decided to draw an image." To utter "e" in "I decided to draw an image," the tongue must move back and forth, and to utter "kimeta" in "I decided to draw an image," the mouth must open and close. The "e" portion of "I decided to draw an image" includes the second formant frequencies of the vowel "e" and the vowel "o," and also includes the amount of change in the second formant frequency because the vowel "e" and the vowel "o" are adjacent to each other. This portion also includes the change in the second formant frequency over time. The "kimeta" part of "I decided to draw a picture" contains the first formant frequencies of the vowel "i", the vowel "e", and the vowel "a", and also contains the amount of change in the first formant frequency because the vowel "i", the vowel "e", and the vowel "a" are adjacent to each other. This part also contains the time change in the first formant frequency. By speaking "I decided to draw a picture", it is possible to extract prosodic features such as sound pressure difference, first formant frequency, second formant frequency, amount of change in first formant frequency, amount of change in second formant frequency, time change in first formant frequency, time change in second formant frequency, and speaking rate.

[0046] For example, a spoken fixed phrase may include a repetition of a phrase consisting of a popping sound and a consonant different from the popping sound. For example, an example of such a fixed phrase in Japanese is "karakara kara...". By repeatedly speaking "karakara kara...", it is possible to extract prosodic features such as sound pressure difference, time change in sound pressure difference, time change in sound pressure, number of repetitions, etc.

[0047] For example, a phrase or a fixed phrase to be uttered may include at least one combination of a vowel and a plosive. For example, in Japanese, an example of such a phrase is "itai" ("whatever"). By uttering "itai" ("whatever"), prosodic features such as sound pressure difference and the duration of the plosive (the duration between vowels) can be extracted.

[0048] Incidentally, since the prosodic feature of the sound pressure difference is easily affected by background noise, there is a possibility that the prosodic feature of the sound pressure difference may adversely affect the accuracy of the estimation of the estimated value, particularly in a sound collection environment with a relatively small S / N ratio. Therefore, in the present invention, an estimation formula is determined so as to change the influence of the feature of the sound pressure difference on the calculation of the estimated value, according to the S / N ratio calculated in the S / N ratio calculation unit 115. In this way, in the present invention, it is possible to estimate the estimated value while reducing the possibility that the prosodic feature of the sound pressure difference may adversely affect the accuracy of the estimation of the estimated value.

[0049] Specific operations such as processing for this purpose will be described with reference to Figs. 3B to 3D. Fig. 3B is a flowchart showing a processing procedure for determining an estimated formula in the oral cavity function evaluation method according to the embodiment. Fig. 3C is a diagram showing an example of information output in the oral cavity function evaluation method according to the embodiment. Fig. 3D is a graph showing the relationship between determining an estimated formula and accuracy (estimation precision) in the oral cavity function evaluation method according to the embodiment.

[0050] As shown in Fig. 3B, the S / N ratio calculation unit 115 measures background noise and calculates a first average intensity (sound pressure) of only the background noise in order to calculate the S / N ratio (step S201). In measuring the background noise, it is sufficient to extract and use the sound during the period when the assessee U is not making a sound. For example, as described above, when the assessee U is speaking a specified phrase or fixed phrase, the sound may be extracted during the period when only background noise exists before and after the phrase or fixed phrase, or if there is a portion in the fixed phrase where the sound is interrupted, the portion may be treated as the period when only background noise exists, and the sound may be extracted.

[0051] Next, the S / N ratio calculation unit 115 calculates the second average intensity (sound pressure) of the subject U's speech in order to calculate the S / N ratio (step S202). At this time, the sound of the subject U speaking the specified phrase or fixed phrase may be used, or an instruction to speak any phrase or fixed phrase may be given separately to collect the sound. Alternatively, if the subject U is in a conversation with someone immediately before the evaluation of the oral function is performed, the first average intensity and the second average intensity may be calculated using that situation.

[0052] Then, the S / N ratio calculation unit 115 calculates the ratio of the second average intensity to the first average intensity to calculate the S / N ratio (step S203). Here, the calculated S / N ratio is output to the information output unit 180. Then, the information output unit 180 judges whether the S / N ratio is greater than a second threshold value (step S204). If it is judged that the S / N ratio is equal to or less than the second threshold value (No in S204), the information output unit 180 generates and outputs information for improving the sound collection environment to increase the S / N ratio (step S205).

[0053] For example, FIG. 3C shows a state in which the mobile terminal 300 displays "Check the microphone connection status or increase the volume of your voice when speaking" as an example of the case where such information is output. In this way, the information is output to instruct the user to increase the S / N ratio by at least one of reducing background noise, i.e., decreasing the second average intensity, and increasing the volume of the voice when speaking, i.e., increasing the first average intensity. Note that the mobile terminal 300 may display "Move the sound collection location" so as to reduce the environmental sound when the subject speaks.

[0054] Returning to FIG. 3B, if it is determined that the S / N ratio is greater than the second threshold (Yes in S204), the information output unit 180 does nothing in particular and proceeds to step S206. Specifically, the calculated S / N ratio is also output to the determination unit 116. The determination unit 116 determines whether the S / N ratio is greater than the first threshold (step S206). If it is determined that the S / N ratio is equal to or less than the first threshold (No in S206), the determination unit 116 determines the second estimation formula, which does not include a feature related to sound pressure among the prosodic features extracted from the speech data, as the estimation formula to be used for estimation (step S208), and causes the estimation formula data 171 to be stored in the storage unit 170. On the other hand, if it is determined that the S / N ratio is greater than the first threshold value (Yes in S206), the determination unit 116 determines the first estimation equation, which includes a feature related to sound pressure among the prosodic features extracted from the speech data, as the estimation equation to be used for estimation (step S207), and stores the estimation equation data 171 in the memory unit 170.

[0055] In this way, an estimation formula that is changed according to the S / N ratio in terms of whether or not the prosodic feature related to sound pressure is included is determined and used to estimate the estimated value. For example, in Fig. 3D, (a) shows the relationship between the S / N ratio and the estimation accuracy when the same estimation formula is used uniformly without considering the S / N ratio, and (b) shows the relationship between the S / N ratio and the estimation accuracy when an estimation formula that is changed according to the S / N ratio in terms of whether or not the prosodic feature related to sound pressure is included is determined.

[0056] As shown in FIG. 3D, when the S / N ratio is greater than the first threshold, both (a) and (b) show the same estimation accuracy. However, when the S / N ratio is in the range of less than the first threshold and greater than the second threshold, the estimation accuracy of (a) of FIG. 3D is lower than that of (b) of FIG. 3D because the prosodic feature related to the sound pressure affected by the influence of background noise reduces the estimation accuracy. Also, as shown by the dashed line in (b) of FIG. 3D, when the S / N ratio is less than the second threshold, an instruction to increase the S / N ratio is issued, so that before the estimated value is estimated, the environment is changed to one with an improved S / N ratio, and processing is performed again from acquisition of speech data, so that the estimated value is difficult to estimate in a state of low estimation accuracy. However, even when the S / N ratio is less than the second threshold, the estimation accuracy is higher than that of (a) of FIG. 3D, so it can be said that estimating the estimated value in this state is useful. Returning to the explanation of FIG. 3A, the voice data may be obtained by collecting the voice of the subject U speaking a phrase or a fixed phrase at least twice at different speaking speeds. For example, the subject U is instructed to speak "I've decided to draw a picture" at a normal speed and at a faster speed. By having the subject U speak "I've decided to draw a picture" at a normal speed and at a faster speed, the degree of retention of the oral function state can be estimated.

[0057] Next, as shown in Fig. 3A, the acquiring unit 110 acquires the voice data of the assessee U who received the instruction in step S101 via the mobile terminal 300 (step S102). As shown in Fig. 4(b), in step S102, for example, the assessee U utters a phrase or a fixed phrase such as "I've decided to draw a picture" toward the mobile terminal 300. The acquiring unit 110 acquires the phrase or fixed phrase uttered by the assessee U as voice data.

[0058] Next, the extraction unit 120 extracts prosodic features from the speech data acquired by the acquisition unit 110 (step S103).

[0059] For example, when the speech data acquired by the acquisition unit 110 is speech data obtained from a speech utterance of "I've decided to draw a picture," the extraction unit 120 extracts the sound pressure difference, the first formant frequency, the second formant frequency, the amount of change in the first formant frequency, the amount of change in the second formant frequency, the time change in the first formant frequency, the time change in the second formant frequency, and the speaking rate as prosodic features. This will be described with reference to Figures 5A and 5B.

[0060] FIG. 5A is a diagram showing an example of voice data showing the voice of the assessee U uttering "I decided to draw a picture." The horizontal axis of the graph shown in FIG. 5A is time, and the vertical axis is power (sound pressure). The unit of power shown on the vertical axis of the graph in FIG. 5A is decibels (dB).

[0061] In the graph shown in FIG. 5A, the changes in sound pressure corresponding to "e", "wo", "ka", "ku", "ko", "to", "ni", "ki", "me", "ta", and "yo" are confirmed. In step S102 shown in FIG. 3A, the acquisition unit 110 acquires the speech data shown in FIG. 5A from the assessee U. In step S103 shown in FIG. 3A, the extraction unit 120 extracts, by a known method, the sound pressures of "k" and "a" in "ka", the sound pressures of "k" and "o" in "ko", the sound pressures of "t" and "o" in "to", and the sound pressures of "t" and "a" in "ta". From the extracted sound pressures of "k" and "a", the extraction unit 120 extracts the sound pressure difference Diff_P(ka) between "k" and "a" as a prosodic feature. Similarly, the extraction unit 120 extracts the sound pressure difference Diff_P(ko) between "k" and "o", the sound pressure difference Diff_P(to) between "t" and "o", and the sound pressure difference Diff_P(ta) between "t" and "a" as prosodic features. For example, the sound pressure difference can be used to evaluate oral cavity function related to swallowing force (pressure of the tongue contacting the palate) or food holding force. In addition, the sound pressure difference including "k" can be used to evaluate oral cavity function related to the ability to prevent food and drink from entering the throat.

[0062] 5B is a diagram showing an example of a change in formant frequency of a voice uttered by the assessee U saying, "I decided to draw a picture." Specifically, FIG. 5B is a graph for explaining an example of a change in the first formant frequency and the second formant frequency.

[0063] The first formant frequency is the first amplitude peak frequency counting from the low frequency side of human voice, and is known to be a good reflector of the characteristics related to opening and closing of the mouth. The second formant frequency is the second amplitude peak frequency counting from the low frequency side of human voice, and is known to be a good reflector of the influence related to the front and back movement of the tongue.

[0064] The extraction unit 120 extracts the first formant frequency and the second formant frequency of each of a plurality of vowels as prosodic features from the speech data representing the speech uttered by the assessee U. For example, the extraction unit 120 extracts the second formant frequency F2e corresponding to the vowel "e" and the second formant frequency F2o corresponding to the vowel "o" in "e wo" as prosodic features. Also, for example, the extraction unit 120 extracts the first formant frequency F1i corresponding to the vowel "i", the first formant frequency F1e corresponding to the vowel "e", and the first formant frequency F1a corresponding to the vowel "a" in "kimeta" as prosodic features.

[0065] Furthermore, the extraction unit 120 extracts the amount of change in the first formant frequency and the amount of change in the second formant frequency of a string of consecutive vowels as prosodic features. For example, the extraction unit 120 extracts the amount of change (F2e-F2o) between the second formant frequency F2e and the second formant frequency F2o, and the amount of change (F1e-F1i, F1a-F1e, F1a-F1i) between the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a as prosodic features.

[0066] Furthermore, the extraction unit 120 extracts, as prosodic features, the time change of the first formant frequency and the time change of the second formant frequency of a character string in which vowels are consecutive. For example, the extraction unit 120 extracts, as prosodic features, the time change of the second formant frequency F2e and the second formant frequency F2o, and the time change of the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a. FIG. 5B shows an example of the time change of the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a, where the time change is ΔF1 / ΔTime. This ΔF1 is F1a-F1i.

[0067] For example, the second formant frequency, the amount of change in the second formant frequency, or the change over time of the second formant frequency can be used to evaluate oral cavity function related to the movement of gathering food (front-back and left-right movement of the tongue). Also, for example, the first formant frequency, the amount of change in the first formant frequency, or the change over time of the first formant frequency can be used to evaluate oral cavity function related to the ability to crush food. Also, the change over time of the first formant frequency can be used to evaluate oral cavity function related to the ability to move the mouth quickly.

[0068] Also, as shown in FIG. 5A, the extraction unit 120 may extract the speech speed as a prosodic feature. For example, the extraction unit 120 may extract the time from when the assessee U starts speaking "I decided to draw a picture" to when he finishes speaking as a prosodic feature. Also, for example, the extraction unit 120 may extract the time from when a specific part of "I decided to draw a picture" starts speaking to when he finishes speaking as a prosodic feature, not limited to the time until the entirety of "I decided to draw a picture" is finished. Also, for example, the extraction unit 120 may extract the average time it takes to speak one or more words of the entirety or a specific part of "I decided to draw a picture" as a prosodic feature. For example, oral functions related to swallowing movements, food gathering movements, or tongue dexterity can be evaluated based on the speech speed.

[0069] For example, when the speech data acquired by the acquisition unit 110 is speech data obtained from a speech repeatedly uttering "karakara kara...", the extraction unit 120 extracts a time change in the sound pressure difference as a prosodic feature. This will be described with reference to FIG.

[0070] Fig. 6 is a diagram showing an example of voice data showing the voice of the assessee U repeatedly uttering "karakara kara...". The horizontal axis of the graph shown in Fig. 6 is time, and the vertical axis is power (sound pressure). The unit of power shown on the vertical axis of the graph in Fig. 6 is decibels (dB).

[0071] In the graph shown in FIG. 6, the change in sound pressure corresponding to "ka" and "ra" can be confirmed. In step S102 shown in FIG. 3A, the acquisition unit 110 acquires the speech data shown in FIG. 6 from the assessee U. For example, in step S103 shown in FIG. 3A, the extraction unit 120 extracts the sound pressures of "k" and "a" in "ka" and the sound pressures of "r" and "a" in "ra" included in the speech data shown in FIG. 6 by a known method. From the extracted sound pressures of "k" and "a", the extraction unit 120 extracts the sound pressure difference Diff_P(ka) between "k" and "a" as a prosodic feature. In the same manner, the extraction unit 120 extracts the sound pressure difference Diff_P(ra) between "r" and "a" as a prosodic feature. For example, the extraction unit 120 extracts the sound pressure difference Diff_P(ka) and the sound pressure difference Diff_P(ra) as prosodic features for each of the repeatedly uttered "kara". Then, from each of the extracted sound pressure differences Diff_P(ka), the extraction unit 120 extracts the time change of the sound pressure difference Diff_P(ka) as a prosodic feature, and from each of the extracted sound pressure differences Diff_P(ra), the extraction unit 120 extracts the time change of the sound pressure difference Diff_P(ra) as a prosodic feature. For example, oral cavity functions related to the swallowing movement, the movement of gathering food, or the ability to crush food can be evaluated based on the time change of the sound pressure difference.

[0072] The extraction unit 120 may extract a time change in sound pressure as a prosodic feature. For example, when "karakara kara..." is repeatedly uttered, the time change in the minimum sound pressure (sound pressure of "k") in each "kara" may be extracted, the time change in the maximum sound pressure (sound pressure of "a") in each "kara" may be extracted, or the time change in sound pressure between "ka" and "ra" in each "kara" (sound pressure of "r") may be extracted. For example, oral functions related to the swallowing movement, the movement of putting food together, or the ability to crush food can be evaluated based on the time change in sound pressure.

[0073] 6, the extraction unit 120 may extract the number of repetitions, which is the number of times that "kara" can be uttered within a predetermined time, as a feature. The predetermined time is not particularly limited, but may be 5 seconds, for example. For example, the oral cavity function related to the movement of swallowing or the movement of gathering food can be evaluated based on the number of repetitions within a predetermined time.

[0074] For example, when the speech data acquired by the acquisition unit 110 is speech data obtained from a speech uttering "itai", the extraction unit 120 extracts the sound pressure difference and the duration of the plosive sound as prosodic features. This will be described with reference to FIG.

[0075] FIG. 7 is a diagram showing an example of voice data showing the speech of the assessee U saying "What is it?" Here, an example of voice data showing the speech repeatedly saying "What is it?" The horizontal axis of the graph shown in FIG. 7 is time, and the vertical axis is power (sound pressure). The unit of power shown on the vertical axis of the graph in FIG. 7 is decibels (dB).

[0076] In the graph shown in FIG. 7, the change in sound pressure corresponding to "i", "tsu", "ta", and "i" is confirmed. In step S102 shown in FIG. 3A, the acquisition unit 110 acquires the speech data shown in FIG. 7 from the assessee U. For example, in step S103 shown in FIG. 3A, the extraction unit 120 extracts the sound pressure of each of "t" and "a" in "ta" included in the speech data shown in FIG. 7 by a known method. From the extracted sound pressures of "t" and "a", the extraction unit 120 extracts the sound pressure difference Diff_P(ta) between "t" and "a" as a prosodic feature. For example, the sound pressure difference can be used to evaluate oral cavity function related to the swallowing power or the power to hold food together. In addition, the extraction unit 120 extracts the time of the plosive sound Time(i-ta) (the time of the plosive sound between "i" and "ta") as a prosodic feature. For example, the duration of pops can assess oral function related to swallowing, gathering food together, or steadying the tongue.

[0077] Although the phrases or phrases to be spoken have been described using Japanese phrases or phrases as examples, the phrases or phrases may be in any language, not limited to Japanese.

[0078] FIG. 8 is a diagram showing an example of a Chinese phrase or a fixed phrase which is similar to a Japanese phrase or a fixed phrase in the degree of tongue movement or opening and closing of the mouth when pronounced.

[0079] There are many languages ​​in the world, but some of them have similar tongue movements or mouth opening and closing when pronouncing words. For example, in Chinese, The degree of tongue movement or mouth opening and closing when pronouncing TIFF0007637922000001.tif5142 (hereinafter referred to as gao dao wu da ka ji ke da yi wu zhe) is similar to the degree of tongue movement or mouth opening and closing when pronouncing the Japanese phrase "I decided to draw an image," so that prosodic features similar to the Japanese phrase "I decided to draw an image" can be extracted. Note that tone codes are omitted in this specification. For reference, FIG. 8 shows examples of phrases or fixed phrases in Japanese and Chinese that have similar tongue movement or mouth opening and closing when pronouncing them.

[0080] In addition, the fact that there are languages ​​in the world that have similar tongue movements or mouth opening and closing degrees when pronouncing words will be briefly described with reference to FIGS. 9A and 9B.

[0081] FIG. 9A is a diagram showing the International Phonetic Alphabet for vowels.

[0082] FIG. 9B is a diagram showing the International Phonetic Alphabet for consonants.

[0083] In the positional relationship of the IPA for vowels shown in Fig. 9A, the horizontal direction indicates the movement of the tongue back and forth, and the closer they are, the more similar the movement of the tongue back and forth is, and the vertical direction indicates the degree of opening and closing of the mouth, and the closer they are, the more similar the degree of opening and closing of the mouth is. In the table of IPA for consonants shown in Fig. 9B, the horizontal direction indicates the parts used in pronunciation, from the lips to the throat, and the same sound can be pronounced using the same parts by using IPAs in the same squares on the table. Therefore, the present invention can be applied to various languages ​​existing in the world.

[0084] For example, if one wishes to enlarge the opening and closing of the mouth, a phrase or a fixed sentence should contain successive vertically spaced international phonetic symbols (e.g., "i" and "a") as shown in FIG. 9A. This makes it possible to increase the amount of change in the first formant frequency as a prosodic feature. Also, if one wishes to enlarge the front and back position of the tongue, a phrase or a fixed sentence should contain successive horizontally spaced international phonetic symbols (e.g., "i" and "u") as shown in FIG. 9A. This makes it possible to increase the amount of change in the second formant frequency as a prosodic feature.

[0085] For example, when the speech data acquired by the acquisition unit 110 is speech data obtained from a speech utterance of "gao dao wu da ka ji ke da yi wu zhe", the extraction unit 120 extracts the sound pressure difference, the first formant frequency, the second formant frequency, the amount of change in the first formant frequency, the amount of change in the second formant frequency, the time change in the first formant frequency, the time change in the second formant frequency, and the speaking rate as prosodic features. This will be described with reference to Figs. 10A and 10B.

[0086] FIG. 10A is a diagram showing an example of speech data showing the speech of the assessee U uttering "gao dao wu da ka ji ke da yi wu zhe." The horizontal axis of the graph shown in FIG. 10A is time, and the vertical axis is power (sound pressure). The unit of power shown on the vertical axis of the graph in FIG. 10A is decibels (dB).

[0087] In the graph shown in FIG. 10A, the changes in sound pressure corresponding to "gao", "dao", "wu", "da", "ka", "ji", "ke", "da", "yi", "wu", and "zhe" are confirmed. In step S102 shown in FIG. 3A, the acquisition unit 110 acquires the speech data shown in FIG. 10A from the assessee U. In step S103 shown in FIG. 3A, the extraction unit 120 extracts, by a known method, the sound pressures of "d" and "a" in "dao", the sound pressures of "k" and "a" in "ka", the sound pressures of "k" and "e" in "ke", and the sound pressures of "zh" and "e" in "zhe" contained in the speech data shown in FIG. 10A. From the extracted sound pressures of "d" and "a", the extraction unit 120 extracts the sound pressure difference Diff_P(da) between "d" and "a" as a prosodic feature. Similarly, the extraction unit 120 extracts the sound pressure difference Diff_P(ka) between "k" and "a", the sound pressure difference Diff_P(ke) between "k" and "e", and the sound pressure difference Diff_P(zhe) between "zh" and "e" as prosodic features. For example, the sound pressure difference can be used to evaluate oral cavity function related to the ability to swallow or hold food together. In addition, the sound pressure difference including "k" can be used to evaluate oral cavity function related to the ability to prevent food and drink from entering the throat.

[0088] Fig. 10B is a diagram showing an example of a change in formant frequency of the speech of the assessee U uttering "gao dao wu da ka ji ke da yi wu zhe". Specifically, Fig. 10B is a graph for explaining an example of a change in the first formant frequency and the second formant frequency.

[0089] The extraction unit 120 extracts the first formant frequency and the second formant frequency of each of a plurality of vowels as prosodic features from the speech data representing the speech uttered by the assessee U. For example, the extraction unit 120 extracts the first formant frequency F1i corresponding to the vowel "i" in "ji", the first formant frequency F1e corresponding to the vowel "e" in "ke", and the first formant frequency F1a corresponding to the vowel "a" in "da" as prosodic features. In addition, for example, the extraction unit 120 extracts the second formant frequency F2i corresponding to the vowel "i" in "yi" and the second formant frequency F2u corresponding to the vowel "u" ​​in "wu" as prosodic features.

[0090] Furthermore, the extraction unit 120 extracts the amount of change in the first formant frequency and the amount of change in the second formant frequency of a string of consecutive vowels as prosodic features. For example, the extraction unit 120 extracts the amount of change in the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a (F1e-F1i, F1a-F1e, F1a-F1i), and the amount of change in the second formant frequency F2i and the second formant frequency F2u (F2i-F2u) as prosodic features.

[0091] Furthermore, the extraction unit 120 extracts, as prosodic features, the time changes of the first formant frequency and the second formant frequency of a string of consecutive vowels. For example, the extraction unit 120 extracts, as prosodic features, the time changes of the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a, and the time changes of the second formant frequency F2i and the second formant frequency F2u.

[0092] For example, the second formant frequency, the amount of change in the second formant frequency, or the change over time of the second formant frequency can be used to evaluate oral function related to the movement of gathering food. Also, for example, the first formant frequency, the amount of change in the first formant frequency, or the change over time of the first formant frequency can be used to evaluate oral function related to the ability to crush food. Also, the change over time of the first formant frequency can be used to evaluate oral function related to the ability to move the mouth quickly.

[0093] Also, as shown in FIG. 10A, the extraction unit 120 may extract the speaking speed as a prosodic feature. For example, the extraction unit 120 may extract the time from when the assessee U starts speaking "gao dao wu da ka ji ke da yi wu zhe" to when he finishes speaking as a prosodic feature. Also, for example, the extraction unit 120 may extract the time from when the assessee U starts speaking a specific part of "gao dao wu da ka ji ke da yi wu zhe" to when he finishes speaking as a prosodic feature, not limited to the time until he finishes speaking the whole of "gao dao wu da ka ji ke da yi wu zhe". Also, for example, the extraction unit 120 may extract the average time it takes to speak one or more words of the whole or a specific part of "gao dao wu da ka ji ke da yi wu zhe" as a prosodic feature. For example, speech rate can assess oral function related to swallowing, food gathering, or tongue dexterity.

[0094] Returning to the explanation in FIG. 3A, the calculation unit 130 calculates an estimate of the oral function of the assessee U based on the extracted prosodic features and an estimation formula for the oral function calculated based on multiple learning data (step S104).

[0095] The estimation formula of the oral cavity function is selectively determined from among a plurality of candidates by the determination unit 116 based on the S / N ratio as described above. Each of the plurality of candidates of the estimation formula is set in advance based on the evaluation results performed on a plurality of subjects. The speech features uttered by the subjects are collected, and the oral cavity functions of the subjects are actually diagnosed, and the correlation between the speech features and the diagnosis results is set by statistical analysis using a multiple regression equation or the like. Different types of estimation formulas can be generated depending on how the speech features used as representative values ​​are selected. In this way, the candidates of the estimation formula can be generated in advance. In addition, since the estimation formula usually suitable for each element of the oral cavity function is different, a plurality of candidates of the estimation formula are set for each element of the oral cavity function. In particular, in the present invention, a first estimation formula and a second estimation formula are set for each element of the oral cavity function.

[0096] In addition, to express the correlation between the voice features and the diagnosis result, machine learning may be used. Examples of machine learning methods include logistic regression, SVM (Support Vector Machine), and random forest.

[0097] For example, the candidate estimated formula can be configured to include coefficients corresponding to elements of oral function and variables into which the extracted prosodic features are substituted and multiplied by the coefficients. The following formulas 1 to 5 are examples of the first estimated formula.

[0098] Oral health estimate = (A1 x F2e) + (B1 x F2o) + (C1 x F1i) + (D1 x F1e) + (E1 x F1a) + (F1 x Diff_P(ka)) + (G1 x Diff_P(ko)) + (H1 x Diff_P(to)) + (J1 x Diff_P(ta)) + (K1 x Diff_P(ka)) + (L1 x Diff_P(ra)) + (M1 x Num(kara)) + (N1 x Diff_P(ta)) + (P1 x Time(i-ta)) + Q1 (Formula 1)

[0099] Estimated dry mouth = (A2 x F2e) + (B2 x F2o) + (C2 x F1i) + (D2 x F1e) + (E2 x F1a) + (F2 x Diff_P(ka)) + (G2 x Diff_P(ko)) + (H2 x Diff_P(to)) + (J2 x Diff_P(ta)) + (K2 x Diff_P(ka)) + (L2 x Diff_P(ra)) + (M2 x Num(kara)) + (N2 x Diff_P(ta)) + (P2 x Time(i-ta)) + Q2 (Formula 2)

[0100] Estimated bite force = (A3 x F2e) + (B3 x F2o) + (C3 x F1i) + (D3 x F1e) + (E3 x F1a) + (F3 x Diff_P(ka)) + (G3 x Diff_P(ko)) + (H3 x Diff_P(to)) + (J3 x Diff_P(ta)) + (K3 x Diff_P(ka)) + (L3 x Diff_P(ra)) + (M3 x Num(kara)) + (N3 x Diff_P(ta)) + (P3 x Time(i-ta)) + Q3 (Formula 3)

[0101] Estimated tongue pressure = (A4 x F2e) + (B4 x F2o) + (C4 x F1i) + (D4 x F1e) + (E4 x F1a) + (F4 x Diff_P(ka)) + (G4 x Diff_P(ko)) + (H4 x Diff_P(to)) + (J4 x Diff_P(ta)) + (K4 x Diff_P(ka)) + (L4 x Diff_P(ra)) + (M4 x Num(kara)) + (N4 x Diff_P(ta)) + (P4 x Time(i-ta)) + Q4 (Equation 4)

[0102] Estimated masticatory function = (A5 x F2e) + (B5 x F2o) + (C5 x F1i) + (D5 x F1e) + (E5 x F1a) + (F5 x Diff_P(ka)) + (G5 x Diff_P(ko)) + (H5 x Diff_P(to)) + (J5 x Diff_P(ta)) + (K5 x Diff_P(ka)) + (L5 x Diff_P(ra)) + (M5 x Num(kara)) + (N5 x Diff_P(ta)) + (P5 x Time(i-ta)) + Q5 (Formula 5)

[0103] A1, B1, C1, ···, P1, A2, B2, C2, ···, P2, A3, B3, C3, ···, P3, A4, B4, C4, ···, P4, A5, B5, C5, ···, P5, are coefficients, specifically, coefficients corresponding to elements of oral function. For example, A1, B1, C1, ..., P1 are coefficients corresponding to oral hygiene, which is one of the elements of oral function, A2, B2, C2, ..., P2 are coefficients corresponding to xerostomia, which is one of the elements of oral function, A3, B3, C3, ..., P3 are coefficients corresponding to bite force, which is one of the elements of oral function, A4, B4, C4, ..., P4 are coefficients corresponding to tongue pressure, which is one of the elements of oral function, and A5, B5, C5, ..., P5 are coefficients corresponding to chewing function, which is one of the elements of oral function.

[0104] Q1 is a constant corresponding to oral hygiene, Q2 is a constant corresponding to xerostomia, Q3 is a constant corresponding to bite force, Q4 is a constant corresponding to tongue pressure, and Q5 is a constant corresponding to masticatory function.

[0105] F2e multiplied by A1, A2, A3, A4, and A5, and F2o multiplied by B1, B2, B3, B4, and B5 are variables to which the second formant frequency, which is a prosodic feature extracted from the speech data when the assessee U uttered, "I've decided to draw a picture," is substituted. F1i multiplied by C1, C2, C3, C4, and C5, F1e multiplied by D1, D2, D3, D4, and D5, and F1a multiplied by E1, E2, E3, E4, and E5 are variables to which the first formant frequency, which is a prosodic feature extracted from the speech data when the assessee U uttered, "I've decided to draw a picture," is substituted. Diff_P(ka) multiplied by F1, F2, F3, F4, F5, Diff_P(ko) multiplied by G1, G2, G3, G4, G5, Diff_P(to) multiplied by H1, H2, H3, H4, H5, and Diff_P(ta) multiplied by J1, J2, J3, J4, J5 are variables substituted with sound pressure differences, which are prosodic features extracted from the speech data when the assessee U uttered "I've decided to draw a picture." Diff_P(ka) multiplied by K1, K2, K3, K4, K5, and Diff_P(ra) multiplied by L1, L2, L3, L4, L5 are variables substituted with sound pressure differences, which are prosodic features extracted from the speech data when the assessee U uttered "kara." Num(kara), which is multiplied by M1, M2, M3, M4, and M5, is a variable substituted with the number of repetitions, which is a prosodic feature extracted from the speech data when the assessee U repeatedly utters "kara" within a certain period of time. Diff_P(ta), which is multiplied by N1, N2, N3, N4, and N5, is a variable substituted with the sound pressure difference, which is a prosodic feature extracted from the speech data when the assessee U utters "itai." Time(i-ta), which is multiplied by P1, P2, P3, P4, and P5, is a variable substituted with the duration of the plosive, which is a prosodic feature extracted from the speech data when the assessee U utters "itai."

[0106] As shown in the above formulas 1 to 5, for example, the calculation unit 130 calculates an estimate for each element (e.g., tongue coating, dry mouth, bite force, tongue pressure, and chewing function) of the oral cavity function of the subject U. Note that these elements of oral cavity function are only examples, and the elements of oral cavity function may include at least one of the tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function of the subject U.

[0107] Also, for example, the extraction unit 120 extracts multiple prosodic features from speech data acquired by collecting speech of multiple types of phrases or fixed phrases (for example, in the above formulas 1 to 5, "I decided to draw a picture," "From," and "What on earth") spoken by the assessee U, and the calculation unit 130 calculates an estimate of the oral function based on the multiple extracted prosodic features and an estimation formula. The calculation unit 130 can accurately calculate an estimate of the oral function by substituting the multiple prosodic features extracted from the speech data of multiple types of phrases or fixed phrases into one estimation formula.

[0108] The following Equations 6 to 10 are examples of the second estimation equation.

[0109] Oral health estimate = (A1 x F2e) + (B1 x F2o) + (C1 x F1i) + (D1 x F1e) + (E1 x F1a) + (M1 x Num(kara)) + (P1 x Time(i-ta)) + Q1 (Equation 6)

[0110] Estimated dry mouth = (A2 x F2e) + (B2 x F2o) + (C2 x F1i) + (D2 x F1e) + (E2 x F1a) + (M2 x Num(kara)) + (P2 x Time(i-ta)) + Q2 (Equation 7)

[0111] Estimated bite force = (A3 x F2e) + (B3 x F2o) + (C3 x F1i) + (D3 x F1e) + (E3 x F1a) + (M3 x Num(kara)) + (P3 x Time(i-ta)) + Q3 (Equation 8)

[0112] Estimated tongue pressure = (A4 x F2e) + (B4 x F2o) + (C4 x F1i) + (D4 x F1e) + (E4 x F1a) + (M4 x Num(kara)) + (P4 x Time(i-ta)) + Q4 (Equation 9)

[0113] Estimated value of masticatory function = (A5 x F2e) + (B5 x F2o) + (C5 x F1i) + (D5 x F1e) + (E5 x F1a) + (M5 x Num(kara)) + (P5 x Time(i-ta)) + Q5 (Equation 10)

[0114] In this way, compared to the first estimation formula, the second estimation formula omits terms for prosodic features related to sound pressure, such as sound pressure difference, making it possible to estimate appropriate estimates with relatively high accuracy even in sound collection environments with low S / N ratios.

[0115] Although a linear expression is shown as the estimation expression, the estimation expression may be a multi-order expression such as a quadratic expression.

[0116] Next, the evaluation unit 140 evaluates the deterioration state of the oral function of the subject U by determining the estimated value calculated by the calculation unit 130 using the oral function evaluation index (step S105). For example, the evaluation unit 140 evaluates the deterioration state of the oral function of the subject U for each element of oral function by determining the calculated estimated value for each element of oral function using the oral function evaluation index defined for each element of oral function. The oral function evaluation index is an index for evaluating oral function, and is, for example, a condition for determining that oral function is deteriorated. The oral function evaluation index will be explained with reference to FIG. 11.

[0117] FIG. 11 is a diagram showing an example of an oral cavity function evaluation index.

[0118] The oral function evaluation index is determined for each element of oral function. For example, an index of 50% or more is determined for oral hygiene, an index of 27 or less is determined for dry mouth, an index of less than 200N is determined for bite force (when using GC Corporation's Dental Prescale II), an index of less than 30kPa is determined for tongue pressure, and an index of less than 100mg / dL is determined for chewing function (for the indexes, see "Basic Concepts on Oral Hypofunction" by the Japanese Dental Association (https: / / www.jads.jp / basic / pdf / document_02.pdf)). The evaluation unit 140 evaluates the deterioration state of the oral function of the subject U for each element of oral function by comparing the calculated estimated value for each element of oral function with the oral function evaluation index determined for each element of oral function. For example, if the calculated estimated value of oral hygiene is 50% or more, it is evaluated that oral hygiene is in a deteriorated state as an element of oral function. Similarly, when the calculated estimated value of dry mouth is 27 or less, dry mouth is evaluated as being in a reduced state as an element of oral function, when the calculated estimated value of occlusal force is less than 200N, occlusal force is evaluated as being in a reduced state as an element of oral function, when the calculated estimated value of tongue pressure is less than 30kPa, occlusal force is evaluated as being in a reduced state as an element of oral function, and when the calculated estimated value of masticatory function is less than 100mg / dL, masticatory function is evaluated as being in a reduced state as an element of oral function. Note that the oral function evaluation indexes set for oral hygiene, dry mouth, occlusal force, tongue pressure, and masticatory function shown in FIG. 11 are examples and are not limited thereto. For example, an index of remaining teeth may be set for masticatory function. Also, oral hygiene, dry mouth, occlusal force, tongue pressure, and masticatory function are shown as elements of oral function, but these are examples. For example, reduced tongue-and-lip motor function involves factors of oral function such as tongue movement, lip movement, and lip strength.

[0119] Returning to the explanation in FIG. 3A, the output unit 150 outputs the evaluation result of the oral function of the assessee U evaluated by the evaluation unit 140 (step S106). For example, the output unit 150 outputs the evaluation result to the mobile terminal 300. In this case, the output unit 150 may include, for example, a communication interface for performing wired or wireless communication, and acquires image data of an image corresponding to the evaluation result from the storage unit 170 and transmits the acquired image data to the mobile terminal 300. An example of the image data (evaluation result) is shown in FIG. 12 and FIG. 13.

[0120] 12 and 13 are diagrams showing an example of an evaluation result for each element of oral function. As shown in FIG. 12, the evaluation result may be a two-stage evaluation result of OK or NG. OK means normal, and NG means abnormal. It is not necessary to indicate normal or abnormal for each element of oral function. For example, only the evaluation result of an element suspected of being deteriorated may be indicated. The evaluation result is not limited to a two-stage evaluation result, and may be a detailed evaluation result in which the degree of evaluation is divided into three or more stages. In this case, the index data 172 stored in the storage unit 170 may include multiple indexes for one element. Also, as shown in FIG. 13, the evaluation result may be expressed in a radar chart. In FIG. 12 and FIG. 13, the elements of oral function are shown to be mouth cleanliness, food holding strength, hard chewing strength, tongue strength, and jaw movement. The evaluation results are presented based on the estimated values ​​of oral hygiene for mouth cleanliness, dry mouth for food holding strength, bite strength for hard chewing strength, tongue strength for tongue pressure, and jaw movement for chewing function. Note that Figures 12 and 13 are merely examples, and the wording of the evaluation items, the oral function items, and the corresponding combinations thereof are not limited to those shown in Figures 12 and 13.

[0121] Returning to the explanation in FIG. 3A, the proposing unit 160 compares the estimated value calculated by the calculating unit 130 with predetermined data (proposal data 173) to make a proposal regarding the oral cavity function of the subject U (step S107). Here, the predetermined data will be explained with reference to FIG.

[0122] FIG. 14 is an example of predetermined data (proposal data 173) used when making a proposal regarding oral cavity function.

[0123] As shown in Fig. 14, the proposal data 173 is data in which the evaluation result and the proposed content are associated with each element of oral function. For example, when the calculated estimated value of oral cleanliness is 50% or more, the proposing unit 160 judges that the index is satisfied and is OK, and makes a proposal according to the proposed content associated with the oral cleanliness. Note that, although the specific proposed content is not described, for example, the storage unit 170 includes data indicating the proposed content (e.g., images, videos, audio, text, etc.), and the proposing unit 160 makes a proposal regarding oral function to the person to be evaluated U using such data.

[0124] [Effects, etc.] As described above, the oral function evaluation method according to this embodiment is executed by a terminal (mobile terminal 300) and an oral function evaluation device 100, and is an oral function evaluation method for evaluating a decline in the oral function of the subject U from the voice uttered by the subject U. The terminal obtains voice data by collecting the voice uttered by the subject U, and the oral function evaluation device 100 obtains the voice data and extracts features from the obtained voice data, calculates a first average intensity of the sound collected during a period when the subject U is not making a sound in the obtained voice data, and a second average intensity of the sound collected during a period when the subject U is making a sound, and calculates a ratio of the second average intensity to the first average intensity. the terminal presents the evaluated state of decline in the oral function of the person being evaluated U, and in determining the estimation formula, if the calculated S / N ratio is greater than a first threshold value, a first estimation formula including a feature related to sound pressure among the features extracted from the voice data is determined as the estimation formula, and if the calculated S / N ratio is equal to or less than the first threshold value, a second estimation formula not including a feature related to sound pressure is determined as the estimation formula.

[0125] This can achieve the same effects as the oral function evaluation device described below.

[0126] Furthermore, for example, as shown in FIG. 3A, the method may include an acquisition step (step S102) of acquiring speech data obtained by collecting speech of the person being evaluated U speaking a phrase or a set phrase consisting of two or more moras including a change in the first formant frequency or a change in the second formant frequency, or including at least one of a pop sound, a plosive sound, an unvoiced sound, a glottal stop, and a fricative sound; an extraction step (step S103) of extracting prosodic features from the acquired speech data; a calculation step (step S104) of calculating an estimate of the oral function of the person being evaluated U based on an estimation formula for oral function calculated based on multiple learning data and the extracted prosodic features; and an evaluation step (step S105) of evaluating the decline in the oral function of the person being evaluated U by judging the calculated estimate using an oral function evaluation index.

[0127] According to this, by acquiring voice data suitable for evaluating oral function, it is possible to easily evaluate the oral function of the person to be evaluated U. In other words, the oral function of the person to be evaluated U can be evaluated simply by the person to be evaluated U speaking the above phrase or fixed phrase toward a sound collection device such as a mobile terminal 300. In particular, since an estimate of the oral function is calculated using an estimation formula calculated based on multiple learning data, the deterioration state of the oral function can be quantitatively evaluated. In addition, the oral function is not evaluated by directly comparing the prosodic feature with a threshold value, but an estimate is calculated from the prosodic feature and the estimation formula, and the estimate is compared with a threshold value (oral function evaluation index), so that the deterioration state of the oral function can be accurately evaluated.

[0128] For example, the estimation formula may include coefficients corresponding to elements of oral function, and variables into which the extracted prosodic features are substituted and which are multiplied by the coefficients.

[0129] According to this, an estimate of oral function can be easily calculated by simply substituting the extracted prosodic features into the estimation formula.

[0130] For example, in the calculation step, an estimated value is calculated for each element of the oral function of the subject U, and in the evaluation step, the calculated estimated value for each element of oral function is judged using an oral function evaluation index defined for each element of oral function, thereby evaluating the decline in the oral function of the subject U for each element of oral function.

[0131] According to this, the deterioration state of the oral function can be evaluated for each element. For example, by preparing an estimation formula for each element of the oral function, in which the coefficients differ according to the element of the oral function, the deterioration state of the oral function can be easily evaluated for each element.

[0132] For example, elements of oral function may include at least one of the subject U's tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function.

[0133] This makes it possible to evaluate the deterioration of at least one of the oral function elements of the subject U, namely tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function.

[0134] For example, the prosodic features may include at least one of speaking rate, sound pressure difference, time change in sound pressure difference, first formant frequency, second formant frequency, amount of change in first formant frequency, amount of change in second formant frequency, time change in first formant frequency, time change in second formant frequency, and time of plosive sound.

[0135] As the deterioration of oral function causes changes in pronunciation, the deterioration of oral function can be evaluated from these prosodic features.

[0136] For example, in the extraction step, multiple prosodic features may be extracted from speech data acquired by collecting speech of multiple types of phrases or fixed phrases spoken by the subject U, and in the calculation step, an estimated value may be calculated based on the extracted multiple prosodic features and an estimation formula.

[0137] According to this, by using a plurality of prosodic features extracted based on a plurality of types of phrases or fixed phrases for one estimation formula, it is possible to improve the accuracy of calculation of an estimate of oral function.

[0138] For example, a phrase or formulaic sentence may include two or more vowels or a combination of vowels and consonants, accompanied by opening and closing the mouth or moving the tongue back and forth to produce speech.

[0139] According to this, prosodic features including the change in the first formant frequency, the change in the first formant frequency over time, the change in the second formant frequency, or the change in the second formant frequency over time can be extracted from the speech of the subject U speaking such a phrase or fixed phrase.

[0140] For example, the voice data may be obtained by collecting the voice of the assessee U speaking a phrase or a standard phrase at least twice at different speaking speeds.

[0141] According to this, the degree to which the oral function state is maintained can be estimated from the voice of the person being evaluated U speaking such phrases or standard phrases.

[0142] For example, a fixed phrase may include a repetition of a phrase consisting of a popping sound and a consonant different from the popping sound.

[0143] This makes it possible to extract prosodic features including the change in sound pressure difference over time, the change in sound pressure over time, and the number of repetitions from the speech of the person being evaluated U speaking such a phrase or fixed phrase.

[0144] For example, a phrase or formula may include at least one combination of a vowel and a plosive.

[0145] This makes it possible to extract prosodic features including sound pressure difference and duration of plosive sounds from the speech of the person being evaluated U speaking such phrases or fixed phrases.

[0146] For example, the oral function evaluation method may further include a suggestion step of making suggestions regarding the oral function of the person being evaluated U by comparing the calculated estimated value with predetermined data.

[0147] According to this, the person being evaluated U can receive suggestions on what measures to take when his / her oral function declines.

[0148] The oral function evaluation device 100 according to this embodiment is an oral function evaluation device that evaluates the deterioration state of the oral function of the person being evaluated U from the voice uttered by the person being evaluated U, and includes an acquisition unit 110 that acquires voice data obtained by collecting the voice uttered by the person being evaluated U, an extraction unit 120 that extracts features from the acquired voice data, an S / N ratio calculation unit 115 that calculates a first average intensity of the sound collected during a period when the person being evaluated U is not making a sound in the acquired voice data and a second average intensity of the sound collected during a period when the person being evaluated U is making a sound, and calculates an S / N ratio that is the ratio of the second average intensity to the first average intensity, and The evaluation system includes a determination unit 116 that determines an estimation formula to be used in evaluating the oral function of the subject U, a calculation unit 130 that calculates an estimate of the oral function of the subject U based on the determined estimation formula and the extracted features, and an evaluation unit 140 that evaluates the deteriorated state of the oral function of the subject U by judging the calculated estimate using an oral function evaluation index, and when the calculated S / N ratio is greater than a first threshold, the determination unit 116 determines a first estimation formula that includes a feature related to sound pressure among the features extracted from the voice data as the estimation formula, and when the calculated S / N ratio is equal to or less than the first threshold, determines a second estimation formula that does not include a feature related to sound pressure as the estimation formula.

[0149] According to this, it is possible to determine whether the conditions are suitable for using the feature quantity related to sound pressure from the S / N ratio, and to determine whether to use the first estimation formula including the feature quantity related to sound pressure or the second estimation formula not including the feature quantity related to sound pressure according to the determination. This makes it possible to evaluate the deterioration state of the oral function of the subject U using an appropriate estimation formula from the viewpoint of whether or not the feature quantity related to sound pressure is included. In other words, by appropriately using the feature quantity related to sound pressure, it is possible to more accurately evaluate the oral function of the subject U.

[0150] For example, the oral function of the subject U may be at least one of the subject U's tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function.

[0151] This makes it possible to evaluate the deterioration of at least one of the subject U's tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function.

[0152] For example, a first estimation equation and a second estimation equation may be set for each of the subject U's tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function.

[0153] According to this, it is possible to determine whether it is more appropriate to use the first estimation formula or the second estimation formula for each of the subject U's tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function, and then evaluate the state of decline.

[0154] For example, the information output section 180 may be further provided, which outputs information for increasing the S / N ratio when the calculated S / N ratio is equal to or less than a second threshold value that is smaller than the first threshold value.

[0155] According to this, when an evaluation of the state of decline in oral function is to be performed in an unsuitable environment where the S / N ratio is even smaller, measures can be taken to improve the environment, thereby preventing the evaluation of the state of decline in oral function from being performed in such an unsuitable environment.

[0156] For example, the information may recommend at least one of the following: checking the connection status of a sound collection device (microphone) used to collect the voice spoken by the person being evaluated U; increasing the volume of the person being evaluated U's voice when speaking; and reducing environmental noise when the person being evaluated U is speaking.

[0157] According to this, as measures to improve the environment, it is possible to recommend at least one of the following: having the person being evaluated check the connection status of the sound collection device (microphone) used to collect the voice spoken by the person being evaluated U, increasing the volume of the person being evaluated U's voice when speaking, and reducing environmental noise when the person being evaluated U is speaking.

[0158] For example, the acquisition unit 110 may acquire first voice data as voice data that is not used in evaluating the oral function of the subject U, and the S / N ratio calculation unit may calculate the S / N ratio for the acquired first voice data.

[0159] According to this, the S / N ratio can be calculated using the first voice data that is not used for evaluating the oral function of the subject U.

[0160] For example, the acquisition unit 110 may acquire second voice data as voice data to be used in evaluating the oral function of the subject U, and the S / N ratio calculation unit may calculate the S / N ratio in the acquired second voice data.

[0161] According to this, the S / N ratio can be calculated using the second voice data used to evaluate the oral function of the subject U.

[0162] For example, the system may further include a suggestion unit 160 that makes suggestions regarding the oral function of the subject U by comparing the calculated estimated value with predetermined data.

[0163] According to this, the person being evaluated U can receive suggestions on what measures to take when his / her oral function declines.

[0164] For example, the device may further include a sound collection device (microphone) used to collect the voice spoken by the subject U, and a presentation device (mobile terminal 300) for presenting the decline in oral function of the evaluated subject U.

[0165] The system may also include an acquisition unit 110 that acquires voice data obtained by collecting the voice of the person being evaluated U speaking a phrase or a set phrase consisting of two or more moras including a change in the first formant frequency or a change in the second formant frequency, or including at least one of a pop sound, a plosive sound, an unvoiced sound, a glottal stop, and a fricative sound; an extraction unit 120 that extracts prosodic features from the acquired voice data; a calculation unit 130 that calculates an estimate of the oral function of the person being evaluated U based on an estimation formula for oral function calculated based on multiple learning data and the extracted prosodic features; and an evaluation unit 140 that evaluates the decline in the oral function of the person being evaluated U by judging the calculated estimate using an oral function evaluation index.

[0166] This makes it possible to provide an oral function assessment device 100 that can easily assess the oral function of the subject U.

[0167] The oral function evaluation system 200 according to this embodiment is an oral function evaluation system that evaluates the deterioration state of the oral function of the person to be evaluated U from the voice uttered by the person to be evaluated U, and includes a terminal (mobile terminal 300) and an oral function evaluation device 100 connected to the terminal. The terminal has a sound collection device (microphone) used to collect the voice uttered by the person to be evaluated U, and a presentation device (part of the mobile terminal 300) for presenting the deterioration state of the oral function of the evaluated person U. The oral function evaluation device 100 includes an acquisition unit 110 that acquires voice data obtained by collecting the voice uttered by the person to be evaluated U, an extraction unit 120 that extracts features from the acquired voice data, and a first average intensity of the sound collected during a period in which the person to be evaluated U is not making a sound in the acquired voice data and a second average intensity of the sound collected during a period in which the person to be evaluated U is making a sound. the first average intensity of the sound collected during the first period and the second average intensity of the sound collected during the first period, respectively, and an S / N ratio that is the ratio of the second average intensity to the first average intensity; a determination unit 116 that determines an estimation formula to be used in evaluating the oral function of the subject U; a calculation unit 130 that calculates an estimate of the oral function of the subject U based on the determined estimation formula and the extracted features; and an evaluation unit 140 that evaluates the deterioration state of the oral function of the subject U by judging the calculated estimate using an oral function evaluation index. When the calculated S / N ratio is greater than a first threshold, the determination unit 116 determines, as the estimation formula, a first estimation formula that includes a feature related to sound pressure from the features extracted from the voice data, and when the calculated S / N ratio is equal to or less than the first threshold, determines, as the estimation formula, a second estimation formula that does not include a feature related to sound pressure.

[0168] This makes it possible to provide an oral cavity function evaluation system 200 that can evaluate oral cavity function more accurately.

[0169] Also, for example, the oral cavity function evaluation device 100 may be provided with a sound collection device (mobile terminal 300) that collects the speech of the person to be evaluated U, such as phrases or fixed phrases, in a non-contact manner.

[0170] This makes it possible to provide an oral function evaluation system 200 that can easily evaluate the oral function of the subject U.

[0171] (Other embodiments) Although the oral cavity function evaluation method and the like according to the embodiments have been described above, the present invention is not limited to the above-mentioned embodiments.

[0172] For example, the candidates of the estimation formula may be updated based on the evaluation results obtained when an expert actually diagnoses the oral function of the assessee U. This can improve the evaluation accuracy of the oral function. Machine learning may be used to improve the evaluation accuracy of the oral function.

[0173] Also, for example, the proposal data 173 may be updated based on the evaluation result after the assessee U evaluates the proposal content. For example, when a proposal is made regarding oral function that is not a problem for the assessee U, the assessee U evaluates the proposal content as incorrect. Then, the proposal data 173 is updated based on the evaluation result, so that the above-mentioned incorrect proposal is not made. In this way, the proposal content regarding oral function for the assessee U can be made more effective. Machine learning may be used to make the proposal content regarding oral function more effective.

[0174] For example, the oral function evaluation results may be stored as big data together with personal information and used for machine learning. Furthermore, the oral function proposal contents may be stored as big data together with personal information and used for machine learning.

[0175] In addition, for example, in the above embodiment, the oral function evaluation method includes a suggestion step (step S107) for making a suggestion regarding the oral function, but this does not have to be included. In other words, the oral function evaluation device 100 does not have to include the suggestion unit 160.

[0176] Also, for example, in the above embodiment, in the acquisition step (step S102), the personal information of the assessee U is acquired, but it is not necessary to acquire it. In other words, the acquisition unit 110 does not need to acquire the personal information of the assessee U.

[0177] For example, the steps in the oral function evaluation method may be executed by a computer (computer system). The present invention can be realized as a program for causing a computer to execute the steps included in the method. Furthermore, the present invention can be realized as a non-transitory computer-readable recording medium, such as a CD-ROM, on which the program is recorded.

[0178] For example, when the present invention is realized by a program (software), each step is executed by executing the program using hardware resources such as a computer's CPU, memory, input / output circuits, etc. In other words, each step is executed by the CPU acquiring data from memory or input / output circuits, etc., performing calculations, and outputting the results of the calculations to memory or input / output circuits, etc.

[0179] Furthermore, each component included in the oral cavity function evaluation device 100 and the oral cavity function evaluation system 200 of the above-mentioned embodiments may be realized as a dedicated or general-purpose circuit.

[0180] Furthermore, each of the components included in the oral cavity function evaluation device 100 and the oral cavity function evaluation system 200 of the above-described embodiments may be realized as an LSI (Large Scale Integration) which is an integrated circuit (IC: Integrated Circuit).

[0181] Furthermore, the integrated circuit is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. A programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor in which the connections and settings of circuit cells within an LSI can be reconfigured may also be used.

[0182] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derived technologies, it is natural that the components included in the oral function evaluation device 100 and the oral function evaluation system 200 can be integrated into integrated circuits using that technology.

[0183] In addition, the present invention also includes forms obtained by applying various modifications to the embodiments that a person skilled in the art may conceive, and forms realized by arbitrarily combining the components and functions of each embodiment without departing from the spirit of the present invention. [Explanation of symbols]

[0184] 100 Oral Function Assessment Device 110 Acquisition Department 115 S / N ratio calculation section 116 Decision Section 120 Extraction part 130 Calculation section 140 Evaluation Department 150 Output section 160 Proposal Department 180 Information Output Section 200 Oral Function Assessment System 300 Mobile terminal (terminal, microphone, presentation device) U Evaluatee

Claims

1. An oral function evaluation device that evaluates a state of deterioration of an oral function of a subject from a voice uttered by the subject, An acquisition unit that acquires voice data obtained by collecting the voice uttered by the subject; An extraction unit that extracts features from the acquired voice data; an S / N ratio calculation unit that calculates a first average intensity of the sound collected during a period when the subject is not making a sound in the acquired voice data and a second average intensity of the sound collected during a period when the subject is making a sound, and calculates an S / N ratio that is the ratio of the second average intensity to the first average intensity; A determination unit that determines an estimation formula used to evaluate the oral function of the subject; A calculation unit that calculates an estimate of the oral function of the subject based on the determined estimation formula and the extracted feature amount; An evaluation unit that evaluates the deterioration state of the oral function of the person to be evaluated by determining the calculated estimated value using an oral function evaluation index, The determination unit is When the calculated S / N ratio is greater than a first threshold, a first estimation formula including a feature quantity related to sound pressure among the feature quantities extracted from the voice data is determined as the estimation formula; When the calculated S / N ratio is equal to or less than the first threshold, a second estimation formula that does not include a feature quantity related to the sound pressure is determined as the estimation formula. Oral function assessment device.

2. The oral function of the subject is at least one of tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function of the subject. The oral cavity function evaluation device according to claim 1 .

3. The first estimation formula and the second estimation formula are set for each of the tongue coating, dry mouth, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function of the subject. The oral cavity function evaluation device according to claim 1 .

4. The information output unit outputs information for increasing the S / N ratio when the calculated S / N ratio is equal to or smaller than a second threshold value that is smaller than the first threshold value. The oral cavity function evaluation device according to claim 1 .

5. The information is information that recommends at least one of the following: checking the connection status of a sound collection device used to collect the sound uttered by the subject; increasing the volume of the voice when the subject speaks; and reducing environmental noise when the subject speaks. The oral cavity function evaluation device according to claim 4.

6. The acquisition unit acquires first voice data as the voice data that is not used for evaluating the oral function of the subject, The S / N ratio calculation unit calculates the S / N ratio in the acquired first voice data. The oral cavity function evaluation device according to claim 4.

7. The acquisition unit acquires second voice data as the voice data used in evaluating the oral function of the subject, The S / N ratio calculation unit calculates the S / N ratio in the acquired second voice data. The oral cavity function evaluation device according to claim 4.

8. The method further includes a suggestion unit that compares the calculated estimated value with predetermined data to make a suggestion regarding the oral function of the person to be evaluated. The oral cavity function evaluation device according to any one of claims 1 to 6.

9. A sound collection device used to collect the voice spoken by the subject; and a presentation device for presenting the evaluated state of deterioration of the oral function of the person to be evaluated. The oral cavity function evaluation device according to claim 8.

10. An oral function evaluation system for evaluating a state of deterioration of an oral function of a subject from a voice uttered by the subject, A terminal, An oral function evaluation device connected to the terminal, The terminal includes: A sound collection device used to collect the voice spoken by the subject; A presentation device for presenting the evaluated state of deterioration of the oral function of the person to be evaluated, The oral function evaluation device includes: An acquisition unit that acquires voice data obtained by collecting the voice uttered by the subject; An extraction unit that extracts features from the acquired voice data; an S / N ratio calculation unit that calculates a first average intensity of the sound collected during a period when the subject is not making a sound in the acquired voice data and a second average intensity of the sound collected during a period when the subject is making a sound, and calculates an S / N ratio that is the ratio of the second average intensity to the first average intensity; A determination unit that determines an estimation formula used to evaluate the oral function of the subject; A calculation unit that calculates an estimate of the oral function of the subject based on the determined estimation formula and the extracted feature amount; An evaluation unit that evaluates the deterioration state of the oral function of the person to be evaluated by determining the calculated estimated value using an oral function evaluation index, The determination unit is When the calculated S / N ratio is greater than a first threshold, a first estimation formula including a feature quantity related to sound pressure among the feature quantities extracted from the voice data is determined as the estimation formula; When the calculated S / N ratio is equal to or less than the first threshold, a second estimation formula that does not include a feature quantity related to the sound pressure is determined as the estimation formula. Oral function assessment system.

11. An oral function evaluation method that is executed by a terminal and an oral function evaluation device, and evaluates a state of deterioration of an oral function of a subject from a voice uttered by the subject, The terminal obtains voice data by collecting the voice uttered by the person to be evaluated, The oral function evaluation device, Acquiring the audio data; Extracting features from the acquired voice data; Calculate a first average intensity of the sound collected during a period when the subject is not making a sound in the acquired voice data, and a second average intensity of the sound collected during a period when the subject is making a sound, and calculate an S / N ratio which is the ratio of the second average intensity to the first average intensity; Determining an estimation equation to be used for evaluating the oral function of the subject; Calculating an estimate of the oral function of the person to be evaluated based on the determined estimation formula and the extracted feature amount; The calculated estimated value is judged using an oral function evaluation index to evaluate the deterioration state of the oral function of the person to be evaluated; The terminal displays the assessed oral function deterioration state of the subject, In determining the estimation formula, When the calculated S / N ratio is greater than a first threshold, a first estimation formula including a feature quantity related to sound pressure among the feature quantities extracted from the voice data is determined as the estimation formula; When the calculated S / N ratio is equal to or less than the first threshold, a second estimation formula that does not include a feature quantity related to the sound pressure is determined as the estimation formula. Methods for assessing oral function.

Images