Oral function evaluation device, oral function evaluation system, and oral function evaluation method

By collecting and analyzing the sound data of the evaluated persons, calculating the signal-to-noise ratio and extracting the characteristic quantity, the discomfort and correctness of oral function evaluation in the prior art is solved, and the accurate assessment of oral function in the elderly is achieved and nutrition prevention is prevented. Bad and health problems.

CN119012956APending Publication Date: 2024-11-22PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380033079.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-18
Filing Date
2023-03-24
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The prior art requires wearing equipment when evaluating oral function, which leads to discomfort and burden, and there are problems with the correctness of sound evaluation. Especially for the elderly, the reduction of oral function is ignored, which may lead to malnutrition and health problems.

Method used

通过收集被评价者说出的声音数据,提取特征量,计算信噪比(S/N比),决定推测式,基于推测式和特征量计算口腔功能的推测值,使用口腔功能评价指标进行评价,区分声压相关和非声压相关的特征量,进行更准确的口腔功能评估。

Benefits of technology

It has achieved more correct evaluation of oral function, especially the oral health status of the elderly, without wearing equipment, and help prevent malnutrition and related health problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119012956A_ABST
    Figure CN119012956A_ABST
Patent Text Reader

Abstract

An oral function evaluation device (100) is provided with: an acquisition unit (110) that acquires voice data; an extraction unit (120) that extracts a feature quantity from the voice data; an S / N ratio calculation unit (115) that calculates an S / N ratio, which is the ratio between a first average intensity of sound collected during a period in which no sound is emitted and a second average intensity of sound collected during a period in which the sound is emitted; a determination unit (116) that determines an estimation formula; and an evaluation unit (140) that evaluates the state of decrease in oral function by performing a determination using an oral function evaluation index on the basis of the determined estimation formula and the extracted feature amount, and wherein, when the S / N ratio is greater than a first threshold value, the determination unit (116) determines a first estimation formula including the feature amount related to sound pressure as the estimation formula, and determines a second estimation formula including the feature amount related to sound pressure as the estimation formula if the S / N ratio is greater than a second threshold value. When the S / N ratio is equal to or less than the first threshold value, a second estimation formula that does not include the characteristic quantity related to the sound pressure is determined as the estimation formula.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an oral function evaluation device, an oral function evaluation system and an oral function evaluation method capable of evaluating the oral function of an evaluated person. Background Art

[0002] A method is disclosed in which a device for evaluating the ingestion and swallowing function is worn on the neck of an evaluation subject, and a pharyngeal movement feature is acquired as an ingestion and swallowing function evaluation marker to evaluate the ingestion and swallowing function of the evaluation subject (for example, see Patent Document 1).

[0003] Prior art literature

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Publication No. 2017-23676 Summary of the invention

[0006] Problems to be solved by the invention

[0007] However, in the method disclosed in the above-mentioned patent document 1, in order to evaluate oral functions such as swallowing function, the person being evaluated needs to wear an instrument, which sometimes causes discomfort to the person being evaluated and creates a burden on the person being evaluated. In addition, although oral function can also be evaluated by visual examination, questioning or palpation by experts such as dentists, dental hygienists, speech and hearing therapists or internists, the elderly, although they have been choking or spilling food due to the influence of aging, ignore the reduction in oral function as a natural symptom of the elderly. Since the reduction in oral function is ignored, for example, malnutrition due to the reduction in food intake will occur, and malnutrition will lead to reduced immunity. In addition, it is easy to swallow by mistake, and the result of swallowing by mistake and reduced immunity may lead to pneumonia caused by swallowing by mistake, creating a vicious circle.

[0008] Even without using such a method, the oral function of the person being evaluated can be evaluated based on the sounds spoken by the person being evaluated. However, since the sounds cannot be appropriately collected, there is a problem in the accuracy of the evaluation.

[0009] Therefore, an object of the present invention is to provide an oral function evaluation device or the like which can more accurately evaluate the oral function while using the voice of the person being evaluated.

[0010] Means used to solve problems

[0011] An oral function evaluation device according to a technical solution of the present invention is an oral function evaluation device for evaluating the reduced state of oral function of the evaluated person based on the sound spoken by the evaluated person, and comprises: an acquisition unit for acquiring sound data obtained by collecting the sound spoken by the evaluated person; an extraction unit for extracting feature quantities from the acquired sound data; an S / N ratio calculation unit for respectively calculating the first average intensity of the sound collected during a period when the evaluated person did not make a sound and the second average intensity of the sound collected during a period when the evaluated person made a sound, and calculating the S / N ratio as the ratio of the second average intensity to the first average intensity; a determination unit for determining the S / N ratio of the sound collected during a period when the evaluated person made a sound. An estimation formula used in evaluating the oral function of an evaluator; a calculation unit, which calculates an estimation value of the oral function of the person being evaluated based on the determined estimation formula and the extracted feature quantity; and an evaluation unit, which uses an oral function evaluation index to determine the calculated estimation value, thereby evaluating the reduced state of the oral function of the person being evaluated, wherein the determination unit, when the calculated S / N ratio is greater than a first threshold value, determines the first estimation formula including the feature quantity related to sound pressure among the feature quantities extracted from the sound data as the estimation formula, and when the calculated S / N ratio is less than the first threshold value, determines the second estimation formula not including the feature quantity related to sound pressure as the estimation formula.

[0012] In addition, an oral function evaluation system according to a technical solution of the present invention is an oral function evaluation system for evaluating the reduced state of oral function of the evaluated person based on the sound spoken by the evaluated person, and comprises: a terminal; and an oral function evaluation device connected to the terminal, the terminal comprising: a sound collecting device used in collecting the sound spoken by the evaluated person; and a prompting device for prompting the reduced state of oral function of the evaluated person, the oral function evaluation device comprising: an acquisition unit for acquiring sound data obtained by collecting the sound spoken by the evaluated person; an extraction unit for extracting feature quantities from the acquired sound data; an S / N ratio calculation unit for respectively calculating the first average intensity of the sound collected during the period when the evaluated person did not make a sound and the S / N ratio collected during the period when the evaluated person made a sound, in the acquired sound data. a second average intensity of the sound, calculating an S / N ratio which is a ratio of the second average intensity to the first average intensity; a determining unit, determining an estimation formula used in evaluating the oral function of the person being evaluated; a calculating unit, calculating an estimation value of the oral function of the person being evaluated based on the determined estimation formula and the extracted feature quantity; and an evaluating unit, using an oral function evaluation index to determine the calculated estimation value, thereby evaluating the reduced state of the oral function of the person being evaluated; the determining unit, when the calculated S / N ratio is greater than a first threshold value, determines the first estimation formula including the feature quantity related to sound pressure among the feature quantities extracted from the sound data as the estimation formula, and when the calculated S / N ratio is less than the first threshold value, determines the second estimation formula not including the feature quantity related to sound pressure as the estimation formula.

[0013] In addition, an oral function evaluation method related to a technical solution of the present invention is an oral function evaluation method that is executed by a terminal and an oral function evaluation device to evaluate the reduced state of the oral function of the above-mentioned person being evaluated based on the sound spoken by the above-mentioned person being evaluated, including: the above-mentioned terminal obtains sound data by collecting the sound spoken by the above-mentioned person being evaluated; the above-mentioned oral function evaluation device obtains the above-mentioned sound data, extracts feature quantities based on the obtained above-mentioned sound data, and calculates the first average intensity of the sound collected during the period when the above-mentioned person being evaluated did not make a sound and the second average intensity of the sound collected during the period when the above-mentioned person being evaluated made a sound in the obtained sound data, respectively, calculates the S / N ratio as the ratio of the above-mentioned second average intensity to the above-mentioned first average intensity, and determines the S / N ratio of the above-mentioned second average intensity to the above-mentioned first average intensity. The inference formula used in the evaluation of the oral function of the evaluator calculates the inference value of the oral function of the person being evaluated based on the determined inference formula and the extracted feature quantity, and uses the oral function evaluation index to determine the calculated inference value, thereby evaluating the reduced state of the oral function of the person being evaluated; the terminal prompts the reduced state of the oral function of the person being evaluated, and in determining the inference formula, when the calculated S / N ratio is greater than the first threshold value, the first inference formula including the feature quantity related to sound pressure among the feature quantities extracted from the sound data is determined as the inference formula, and when the calculated S / N ratio is less than the first threshold value, the second inference formula not including the feature quantity related to sound pressure is determined as the inference formula.

[0014] Effects of the Invention

[0015] According to the oral function evaluation method and the like of the present invention, the oral function can be evaluated more accurately while using the voice of the person being evaluated. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a diagram showing the structure of an oral function evaluation system according to an embodiment.

[0017] Figure 2 This is a block diagram showing a characteristic functional structure of the oral function evaluation system according to the embodiment.

[0018] Figure 3A This is a flowchart showing a processing procedure for evaluating the oral function of an evaluation subject based on the oral function evaluation method according to the embodiment.

[0019] Figure 3B This is a flowchart showing a processing procedure for determining an estimation formula in the oral function evaluation method according to the embodiment.

[0020] Figure 3CThis is a diagram showing an example of information output in the oral function evaluation method according to the embodiment.

[0021] Figure 3D This is a graph showing the relationship between the process of determining an estimation formula and the correctness (estimation accuracy) in the oral function evaluation method according to the embodiment.

[0022] Figure 4 This is a diagram showing an overview of a method for acquiring the voice of an evaluation subject in the oral function evaluation method according to the embodiment.

[0023] Figure 5A This is a diagram showing an example of voice data in which the evaluee speaks "えをかくことにきめたよ".

[0024] Figure 5B This is a graph showing an example of the change in the formant frequency of the voice of the person being evaluated saying "えをかくことにきめたよ".

[0025] Figure 6 This is a diagram showing an example of audio data in which the subject of evaluation repeatedly utters “からからから…”.

[0026] Figure 7 1 is a diagram showing an example of audio data of the subject saying “いったい”.

[0027] Figure 8 This is a diagram showing an example of a Japanese sentence segment or fixed sentence pattern and a Chinese sentence segment or fixed sentence pattern that is similar to the degree of tongue movement or mouth opening and closing when pronouncing.

[0028] Fig. 9A It is a diagram showing the International Phonetic Alphabet of vowels.

[0029] Fig. 9B It is a diagram showing the International Phonetic Alphabet of consonants.

[0030] Fig. 10A 2 is a diagram showing an example of audio data showing the audio of the person being evaluated saying “gao dao wu da ka ji ke da yi wu zhe”.

[0031] Fig. 10B : is a graph showing an example of changes in the formant frequency of the sound of the evaluated person uttering “gao dao wu da ka ji ke da yi wu zhe”.

[0032] Fig.11 This is a diagram showing an example of oral function evaluation index.

[0033] Fig.12This is a diagram showing an example of evaluation results for each element of oral function.

[0034] Fig.13 This is a diagram showing an example of evaluation results for each element of oral function.

[0035] Fig.14 This is an example of preset data used when providing advice on oral function. DETAILED DESCRIPTION

[0036] Hereinafter, the embodiments are described with reference to the accompanying drawings. In addition, the embodiments described below all represent inclusive or specific examples. The numerical values, shapes, materials, constituent elements, configuration positions of constituent elements and connection forms, steps, the order of steps, etc. shown in the following embodiments are examples and are not intended to limit the present invention. In addition, regarding the constituent elements of the following embodiments that are not recorded in the independent claims representing the highest concept, they are described as arbitrary constituent elements.

[0037] In addition, each figure is a schematic diagram, and does not necessarily illustrate the exact figure. In addition, in each figure, the same reference numeral is given to substantially the same structure, and the repeated description may be omitted or simplified.

[0038] (Implementation Method)

[0039] [Elements of oral function]

[0040] The present invention relates to a method for evaluating a decrease in oral function, etc. Oral function includes various elements.

[0041] For example, the elements of oral function include tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function. Here, tongue coating, oral dryness, bite force, tongue pressure, and chewing function are briefly described.

[0042] The tongue coating indicates the extent of bacterial or food deposition on the tongue (i.e., oral hygiene). In the case of no tongue coating or a thin tongue coating, it is known that there is a mechanical friction environment (such as food intake), the cleaning effect of saliva, and the swallowing movement (tongue movement) is normal. On the other hand, in the case of a thick tongue coating, it is known that because the tongue movement is poor and it is difficult to take in food, it may lead to malnutrition or insufficient muscle strength. Dry mouth is the degree of dryness of the tongue. If it is dry, the movement used for speaking is hindered. In addition, food is crushed after being taken into the mouth, but it is difficult to swallow directly, so in order to make it easy to swallow the crushed food, saliva plays a role in gathering the crushed food. However, if the mouth is dry, it is difficult to form food blocks (blocks of crushed food gathered). Bite force is the force of biting hard objects, which is the strength of the muscle strength of the lower jaw. Tongue pressure is an indicator of the force of the tongue pushing the palate. If the tongue pressure weakens, there are cases where the swallowing movement is difficult to carry out. In addition, if the tongue pressure weakens, there are cases where the speed of tongue movement slows down, and there are cases where the speech speed slows down. Chewing function is a comprehensive function of the oral cavity.

[0043] In the present invention, the reduced state of the oral function of the person being evaluated (for example, the reduced state of the elements of the oral function) can be evaluated based on the sound uttered by the person being evaluated. This is because specific features can be seen in the sound uttered by the person being evaluated whose oral function is reduced, and by extracting it as a rhythmic feature, the oral function of the person being evaluated can be evaluated. The present invention is implemented by an oral function evaluation method, a program that enables a computer or the like to execute the method, an oral function evaluation device as an example of the computer, and an oral function evaluation system having the oral function evaluation device. Below, an oral function evaluation system is shown to illustrate the oral function evaluation method, etc.

[0044] [Structure of the oral function evaluation system]

[0045] The configuration of the oral function evaluation system 200 according to the embodiment will be described.

[0046] Figure 1 1 is a diagram showing a configuration of an oral function evaluation system 200 according to an embodiment.

[0047] The oral function evaluation system 200 is a system for evaluating the oral function of the evaluated person U by analyzing the voice of the evaluated person U. Figure 1 As shown, an oral function evaluation device 100 and a portable terminal 300 (an example of a terminal) are provided.

[0048] The oral function evaluation device 100 is a device that acquires voice data representing voice uttered by the evaluation subject U through the portable terminal 300 and evaluates the oral function of the evaluation subject U based on the acquired voice data.

[0049] The portable terminal 300 is a sound collection device that collects the sound of the evaluated person U speaking a sentence or a fixed sentence pattern in a non-contact manner, and outputs the sound data representing the collected sound to the oral function evaluation device 100, wherein the sentence or fixed sentence pattern includes a change in the first formant frequency or a change in the second formant frequency and is composed of two or more beats (mora), or includes at least one of a flash, a plosive, a voiceless sound, a glottal sound, and a fricative sound. For example, the portable terminal 300 is a smart phone or a tablet computer with a microphone. In addition, the portable terminal 300 is not limited to a smart phone or a tablet computer as long as it is a device with a sound collection function, and it can also be a notebook PC, etc. In addition, the oral function evaluation system 200 can also be equipped with a sound collection device (microphone) instead of the portable terminal 300. In addition, the oral function evaluation system 200 can also be equipped with an input interface for obtaining personal information of the evaluated person U. The input interface is not particularly limited as long as it has an input function such as a keyboard or a touch. In addition, the volume of the microphone can also be set in the oral function evaluation system 200.

[0050] The portable terminal 300 may also be a display device having a display and displaying an image based on the image data output from the oral function evaluation device 100. That is, the portable terminal 300 is an example of a prompting device for prompting the information output from the oral function evaluation device 100 as an image. In addition, the display device may not be the portable terminal 300, but may be a monitor device composed of a liquid crystal panel or an organic EL panel. That is, in this embodiment, the portable terminal 300 is both a sound collection device and a display device, but the sound collection device (microphone), the input interface and the display device may also be separately provided.

[0051] The oral function evaluation device 100 and the portable terminal 300 may be connected by wire or wirelessly as long as they can transmit and receive audio data or image data for displaying an image showing an evaluation result described later.

[0052] The oral function evaluation device 100 analyzes the voice of the person to be evaluated U based on the voice data collected by the portable terminal 300, evaluates the oral function of the person to be evaluated U based on the analysis result, and outputs the evaluation result. For example, the oral function evaluation device 100 outputs image data for displaying an image representing the evaluation result or data for providing oral advice to the person to be evaluated U generated based on the evaluation result to the portable terminal 300. In this way, the oral function evaluation device 100 can notify the person to be evaluated U of the degree of oral function and advice for preventing the reduction of oral function, so that, for example, the person to be evaluated U can prevent and improve the reduction of oral function.

[0053] The oral function evaluation device 100 is, for example, a personal computer, but may also be a server device. In addition, the oral function evaluation device 100 may also be a portable terminal 300. That is, the portable terminal 300 may also have the functions of the oral function evaluation device 100 described below.

[0054] Figure 2 1 is a block diagram showing a characteristic functional structure of an oral function evaluation system 200 according to an embodiment. The oral function evaluation device 100 includes an acquisition unit 110, an S / N ratio calculation unit 115, a determination unit 116, an extraction unit 120, a calculation unit 130, an evaluation unit 140, an output unit 150, a suggestion unit 160, a storage unit 170, and an information output unit 180.

[0055] The acquisition unit 110 acquires voice data obtained by collecting voices spoken by the evaluated person U in a non-contact manner through the portable terminal 300. The voice is a voice in which the evaluated person U speaks a sentence or a fixed sentence pattern that includes a change in the first formant frequency or a change in the second formant frequency and is composed of two or more beats. Alternatively, the voice may be a voice in which at least one of a flash, a plosive, a voiceless, a glottal, and a fricative is spoken or a fixed sentence pattern is spoken. However, in some situations described later, the voice may be a voice in which an arbitrary sentence is spoken. In addition, the acquisition unit 110 may also acquire personal information of the evaluated person U. For example, personal information is information input into the portable terminal 300, such as age, weight, height, sex, BMI (Body Mass Index), dental information (for example, the number of teeth, the presence or absence of dentures, the location of occlusal support, the number of functional teeth, the number of remaining teeth, etc.), serum albumin value, or eating rate, etc. In addition, personal information may be acquired through a swallowing screening tool called EAT-10 (10-item swallowing dysphagia screening form), a Saint-style swallowing questionnaire, a medical interview, the Barthel Index, or a basic checklist, etc. The acquisition unit 110 is, for example, a communication interface for wired or wireless communication.

[0056] The S / N ratio calculation unit 115 is a processing unit that calculates the S / N ratio of the acquired sound data. The S / N ratio of the sound data is the ratio of the second average intensity of the sound collected during the period when the evaluated person U made a sound in the acquired sound data to the first average intensity of the sound collected during the period when the evaluated person U did not make a sound (a period of only dark noise). Therefore, the S / N ratio calculation unit 115 is configured to be able to extract the sound during the period when the evaluated person U did not make a sound from the sound data and calculate the first average intensity, and to extract the sound during the period when the evaluated person U made a sound from the sound data and calculate the second average intensity. Specifically, the S / N ratio calculation unit 115 is implemented by a processor, a microcomputer, or a dedicated circuit.

[0057] The decision unit 116 is a processing unit that determines the estimation formula used when the estimation value of the oral function of the evaluated person U is calculated by the calculation unit 130 described later, based on the S / N ratio calculated in the S / N ratio calculation unit 115. Specifically, the decision unit 116 determines the estimation formula to be used in the estimation from among the candidates of the estimation formulas that are preset, including at least the first estimation formula and the second estimation formula, taking into account the influence of the dark noise assumed based on the S / N ratio. In addition, the candidates of the estimation formulas are stored in the storage unit 170, etc. in a state that they are pre-calculated based on a plurality of learning data. And, the decision unit 116 determines the estimation formula used in the estimation based on the candidates of the estimation formulas stored in the storage unit 170, and stores it separately in the storage unit 170 as estimation formula data 171. Specifically, the decision unit 116 is implemented by a processor, a microcomputer, or a dedicated circuit.

[0058] The extraction unit 120 is a processing unit that analyzes the voice data of the evaluation subject U acquired by the acquisition unit 110. Specifically, the extraction unit 120 is implemented by a processor, a microcomputer, or a dedicated circuit.

[0059] The extraction unit 120 calculates a prosodic feature value based on the voice data acquired by the acquisition unit 110. The prosodic feature value is a numerical value representing a feature of the voice of the person to be evaluated U extracted from the voice data and used by the evaluation unit 140 to evaluate the oral function of the person to be evaluated U. The prosodic feature value may include at least one of a speech rate, a relative difference in sound pressure, a temporal change in the relative difference in sound pressure, a first formant frequency, a second formant frequency, a change in the first formant frequency, a change in the second formant frequency, a temporal change in the first formant frequency, a temporal change in the second formant frequency, and a time of a plosive sound.

[0060] The calculation unit 130 calculates the estimated value of the oral function of the evaluation subject U based on the prosodic feature extracted by the extraction unit 120 and the determined estimation formula. Specifically, the calculation unit 130 is implemented by a processor, a microcomputer, or a dedicated circuit.

[0061] The evaluation unit 140 evaluates the deterioration state of the oral function of the evaluated person U by determining the estimated value calculated by the calculation unit 130 using the oral function evaluation index. Index data 172 indicating the oral function evaluation index is stored in the storage unit 170. Specifically, the evaluation unit 140 is implemented by a processor, a microcomputer, or a dedicated circuit.

[0062] The output unit 150 outputs the estimated value calculated by the calculation unit 130 to the suggestion unit 160. In addition, the output unit 150 may also output the evaluation result of the oral function of the evaluated person U evaluated by the evaluation unit 140 to the portable terminal 300 or the like. Specifically, the output unit 150 is implemented by a processor, a microcomputer or a dedicated circuit, and a communication interface for wired communication or wireless communication.

[0063] The suggestion unit 160 compares the estimated value calculated by the calculation unit 130 with the preset data, and makes suggestions related to the oral function of the person to be evaluated U. The suggestion data 173, which is the preset data, is stored in the storage unit 170. In addition, the suggestion unit 160 may also compare the personal information acquired by the acquisition unit 110 with the suggestion data 173, and make suggestions related to the oral cavity to the person to be evaluated U. The suggestion unit 160 outputs the suggestions to the portable terminal 300. The suggestion unit 160 is realized by, for example, a processor, a microcomputer or a dedicated circuit, and a communication interface for wired communication or wireless communication.

[0064] The storage unit 170 is a storage device that stores candidate data (not shown) of an estimated formula of oral function calculated based on a plurality of learning data, estimated formula data 171 indicating the estimated formula determined by the determination unit 116, index data 172 indicating an oral function evaluation index for determining an estimated value of the oral function of the evaluated person U, advice data 173 indicating a relationship between the estimated value of the oral function and the content of advice, and personal information data 174 indicating the above-mentioned personal information of the evaluated person U. The estimated formula data 171 is referred to by the calculation unit 130 when calculating the estimated value of the oral function of the evaluated person U. The index data 172 is referred to by the evaluation unit 140 when evaluating the deterioration state of the oral function of the evaluated person U. The advice data 173 is referred to by the advice unit 160 when providing advice related to the oral function to the evaluated person U. The personal information data 174 is, for example, data acquired via the acquisition unit 110. In addition, the personal information data 174 may be stored in the storage unit 170 in advance. The storage unit 170 is implemented by, for example, ROM (Read Only Memory), RAM (Random Access Memory), a semiconductor memory, HDD (Hard Disk Drive), or the like.

[0065] The information output unit 180 is a processing unit that outputs information for increasing the S / N ratio. When the calculated S / N ratio does not satisfy a certain standard, the information output unit 180 generates and outputs information such as instructions for improving the environment for collecting the voice spoken by the evaluated person U. Specifically, the information output unit 180 is implemented by a processor, a microcomputer, or a dedicated circuit.

[0066] In addition, the storage unit 170 may also store a program executed by a computer to realize the S / N ratio calculation unit 115, the determination unit 116, the extraction unit 120, the calculation unit 130, the evaluation unit 140, the output unit 150, the suggestion unit 160, and the information output unit 180, image data representing the evaluation result used when outputting the evaluation result of the oral function of the evaluated person U, and data such as an image, a moving image, a sound, or a text representing the content of the suggestion. In addition, the storage unit 170 may also store an image for instruction described later.

[0067] Although not shown in the figure, the oral function evaluation device 100 may also include an indication unit for indicating that the subject U pronounces a sentence or a fixed sentence pattern including a change in the first formant frequency or a change in the second formant frequency and composed of two or more beats, or including at least one of a flash, a plosive, a voiceless sound, a glottal, and a fricative. Specifically, the indication unit obtains image data of an indication image or sound data of an indication sound stored in the storage unit 170 for indicating the pronunciation of the above-mentioned sentence or fixed sentence pattern, and outputs the image data or the sound data to the portable terminal 300.

[0068] [Oral function evaluation method processing order]

[0069] Next, a specific processing procedure of the oral function evaluation method executed by the oral function evaluation device 100 will be described.

[0070] Figure 3A This is a flowchart showing a processing procedure for evaluating the oral function of an evaluation subject U based on the oral function evaluation method according to the embodiment. Figure 4 1 is a diagram showing an overview of a method for acquiring the voice of the person to be evaluated U based on the oral function evaluation method.

[0071] First, the instruction unit instructs to pronounce a sentence or a fixed sentence pattern, wherein the sentence or fixed sentence pattern includes a change in the first formant frequency or a change in the second formant frequency and is composed of two or more beats, or includes at least one of a flash, a plosive, a voiceless sound, a glottal, and a fricative sound (step S101). For example, in step S101, the instruction unit obtains image data of an image for indicating to the evaluated person U stored in the storage unit 170, and outputs the image data to the portable terminal 300. Then, as Figure 4 As shown in (a) of FIG. 1 , an image for indicating to the evaluated person U is displayed on the portable terminal 300. Figure 4In (a), an example of a fixed phrase is "I decided to draw a picture". However, it can also be used as an indicative phrase such as "The old man who made the flowers bloom and the monkey and the crab compete", "Draw a picture of the flowers blooming", "The sunflowers have bloomed", etc. In addition, it can also be used as an indicative phrase such as "Ippai", "Ittai", "Ikkai", "Pattan", "Kappa", "Tail", "Kikkari", "Only", etc. In addition, it can also be used as an indicative phrase such as "kara", "sara", "chara", "jara", "shara", "kyara", "pura", etc. In addition, you can also point out phrases such as "aei", "iea", "ai", "ia", "kakeki", "kikeka", "naneni", "chiteta", "papepi", "pipepapa", "katepi", "chipeka", "kaki", "tachi", "papi", "misa", "rari", "wani", "niwa", "eo", "io", "iu", "teko", "kiro", "teru", "peko", "memo", "emo", etc. Pronunciation instructions can also be repeated to point out such phrases.

[0072] In addition, the instruction section can also be passed through to obtain the existing information section 170 intermediate uses for the user to be referred to. However, it is also possible not to use the vocal image and voice for instructions, and it is the idea that the recipient (family members, medical students, etc.) can use the voice to direct the recipient U to the above-mentioned instructions.

[0073] For example, the uttered sentence or fixed sentence pattern may include two or more vowels, or a combination of vowels and consonants that are accompanied by the opening and closing of the mouth or the forward and backward movement of the tongue for speaking. For example, as such a sentence section or fixed sentence pattern, Japanese has "えをかくことにきめたよ" and so on. In order to say "えを" in "えをかくことにきめたよ", the tongue moves back and forth, and in order to say "きめた" in "えをかくことにきめたよ", the mouth opens and closes. The "えを" part of "えをかくことにきめたよ" includes the second formant frequencies of the vowel "e" and the vowel "o", and because the vowel "e" and the vowel "o" are adjacent, it includes the change amount of the second formant frequency. In addition, the time change of the second formant frequency is included in this part. The "きめた" part of "えをかくことにきめたよ" includes the first formant frequencies of the vowel "i", the vowel "e", and the vowel "a", and because the vowel "i", the vowel "e", and the vowel "a" are adjacent, it includes the change amount of the first formant frequency. In addition, the time change of the first formant frequency is included in this part. By saying "えをかくことにきめたよ", prosodic features such as relative sound pressure difference, first formant frequency, second formant frequency, change in first formant frequency, change in second formant frequency, time change in first formant frequency, time change in second formant frequency, and speech rate can be extracted.

[0074] For example, the fixed sentence pattern may include the repetition of a segment consisting of a flop and a consonant different from the flop. For example, Japanese has such a fixed sentence pattern as "からからから..." By repeatedly saying "からからから...", it is possible to extract prosodic features such as the relative difference in sound pressure, the temporal change in the relative difference in sound pressure, the temporal change in sound pressure, and the number of repetitions.

[0075] For example, a spoken sentence or fixed sentence pattern may include at least one combination of a vowel and a plosive. For example, Japanese has such a sentence. By speaking "いったい", prosodic features such as a relative difference in sound pressure and the time of the plosive (the time between vowels) can be extracted.

[0076] In addition, since the rhythmic feature of the relative sound pressure difference is easily affected by dark noise, the rhythmic feature of the relative sound pressure difference may have an adverse effect on the accuracy of the estimation of the estimated value, especially in the case of a sound collection environment with a relatively small S / N ratio. Therefore, in the present invention, the estimation formula is determined based on the S / N ratio calculated in the S / N ratio calculation unit 115 to change the influence of the feature of the relative sound pressure difference on the calculation of the estimated value. In the present invention, by doing so, the estimated value can be estimated by reducing the possibility that the rhythmic feature of the relative sound pressure difference will have an adverse effect on the accuracy of the estimation of the estimated value.

[0077] use Figure 3B to Figure 3D Specific processing and other operations for this will be described. Figure 3B This is a flowchart showing a processing procedure for determining an estimation formula in the oral function evaluation method according to the embodiment. Figure 3C This is a diagram showing an example of information output in the oral function evaluation method according to the embodiment. Figure 3D This is a graph showing the relationship between the process of determining an estimation formula and the correctness (estimation accuracy) in the oral function evaluation method according to the embodiment.

[0078] like Figure 3B As shown, the S / N ratio calculation unit 115 measures the dark noise for calculating the S / N ratio, and calculates the first average intensity (sound pressure) of only the dark noise (step S201). In the measurement of the dark noise, the sound during the period when the person being evaluated U does not make a sound can be extracted for use. For example, as described above, when the person being evaluated U speaks the indicated sentence or fixed sentence pattern, the sound can be extracted during the period of only dark noise before and after the sentence or fixed sentence pattern. If there is a part where the sound is interrupted in the fixed sentence pattern, the sound can be extracted as a period of only dark noise at that part.

[0079] Next, the S / N ratio calculation unit 115 calculates the second average intensity (sound pressure) of the person being evaluated U when speaking in order to calculate the S / N ratio (step S202). At this time, the sound when the instructed sentence or fixed sentence pattern is spoken may be used, or an instruction may be given to separately speak an arbitrary sentence or fixed sentence pattern in order to collect the sound. Alternatively, if the person being evaluated U is talking to someone before the oral function evaluation is about to be performed, the first average intensity and the second average intensity may be calculated using this situation.

[0080] Furthermore, the S / N ratio calculation unit 115 calculates the S / N ratio by calculating the ratio of the second average intensity to the first average intensity (step S203). Here, the calculated S / N ratio is output to the information output unit 180. Next, the information output unit 180 determines whether the S / N ratio is greater than the second threshold value (step S204). When it is determined that the S / N ratio is below the second threshold value ("No" in S204), the information output unit 180 generates and outputs information on improving the sound collection environment to increase the S / N ratio (step S205).

[0081] For example, in Figure 3CAs an example of the case where such information is output, the state where "Please check the connection status of the microphone or increase the volume when speaking" is displayed on the portable terminal 300 is shown. By outputting information in this way, an instruction to increase the S / N ratio is given by at least one of reducing the dark noise, that is, reducing the second average intensity, and increasing the volume when speaking, that is, increasing the first average intensity. In addition, the portable terminal 300 may be caused to display "Please move the sound collection location" to reduce the ambient sound when the person being evaluated speaks.

[0082] Back to Figure 3B When it is determined that the S / N ratio is greater than the second threshold value ("Yes" in S204), the information output unit 180 does nothing in particular and proceeds to step S206. Specifically, the calculated S / N ratio is also output to the decision unit 116. The decision unit 116 determines whether the S / N ratio is greater than the first threshold value (step S206). When it is determined that the S / N ratio is less than the first threshold value ("No" in S206), the decision unit 116 determines the second estimation formula that does not include the feature quantity related to sound pressure among the prosody feature quantities extracted from the voice data as the estimation formula used for estimation (step S208), and stores the estimation formula data 171 in the storage unit 170. On the other hand, when it is determined that the S / N ratio is greater than the first threshold value ("Yes" in S206), the decision unit 116 determines the first inference formula including the feature quantity related to sound pressure among the prosodic feature quantities extracted from the sound data as the inference formula used in the inference (step S207), and stores the inference formula data 171 in the storage unit 170.

[0083] In this way, an estimation formula that changes depending on whether or not the prosodic feature quantity related to sound pressure is determined based on the S / N ratio, and is used for estimating the estimation value. Figure 3D In the figure, (a) shows the relationship between the S / N ratio and the estimation accuracy when the same estimation formula is used without considering the S / N ratio, and (b) shows the relationship between the S / N ratio and the estimation accuracy when the estimation formula is changed according to the S / N ratio to include or not include the rhythmic feature quantity related to sound pressure.

[0084] like Figure 3D As shown in FIG. 1 , if the S / N ratio is greater than the first threshold, the estimation accuracy is the same in (a) and (b). However, in the range where the S / N ratio is less than the first threshold and greater than the second threshold, Figure 3D In (a), the rhythmic feature quantity related to sound pressure is affected by the influence of dark noise, which reduces the estimation accuracy, thus Figure 3D The inference accuracy will be lower than that of (b). Figure 3DAs shown by the single-dot chain line in (b), if the S / N ratio is below the second threshold, an instruction to increase the S / N ratio is given, so before the estimated value is estimated, the environment is transferred to an improved S / N ratio and the processing is performed again from the acquisition of the sound data. Therefore, it is not easy to estimate the estimated value in a state with low estimation accuracy. However, in the state where the S / N ratio is below the second threshold, Figure 3D The estimation accuracy is higher than that of (a), so it can be said that it is also useful to estimate the estimated value in this state. Figure 3A As explained in the description, the sound data can also be obtained by collecting the sound of the evaluated person U speaking at least twice a sentence or a fixed sentence pattern at different speech speeds. For example, the evaluated person U is instructed to speak "えをかくことにきめたよ" at a normal speed and a speed faster than that. By speaking "えをかくことにきめたよ" at a normal speed and a speed faster than that, the degree of maintenance of the state of the oral function can be estimated.

[0085] Then, if Figure 3A As shown, the acquisition unit 110 acquires the voice data of the evaluated person U who received the instruction in step S101 via the portable terminal 300 (step S102). Figure 4 As shown in (b) of FIG. 1 , in step S102 , for example, the evaluated person U utters a sentence or a fixed sentence pattern such as “えをかくことにきめたよ” toward the portable terminal 300. The acquisition unit 110 acquires the sentence or fixed sentence pattern uttered by the evaluated person U as voice data.

[0086] Next, the extraction unit 120 extracts prosodic features from the audio data acquired by the acquisition unit 110 (step S103 ).

[0087] For example, when the sound data acquired by the acquisition unit 110 is obtained from the sound of saying "えをかくことにきめたよ", the extraction unit 120 extracts the relative difference in sound pressure, the first formant frequency, the second formant frequency, the amount of change in the first formant frequency, the amount of change in the second formant frequency, the time change of the first formant frequency, the time change of the second formant frequency, and the speech rate as prosodic feature quantities. Figure 5A and Figure 5B This is explained.

[0088] Figure 5A This is a diagram showing an example of voice data showing the voice of the evaluee U speaking "えをかくことにきめたよ". Figure 5A The horizontal axis of the graph shown is time, and the vertical axis is power (sound pressure). Figure 5A The unit of power shown on the vertical axis of the graph is decibel (dB).

[0089] exist Figure 5AIn the graph shown, the changes in sound pressure corresponding to "え", "を", "か", "く", "こ", "と", "に", "き", "め", "た", and "よ" can be confirmed. Obtain Department 110 at Figure 3A In step S102 shown in the figure, the user U is obtained Figure 5A The extraction unit 120, for example, Figure 3A In step S103 shown in the figure, the extraction is performed by a known method. Figure 5A The sound data shown includes the sound pressures of "k" and "a" in "か(ka)", the sound pressures of "k" and "o" in "こ(ko)", the sound pressures of "t" and "o" in "と(to)", and the sound pressures of "t" and "a" in "た(ta)". The extraction unit 120 extracts the relative difference Diff_P(ka) of the sound pressures of "k" and "a" as a prosodic feature based on the extracted sound pressures of "k" and "a". Similarly, the extraction unit 120 extracts the relative difference Diff_P(ko) of the sound pressures of "k" and "o", the relative difference Diff_P(to) of the sound pressures of "t" and "o", and the relative difference Diff_P(ta) of the sound pressures of "t" and "a" as prosodic features. For example, based on the relative difference in sound pressure, the oral function related to the force of swallowing (the pressure of the tongue contacting the palate) or the force of gathering food can be evaluated. Furthermore, based on the relative difference in sound pressure including "k", the oral function related to the ability to prevent food and drink from flowing into the throat can be evaluated.

[0090] Figure 5B A graph showing an example of change in the formant frequency of the voice of the evaluee U saying "えをかくことにきめたよ". Specifically, Figure 5B This is a graph for explaining an example of changes in the first formant frequency and the second formant frequency.

[0091] The first resonance peak frequency is the peak frequency of the amplitude that can be seen first from the low frequency side of the human voice, and is known to easily reflect the characteristics related to the opening and closing of the mouth. The second resonance peak frequency is the peak frequency of the amplitude that can be seen second from the low frequency side of the human voice, and is known to easily reflect the influence related to the forward and backward movement of the tongue.

[0092] The extraction unit 120 extracts the first formant frequency and the second formant frequency of each of the plurality of vowels as prosodic feature quantities from the voice data representing the voice spoken by the evaluator U. For example, the extraction unit 120 extracts the second formant frequency F2e corresponding to the vowel "e" in "えを" and the second formant frequency F2o corresponding to the vowel "o" as prosodic feature quantities. In addition, for example, the extraction unit 120 extracts the first formant frequency F1i corresponding to the vowel "i" in "きめた", the first formant frequency F1e corresponding to the vowel "e", and the first formant frequency F1a corresponding to the vowel "a" as prosodic feature quantities.

[0093] Furthermore, the extraction unit 120 extracts the variation of the first formant frequency and the variation of the second formant frequency of the continuous vowel character string as the prosodic feature quantity. For example, the extraction unit 120 extracts the variation of the second formant frequency F2e and the second formant frequency F2o (F2e-F2o), and the variation of the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a (F1e-F1i, F1a-F1e, F1a-F1i) as the prosodic feature quantity.

[0094] Furthermore, the extraction unit 120 extracts the time variation of the first formant frequency and the time variation of the second formant frequency of the continuous vowel string as the prosodic feature. For example, the extraction unit 120 extracts the time variation of the second formant frequency F2e and the second formant frequency F2o, and the time variation of the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a as the prosodic feature. Figure 5B , an example of the time change of the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a is shown in FIG. 1 , and the time change is ΔF1 / ΔTime. ΔF1 is F1a-F1i.

[0095] For example, based on the second resonance peak frequency, the amount of change in the second resonance peak frequency, or the time change in the second resonance peak frequency, the oral function related to the movement of gathering food (the front, back, left, and right movement of the tongue) can be evaluated. In addition, for example, based on the first resonance peak frequency, the amount of change in the first resonance peak frequency, or the time change in the first resonance peak frequency, the oral function related to the ability to crush food can be evaluated. In addition, based on the time change in the first resonance peak frequency, the oral function related to the ability to move the mouth faster can be evaluated.

[0096] In addition, if Figure 5AAs shown, the extraction unit 120 may also extract the speech rate as the prosodic feature. For example, the extraction unit 120 may also extract the time from when the evaluated person U starts to say "えをかくことにきめたよ" to when the person finishes saying it as the prosodic feature. In addition, for example, the extraction unit 120 is not limited to the time from when the entire "えをかくことにきめたよ" is said to be finished, and the extraction unit 120 may also extract the time from when a specific part of "えをかくことにきめたよ" is said to be finished as the prosodic feature. In addition, for example, the extraction unit 120 may also extract the average time taken to say one word or a plurality of words of the entire or specific part of "えをかくことにきめたよ" as the prosodic feature. For example, based on the speech rate, oral functions related to swallowing movements, movements to bring food together, or tongue dexterity can be evaluated.

[0097] For example, when the speech data acquired by the acquisition unit 110 is speech data obtained by repeatedly saying "からからから...", the extraction unit 120 extracts the temporal change of the relative difference in sound pressure as the prosodic feature value. Figure 6 This is explained.

[0098] Figure 6 1 is a diagram showing an example of audio data showing the audio of the evaluated person U repeatedly saying “Kara Kara Kara…”. Figure 6 The horizontal axis of the graph shown is time, and the vertical axis is power (sound pressure). Figure 6 The unit of power shown on the vertical axis of the graph is decibel (dB).

[0099] exist Figure 6 In the graph shown in FIG. 1 , the change in sound pressure corresponding to “か” and “ら” can be confirmed. Figure 3A In step S102 shown in the figure, the user U is obtained Figure 6 The extraction unit 120, for example, Figure 3A In step S103 shown in the figure, the extraction is performed by a known method. Figure 6The sound data shown includes the sound pressures of "k" and "a" in "か(ka)", and the sound pressures of "r" and "a" in "ら(ra)". The extraction unit 120 extracts the relative difference Diff_P(ka) between the sound pressures of "k" and "a" as a prosodic feature based on the extracted sound pressures of "k" and "a". Similarly, the extraction unit 120 extracts the relative difference Diff_P(ra) between the sound pressures of "r" and "a" as a prosodic feature. For example, the extraction unit 120 extracts the relative difference Diff_P(ka) and the relative difference Diff_P(ra) as prosodic features for each of the repeatedly spoken "から". Next, the extraction unit 120 extracts the temporal variation of the relative sound pressure difference Diff_P(ka) as a prosodic feature quantity based on the extracted relative sound pressure differences Diff_P(ka), and extracts the temporal variation of the relative sound pressure difference Diff_P(ra) as a prosodic feature quantity based on the extracted relative sound pressure differences Diff_P(ra). For example, based on the temporal variation of the relative sound pressure difference, the oral function related to the movement of swallowing, the movement of gathering food, or the ability to crush food can be evaluated.

[0100] In addition, the extraction unit 120 may extract the temporal change of the sound pressure as the prosodic feature. For example, the temporal change of the minimum sound pressure (the sound pressure of "k") in each "kara" when "kara kara..." is repeatedly said may be extracted, the temporal change of the maximum sound pressure (the sound pressure of "a") in each "kara" may be extracted, and the temporal change of the sound pressure (the sound pressure of "r") between "kara" and "ra" in each "kara" may be extracted. For example, based on the temporal change of the sound pressure, the oral function related to the swallowing movement, the movement of gathering food, or the ability to crush food can be evaluated.

[0101] In addition, if Figure 6 As shown, the extraction unit 120 may also extract the number of times "から" is uttered per a predetermined time, that is, the number of repetitions as a feature quantity. The predetermined time is not particularly limited, and is 5 seconds, etc. For example, based on the number of repetitions per a predetermined time, the oral function related to the movement of swallowing or the movement of gathering food can be evaluated.

[0102] For example, when the sound data acquired by the acquisition unit 110 is sound data obtained from the sound of saying "いったい", the extraction unit 120 extracts the relative difference in sound pressure and the time of the plosive sound as the prosodic feature amount. Figure 7 Explain it.

[0103] Figure 7 : is a figure which shows an example of the audio data which shows the sound which the evaluation subject U said "hittai". Here, an example of the audio data which shows the sound which repeatedly said "hittaihittai..." is shown. Figure 7The horizontal axis of the graph shown is time, and the vertical axis is power (sound pressure). Additionally, Figure 7 the unit of the power shown on the vertical axis of the graph is decibels (dB).

[0104] In Figure 7 the graph shown, changes in sound pressure corresponding to "i", "tsu", "ta", and "i" can be confirmed. The acquisition unit 110 obtains Figure 3A in the step S102 shown, sound data as shown in Figure 7 from the evaluatee U. The extraction unit 120 extracts, for example, in Figure 3A the step S103 shown, the sound pressures of "t" and "a" in "ta" included in the sound data shown in Figure 7 by a known method. The extraction unit 120 extracts the relative sound pressure difference Diff_P(ta) of "t" and "a" as a prosodic feature quantity based on the extracted sound pressures of "t" and "a". For example, based on the relative sound pressure difference, the oral functions related to the force of swallowing or the force of gathering food can be evaluated. In addition, the extraction unit 120 extracts the time Time(i - ta) of the plosive sound (the time of the plosive sound between "i" and "ta") as a prosodic feature quantity. For example, based on the time of the plosive sound, the oral functions related to the movement of swallowing, the movement of gathering food, or the stable movement of the tongue can be evaluated.

[0105] In addition, Japanese sentence segments or fixed phrases have been exemplified as the spoken sentence segments or fixed phrases, but it is not limited to Japanese, and any language can be used.

[0106] Figure 8 is a diagram showing an example of a Chinese sentence segment or fixed phrase that is similar to a Japanese sentence segment or fixed phrase in terms of the movement of the tongue or the degree of opening and closing of the mouth during pronunciation.

[0107] There are various languages in the world, but there are languages with similar degrees of movement of the tongue or opening and closing of the mouth during pronunciation. For example, when pronouncing the Chinese sentence "高岛屋打卡即刻打一五折" (hereinafter denoted as gao dao wu da ka ji ke dayi wu zhe), the degree of movement of the tongue or opening and closing of the mouth during pronunciation is similar to that when pronouncing the Japanese sentence "えをかくことにきめたよ", so prosodic feature quantities similar to those of the Japanese sentence "えをかくことにきめたよ" can be extracted. Additionally, the notation of tone marks is omitted in this specification. In Figure 8 several examples of sentence segments or fixed phrases with similar degrees of movement of the tongue or opening and closing of the mouth during pronunciation in Japanese and Chinese are shown for reference only.

[0108] Furthermore, using Fig. 9A and Fig. 9B , a brief explanation is given of the various languages ​​in the world that have similar tongue movements or the degree of opening and closing of the mouth during pronunciation.

[0109] Fig. 9A It is a diagram showing the International Phonetic Alphabet of vowels.

[0110] Fig. 9B It is a diagram showing the International Phonetic Alphabet of consonants.

[0111] Fig. 9A In the positional relationship of the International Phonetic Symbols of the vowels shown, the horizontal direction represents the forward and backward movement of the tongue. The closer the tongue is, the more similar the forward and backward movement of the tongue is. The vertical direction represents the degree of opening and closing of the mouth. The closer the tongue is, the more similar the degree of opening and closing of the mouth is. Fig. 9B In the table of international phonetic symbols of consonants shown, the horizontal direction indicates the parts used for pronunciation from lips to throat, and the international phonetic symbols in the same grid of the table can be used to pronounce the same sound using the same part. Therefore, the present invention can be applied to various languages ​​existing in the world.

[0112] For example, if you want to increase the opening and closing of the mouth, Fig. 9A The international phonetic symbols (e.g., "i" and "a") shown as being separated in the vertical direction are included in a sentence or a fixed sentence pattern. As a result, the variation of the first formant frequency can be increased as a prosodic feature. In addition, for example, when the front and rear positions of the tongue are to be increased, Fig. 9A The international phonetic symbols (such as "i" and "u") that are separated in the horizontal direction are included in a sentence or a fixed sentence pattern. This makes it possible to increase the variation of the second formant frequency as a prosodic feature.

[0113] For example, when the sound data acquired by the acquisition unit 110 is sound data obtained from the sound of saying "gao dao wu da ka ji ke da yiwu zhe", the extraction unit 120 extracts the relative difference in sound pressure, the first formant frequency, the second formant frequency, the change in the first formant frequency, the change in the second formant frequency, the time change of the first formant frequency, the time change of the second formant frequency, and the speech rate as prosodic feature quantities. Fig. 10A and Fig. 10B This is explained.

[0114] Fig. 10A 2 is a diagram showing an example of audio data showing the audio of the evaluated person U uttering “gao dao wu da ka ji ke da yi wu zhe”. Fig. 10A The horizontal axis of the graph shown is time, and the vertical axis is power (sound pressure). Fig. 10AThe unit of power shown on the vertical axis of the graph is decibel (dB).

[0115] exist Fig. 10A In the graph shown in FIG. 1 , the changes in sound pressure corresponding to “gao”, “dao”, “wu”, “da”, “ka”, “ji”, “ke”, “da”, “yi”, “wu”, and “zhe” can be confirmed. Figure 3A In step S102 shown in the figure, the user U is obtained Fig. 10A The extraction unit 120, for example, Figure 3A In step S103 shown in the figure, the extraction is performed by a known method. Fig. 10A The sound data shown includes the sound pressures of "d" and "a" in "dao", the sound pressures of "k" and "a" in "ka", the sound pressures of "k" and "e" in "ke", and the sound pressures of "zh" and "e" in "zhe". The extraction unit 120 extracts the relative difference Diff_P(da) between the sound pressures of "d" and "a" as a prosodic feature based on the extracted sound pressures of "d" and "a". Similarly, the extraction unit 120 extracts the relative difference Diff_P(ka) between the sound pressures of "k" and "a", the relative difference Diff_P(ke) between the sound pressures of "k" and "e", and the relative difference Diff_P(zhe) between the sound pressures of "zh" and "e" as prosodic feature. For example, based on the relative difference in sound pressure, the oral function related to the force of swallowing or the force of gathering food can be evaluated. Furthermore, based on the relative difference in sound pressure including "k", the oral function related to the ability to prevent food and drink from flowing into the throat can be evaluated.

[0116] Fig. 10B : is a diagram showing an example of changes in the formant frequency of the sound of the evaluated person U saying "gao dao wu da ka ji ke da yi wu zhe". Specifically, Fig. 10B This is a graph for explaining an example of changes in the first formant frequency and the second formant frequency.

[0117] The extraction unit 120 extracts the first formant frequency and the second formant frequency of each of the plurality of vowels as prosodic feature quantities from the voice data representing the voice spoken by the evaluator U. For example, the extraction unit 120 extracts the first formant frequency F1i corresponding to the vowel "i" in "ji", the first formant frequency F1e corresponding to the vowel "e" in "ke", and the first formant frequency F1a corresponding to the vowel "a" in "da" as prosodic feature quantities. In addition, for example, the extraction unit 120 extracts the second formant frequency F2i corresponding to the vowel "i" in "yi" and the second formant frequency F2u corresponding to the vowel "u" ​​in "wu" as prosodic feature quantities.

[0118] Furthermore, the extraction unit 120 extracts the variation of the first formant frequency and the variation of the second formant frequency of the character string of consecutive vowels as prosodic feature quantities. For example, the extraction unit 120 extracts the variation of the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a (F1e-F1i, F1a-F1e, F1a-F1i), and the variation of the second formant frequency F2i and the second formant frequency F2u (F2i-F2u) as prosodic feature quantities.

[0119] Furthermore, the extraction unit 120 extracts the time variation of the first formant frequency and the time variation of the second formant frequency of the continuous vowel character string as the prosodic feature quantity. For example, the extraction unit 120 extracts the time variation of the first formant frequency F1i, the first formant frequency F1e, and the first formant frequency F1a, and the time variation of the second formant frequency F2i and the second formant frequency F2u as the prosodic feature quantity.

[0120] For example, based on the second resonance peak frequency, the amount of change in the second resonance peak frequency, or the time change in the second resonance peak frequency, the oral function related to the movement of gathering food can be evaluated. In addition, based on the first resonance peak frequency, the amount of change in the first resonance peak frequency, or the time change in the first resonance peak frequency, the oral function related to the ability to crush food can be evaluated. In addition, based on the time change in the first resonance peak frequency, the oral function related to the ability to move the mouth faster can be evaluated.

[0121] In addition, if Fig. 10A As shown, the extraction unit 120 may also extract the speech rate as the prosodic feature. For example, the extraction unit 120 may also extract the time from when the person being evaluated U starts to say "gao dao wu da ka ji ke da yi wu zhe" to when the person being evaluated finishes saying it as the prosodic feature. In addition, for example, it is not limited to the time from when the entire "gao dao wu da ka ji ke da yi wu zhe" is said to be finished. The extraction unit 120 may also extract the time from when a specific part of "gao dao wu da ka ji ke dayi wu zhe" is said to be finished as the prosodic feature. In addition, for example, the extraction unit 120 may also extract the average time taken to say one word or a plurality of words of the entire or specific part of "gao dao wu da ka ji ke da yi wu zhe" as the prosodic feature. For example, based on the speech rate, it is possible to evaluate the oral function related to the movement of swallowing, the movement of gathering food, or the dexterity of the tongue.

[0122] Back to Figure 3AAs described in , the calculation unit 130 calculates the estimated value of the oral function of the evaluated person U based on the extracted prosodic feature value and the estimation formula of the oral function calculated based on a plurality of learning data (step S104).

[0123] As described above, the inference formula of the oral function is determined by the determination unit 116 from a plurality of candidates based on the S / N ratio. The plurality of candidates for the inference formula are respectively set in advance based on the evaluation results of a plurality of subjects. The sound feature quantities spoken by the subjects are collected, and the oral function of the subjects is actually diagnosed, and the correlation between the sound feature quantities and the diagnosis results is set by statistical analysis such as multiple regression. Different types of inference formulas can be generated according to the selection method of the sound feature quantities used as representative values. In this way, candidates for the inference formula can be generated in advance. In addition, the inference formulas suitable for each element of the oral function are usually different, so a plurality of candidates for the inference formula are set for each element of the oral function. In particular, in the present invention, each of the first inference formula and the second inference formula is set for each element of the oral function.

[0124] Furthermore, in order to express the correlation between the sound feature quantity and the diagnosis result, machine learning may be used for setting. Machine learning methods include logistic regression, SVM (Support Vector Machine), random forest, and the like.

[0125] For example, the candidate of the estimation formula may be configured to include coefficients corresponding to elements of oral function and variables into which the extracted prosodic feature values ​​are substituted and multiplied by the coefficients. The following equations 1 to 5 are examples of the first estimation formula.

[0126] Estimated value of oral hygiene = (A1×F2e)+(B1×F2o)+(C1×F1i)+(D1×F1e)+(E1×F1a)+(F1×Diff_P(ka))+(G1×Diff_P(ko))+(H1×Diff_P(to))+(J1×Diff_P(ta))+(K1×Diff_P(ka))+(L1×Diff_P(ra))+(M1×Num(kara))+(N1×Diff_P(ta))+(P1×Time(i-ta))+Q1 (Formula 1)

[0127] Estimated value of oral dryness = (A2×F2e)+(B2×F2o)+(C2×F1i)+(D2×F1e)+(E2×F1a)+(F2×Diff_P(ka))+(G2×Diff_P(ko))+(H2×Diff_P(to))+(J2×Diff_P(ta))+(K2×Diff_P(ka))+(L2×Diff_P(ra))+(M2×Num(kara))+(N2×Diff_P(ta))+(P2×Time(i-ta))+Q2 (Formula 2)

[0128] Estimated value of bite force = (A3×F2e)+(B3×F2o)+(C3×F1i)+(D3×F1e)+(E3×F1a)+(F3×Diff_P(ka))+(G3×Diff_P(ko))+(H3×Diff_P(to))+(J3×Diff_P(ta))+(K3×Diff_P(ka))+(L3×Diff_P(ra))+(M3×Num(kara))+(N3×Diff_P(ta))+(P3×Time(i-ta))+Q3 (Formula 3)

[0129] Estimated value of tongue pressure = (A4×F2e)+(B4×F2o)+(C4×F1i)+(D4×F1e)+(E4×F1a)+(F4×Diff_P(ka))+(G4×Diff_P(ko))+(H4×Diff_P(to))+(J4×Diff_P(ta))+(K4×Diff_P(ka))+(L4×Diff_P(ra))+(M4×Num(kara))+(N4×Diff_P(ta))+(P4×Time(i-ta))+Q4 (Formula 4)

[0130] Estimated value of chewing function = (A5×F2e)+(B5×F2o)+(C5×F1i)+(D5×F1e)+(E5×F1a)+(F5×Diff_P(ka))+(G5×Diff_P(ko))+(H5×Diff_P(to))+(J5×Diff_P(ta))+(K5×Diff_P(ka))+(L5×Diff_P(ra))+(M5×Num(kara))+(N5×Diff_P(ta))+(P5×Time(i-ta))+Q5 (Formula 5)

[0131] A1, B1, C1, ..., P1, A2, B2, C2, ..., P2, A3, B3, C3, ..., P3, A4, B4, C4, ..., P4, A5, B5, C5, ..., P5 are coefficients, specifically, coefficients corresponding to elements of oral function. For example, A1, B1, C1, ..., P1 are coefficients corresponding to oral hygiene, which is one of the elements of oral function, A2, B2, C2, ..., P2 are coefficients corresponding to oral dryness, which is one of the elements of oral function, A3, B3, C3, ..., P3 are coefficients corresponding to bite force, which is one of the elements of oral function, A4, B4, C4, ..., P4 are coefficients corresponding to tongue pressure, which is one of the elements of oral function, and A5, B5, C5, ..., P5 are coefficients corresponding to masticatory function, which is one of the elements of oral function.

[0132] Q1 is a constant corresponding to oral hygiene, Q2 is a constant corresponding to oral dryness, Q3 is a constant corresponding to bite force, Q4 is a constant corresponding to tongue pressure, and Q5 is a constant corresponding to chewing function.

[0133] F2e multiplied by A1, A2, A3, A4, A5 and F2o multiplied by B1, B2, B3, B4, B5 are variables into which the second formant frequency as the prosodic feature quantity extracted from the utterance data when the evaluated person U utters “えをかくことにきめたよ” is substituted. F1i multiplied by C1, C2, C3, C4, C5, F1e multiplied by D1, D2, D3, D4, D5 and F1a multiplied by E1, E2, E3, E4, E5 are variables into which the first formant frequency as the prosodic feature quantity extracted from the utterance data when the evaluated person U utters “えをかくことにきめたよ” is substituted. Diff_P(ka) multiplied by F1, F2, F3, F4, F5, Diff_P(ko) multiplied by G1, G2, G3, G4, G5, Diff_P(to) multiplied by H1, H2, H3, H4, H5, and Diff_P(ta) multiplied by J1, J2, J3, J4, J5 are variables substituted as the relative difference in sound pressure of the prosodic feature extracted from the speech data when the person being evaluated U utters "えをかくことにきめたよ". Diff_P(ka) multiplied by K1, K2, K3, K4, K5 and Diff_P(ra) multiplied by L1, L2, L3, L4, L5 are variables into which the relative difference in sound pressure of the prosodic feature extracted from the speech data when the evaluated person U utters "から". Num(kara) multiplied by M1, M2, M3, M4, M5 is a variable into which the number of repetitions of the prosodic feature extracted from the speech data when the evaluated person U repeatedly utters "から" within a certain period is substituted. Diff_P(ta) multiplied by N1, N2, N3, N4, N5 is a variable into which the relative difference in sound pressure of the prosodic feature extracted from the speech data when the evaluated person U utters "いったい". Time (i-ta) multiplied by P1, P2, P3, P4, and P5 is a variable into which the time of the plosive sound, which is a prosodic feature extracted from the utterance data when the evaluated person U utters “いったい”, is substituted.

[0134] As shown in the above formulas 1 to 5, for example, the calculation unit 130 calculates the estimated value for each element of the oral function of the evaluated person U (for example, tongue coating, oral dryness, bite force, tongue pressure, and chewing function). In addition, these elements of oral function are examples, and the elements of oral function only need to include at least one of the tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function of the evaluated person U.

[0135] In addition, for example, the extraction unit 120 extracts a plurality of prosodic features from the voice data obtained by collecting voices of the evaluated person U speaking a plurality of types of sentences or fixed sentence patterns (for example, "えをかくことにきめたよ", "から", and "いったい" in the above-mentioned Formulas 1 to 5), and the calculation unit 130 calculates the estimated value of the oral function based on the extracted plurality of prosodic features and the estimation formula. The calculation unit 130 can calculate the estimated value of the oral function with high accuracy by substituting the plurality of prosodic features extracted from the voice data of the plurality of types of sentences or fixed sentence patterns into one estimation formula.

[0136] The following equations 6 to 10 are examples of the second estimation equation.

[0137] Estimated value of oral hygiene = (A1×F2e)+(B1×F2o)+(C1×F1i)+(D1×F1e)+(E1×F1a)+(M1×Num(kara))+(P1×Time(i-ta))+Q1 (Formula 6)

[0138] Estimated value of oral dryness = (A2×F2e)+(B2×F2o)+(C2×F1i)+(D2×F1e)+(E2×F1a)+(M2×Num(kara))+(P2×Time(i-ta))+Q2 (Formula 7)

[0139] Estimated value of bite force = (A3×F2e)+(B3×F2o)+(C3×F1i)+(D3×F1e)+(E3×F1a)+(M3×Num(kara))+(P3×Time(i-ta))+Q3 (Formula 8)

[0140] Estimated value of tongue pressure = (A4×F2e)+(B4×F2o)+(C4×F1i)+(D4×F1e)+(E4×F1a)+(M4×Num(kara))+(P4×Time(i-ta))+Q4 (Formula 9)

[0141] Estimated value of masticatory function = (A5×F2e)+(B5×F2o)+(C5×F1i)+(D5×F1e)+(E5×F1a)+(M5×Num(kara))+(P5×Time(i-ta))+Q5 (Formula 10)

[0142] Thus, compared with the first inference formula, the second inference formula omits the terms of rhythmic feature quantities related to sound pressure, such as relative sound pressure difference, and can estimate appropriate inference values ​​with relatively high estimation accuracy even in a sound collection environment with a smaller S / N ratio.

[0143] Furthermore, although a linear expression is shown as the estimated expression, the estimated expression may be a multi-level expression such as a quadratic expression.

[0144] Next, the evaluation unit 140 evaluates the reduced state of the oral function of the evaluated person U by using the oral function evaluation index to determine the estimated value calculated by the calculation unit 130 (step S105). For example, the evaluation unit 140 evaluates the reduced state of the oral function of the evaluated person U according to each element of the oral function by using the oral function evaluation index set for each element of the oral function to determine the estimated value of each element of the oral function. The oral function evaluation index is an index used to evaluate the oral function, for example, a condition for determining that the oral function is reduced. Fig.11 The oral function evaluation index is described.

[0145] Fig.11 This is a diagram showing an example of oral function evaluation index.

[0146] The oral function evaluation index is set for each element of the oral function. For example, an index of 50% or more is set for oral hygiene, an index of 27 or less is set for oral dryness, an index of less than 200N is set for bite force (when using Dental Prescale II of GC Co., Ltd.), an index of less than 30kPa is set for tongue pressure, and an index of less than 100mg / dL is set for chewing function (for the index, refer to "Basic Considerations on Oral Function Decrease" (https: / / www.jads.jp / basic / pdf / document_02.pdf) of the Japanese Dental Association). The evaluation unit 140 evaluates the state of oral function decline of the person being evaluated U according to each element of the oral function by comparing the calculated estimated value of each element of the oral function with the oral function evaluation index set for each element of the oral function. For example, when the calculated estimated value of oral hygiene is 50% or more, oral hygiene as an element of oral function is evaluated to be in a reduced state. Similarly, when the calculated estimated value of oral dryness is 27 or less, oral dryness as an element of oral function is evaluated to be in a reduced state, when the calculated estimated value of bite force is less than 200N, the bite force as an element of oral function is evaluated to be in a reduced state, when the calculated estimated value of tongue pressure is less than 30kPa, the bite force as an element of oral function is evaluated to be in a reduced state, and when the calculated estimated value of chewing function is less than 100mg / dL, the chewing function as an element of oral function is evaluated to be in a reduced state. In addition, as oral function evaluation indicators set for oral hygiene, oral dryness, bite force, tongue pressure and chewing function, Fig.11What is shown is an example, and the present invention is not limited to this. For example, the index of remaining teeth can also be set for chewing function. In addition, oral hygiene, oral dryness, bite force, tongue pressure, and chewing function are shown as elements of oral function, but these are examples. For example, for the reduction of tongue, mouth and lip movement function, there are elements of oral function such as tongue movement, lip movement, and lip strength.

[0147] Back to Figure 3A As described in , the output unit 150 outputs the evaluation result of the oral function of the evaluated person U evaluated by the evaluation unit 140 (step S106). For example, the output unit 150 outputs the evaluation result to the portable terminal 300. In this case, the output unit 150 may also include a communication interface for wired communication or wireless communication, obtain image data of an image corresponding to the evaluation result from the storage unit 170, and send the obtained image data to the portable terminal 300. Fig.12 and Fig.13 An example of the image data (evaluation result) is shown in FIG.

[0148] Fig.12 and Fig.13 : is a diagram showing an example of the evaluation results of each element of oral function. Fig.12 As shown, the evaluation result may be an evaluation result of two levels, OK or NG. OK means normal, and NG means abnormal. In addition, normal and abnormal may not be indicated for each element of the oral function. For example, the evaluation result may only indicate the element with reduced suspicion. In addition, the evaluation result is not limited to the evaluation result of two levels, and may be a more detailed evaluation result in which the degree of evaluation is divided into three or more levels. In this case, the indicator data 172 stored in the storage unit 170 may include multiple indicators for one element. In addition, Fig.13 The evaluation results are shown in a radar chart. Fig.12 and Fig.13 In the data, oral cleanliness, food holding force, force of biting hard objects, tongue force, and jaw movement are shown as elements of oral function. Regarding oral cleanliness, the evaluation result is given based on oral hygiene; regarding food holding force, the evaluation result is given based on oral dryness; regarding force of biting hard objects, the evaluation result is given based on bite force; regarding tongue force, the evaluation result is given based on tongue pressure; and regarding jaw movement, the evaluation result is given based on the estimated value of chewing function. In addition, Fig.12 and Fig.13 This is an example, and the evaluation item sentences, oral function items, and their corresponding combinations are not limited to Fig.12 and Fig.13 The content shown.

[0149] Back to Figure 3AAs described in , the suggestion unit 160 compares the estimated value calculated by the calculation unit 130 with the pre-set data (suggestion data 173) to make suggestions related to the oral function of the evaluated person U (step S107). Fig.14 Describe the preset data.

[0150] Fig.14 This is an example of preset data (advice data 173 ) used when providing advice on oral function.

[0151] like Fig.14 As shown, the recommendation data 173 is data that establishes a correspondence between the evaluation result and the recommendation content for each element of the oral function. For example, when the estimated value of the calculated oral cleanliness is 50% or more, the recommendation unit 160 determines that it is OK because the index is met, and makes a recommendation based on the recommendation content established in correspondence with the oral cleanliness. In addition, the specific recommendation content is omitted, but for example, the storage unit 170 includes data (for example, images, moving images, sounds, texts, etc.) indicating the recommendation content, and the recommendation unit 160 uses such data to make recommendations related to the oral function to the evaluated person U.

[0152] [Effects, etc.]

[0153] As described above, the oral function evaluation method related to the present embodiment is performed by a terminal (portable terminal 300) and an oral function evaluation device 100, and the oral function evaluation method evaluates the reduced state of the oral function of the evaluated person U based on the sound spoken by the evaluated person U, including: the terminal obtains sound data by collecting the sound spoken by the evaluated person U; the oral function evaluation device 100 obtains the sound data, extracts a feature value from the obtained sound data, and calculates the first average intensity of the sound collected during the period when the evaluated person U did not make a sound and the second average intensity of the sound collected during the period when the evaluated person U made a sound in the obtained sound data, and calculates the ratio of the second average intensity to the first average intensity. The S / N ratio of the evaluated person U is determined, and an inference formula used in the evaluation of the oral function of the evaluated person U is determined; based on the determined inference formula and the extracted feature quantity, an inference value of the oral function of the evaluated person U is calculated; the calculated inference value is judged by using an oral function evaluation index, and the reduced state of the oral function of the evaluated person U is evaluated; the terminal prompts the evaluated reduced state of the oral function of the evaluated person U; in determining the inference formula, when the calculated S / N ratio is greater than the first threshold value, the first inference formula including the feature quantity related to the sound pressure among the feature quantities extracted from the sound data is determined as the inference formula, and when the calculated S / N ratio is less than the first threshold value, the second inference formula not including the feature quantity related to the sound pressure is determined as the inference formula.

[0154] This can produce the same effects as the oral function evaluation device described later.

[0155] In addition, for example Figure 3A As shown, it may also include: an acquisition step (step S102), acquiring sound data obtained by collecting the sound of the evaluated person U speaking a sentence or a fixed sentence pattern, wherein the sentence or fixed sentence pattern includes a change in the first resonance peak frequency or a change in the second resonance peak frequency and is composed of more than 2 beats, or includes at least one of a flash, a plosive, a voiceless sound, a glottal and a fricative sound; an extraction step (step S103), extracting a rhythmic feature from the acquired sound data; a calculation step (step S104), calculating an estimated value of the oral function of the evaluated person U based on an estimated formula of the oral function and the extracted rhythmic feature, wherein the estimated formula of the oral function is calculated based on multiple learning data; and an evaluation step (step S105), evaluating the reduced state of the oral function of the evaluated person U by using an oral function evaluation index to determine the calculated estimated value.

[0156] Thus, by obtaining sound data suitable for the evaluation of oral function, the oral function of the person U to be evaluated can be easily evaluated. That is, the oral function of the person U to be evaluated can be evaluated only by the person U to be evaluated speaking the above-mentioned sentence or fixed sentence pattern toward a sound collecting device such as the portable terminal 300. In particular, since the estimated value of the oral function is calculated using an inference formula calculated based on a plurality of learning data, the reduced state of the oral function can be quantitatively evaluated. In addition, since the oral function is not evaluated by directly comparing the prosodic feature value with the threshold value, but the inference value is calculated based on the prosodic feature value and the inference formula, and the inference value is compared with the threshold value (oral function evaluation index), the reduced state of the oral function can be evaluated with high accuracy.

[0157] For example, the estimation formula may include coefficients corresponding to elements of oral function and variables into which the extracted prosodic feature values ​​are substituted and multiplied by the coefficients.

[0158] Thus, the estimated value of the oral cavity function can be easily calculated simply by substituting the extracted prosodic feature into the estimation formula.

[0159] For example, in the calculation step, an estimated value is calculated for each element of the oral function of the evaluated person U; in the evaluation step, the calculated estimated value of each element of the oral function is determined using an oral function evaluation index set for each element of the oral function, thereby evaluating the reduced state of the oral function of the evaluated person U for each element of the oral function.

[0160] Thus, the state of oral function decline can be evaluated for each element. For example, by preparing an estimation formula for each element of oral function, in which the coefficient differs according to the element of oral function, the state of oral function decline can be easily evaluated for each element.

[0161] For example, the oral function elements may include at least one of the tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function of the evaluated person U.

[0162] Thus, the decline state of at least one oral function element of the evaluated person U among tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function and chewing function can be evaluated.

[0163] For example, the prosodic feature may include at least one of speech rate, relative sound pressure difference, time change of relative sound pressure difference, first formant frequency, second formant frequency, change of first formant frequency, change of second formant frequency, time change of first formant frequency, time change of second formant frequency and time of plosive sound.

[0164] Since pronunciation changes due to decreased oral function, the decreased state of oral function can be evaluated based on these prosodic feature quantities.

[0165] For example, in the extraction step, multiple prosodic features may be extracted from voice data obtained by collecting voices of the evaluated person U speaking multiple types of sentence segments or fixed sentence patterns; and in the calculation step, an inference value may be calculated based on the extracted multiple prosodic features and an inference formula.

[0166] Thus, by using a plurality of prosodic feature quantities extracted based on a plurality of types of sentence segments or fixed sentence patterns for one estimation formula, it is possible to improve the calculation accuracy of the estimation value of the oral cavity function.

[0167] For example, a sentence segment or a fixed sentence pattern may include two or more vowels or a combination of vowels and consonants, which are accompanied by the opening and closing of the mouth or the forward and backward movement of the tongue for speaking.

[0168] Thus, prosodic features including the change in the first formant frequency, the temporal change in the first formant frequency, the change in the second formant frequency or the temporal change in the second formant frequency can be extracted from the sound of the evaluated person U speaking such a sentence or fixed sentence.

[0169] For example, the audio data may be obtained by collecting audio of the evaluated person U speaking a sentence or a fixed sentence pattern at least twice at different speech speeds.

[0170] Therefore, the degree to which the state of the oral function is maintained can be estimated based on the sound of the evaluated person U speaking such a sentence or a fixed sentence pattern.

[0171] For example, the fixed sentence pattern may include repetition of a sentence segment consisting of a flap and a consonant different from the flap.

[0172] Thus, prosodic feature quantities including temporal changes in the relative difference in sound pressure, temporal changes in sound pressure, and the number of repetitions can be extracted from the sound of the evaluated person U speaking such a sentence or a fixed sentence pattern.

[0173] For example, a sentence segment or a fixed sentence pattern may include at least one combination of a vowel and a plosive sound.

[0174] Thus, it is possible to extract prosodic feature quantities including the relative difference in sound pressure and the timing of plosive sounds from the sound of the evaluated person U speaking such a sentence segment or a fixed sentence pattern.

[0175] For example, the oral function evaluation method may further include an advice step of providing advice on the oral function of the person to be evaluated U by comparing the calculated estimated value with preset data.

[0176] Thus, the person being evaluated U can receive advice on what kind of countermeasures should be taken when the oral function is reduced.

[0177] The oral function evaluation device 100 of the present embodiment is an oral function evaluation device 100 for evaluating the reduced state of the oral function of the evaluated person U based on the sound spoken by the evaluated person U, and comprises: an acquisition unit 110, which acquires sound data obtained by collecting the sound spoken by the evaluated person U; an extraction unit 120, which extracts feature quantities from the acquired sound data; an S / N ratio calculation unit 115, which respectively calculates the first average intensity of the sound collected during the period when the evaluated person U did not make a sound and the second average intensity of the sound collected during the period when the evaluated person U made a sound, and calculates the S / N ratio which is the ratio of the second average intensity to the first average intensity, from the acquired sound data; The unit 116 determines an inference formula used in the evaluation of the oral function of the evaluated person U; the calculation unit 130 calculates the inference value of the oral function of the evaluated person U based on the determined inference formula and the extracted feature quantity; and the evaluation unit 140 uses the oral function evaluation index to determine the calculated inference value, thereby evaluating the reduced state of the oral function of the evaluated person U. The determination unit 116 determines the first inference formula including the feature quantity related to the sound pressure among the feature quantities extracted from the sound data as the inference formula when the calculated S / N ratio is greater than the first threshold value, and determines the second inference formula not including the feature quantity related to the sound pressure as the inference formula when the calculated S / N ratio is less than the first threshold value.

[0178] Thus, it is possible to determine whether the condition is suitable for using the feature quantity related to sound pressure based on the S / N ratio, and according to the determination, it is determined whether to use the first estimation formula including the feature quantity related to sound pressure or the second estimation formula not including the feature quantity related to sound pressure. Thus, the state of the oral function reduction of the evaluated person U can be evaluated by an appropriate estimation formula from the perspective of including or not including the feature quantity related to sound pressure. That is, by appropriately using the feature quantity related to sound pressure, the oral function of the evaluated person U can be evaluated more accurately.

[0179] For example, the oral function of the person to be evaluated U may be at least one of tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function of the person to be evaluated U.

[0180] Thus, the decline state of at least one of the tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function of the evaluation subject U can be evaluated.

[0181] For example, the first estimation formula and the second estimation formula may be set for each of the tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function, and chewing function of the evaluated person U.

[0182] Therefore, for each of the tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function and chewing function of the evaluated person U, an evaluation of the reduced state can be performed based on whether it is suitable to use the first estimation formula or the second estimation formula.

[0183] For example, the information output unit 180 may be further provided to output information for increasing the S / N ratio when the calculated S / N ratio is equal to or smaller than a second threshold value which is smaller than the first threshold value.

[0184] Therefore, when it is desired to evaluate the deterioration of oral function in an unsuitable environment such as one with a smaller S / N ratio, measures can be taken to improve the environment, thereby suppressing the evaluation of the deterioration of oral function in such an unsuitable environment.

[0185] For example, the information may be information recommending at least one of the following actions: confirming the connection status of the sound collecting device (microphone) used in collecting the sound spoken by the evaluated person U, increasing the volume of the voice when the evaluated person U speaks, and reducing the ambient sound when the evaluated person U speaks.

[0186] Therefore, as measures to improve the environment, at least one of the following actions can be recommended: confirming the connection status of the sound collection device (microphone) used in collecting the sound spoken by the evaluated person U, increasing the volume of the voice when the evaluated person U speaks, and reducing the ambient sound when the evaluated person U speaks.

[0187] For example, the acquisition unit 110 may acquire the first voice data as voice data not used in the evaluation of the oral function of the person to be evaluated U, and the S / N ratio calculation unit may calculate the S / N ratio of the acquired first voice data.

[0188] This makes it possible to calculate the S / N ratio using the first audio data that is not used in the evaluation of the oral function of the person U to be evaluated.

[0189] For example, the acquisition unit 110 may acquire the second audio data as audio data used in the evaluation of the oral function of the person to be evaluated U, and the S / N ratio calculation unit may calculate the S / N ratio of the acquired second audio data.

[0190] Thereby, the S / N ratio can be calculated using the second audio data used for evaluating the oral cavity function of the person U to be evaluated.

[0191] For example, the system may further include an advice unit 160 that provides advice on the oral function of the person to be evaluated U by comparing the calculated estimated value with preset data.

[0192] Thus, the person being evaluated U can receive advice on what kind of countermeasures should be taken when the oral function is reduced.

[0193] For example, the device may further include: a sound collecting device (microphone) used for collecting the voice spoken by the person to be evaluated U; and a presentation device (portable terminal 300) for presenting the deterioration state of the oral function of the person to be evaluated U.

[0194] In addition, for example, it may also include: an acquisition unit 110, which acquires sound data obtained by collecting the sound of the evaluated person U speaking a sentence or a fixed sentence pattern, wherein the sentence or fixed sentence pattern includes a change in the first resonance peak frequency or a change in the second resonance peak frequency and is composed of more than 2 beats, or includes at least one of a flash sound, a plosive sound, a voiceless sound, a glottal sound and a fricative sound; an extraction unit 120, which extracts a prosodic feature from the acquired sound data; a calculation unit 130, which calculates an estimated value of the oral function of the evaluated person U based on an estimated formula of the oral function and the extracted prosodic feature, wherein the estimated formula of the oral function is calculated based on multiple learning data; and an evaluation unit 140, which uses an oral function evaluation index to determine the calculated estimated value, thereby evaluating the reduced state of the oral function of the evaluated person U.

[0195] Thus, the oral function evaluation device 100 capable of easily evaluating the oral function of the person to be evaluated U can be provided.

[0196] The oral function evaluation system 200 related to the present embodiment is an oral function evaluation system 200 that evaluates the reduced state of the oral function of the evaluated person U based on the sound spoken by the evaluated person U, and is equipped with a terminal (portable terminal 300) and an oral function evaluation device 100 connected to the terminal; the terminal has: a sound collecting device (microphone) used in collecting the sound spoken by the evaluated person U; and a prompting device (a part of the portable terminal 300) used to prompt the reduced state of the oral function of the evaluated person U being evaluated, and the oral function evaluation device 100 has: an acquisition unit 110 that acquires sound data obtained by collecting the sound spoken by the evaluated person U; an extraction unit 120 that extracts feature values ​​from the acquired sound data; an S / N ratio calculation unit 115 that calculates, in the acquired sound data, the first average value of the sound collected during the period when the evaluated person U did not make any sound. intensity, and the second average intensity of the sound collected during the period when the evaluated person U makes the sound, and calculates the S / N ratio as the ratio of the second average intensity to the first average intensity; the determination unit 116 determines the estimation formula used in the evaluation of the oral function of the evaluated person U; the calculation unit 130 calculates the estimated value of the oral function of the evaluated person U based on the determined estimation formula and the extracted feature quantity; and the evaluation unit 140 uses the oral function evaluation index to determine the calculated estimation value, thereby evaluating the reduced state of the oral function of the evaluated person U; the determination unit 116 determines the first estimation formula that includes the feature quantity related to the sound pressure among the feature quantities extracted from the sound data as the estimation formula when the calculated S / N ratio is greater than the first threshold value, and determines the second estimation formula that does not include the feature quantity related to the sound pressure as the estimation formula when the calculated S / N ratio is less than the first threshold value.

[0197] Thus, it is possible to provide the oral function evaluation system 200 that can evaluate the oral function more accurately.

[0198] Furthermore, for example, the device may include the oral function evaluation device 100 and a sound collection device (portable terminal 300 ) to collect the sound of the person to be evaluated U speaking a sentence or a fixed sentence pattern in a non-contact manner.

[0199] Thus, the oral function evaluation system 200 capable of easily evaluating the oral function of the person to be evaluated U can be provided.

[0200] (Other embodiments)

[0201] As mentioned above, although the oral function evaluation method etc. concerning embodiment were demonstrated, this invention is not limited to the said embodiment.

[0202] For example, the candidate of the inference formula may be updated based on the evaluation result obtained when the expert actually diagnoses the oral function of the person to be evaluated U. In this way, the evaluation accuracy of the oral function can be improved. Machine learning may also be used to improve the evaluation accuracy of the oral function.

[0203] In addition, for example, regarding the suggestion data 173, the suggestion content may be evaluated by the evaluated person U, and updated based on the evaluation result. For example, in the case where a suggestion is made regarding an oral function that is not a problem for the evaluated person U, the evaluated person U evaluates the suggestion content as wrong. And, by updating the suggestion data 173 based on the evaluation result, the wrong suggestion as described above is not made. In this way, the suggestion content related to the oral function for the evaluated person U can be made more effective. In addition, in order to make the suggestion content related to the oral function more effective, machine learning can also be used.

[0204] In addition, for example, the evaluation results of oral function may be stored together with personal information as big data and used for machine learning. In addition, the content of advice related to oral function may be stored together with personal information as big data and used for machine learning.

[0205] In addition, for example, in the above-mentioned embodiment, the oral function evaluation method includes the advice step (step S107 ) of giving advice on the oral function, but this step may not be included. In other words, the oral function evaluation device 100 may not include the advice unit 160 .

[0206] In addition, for example, in the above-described embodiment, the acquisition step (step S102 ) acquires the personal information of the person to be evaluated U, but this information may not be acquired. In other words, the acquisition unit 110 may not acquire the personal information of the person to be evaluated U.

[0207] In addition, for example, the steps in the oral function evaluation method can also be executed by a computer (computer system). Furthermore, the present invention can also be implemented as a program for causing a computer to execute the steps included in these methods. Furthermore, the present invention can also be implemented as a computer-readable non-temporary recording medium such as a CD-ROM recording the program.

[0208] For example, when the present invention is implemented by a program (software), each step is performed by executing the program using hardware resources such as a CPU, a memory, and an input / output circuit of a computer. That is, each step is performed by the CPU obtaining data from a memory or an input / output circuit and performing calculations, or outputting calculation results to a memory or an input / output circuit.

[0209] Furthermore, each component included in the oral function evaluation device 100 and the oral function evaluation system 200 of the above-described embodiment may be realized as a dedicated or general-purpose circuit.

[0210] Furthermore, each component included in the oral function evaluation device 100 and the oral function evaluation system 200 of the above-described embodiment may be realized as an LSI (Large Scale Integration) which is an integrated circuit (IC).

[0211] Furthermore, the integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. A programmable FPGA (Field Programmable Gate Array) or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may also be used.

[0212] Furthermore, if integrated circuit technology that replaces LSI emerges due to advances in semiconductor technology or other derived technologies, it is of course possible to integrate the components included in the oral function evaluation device 100 and the oral function evaluation system 200 using this technology.

[0213] In addition, various modifications that can be conceived by those skilled in the art to the embodiments and embodiments achieved by arbitrarily combining the components and functions of the embodiments without departing from the gist of the present invention are also included in the present invention.

[0214] Description of Reference Numerals

[0215] 100 Oral function evaluation device

[0216] 110 Acquisition Department

[0217] 115S / N ratio calculation unit

[0218] 116 Decision Department

[0219] 120 Extraction Department

[0220] 130 Computing Department

[0221] 140 Evaluation Department

[0222] 150 Output

[0223] 160 Recommendation Department

[0224] 180 Information output unit

[0225] 200 Oral Function Evaluation System

[0226] 300 portable terminal (terminal, microphone, prompt device)

[0227] U Evaluated

Claims

1. An oral function evaluation device for evaluating the reduced state of oral function of a person being evaluated based on the sound spoken by the person being evaluated, wherein: have: an acquisition unit that acquires voice data obtained by collecting voices spoken by the person to be evaluated; An extraction unit extracts a feature value from the acquired sound data; An S / N ratio calculation unit that calculates, from the acquired sound data, a first average intensity of sounds collected during a period when the subject did not make a sound, and a second average intensity of sounds collected during a period when the subject made a sound, and calculates an S / N ratio that is a ratio of the second average intensity to the first average intensity; a determination unit that determines an estimation formula used in the evaluation of the oral function of the person to be evaluated; a calculation unit that calculates an estimated value of the oral function of the subject based on the determined estimation formula and the extracted feature amount; as well as The evaluation unit determines the calculated estimated value using an oral function evaluation index, thereby evaluating the deterioration state of the oral function of the person being evaluated, The above decision department, When the calculated S / N ratio is greater than a first threshold value, a first estimation formula including a feature quantity related to sound pressure among the feature quantities extracted from the sound data is determined as the estimation formula, When the calculated S / N ratio is equal to or less than the first threshold value, a second estimation formula that does not include the feature amount related to the sound pressure is determined as the estimation formula.

2. The oral function evaluation device according to claim 1, wherein: The oral function of the subject to be evaluated is at least one of tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function and chewing function of the subject to be evaluated.

3. The oral function evaluation device according to claim 1, wherein: The first estimation formula and the second estimation formula are respectively set for each of the tongue coating, oral dryness, bite force, tongue pressure, cheek pressure, number of remaining teeth, swallowing function and chewing function of the above-mentioned person being evaluated.

4. The oral function evaluation device according to claim 1, wherein: The device further includes an information output unit configured to output information for increasing the S / N ratio when the calculated S / N ratio is equal to or less than a second threshold value that is smaller than the first threshold value.

5. The oral function evaluation device according to claim 4, wherein: The information is information recommending at least one of the following actions: confirming the connection status of a sound collecting device used to collect the sound spoken by the person being evaluated; increasing the volume of the person being evaluated when speaking; and reducing the ambient sound when the person being evaluated speaks.

6. The oral function evaluation device according to claim 4, wherein: The acquisition unit acquires first audio data as the audio data not used in the evaluation of the oral function of the person to be evaluated, The S / N ratio calculation unit calculates the S / N ratio of the acquired first audio data.

7. The oral function evaluation device according to claim 4, wherein: The acquisition unit acquires second audio data as the audio data used in evaluating the oral function of the person to be evaluated, The S / N ratio calculation unit calculates the S / N ratio of the acquired second audio data.

8. The oral function evaluation device according to any one of claims 1 to 6, wherein: The system further includes an advice unit for providing advice on the oral function of the person to be evaluated by comparing the calculated estimated value with preset data.

9. The oral function evaluation device according to claim 8, wherein: Also available: A sound collecting device used to collect the sound spoken by the above-mentioned person being evaluated; and The prompting device is used to prompt the evaluated person's oral function to be reduced.

10. An oral function evaluation system for evaluating the deterioration of the oral function of a person being evaluated based on the sounds spoken by the person being evaluated, wherein: have: Terminal; and An oral function evaluation device is connected to the above terminal. The above terminal has: A sound collecting device used to collect the sound spoken by the above-mentioned person being evaluated; and a prompting device for prompting the evaluated person's oral function to be reduced; The oral function evaluation device has: an acquisition unit that acquires voice data obtained by collecting voices spoken by the person to be evaluated; An extraction unit extracts a feature value from the acquired sound data; An S / N ratio calculation unit that calculates, from the acquired sound data, a first average intensity of sounds collected during a period when the subject did not make a sound, and a second average intensity of sounds collected during a period when the subject made a sound, and calculates an S / N ratio that is a ratio of the second average intensity to the first average intensity; a determination unit that determines an estimation formula used in the evaluation of the oral function of the person to be evaluated; a calculation unit that calculates an estimated value of the oral function of the subject based on the determined estimation formula and the extracted feature amount; as well as The evaluation unit determines the calculated estimated value using an oral function evaluation index, thereby evaluating the deterioration state of the oral function of the person being evaluated, The above decision department, When the calculated S / N ratio is greater than a first threshold value, a first estimation formula including a feature quantity related to sound pressure among the feature quantities extracted from the sound data is determined as the estimation formula, When the calculated S / N ratio is equal to or less than the first threshold value, a second estimation formula that does not include the feature amount related to the sound pressure is determined as the estimation formula.

11. An oral function evaluation method, executed by a terminal and an oral function evaluation device, for evaluating the reduced state of the oral function of the person being evaluated based on the sound spoken by the person being evaluated, wherein: include: The terminal obtains voice data by collecting the voice spoken by the person being evaluated; The oral function evaluation device obtains the sound data; The oral function evaluation device extracts a feature value from the acquired sound data; The oral function evaluation device calculates, from the acquired sound data, a first average intensity of the sound collected during a period when the subject did not make a sound, and a second average intensity of the sound collected during a period when the subject made a sound, and calculates an S / N ratio which is a ratio of the second average intensity to the first average intensity; The oral function evaluation device determines an estimation formula used in evaluating the oral function of the person to be evaluated; The oral function evaluation device calculates an estimated value of the oral function of the person being evaluated based on the determined estimation formula and the extracted feature quantity; The oral function evaluation device uses the oral function evaluation index to determine the calculated estimated value, thereby evaluating the reduced state of the oral function of the person being evaluated; The terminal indicates the deterioration of the oral function of the person being evaluated. In the above speculative decision, When the calculated S / N ratio is greater than a first threshold value, a first estimation formula including a feature quantity related to sound pressure among the feature quantities extracted from the sound data is determined as the estimation formula, When the calculated S / N ratio is equal to or less than the first threshold value, a second estimation formula that does not include the feature amount related to the sound pressure is determined as the estimation formula.