Cognitive ability assessment device, mobile terminal, and speech acquisition device

Abstract

The embodiments of the present disclosure can provide a user's cognitive ability assessment results in a simple manner by analyzing the linguistic characteristics of the user's speech data, and through simple tests in daily life, can easily detect mild cognitive impairment or dementia in the user, allowing for early medical management and improving the quality of life of patients with degenerative brain diseases.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to Speech and EEG (Electroencephalography) a cognitive ability evaluation device, a mobile terminal, and an utterance acquisition device.

Background Art

[0002] With the development of medical technology, the average lifespan of humans is increasing. As a result of the increase in lifespan, the proportion of the elderly population in the overall population is globally on the rise. The elderly population is vulnerable to various geriatric diseases, and healthcare for the elderly population has emerged as a national and social issue.

[0003] In particular, due to the continuous increase in the elderly population, the number of elderly people suffering from degenerative brain diseases such as Alzheimer's -( (AD) disease and Parkinson's (PKS) disease is constantly increasing. Once these degenerative brain diseases develop, there are no treatment drugs, and not only the cost for management and maintenance, but also the quality of life deteriorates rapidly. Therefore, medical management for the elderly population suffering from such degenerative brain diseases poses a problem of a significant burden, either personally or socially.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Embodiments of the present disclosure can provide a method for easily confirming a decline in cognitive ability by using linguistic (linguistic or language) and acoustic (acoustic or speech) characteristics detected through utterances in a person's daily life, and using this to easily confirm early mild cognitive impairment (MCI) and early dementia. or EEG Embodiments of this disclosure can provide a device that allows elderly people to conveniently collect speech and brainwave data in their daily lives.

Means for Solving the Problems

[0005] Embodiments of this disclosure can provide a cognitive ability evaluation device that includes a task provisioning unit that provides a plurality of tasks to a user between a first test period and a second test period following the first test period; a speech data collection unit that acquires speech data in digital format converted from test speech in which the volume of at least a portion of the base speech has been adjusted according to the user's response to at least one of the plurality of tasks; and a cognitive ability measurement unit that calculates an evaluation result regarding the user's cognitive ability based on at least one linguistic characteristic of the speech data.

[0006] Embodiments of this disclosure include a task delivery module that provides a user with multiple tasks during a test period, and acquiring speech data in digital format converted from test speech in which the volume of at least a portion of the base speech is adjusted according to the user's response to at least one of the multiple tasks. EEG acquisition around the ear and A mobile device including a speech data collection module can be provided.

[0007] The mobile device may further include a cognitive ability measurement module that calculates an assessment result regarding the user's cognitive ability based on at least one linguistic characteristic of the speech data.

[0008] Embodiments of this disclosure provide a speech acquisition device that includes an output module that outputs speech corresponding to a plurality of tasks provided to the user, an input module that acquires basic speech corresponding to the user's response to at least one of the plurality of tasks, and a processing module that analyzes the magnitude or frequency band of the basic speech, amplifies the magnitude of the low-magnitude basic speech based on the analysis, adjusts so as not to amplify the large-magnitude basic speech that is easy to recognize, generates test speech so that even the user's small voice can be recognized, and converts the test speech into speech data in digital format.

[0009] The speech acquisition device may further include a communication module for transmitting speech data to an external source. [Effects of the Invention]

[0010] According to embodiments of this disclosure, changes in cognitive ability can be easily confirmed from utterances made by the user in a daily life environment, rather than in a medical facility. Evaluation results for diseases such as mild cognitive impairment or dementia can be provided through absolute evaluation of the individual's cognitive ability based on the user's individual utterance information, comparative evaluation of the individual's cognitive ability over time based on periodic evaluation information, or relative comparison of cognitive ability based on the individual's age, educational background, etc. [Brief explanation of the drawing]

[0011] [Figure 1] This figure schematically shows an example of the configuration of a cognitive ability assessment system according to an embodiment of the present disclosure. [Figure 2] This figure shows an example of how a cognitive ability assessment system according to an embodiment of this disclosure performs a cognitive ability assessment. [Figure 3] This figure shows an example of how a cognitive ability assessment system according to an embodiment of this disclosure performs a cognitive ability assessment. [Figure 4] This figure shows an example of how a cognitive ability assessment system according to an embodiment of this disclosure performs a cognitive ability assessment. [Figure 5] This figure shows an example of how a cognitive ability assessment system according to an embodiment of this disclosure performs a cognitive ability assessment. [Figure 6] This figure schematically illustrates another example of the configuration of a cognitive ability assessment system according to an embodiment of the present disclosure. [Figure 7] Figure 6 is a diagram illustrating the configuration of the speech acquisition device shown. [Figure 8] Figure 6 is a diagram illustrating the configuration of the speech acquisition device shown. [Figure 9] This figure schematically illustrates yet another example of the configuration of a cognitive ability assessment system according to the embodiments of this disclosure. [Figure 10] This figure shows an example of how a cognitive ability assessment system according to an embodiment of this disclosure performs a cognitive ability assessment. [Figure 11] This figure shows an example of how a cognitive ability assessment system according to an embodiment of this disclosure performs a cognitive ability assessment. [Figure 12] It is a flowchart showing an example of the process of a cognitive ability evaluation method according to an embodiment of the present disclosure. [Figure 13] It is a flowchart showing an example of the process of a cognitive ability evaluation method according to an embodiment of the present disclosure.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When adding reference numerals to the components of each drawing, for the same components, even if they are shown on other drawings, they may be assigned the same reference numerals as much as possible. In the description of the present disclosure, when it is determined that a specific description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description thereof will be omitted. When terms such as "including", "having", and "being composed of" mentioned in this specification are used, other parts may be added unless "only" is used. When a component is expressed in the singular, it can include the case of including a plurality, unless there are particularly explicit descriptions.

[0013] Also, when describing the components of the present disclosure, terms such as first, second, A, B, (a), (b), etc. can be used. These terms are only for distinguishing the components from other components, and the essence, order, sequence, number, etc. of the components are not limited by these terms.

[0014] In the description of the positional relationship of components, when it is described that two or more components are "connected", "coupled", or "joined", it should be understood that two or more components can be directly "connected", "coupled", or "joined", but it is also possible that two or more components and other components are further "interposed" and "connected", "coupled", or "joined". Here, the other components may be included in one or more of the two or more components that are "connected", "coupled", or "joined" to each other.

[0015] In the description of the relationship of the time flow regarding components, operation methods, manufacturing methods, etc., for example, when the chronological relationship or the sequential relationship of the flow is described using "after ~", "subsequent to ~", "next to ~", "before ~", etc., it may include cases where it is not continuous, unless "immediately" or "directly" is used.

[0016] On the other hand, when a numerical value regarding a component or its corresponding information (for example, level, etc.) is mentioned, even without a separate explicit description, the numerical value or its corresponding information can be interpreted as including the range of errors that can occur due to various factors (for example, process factors, internal or external impacts, noise, etc.).

[0017] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0018] Recently, various methods for the early detection of dementia have been studied. According to the 4th Comprehensive Dementia Management Plan, dementia management is divided into three stages: screening tests, diagnostic tests, and differential diagnoses. Screening tests are conducted at dementia support centers and other facilities, using diagnostic tools such as MMSE-KC, MMSE-DS, and CIST. The two-stage diagnostic testing utilizes CERAD-K and SNSB. If abnormalities are found during these screening and diagnostic tests, further examinations, such as beta-protein analysis using PET imaging, are conducted to ensure a precise diagnosis of the disease. These detailed examinations can be obtained through direct diagnosis at a hospital.

[0019] The cognitive ability evaluation method by the cognitive ability evaluation system according to the embodiment of the present disclosure can be performed in a short time at a welfare center or a place of residence where the elderly frequently visit, rather than at a designated medical institution. Also, the proposed method can screen a high-risk group for mild cognitive impairment (Perform screening) and by performing a detailed examination and a diagnostic examination only on the selected high-risk group, unnecessary examination costs for subjects who are not in the high-risk group can be reduced, and the state only needs to provide state support only for the high-risk group, so national costs can also be reduced. Furthermore, the proposed examination method can be easily and conveniently repeated regularly (for example, every six months) at any time and anywhere by the elderly, and can continuously progress, rather than being a one-time method at an existing medical institution, and can continuously manage and monitor the high-risk group with symptoms of early dementia over time.

[0020] As is currently known, once dementia progresses to moderate levels, it becomes difficult to treat or alleviate symptoms. Therefore, early detection of dementia and slowing its progression through drug therapy and hospital treatment, thereby enabling continued daily activities, is being promoted as a good approach.

[0021] An efficient method for detecting early symptoms of dementia is to identify high-risk groups through screening tests and then periodically test and diagnose these high-risk individuals. This method is far more efficient than existing random (approximately 1 in 10 elderly people) or all-elderly testing methods. In particular, it would be extremely useful if high-risk groups with dementia could be detected early through continuous screening tests, and if these high-risk elderly individuals could be actively managed by the state, delaying their progression to dementia stages requiring national intervention.

[0022] Recently, several methods have been proposed for identifying such high-risk groups. For example, methods have been suggested for monitoring the walking patterns of elderly individuals in their daily lives and for monitoring their sleep patterns over extended periods.

[0023] Another approach involves actively researching the monitoring of degenerative brain diseases such as dementia based on the speech (utterrance) of elderly individuals. Acoustic data or speech data, such as conversations, contains various information about a person's health status. For example, one method can be proposed that involves analyzing the characteristics of speech (timbre, speed, volume, etc.). Alternatively, speech information can be converted into the frequency domain, and artificial intelligence technology can be integrated into the frequency domain to find disease-related acoustic features. Here, speech involves learning various information that should be expressed in words, such as hearing sounds or seeing questions, through the five senses. When you gain something, it is expressed in voice. Response to this That could be the answer.

[0024] This method involves either prompting speech by having the subject listen to or see pre-prepared questions, or conveying undecided questions (e.g., questions or conversations about daily life) to the subject using audio or images, and saving the speech data, recording, or videotaping it. Tasks such as This involves real-time or retrospective analysis. Analysis of speech features may primarily involve analyzing timbre, speed, and volume. Methods utilizing such acoustic features can be highly effective in identifying dementia. Furthermore, it may be necessary to extract common phonetic features, taking into account differences due to dialect usage and individual differences in vocal structure.

[0025] Another method using dialogue or speech data from the elderly is to analyze audio data such as language comprehension, vocabulary used, descriptions of shapes, and descriptions of daily life. or dialogue The data is used to analyze response time to questions, interjections used, words used, and sentence structure. Linguistic characteristics in answers to questions such as these (Linguistic features )of There is a way to analyze it.

[0026] In this method, the speech data of elderly individuals is obtained from questions that have been pre-validated by the examiner in a one-on-one face-to-face setting. (called a task)The elderly are shown or made to listen to a video, and their responses to each question are recorded or saved in a specific memory area (e.g., the memory of a smart device). To analyze the results, the recorded audio is transcribed into text (this is called transcription), and each response is saved so that it matches the question used. During transcription, each response is transcribed along with the response time to the question, the number of words used, the accuracy of the sentence structure (e.g., whether the subject, object, and predicate are in the correct order), the appropriateness of the words used, the difficulty level of the words used, and any interjections used during the response. To date, the audio recorded through the test is manually converted into text, and the transcribed material is organized according to analysis rules. These rules may include, in addition to those mentioned above, the degree of repetition of similar words, the limits of the words used, and, if a word needed for explanation cannot be recalled, the interjections used when a word cannot be recalled.

[0027] Furthermore, a process is needed to synchronize the recorded audio segments with the transcribed text for analysis. This is called labeling. Tools for labeling may also exist for Hangul (Korean alphabet).

[0028] These methods require a significant amount of time, often several hours, to collect speech data. Furthermore, they necessitate face-to-face testing of the speaker and the person seeking advice in specific locations such as hospitals. Additionally, transcribing the collected audio data requires considerable manpower and time.

[0029] In recent years, research has been actively conducted on integrating artificial intelligence (AI) technology to recognize spoken audio and analyze its linguistic and acoustic structure. For example, it is possible to convert spoken audio data into text using speech recognition systems. Furthermore, AI speech recognition systems can be used for labeling or transcription.

[0030] By using such speech recognition devices, various speech recognition devices and equipment have been developed, and services utilizing them have been revitalized. Devices that can acquire voice or speech information include smart devices that allow two-way information exchange, such as smartphones and tablets, conversational robots, and artificial intelligence speakers. The basic configuration of these devices can consist of a microphone for listening to speech for conversation, an artificial intelligence-based speech recognition device for recognizing the input speech, response technology based on the recognized speech, and finally, a speaker for explaining or listening to questions.

[0031] Current methods of collecting speech information using microphones built into smart devices such as portable tablets require the subject to look at or listen to the smart device. Alternatively, speech information can be acquired by providing questions for speech through fixed devices with built-in artificial intelligence speech recognition capabilities, such as AI speakers, social robots, and chatbots. In both portable and fixed devices, there is a certain distance between the microphone used to collect speech data and the speaker. This means the microphone receives not only the speaker's voice but also other voices and ambient noise. In this case, the speech recognition capabilities of AI speakers and social robots may deteriorate in noisy environments, failing to correctly recognize the speaker's voice and convert it to text accurately. Alternatively, accurate speech recognition or text conversion may not occur. For example, speech spoken close to the microphone used for speech collection has a good recognition rate, but the recognition rate decreases significantly with even a slight distance. This is because the distance between the speaker and the microphone is large, resulting in a lower volume of input sound, and other ambient noises (environmental noise, everyday noises, voices other than the speaker's) are also included. In other words, the spectrum of the input sound contains noise components in addition to the speaker's original voice, leading to many errors in acoustic analysis. This is because the noise and the speaker's voice mix together and cannot be physically separated.

[0032] One way to solve this is to use multiple directional microphones in a smart device to collect only the speaker's voice. Alternatively, a noise reduction algorithm can be incorporated into the smart device to remove unwanted noise from the audio input via the microphone.

[0033] Furthermore, if the distance is slightly greater, the subject may not be well-positioned to hear the questions or dialogue. In the absence of common assistive devices or in environments with external environmental noise, the subject may have difficulty correctly hearing the questions. This could lead to situations where the subject was unable to answer because they couldn't hear, but the analysis reveals errors that can be identified as cognitive impairment.

[0034] Furthermore, in order for the subject (speaker) to hear the question and be guided to speak, the process of the subject visually and audibly seeing and hearing the content of the question can be extremely important. Embodiments of this disclosure can provide a function that amplifies the sound of the question to enable the subject (e.g., an elderly person) to hear the question well, thereby amplifying the sound of the question to a level suitable for the user so that the question can be heard well not only by elderly people with hearing loss but also by elderly people with normal hearing.

[0035] In quiet spaces free from external noise, existing proposed methods can yield accurate evaluation results. However, applying these technologies to spaces where elderly individuals are actually active presents various challenges. Alternatively, wireless earphones with microphone functionality can be used. These earphones allow for clear listening to the questionnaire's explanations and the transmitted audio to a smart device (smartphone, tablet, computer, etc.). Furthermore, the volume of the questions can be kept constant regardless of the speaker's state, thereby increasing the reliability of the test.

[0036] However, when this method is used in everyday life, the speaker's voice is transmitted along with external noise, in addition to the speaker's speech data.

[0037] The embodiments of this disclosure can provide a solution that can overcome the aforementioned problems in the practical application of screening test technology for cognitive impairment in the elderly, which actually applies linguistic / acoustic analysis technology based on speech data from the daily lives of elderly people.

[0038] Therefore, embodiments of this disclosure can provide a method for easily and conveniently recording speech data while listening to and viewing questions for testing, not only in non-medical public institutions such as elderly welfare centers that are easily accessible to the elderly, but also in actual individual living spaces, for the practical application of a convenient and efficient screening test technology for cognitive impairment and early dementia using language testing technology. Furthermore, it can enable testing and evaluation even in everyday living spaces, even with background noise and other sounds.

[0039] Most elderly people experience age-related hearing loss, and methods should be proposed to help them acquire information for speech more effectively. In the testing, it is preferable to be able to amplify the sound according to the degree of hearing loss. Examples of such methods include adjusting the volume when using earphones, or appropriately amplifying the sound when there is no assistance from devices such as earphones. Therefore, embodiments of this disclosure may also include adjusting the volume of the sound to a level suitable for elderly individuals to hear questions.

[0040] Furthermore, a method is needed to transmit speech data to a cloud server as quickly as possible, and to analyze the speech data, so that the results of the speech data analysis can be provided to the user immediately after the test.

[0041] In particular, techniques for selecting the optimal set of questions for various tasks are necessary to conduct the test as efficiently and quickly as possible. Furthermore, techniques for shortening the test time and guiding the test subject to concentrate on the test while it is being conducted may also be necessary.

[0042] In general, information gathering methods in such phonetic approaches require a method and equipment that provides the speaker with questions for speaking, and equipment that stores the spoken information when the speaker speaks.

[0043] Generally, in language tests, questions are heard via a smart device, and, if necessary, the content of the questions is viewed on the smart device's screen before answering. These smart devices can take several forms. For example, they may be mobile-assistant tablets, smartphones, notebooks, or mobile robots. Alternatively, they may be fixed devices such as personal computers.

[0044] [Method for collecting speech data using smart devices and wearable devices, and analysis method using the cloud and AI]

[0045] Embodiments of this disclosure propose a method for performing cognitive impairment or language tests via an app or the web using wearable devices and smart devices. Furthermore, the invention proposes a method that utilizes the cloud and integrates AI technology to screen for mild cognitive impairment and early-stage dementia. This method allows for testing not only in service settings such as senior citizen centers, dementia support centers, and elderly welfare centers, but also in the daily living spaces of the elderly, enabling them to conveniently, easily, and periodically utilize the testing service by using their own smart devices.

[0046] In particular, embodiments of the present disclosure use a wearable device that wirelessly listens to questions asked on a smart device's web or app, receives audio of the answers to the questions heard, and transmits them to the smart device without distortion, in order to solve these problems. The audio answers transmitted from the wearable device to the smart device are stored on the smart device, and the stored audio is transmitted to the cloud.

[0047] As another example, while the questioning is in progress, the audio collected via the wearable device's microphone is stored directly on the wearable device and the audio data is distributed all at once.

[0048] By listening to questions and saving answers via a smart device's web or app through a wearable device, labeling can be easily handled. This is because questions and answers are stored consistently. Furthermore, by incorporating gyroscopes and accelerometers, it's possible to use acceleration information to record answers in the case of multiple-choice questions. The timing of the response to a question is particularly important, and providing information about the end and duration of the response allows for more accurate information about the utterance of the answer.

[0049] The proposed method constructs a standardized big data set of speech data from elderly Koreans and uses it to learn the language characteristics of subjects with symptoms of mild cognitive impairment and early dementia. Furthermore, the big data can accumulate long-term test data from the same individuals, allowing for the extraction of language characteristics that change over time. The classifier thus trained is preferably able to present the degree of normality and abnormality in cognitive impairment. For example, it can show how much the performance deviates from the average value for healthy elderly individuals. It can also present, based on speech data extracted from patients with mild cognitive impairment, how much the current state of mild cognitive impairment deviates from existing accumulated database data. Specifically, the evaluation results of the language test are displayed by converting the evaluation of each individual question into a score, and the score for each question may differ depending on the type of task used to induce speech. As an example of the score for each question, weights and importance scores for language features such as the number of words used, frequency, sentence structure, and time to the start of speech after the question are prepared in advance and applied to the collected speech data. The sum of the weighted language features quantifies the speech data for each question.

[0050] In speech testing methods, participants can choose how to speak about a given topic within a limited time. This can be done using audio, images, or text to present the topic. The topics can vary widely, including family names, animal names, place names, and country names. Any topic word can be selected as long as its number of single-word responses can be counted. Therefore, not only nouns, but also predicates such as "beautiful," "pretty," and "adorable," or verbs such as "run," "sprint," and "flee," can be counted. Thus, any topic word that allows for the deduction of related words or can be spoken is acceptable.

[0051] The time limit is approximately a few minutes for common topic words. However, utterances of at least 30 seconds are possible, and it is desirable to design each topic word to be completed within 3 minutes.

[0052] A simple practical evaluation method involves assessing cognitive function by briefly evaluating the number of words in each topic, the difficulty level of the words, and the number of times the same word is repeated.

[0053] Furthermore, evaluating two or more topic words rather than just one can improve the accuracy of the assessment of cognitive function. In other words, by evaluating multiple topic words, the results of the evaluation for one topic word and the evaluation for the second or third topic word can be presented separately, and the three results can be combined to make a comprehensive evaluation.

[0054] The tasks include comprehension tests, expression / speech tests, and memory tests, and each task contains a variety of questions. For quantification, the speech data from the questions used within each task can be quantified and the average value calculated. Alternatively, the test results can be quantified independently for each task.

[0055] When the results of each test are quantified as described above, the speech data can be translated into linguistic data collected from prepared language tests conducted on healthy individuals, MCI patients, dementia patients, etc., and by transcribing this data, it is possible to derive data that shows linguistic characteristics (word frequency, speech time after question, words used, interjections, sentence structure, etc.).

[0056] In particular, current language assessments include a variety of tasks, each with several questions corresponding to that task. For example, language assessments include comprehension tasks, expression tasks, and memory tasks. Expression tasks include a variety of questions, such as describing shapes or telling fairy tales. Memory tasks involve reading a series of words about animals or fruits and then speaking the words that are recalled. The questions in each task include questions of similar difficulty levels, as well as more difficult questions. However, depending on the actual subject, it may not be necessary to administer the same questions for all tasks. To address this, it is desirable to select the most appropriate questions for each task. Methods for selecting these optimal tasks and questions may also be derived from big data on language assessments. For example, the difficulty level of tasks could be varied depending on the subject's condition. Alternatively, tasks with overlapping features could be optimized and used. To date, there has been little academic research on question sets that consider task-specific correlations. This optimal set of questions, similar to currently used methods, can extract characteristic correlations between questions through artificial intelligence learning, based on big data collected through almost all tasks and the questions within each task, taking into account factors such as the participant's gender, age, and educational background. For example, it can select a set of questions for tasks that match the difficulty level of describing pictures based on educational background.

[0057] Another approach involves pre-prepared, task-specific questions, where the selection of the next question may be based on information and answers from previous questions. This information and answers are analyzed in real time. The analysis includes the speaker's understanding of the question, their ability to answer, and their linguistic characteristics. Therefore, the selection of the next question is based on basic information about the speaker, such as gender, age, and education level, along with the analysis results from previous questions. Artificial intelligence techniques can also be applied to create appropriate models for selecting these next questions.

[0058] In this way, the optimal task set can be selected for each task (the selection method utilizes an AI model that selects the optimal task set based on big data on existing subjects), and the examination can proceed. This shortens the examination time and makes it easier for elderly people to undergo the examination.

[0059] Therefore, in order to obtain utterance information for each task, it is necessary to determine the optimal task list for each individual question task.

[0060] [Functional Definition of Wearable Devices]

[0061] As mentioned earlier, in voice reception methods using microphones built into smart devices, there is a certain distance between the smart device and the speaker, and in addition to the speaker's utterances, ambient noise and the voices of other people are added. The accuracy of the linguistic and acoustic analysis results using the input speech information in this way deteriorates.

[0062] Embodiments of this disclosure can provide a method for using a wearable device to accurately collect spoken voice data. The proposed wearable device is preferably connected wirelessly to a smart device. Alternatively, a separate app / web may be used for the connection between the wearable device and the smart device.

[0063] Wearable speech collection devices for the elderly may include noise reduction technology to eliminate external noise. Noise reduction technology not only improves the accuracy of speaker speech recognition by directly removing various background noises and irregular sounds, but also allows for clearer confirmation and analysis of the characteristics of the speech itself by removing unwanted signals, making it possible to easily conduct language tests even in noisy environments.

[0064] To improve the speech recognition rate of AI speech recognition devices in wearable devices and to secure as much speech information from the speaker as possible, speech signal processing can have two possible functions.

[0065] Firstly, there may be a function to remove noise depending on the environment. For example, if background noise or voices other than the speaker's are mixed in from the surrounding environment, removal may be necessary. This can improve the speech recognition characteristics of the speaker in the final evaluation stage.

[0066] Secondly, if the speaker's voice volume is low, it needs to be increased. Also, if there are quiet sounds within the speaker's speech, they need to be searched further in the final speech recognition process. When a person listens and transcribes, they can hear them if they listen carefully, but in everyday life, the situation can be very different when using speech information from a speaker to recognize speech with an AI speech recognition device. In other words, internal monologues and interjections included in the speech are all quiet and may be excluded by the AI ​​speech recognition device, in which case the information necessary for analysis may be lost. Therefore, in order to secure speech data that is displayed quietly in addition to the voice that is accurately spoken by the wearable device, it is necessary to amplify the voice in question to improve the speech recognition rate of the AI ​​speech recognition device. Also, since noise is amplified along with the sound, it is sometimes desirable to perform noise reduction and amplification simultaneously.

[0067] The noise reduction process involves converting the sound signal input via the microphone into a frequency domain and removing the noise power in the frequency domain.

[0068] One method for removing noise is to pre-learn and store the speaker's voice. At this time, the speaker's voice characteristics, such as frequency characteristics and loudness information, can be extracted and utilized. It is also possible to use a method that removes both speech containing information other than the secured speaker's voice characteristics, or background noise. For noise reduction, the speech data input in real time should be converted to the frequency domain, and background noise and speech information not related to the speaker's characteristics should be removed using the already stored speaker voice information.

[0069] Another method involves using a classifier or detector capable of classifying noise and the speech environment. That is, if noise and speech can be efficiently separated, misrecognition of speech will not occur, and noise power can be removed during speech frequency analysis. Therefore, by improving the signal-to-noise ratio (SNR), speech can be heard more clearly (Clarification), and the content of the noise can be heard with clearer speech (Clearance). Furthermore, by utilizing an intelligent classifier that has been trained on traffic noise, everyday noise, etc., a method can be used to remove data other than the speaker's voice from the speech data input via the microphone.

[0070] In addition, various noise reduction techniques can be applied. In particular, several methods can be applied to remove noise from the frequency band by converting the input signal into a frequency band.

[0071] The wearable device may have a receiver (speaker) that can hear task questions for speech from a smart device. The speaker is preferably inserted into the ear. It can also be manufactured as a bone conduction type. This is advantageous for testing because, when listening to task questions using only a smart device, the questions may be difficult to hear due to external noise, but by using the proposed method, the questions can be heard more accurately.

[0072] The microphone of a wearable device is advantageously positioned near where the subject speaks. For example, it could be a neckband-type speech collector or an earphone-type device. A lightweight wearable device should be used. This device, if possible, can collect speech information directly from the subject.

[0073] Wearable devices can utilize directional microphones. Directional microphones allow for clearer reception of only the speaker's voice.

[0074] Also, When audio is transmitted, the sound quality itself may be distorted, potentially leading to a lower voice recognition rate. It transmits digitized data instead of voice. It can be converted from smart devices to audio. 。

[0075] Wearable devices can selectively amplify audio received via a microphone. For example, they can selectively amplify specific frequency bands, such as high-frequency ranges or low-frequency ranges. .Ma The audio characteristics of audio received via microphone change slightly depending on the microphone's characteristics or the ADC (analog-to-digital converter) that converts analog to digital signals. The digitized audio source, depending on the characteristics of wireless communication and the signal-to-noise ratio, may have small signals in certain frequency bands removed before being transmitted wirelessly to a smart device. For example, an elderly person with cognitive impairment may mutter softly to themselves when considering an answer to a question if they cannot recall it. This information can be used as an important item in linguistic characteristics analysis, and it is necessary to preserve the speaker's original data as much as possible. Alternatively, in certain expressions, if the sound is too quiet, the answer to the question may not be recognized, potentially leading to an inaccurate evaluation. In yet another case, the elderly person's voice itself may be quiet, resulting in a small audio signal being input to the smart device.

[0076] Generally, when signals input from a microphone are digitized, faint sounds may be removed due to the characteristics of the equipment and the process of audio signal processing. While this removal may not pose a major problem for everyday communication, it can result in the loss of various information in terms of linguistic and acoustic characteristics. If faint sounds are removed, they may not be included in the evaluation results, even though they should be reflected in the evaluation score, potentially leading to an inaccurate final evaluation.

[0077] Therefore, this problem can be solved by using a method that amplifies quiet sounds and does not amplify loud sounds in the speech data received by the microphone. Furthermore, by analyzing the frequency band of the input sound and selectively amplifying according to frequency, the degree of speech recognition can be improved, expressions with speech features can not be removed, and quiet sounds can be amplified and used to accurately evaluate the speech data.

[0078] Amplifying the signal using a wearable device offers the advantage of transmitting and analyzing the original speaker's voice signal with the least distortion. In other words, while this could also be done on a server with very high computing power, there is a problem in that a lot of linguistic and phonetic information may be lost during transmission to the server.

[0079] Wearable devices can directly store answers to individual task questions on the device itself, and can also send multiple answers together to a smart device. Furthermore, the speech data received for each question can be wirelessly transmitted to the smart device before the next question begins.

[0080] [App / Web Functionality Definition]

[0081] The speech collection app / web must include the following features: It must be able to transmit voice instructions for a task to a wearable device as the task is performed. It must also have features to monitor the process performed by the elderly via the smart device, which can be helpful if the test is erroneous due to the elderly's mistakes or errors. Generally, if an error occurs or a retest is required during the test via the app / web, the test should be able to be rerun. Also, if the test is interrupted due to pressing the wrong button during the test, the user should be able to resolve the relevant issue with the help of the examiner.

[0082] The system will synchronize and save the speech data for each question. The data stored on the smart device will be sent to the cloud according to a specified protocol.

[0083] The results of the answers to the questions for each task sent to the cloud are used to determine mild cognitive impairment, dementia symptoms, and steady state through a model that has already learned about the disease. Such determinations can also be made using scores. It can indicate the degree of risk compared to the normal standard. For example, if a high-risk result is obtained, it can be linked to preventative treatment through detailed screening tests and diagnostic tests at a hospital.

[0084] While cloud-based evaluation methods can derive highly reliable results by using a large amount of data, they require the direct transmission of speech data to the cloud, thus necessitating cloud access. Therefore, we propose a method that can provide services using only wearable devices, even in environments where the cloud is unavailable.

[0085] This involves using a speech database classifier for mild cognitive impairment or cognitive impairment embedded in a wearable device. In other words, the user speaks while viewing and listening to the task's questions via a smart device. Speech data is input via the wearable device's microphone, and noise is removed and sound is amplified as needed. The wearable device has a pre-trained cognitive impairment classifier built in, and cognitive impairment is detected early through this classifier, utilizing the speech data input to the microphone. The classifier used in this case preferably uses smaller linguistic and volumetric features than a cloud-based classifier, taking into account the computing power and memory capacity of the wearable device.

[0086] As a method for collecting speech data, various tasks can be automatically saved and labeled to facilitate their identification.

[0087] For example, methods can be used to skip if there is no sound for a certain period of time, to record answers using physical sensor information by incorporating a gyroscope or accelerometer for multiple-choice questions, and to skip using specific interactions with the device (e.g., by command, by questioning, such as "Next" or "Do you want to proceed to the next step?", or by using physical functions unique to the device (touch, buttons, etc.). Messages may also be output at regular intervals to alert the user to help them concentrate on the task. The content and format of the messages can vary; for example, the user's name may be called, or comments such as "Please concentrate on the test" may be output in the form of sound. Alternatively, the device may output light, images, or other forms of light that are advantageous for concentration, such as flashing lights.

[0088] This proposed method not only facilitates labeling when storing data, but also facilitates the separation of each task, allowing for the subdivision of various questions included in a task and the selection of the optimal set of questions for each task. This significantly reduces the time required for screening and verification.

[0089] Such wearable devices can be used solely to store a speaker's utterances and transmit them to a smart device. That is, the subject can directly hear and see the questions through the smart device. The wearable device collects, stores, and transmits the utterances for each question to the smart device. Wireless communication is preferred for the connection between the wearable device and the smart device, and the utterance data collected via the microphone in the wearable device can be converted to digital via an ADC, transmitted as digital data, or converted back to speech and transmitted as speech. In this process as well, background noise and voices other than the speaker's are removed to facilitate speaker speech recognition and analysis.

[0090] [About the service]

[0091] The service method scenario is as follows:

[0092] Participants wear a wearable device and can view or listen to test questions via an app on their smart device or via the web on a PC. They proceed through the test by listening to and viewing the test questions administered via the smart device's app through the wearable device. During the questions, the speaker's voice is collected via the wearable device's microphone and transmitted to the smart device as digital or audio data. The smart device sends the speaker's answers to each question to the cloud, where an evaluation model is used to score each answer. The evaluation and analysis of each answer are performed via a separate speech recognition system and a normal / mild cognitive impairment classifier. Specifically, the input speech data is converted to text via the speech recognition system. The converted text is then analyzed and evaluated according to certain rules. The classifier learns and stores linguistic and phonetic characteristics of normal individuals and mild cognitive impairment patients using answers to previously presented questions. Once the entire test is completed, the scores are totaled and the speaker is provided with information indicating how much their level of mild cognitive impairment, depression, etc., deviates from that of a person with normal cognitive function.

[0093] Alternatively, a wearable device can utilize a speech recognition system and a classifier capable of identifying mild cognitive impairment, depression, and other conditions. In this method, the subject wears the wearable device and can view, listen to, and answer questions while looking at the smart device. Specifically, speech data is recorded via the wearable device's microphone. The wearable device incorporates both a speech recognition system and a classifier. The speech recognition system converts the input speech data into text. The converted text is then analyzed and evaluated according to certain rules. While it is desirable for the wearable device to know the speaker's question information, it is not required. The wearable device incorporates a classifier that has already learned the linguistic and vocal characteristics of normal individuals and those with mild cognitive impairment in response to previously presented questions. This method allows for immediate and rapid verification of the test results without the use of the cloud. Furthermore, there is no need to transmit the subject's voice data to the cloud.

[0094] The embodiments of the present disclosure described above will be explained with reference to the accompanying drawings to illustrate feasible examples.

[0095] Figure 1 is a schematic diagram showing an example of the configuration of a cognitive ability assessment system according to an embodiment of the present disclosure. Figures 2 to 5 show examples of how the cognitive ability assessment system according to an embodiment of the present disclosure performs cognitive ability assessment.

[0096] Referring to Figure 1, the cognitive ability evaluation system according to the embodiment of the present disclosure may, for example, include a speech acquisition device 200 and a cognitive ability evaluation device 300.

[0097] The speech acquisition device 200 can acquire speech from the user 100 (or subject). The speech acquisition device 200 can output questions to guide the user 100's speech.

[0098] The speech acquisition device 200 may be implemented in a wearable form or in the form of a movable speaker, but is not limited to these.

[0099] The cognitive ability evaluation device 300 communicates with the speech acquisition device 200 via wired or wireless means and can provide tasks to the user 100 via the speech acquisition device 200. The cognitive ability evaluation device 300 can also receive speech data based on the user 100's voice acquired by the speech acquisition device 200.

[0100] Figure 1 illustrates a configuration in which the cognitive ability assessment device 300 performs an assessment on the user 100 via the speech acquisition device 200. In some cases, the cognitive ability assessment device 300 can directly provide tasks to the user 100 and receive the user 100's speech data.

[0101] The cognitive ability assessment device 300 may be implemented in the form of a portable terminal or a server, but is not limited to these forms. In some cases, some components of the cognitive ability assessment device 300 may be included in the terminal, and the remaining components may be included in the server.

[0102] The cognitive ability evaluation device 300 may, for example, include a task provision unit 310, a speech data collection unit 320, a cognitive ability measurement unit 330, and a control unit 340.

[0103] The task provision unit 310 can provide tasks for evaluating the user's cognitive abilities. The speech data collection unit 320 can acquire speech data based on the user's responses to the tasks. The cognitive ability measurement unit 330 can calculate the evaluation result of the user 100's cognitive abilities based on the speech data. The control unit 340 can control the operation of each component included in the cognitive ability evaluation device 300.

[0104] The task provision unit 310 can select tasks to provide to user 100 during a test period to evaluate user 100's cognitive abilities, and provide the selected tasks to user 100 via the speech acquisition device 200.

[0105] The task provision unit 310 can select tasks to provide to user 100 in different ways based on user 100's personal information such as educational background, gender, and age.

[0106] The task provision unit 310 can provide questions generated based on information recognized by user 100 as tasks. The information recognized by user 100 may be information that user 100 personally recognizes, or it may be information that is commonly recognized by the general public.

[0107] Information that user 100 personally recognizes can be obtained through devices used by user 100, such as smart devices. Commonly recognized information among the general public might include, for example, well-known fairy tales. This commonly recognized information can be used to generate tasks after confirming whether user 100 is aware of it beforehand.

[0108] The task provisioning unit 310 can generate tasks that include questions for evaluating the cognitive abilities of user 100.

[0109] The task provider 310 can provide basic questions used to assess the cognitive abilities of user 100. The task provider 310 can provide questions that require user 100 to make multiple judgments simultaneously. For example, the task provider 310 can provide a task that requires the user to make judgments on two or more elements simultaneously in order to solve the task, using content expressed through text, pictures, colors, etc.

[0110] The task provider 310 may modify some of the information recognized by the user 100 and provide it to the user 100, and then provide the user 100 with a question generated from the modified cognitive information. For example, the task provider 310 may modify some of the words in the information recognized by the user 100. The task provider 310 may also modify words corresponding to specific parts of speech (e.g., nouns, pronouns, particles, etc.). By providing the user 100 with a question based on information that has been slightly modified from the information recognized by the user 100, the task provider 310 can verify whether the user 100 recognizes the modified content and provides an accurate answer.

[0111] The task provider 310 can provide the user 100 with voice-based questions. The task provider 310 may, in some cases, provide tasks that require images to be displayed on the screen or actions to be taken by the user 100.

[0112] The task provider 310 can guide the user 100's speech through the tasks it provides.

[0113] The task provider 310 can provide tasks that induce specific utterances from user 100. For example, the task provider 310 can provide tasks that cause user 100 to pronounce voiced sounds or plosives. The task provider 310 can also provide tasks that cause user 100 to repeat specific utterances for a certain period of time. In this way, a variety of tasks can be provided to user 100 that can induce pronunciations that are distinguishable between those of a normal person and those of a person with mild cognitive impairment.

[0114] Furthermore, the task provider 310 may adjust the tasks provided thereafter in accordance with the user 100's responses. By adjusting the tasks according to the user 100's responses, it is possible to continuously guide the user 100's speech and acquire speech data for evaluating cognitive abilities. In addition, in order to maintain the user 100's concentration on the tasks, the task provider 310 can output sounds, images, etc., at regular intervals while the tasks are being provided to induce the user 100's concentration. This makes it possible to maintain the user 100's concentration at a certain level or higher while the tasks are being performed.

[0115] When a speech is generated according to the user 100's response to the task, the speech data collection unit 320 can acquire the resulting audio.

[0116] The speech data collection unit 320 can receive speech data acquired from the user via the speech acquisition device 200.

[0117] When the speech acquisition device 200 acquires speech from the user 100, it can convert the speech into digital speech data. The speech acquisition device 200 can convert the acquired speech directly into digital speech data and transmit it to the speech data collection unit 320.

[0118] Alternatively, the speech acquisition device 200 may adjust the acquired speech in a format that can improve the performance of cognitive ability assessment, and then convert it into digital speech data. For example, the speech acquisition device 200 may remove noise from the acquired speech before converting it into speech data. As another example, the speech acquisition device 200 may amplify at least a portion of the acquired speech before converting it into speech data.

[0119] Noise reduction or sound amplification may be performed in stages during the period in which tasks are provided by the task provision unit 310 and user 100 provides responses to the tasks.

[0120] As an example, referring to Figure 2, the task provision unit 310 can provide tasks during a test period that is divided into a first test period T1 and a second test period T2.

[0121] The first test period T1 and the second test period T2 may be temporally distinct periods. The first test period T1 may be a period prior to the second test period T2.

[0122] The first test period T1 may, for example, be the period during which preliminary tasks for the main test for cognitive ability assessment are performed before the main test for cognitive ability assessment is administered. The second test period T2 may, for example, be the period during which the main test for cognitive ability assessment is administered.

[0123] The first test period T1 may, for example, be a period for providing guidance to the main test before the main test is administered in the second test period T2. Alternatively, the first test period T1 may be a period during which a preliminary test is conducted before the second test period T2. Alternatively, the first test period T1 may be a period during which tests or tasks with purposes different from the cognitive ability assessment test are administered simultaneously.

[0124] For example, guidance for this test can be provided during the first test period T1. If guidance is provided during the first test period T1, it is possible that no voice output from user 100 will be generated.

[0125] The speech acquisition device 200 can acquire noise generated around the user 100 during the first test period T1. The speech acquisition device 200 can analyze the frequency band of the noise acquired during the first test period T1. Based on the frequency band of the noise acquired during the first test period T1, the speech acquisition device 200 can remove noise generated during the second test period T2 in which this test is performed.

[0126] For example, the main test for cognitive ability assessment can be administered during the second test period T2. User 100 can provide answers to the tasks provided according to the main test during the second test period T2. The speech acquisition device 200 can acquire speech corresponding to User 100's answers during the second test period T2. During the second test period T2, it can acquire sound that is a mixture of User 100's speech and ambient noise.

[0127] Since the speech acquisition device 200 has confirmed the frequency band of noise around the user 100 during the first test period T1, it can remove noise from the sound obtained during the second test period T2 by analyzing the frequency band of the sound acquired during the second test period T2. The speech acquisition device 200 can convert the noise-free speech into digital speech data and transmit it to the speech data acquisition unit 320. The cognitive ability evaluation device 300 can acquire speech data based on the speech from which noise has been removed by the external device.

[0128] Furthermore, the speech acquisition device 200 can adjust the volume of at least a portion of the user 100's voice acquired during the second test period T2, and then convert it into speech data.

[0129] Referring to Figure 3, noise around user 100 can be acquired during the first test period T1. Based on the noise acquired during the first test period T1, the speech acquisition device 200 can determine the frequency band of the noise.

[0130] During the second test period T2, the voice of user 100 can be acquired. Based on the noise frequency band identified during the first test period T1, the noise can be removed from the sound obtained during the second test period T2.

[0131] The speech acquisition device 200 can adjust the volume of at least a portion of the speech before converting the user 100's speech acquired during the second test period T2 into digital speech data.

[0132] For example, if the voice of user 100 acquired during the second test period T2 is called the base voice, the speech acquisition device 200 can generate a test voice in which the volume of at least a portion of the base voice is adjusted. For example, the speech acquisition device 200 can amplify the volume of the first portion P1 of the base voice. The speech acquisition device 200 does not need to amplify the volume of the second portion P2 or the third portion P3, which are portions of the base voice other than the first portion P1.

[0133] The cognitive ability evaluation system according to the embodiments of this disclosure can evaluate the cognitive abilities of user 100 through speech analysis of user 100. Even low-volume sounds that can be removed as noise may be necessary for firing analysis. Based on the frequency band of noise analyzed during the first test period T1, the noise is removed from the sound obtained during the second test period T2, so the remaining sound can be seen as user 100's speech. By amplifying the volume of a portion of user 100's speech, analysis can be performed using speech data based on user 100's overall speech acquired during the second test period T2.

[0134] The speech acquisition device 200 may amplify portions of the user's speech acquired during the second test period T2 where the volume is less than or equal to a previously set volume. Alternatively, the speech acquisition device 200 may amplify portions of the speech where the length of the portion where the volume is less than or equal to a previously set volume is greater than or equal to a previously set length. Examples of the speech acquisition device 200 amplifying a portion of the user 100's speech are not limited to these. All methods for amplifying the volume of at least a portion of the user 100's speech acquired during the second test period T2 in order to enable speech analysis by the cognitive ability evaluation device 300 may be included in embodiments of this disclosure.

[0135] As an example, see Figure 4, which illustrates the compression amplification technique (WDRC). Figure 4 shows an example of partially amplifying the original sound. In Figure 4, the horizontal axis represents time, and the vertical axis represents the volume of the sound.

[0136] As shown in the example in Figure 4, the original sound can include both very quiet parts and clean, loud parts sufficient for speech recognition. Sometimes, a very quiet sound is heard in the first part, and a loud sound is heard at the end. In everyday life, quiet sounds like those in the first part may not be recognized by actual speech recognition systems and therefore cannot be used as information.

[0137] To compensate for this, one example is to apply different gain levels depending on the section of the audio to partially amplify the sound. As shown in the example in Figure 4, the gain may be around 25 dB in sections with low volume, while the amplification may be minimal in sections with high volume.

[0138] Loud sounds are recognized as they are, while quiet sounds are amplified to an appropriate level, resulting in better speech recognition. The amplification method can simply amplify the entire sound, or it can select and amplify specific frequency bands.

[0139] In this way, amplification allows us to hear even faint sounds. In particular, noise is also amplified, and when noise reduction is applied, sounds that were previously inaudible become audible, ensuring that we can obtain sufficient necessary information through speech.

[0140] In this way, the speech acquisition device 200 can provide the user 100 with preparation time for the test via the first test period T1, and at the same time perform a process to remove noise from the user 100's voice.

[0141] In addition, depending on the circumstances, speech recordings of user 100 may be acquired during the first test period T1. The speech data corresponding to user 100's speech acquired during the first test period T1 may or may not be used to evaluate user 100's cognitive abilities.

[0142] Referring to Figure 5, during the first test period T1, user 100 can be provided with instructions for the main test or a preliminary test. Depending on the circumstances, the decision of whether or not to conduct the main test in the second test period T2, the difficulty level of the main test, etc., can be determined based on the responses of user 100 obtained during the first test period T1.

[0143] During the first test period T1, it is possible to acquire audio recordings of 100 user utterances in response to the tasks provided.

[0144] The speech acquisition device 200 can generate first speech data based on the speech acquired during the first test period T1. The speech acquisition device 200 can generate second speech data based on the speech acquired during the second test period T2.

[0145] The speech acquisition device 200 can transmit the first speech data and the second speech data to the speech data collection unit 320.

[0146] The cognitive ability evaluation device 300 can calculate an evaluation result regarding user 100's cognitive ability based on second speech data based on speech acquired during the second test period T2. In some cases, the cognitive ability evaluation device 300 may also use first speech data based on speech acquired during the first test period T1 to calculate an evaluation result regarding user 100's cognitive ability.

[0147] For example, in the process of calculating an evaluation result regarding the cognitive ability of user 100 based on second speech data, if there is insufficient data to calculate the evaluation result, the cognitive ability evaluation device 300 can include first speech data and calculate the evaluation result regarding user 100's cognitive ability.

[0148] Alternatively, the cognitive ability evaluation device 300 may calculate an evaluation result regarding the user 100's cognitive ability by including both the first speech data and the second speech data.

[0149] The cognitive ability evaluation device 300 can reflect the first speech data and the second speech data at the same level when calculating the evaluation results regarding the cognitive ability of user 100.

[0150] Alternatively, in some cases, the first test period T1 can be considered a period in which the confidence level of the test results for user 100 is lower than that of the second test period T2. In such cases, different weights may be applied to the first and second speech data to calculate the evaluation results regarding cognitive ability. For example, by applying a first weight to the first speech data and a second weight to the second speech data, the evaluation results regarding user 100's cognitive ability can be calculated. The first weight may be smaller than the second weight.

[0151] Thus, speech data based on the utterances of 100 users acquired during the first test period T1 can also be used to calculate evaluation results regarding cognitive ability.

[0152] Furthermore, even if the speech acquisition device 200 acquires voice from the user 100's speech during both the first test period T1 and the second test period T2, it can remove noise from the user 100's voice acquired during each test period or adjust the volume of the voice.

[0153] For example, the speech acquisition device 200 can acquire speech from user 100 during the first test period T1. The speech acquisition device 200 can identify the frequency band of user 100's speech by analyzing the frequency band of user 100's speech acquired during the first test period T1.

[0154] The speech acquisition device 200 can remove noise corresponding to frequency bands other than the frequency band of the user 100's voice from the sound acquired during the first test period T1 and the second test period T2. The speech acquisition device 200 can amplify the volume of a portion of the user 100's voice from which the noise has been removed.

[0155] Therefore, even if speech from user 100 is acquired during the first test period T1 and the second test period T2, it is possible to provide speech data based on speech that has been noise-free and partially amplified.

[0156] The speech data transmitted by the speech acquisition device 200 may be provided to the cognitive ability measurement unit 330.

[0157] The cognitive ability measurement unit 330 can calculate evaluation results regarding the cognitive abilities of user 100 by analyzing speech data.

[0158] The cognitive ability measurement unit 330 can calculate an evaluation result regarding the cognitive ability of user 100 using at least one of the various elements contained in the speech data of user 100.

[0159] For example, the cognitive ability measurement unit 330 can calculate an evaluation result regarding user 100's cognitive ability based on the number of words used by user 100 during a pre-set period included in the speech data. The cognitive ability measurement unit 330 may also calculate an evaluation result regarding user 100's cognitive ability based on the number of words used in each of multiple intervals included in the pre-set period.

[0160] The cognitive ability measurement unit 330 can calculate an evaluation result regarding user 100's cognitive ability based on the types of words used by user 100 during a pre-set period included in the speech data. The "type of word" may refer to the part of speech of the word. An evaluation result regarding user 100's cognitive ability can be calculated based on the frequency with which words of a specific part of speech are used.

[0161] The cognitive ability measurement unit 330 can calculate an evaluation result regarding user 100's cognitive ability based on the number of repeated words used by user 100 during a pre-set period included in the speech data. The number of repeated words may be counted separately for each of the multiple segments included in the pre-set period. An evaluation result regarding user 100's cognitive ability can be calculated based on the number of repeated words used in each segment.

[0162] The cognitive ability measurement unit 330 can calculate an evaluation result regarding the cognitive ability of user 100 based on the number of words used by user 100 for each pre-set time interval included in the speech data.

[0163] The cognitive ability measurement unit 330 can calculate an evaluation result regarding the user 100's cognitive ability based on the speech data, specifically the speech periods and pause periods included in a pre-set period. A pause period can mean a period during which no speech occurs. Alternatively, a pause period may mean a period of a certain duration or longer within the interval between speech periods.

[0164] The cognitive ability measurement unit 330 can calculate an evaluation result regarding the user 100's cognitive ability based on the rate of change of at least one length of speech period or pause period included in a pre-set period of speech data. For example, the cognitive ability measurement unit 330 can calculate an evaluation result regarding the user 100's cognitive ability based on the rate of change of the length of the pause period. Alternatively, the evaluation result regarding the user 100's cognitive ability may be calculated based on the rate of change of the length of the pause period in each of several intervals included in a pre-set period.

[0165] The cognitive ability measurement unit 330 can calculate an evaluation result regarding the user 100's cognitive ability based on the change in the ratio of speaking periods to pause periods included in the speech data within a pre-set period. The ratio of speaking periods to pause periods can refer to the proportion of both in a single interval in which speaking periods and pause periods are repeated. Alternatively, it may refer to the ratio of speaking periods to pause periods in each of multiple intervals included within a pre-set period. An evaluation result regarding the user 100's cognitive ability can be calculated based on the uniformity of the proportion of pause periods.

[0166] The cognitive ability measurement unit 330 can calculate an evaluation result regarding the cognitive ability of user 100 based on one or more combinations of the examples described above.

[0167] The cognitive ability measurement unit 330 can provide the user 100 with evaluation results regarding the user's cognitive abilities.

[0168] The cognitive ability measurement unit 330 may present the evaluation results regarding cognitive ability as a score, or it may present them divided into two or more grades, or it may provide them as whether or not there is a warning regarding mild cognitive impairment.

[0169] The cognitive ability measurement unit 330 can calculate an evaluation result regarding user 100's cognitive ability by comparing the analysis results of user 100's speech data with user 100's past data. Alternatively, the cognitive ability measurement unit 330 can calculate an evaluation result regarding user 100's cognitive ability by comparing the analysis results of user 100's speech data with the average data of other users other than user 100. Alternatively, the cognitive ability measurement unit 330 can calculate an evaluation result regarding user 100's cognitive ability by comparing the analysis results of user 100's speech data with data derived from patients with mild cognitive impairment or dementia. In some cases, the cognitive ability measurement unit 330 may combine two or more of the above-described comparison methods to calculate an evaluation result regarding user 100's cognitive ability.

[0170] Thus, according to the embodiments of this disclosure, the speech acquisition device 200 and the cognitive ability evaluation device 300 can be used to easily evaluate the cognitive ability of the user 100, and if there is a risk of mild cognitive impairment, the relevant information can be provided to the user 100, thereby enabling early detection of mild cognitive impairment or dementia.

[0171] As described above, the cognitive ability evaluation device 300 can be implemented in various forms such as terminals and servers, and in some cases, the configuration of the cognitive ability evaluation device 300 may be distributed between terminals and servers to form a cognitive ability evaluation system.

[0172] Figure 6 is a schematic diagram illustrating another example of the configuration of a cognitive ability assessment system according to an embodiment of the present disclosure. Figures 7 and 8 are illustrative diagrams illustrating the configuration of the speech acquisition device 200 shown in Figure 6.

[0173] Referring to Figure 6, the cognitive ability evaluation system according to the embodiment of this disclosure can consist of a speech acquisition device 200, a mobile terminal 400, and a cognitive ability evaluation management server 500.

[0174] The speech acquisition device 200 may, for example, include an output module 210, an input module 220, a processing module 230, and a communication module 240.

[0175] The output module 210 can output audio corresponding to multiple tasks provided to the user 100. . Out As in the example described above, the power module 210 can adjust and provide the volume of the output audio related to the task, taking into account the auditory ability of the user 100.

[0176] The input module 220 can acquire audio corresponding to the user 100's responses to multiple tasks provided by the output module 210.

[0177] The processing module 230 can convert the audio acquired by the input module 220 into digital speech data. The processing module 230 can also convert the acquired audio into speech data after adjusting the volume of at least a portion of it.

[0178] The processing module 230 can remove noise from the audio acquired by the input module 220 through frequency band analysis. The processing module 230 can also select the portion of the audio acquired by the input module 220 to be amplified through frequency band analysis.

[0179] The communication module 240 can transmit the speech data generated by the processing module 230 to the mobile terminal 400.

[0180] The speech acquisition device 200 can be implemented in various forms, such as a speaker. In some cases, the speech acquisition device 200 may be implemented in the form of a wearable device.

[0181] In one example, the speech acquisition device 200 can be implemented in the form of a wearable device including an earphone 201, a microphone 202, a main unit 203, and a connector 204. The microphone 202 can be attached to the main unit 203. The earphone 201 can be connected to the main unit 203 via the connector 204.

[0182] The main unit 203 may be in a form that wraps around the neck of the user 100. The main unit 203 of the speech acquisition device 200 can wrap around the neck of the user 100 and, while worn by the user 100, provide tasks to the user 100 and acquire voice from the user 100's speech.

[0183] The microphone 202 may, in some cases, be implemented to be closely attached to the user 100's neck and to acquire sound corresponding to the user 100's speech by sensing vibrations caused by the user 100's speech. Even if the user 100's speech ability deteriorates, cognitive ability can be evaluated by analyzing the user 100's speech data.

[0184] The earphone 201 may be provided connected to the main unit 203 via the connector 204. This can provide convenience for the user 100. In some cases, the earphone 201 may be provided in a form that includes a function to measure the user 100's brainwaves.

[0185] Furthermore, the speech acquisition device 200 can provide a hearing aid function to compensate for hearing loss, enabling users 100 with hearing impairments to hear speech more easily.

[0186] Referring to Figure 7, an illustrative example of the configuration of the speech acquisition device 200 shows that the input module 220 can include multiple microphones. The input module 220 can use directional microphones or apply algorithms to more efficiently listen to audio output from external devices such as smart devices. In some cases, omnidirectional microphones can be used, but using directional microphones improves noise reduction and allows for better capture of the speaker's voice.

[0187] The processing module 230 can convert the input audio into the frequency domain, process it, and provide amplified and de-noised audio. For example, the input audio may include ambient noise corresponding to the current location of user 100, and such ambient noise may include characteristics of various outdoor environments such as office environments, subways, and factory environments. The characteristics of the ambient noise can be classified using information obtained by fast Fourier transforming the input signal, as shown in the example in Figure 7. Furthermore, the fast Fourier transformed signal can be amplified by varying the amplification gain according to the frequency domain. This amplification can amplify quiet sounds, making them clearer to users 100 (elderly people) with hearing impairments. The amplification method may be similar to the example described with reference to Figure 4. The module can also perform ambient noise removal. This allows for amplification of the audio frequency band while removing noise from the received audio. If necessary, the frequency-converted signal can undergo spectral enhancement.

[0188] The noise reduction and audio amplification functions of the processing module 230 may, for example, be controlled by a remote control device (e.g., a smartphone or PC). Remote control of the compression amplification size for each frequency band, the audio-to-noise ratio for noise reduction, and the inversion suppression parameters can be performed via the communication module 240.

[0189] Furthermore, the speech acquisition device 200 can provide speech processing functions to improve the speech recognition rate of the user 100.

[0190] Referring to Figure 8, an illustrative configuration of the speech acquisition device 200 can be seen, where the input module 220 may include at least one microphone. The input module 220 may include a directional microphone or use multiple microphones to apply a directional algorithm in order to efficiently hear the speech of the user 100. In some cases, an omnidirectional microphone may be included, but a directional microphone can improve the noise reduction effect and receive the speaker's voice better. After initial processing of the sound acquired through the microphone, the input module 220 can transmit it to the processing module 230. For example, the input module 220 may remove noise from the sound acquired through the microphone or amplify at least a portion of the acquired sound to transmit a clear sound to the processing module 230.

[0191] The processing module 230 may remove noise from the audio input from the microphone, and may analyze ambient noise to extract information through classification of the external environment in which the speaker is located. The processing module 230 can also perform a function to amplify the speaker's quiet voice. Alternatively, the processing module 230 may pass the audio signal through without performing any further audio signal processing.

[0192] The processing module 230 can perform a Fast Fourier Transform on the signal input via the microphone. By analyzing the frequency characteristics of the transformed signal, it can provide information about the environment in which the speaker is located. For example, it can analyze the environment in which the speaker is located, such as a quiet office environment, a subway environment, or an external environment. This information can be used to remove environmental noise other than the speaker's voice information input via the microphone.

[0193] Furthermore, the processing module 230 amplifies the interjections or whispers input via the microphone from the Fast Fourier Transformed signal, as well as sounds that are quietly expressed depending on the speaker's characteristics, thereby improving the recognition rate of the speech recognition system. When quiet sounds are amplified, noise contained in the sound is also amplified, which can reduce the speech recognition rate, so noise reduction may be desirable in some cases.

[0194] The processing module 230 can process and digitize analog sound received from the input module 220. For example, the processing module 230 can classify the acquired sound and either allow all of it to pass through or delay it. The processing module 230 can convert the acquired sound into a frequency spectrum and perform processing to amplify a portion of the converted sound (dynamic range compression (WDRC)) and noise reduction (noise suppression). The processing module 230 may further include a configuration for selectively amplifying sound to further amplify specific sounds. Once processing of the sound is complete, the processing module 230 can convert it back into the time domain and output it. The output sound can be provided as feedback and reflected in the sound output by the processing module 230. The processing module 230 can provide the digitized audio data to the communication module 240.

[0195] When the communication module 240 receives digitized voice data, i.e., speech data, it can transmit the speech data to a mobile terminal 400 such as a smart device.

[0196] The mobile terminal 400 may include a task provision module 410, a speech data collection module 420, and a cognitive ability measurement module 430.

[0197] The task provision module 410 can select tasks to be provided to the user 100 via the speech acquisition device 200 and provide the selected tasks to the user 100 via the speech acquisition device 200 according to the test period. The task provision module 410 may, in some cases, be located on the cognitive ability assessment management server 500.

[0198] The speech data acquisition module 420 can acquire speech data transmitted by the speech acquisition device 200.

[0199] The speech data collection module 420 can receive speech data based on test speech from which noise has been removed and the volume of at least a portion of the base speech has been adjusted, corresponding to the user 100's responses.

[0200] The speech data acquisition module 420 can receive speech data that has been digitally processed by the speech acquisition device 200. In some cases, the speech data acquisition module 420 can receive analog audio from the speech acquisition device 200 and process the received audio as digital speech data.

[0201] The cognitive ability measurement module 430 can calculate evaluation results regarding the cognitive abilities of user 100 based on speech data.

[0202] The cognitive ability measurement module 430 can analyze the speech data acquired by the speech data collection module 420 and calculate an evaluation result regarding the user 100's cognitive ability. The method by which the cognitive ability measurement module 430 calculates the evaluation result regarding the user 100's cognitive ability may be the same as the method performed by the cognitive ability measurement unit 330 included in the cognitive ability evaluation device 300 described above.

[0203] The cognitive ability measurement module 430 may, in some cases, be located on the cognitive ability assessment management server 500. In other words, the cognitive ability assessment may be performed on the cognitive ability assessment management server 500. In this case, the assessment results may be transmitted from the cognitive ability assessment management server 500 to the mobile terminal 400.

[0204] As shown in the example in Figure 6, if the task provision module 410, speech data collection module 420, and cognitive ability measurement module 430 are included in the mobile terminal 400, the cognitive ability evaluation management server 500 can provide a function to update tasks that can be provided to the user 100, or a function to save and manage evaluation results regarding the user 100's cognitive ability. In some cases, some of the functions performed on the mobile terminal 400 may be performed on the cognitive ability evaluation management server 500.

[0205] In this way, a system can be provided in which a mobile terminal 400 and a cognitive ability assessment management server 500 communicate via a network 600, making it easy to perform cognitive ability tests on a user 100.

[0206] Furthermore, in some cases, embodiments of this disclosure can acquire the user 100's electroencephalogram (EEG) data, along with the user 100's speech data, during the test period, the speech period, or periods other than the speech period, and calculate evaluation results regarding the user 100's cognitive abilities.

[0207] Figure 9 schematically illustrates yet another example of the configuration of a cognitive ability assessment system according to an embodiment of the present disclosure. Figures 10 and 11 illustrate an example of how the cognitive ability assessment system according to an embodiment of the present disclosure performs a cognitive ability assessment.

[0208] Referring to Figure 9, the cognitive ability evaluation system according to the embodiment of this disclosure can consist of a speech acquisition device 200, a mobile terminal 400, and a cognitive ability evaluation management server 500.

[0209] The speech acquisition device 200 may include an output module 210, a processing module 230, and a communication module 240, as in the example described with reference to Figure 6. The speech acquisition device 200 may also include a speech input module 221 and an electroencephalogram (EEG) input module 222. The speech acquisition device 200 can acquire speech from the user 100 and acquire the user 100's brainwaves. The speech acquisition device 200 can acquire the user 100's brainwaves via a part connected to the user 100's head or ears.

[0210] The speech acquisition device 200 can acquire the brainwaves of user 100 for at least a portion of the period during which the voice produced by user 100's speech is acquired. Alternatively, the speech acquisition device 200 may acquire the brainwaves of user 100 during a period distinct from the period during which the voice produced by user 100's speech is acquired.

[0211] The speech input module 221 and the electroencephalogram (EEG) input module 222 can acquire the voice and brainwaves of user 100. The communication module 240 can transmit speech data corresponding to user 100's voice and EEG data corresponding to the brainwaves to the mobile terminal 400.

[0212] The mobile terminal 400 may include a task provision module 410, a speech data collection module 420, and a cognitive ability measurement module 430, and may further include an electroencephalogram (EEG) data collection module 440.

[0213] The mobile terminal 400 can utilize the acquired electroencephalogram (EEG) data in various ways during the process of calculating evaluation results regarding the cognitive abilities of user 100 based on speech data.

[0214] As an example, referring to Figure 10, tasks may be provided during a test period that includes a first test period T1 and a second test period T2, and a test may be performed to evaluate the cognitive abilities of user 100.

[0215] During the second test period T2, this test can be administered to evaluate the cognitive abilities of user 100. During the first test period T1, instructions for this test can be provided.

[0216] During the first test period T1, the first electroencephalogram (EEG) data can be acquired. During the second test period T2, the second EEG data can be acquired.

[0217] The first electroencephalogram (EEG) data may be data obtained from the EEG related to user 100's level of understanding. The task provision module 410 or cognitive ability measurement module 430 can confirm user 100's level of understanding of the instructions for this test based on the first EEG data. Based on the first EEG data, it is possible to determine whether or not to conduct this test during the second test period T2, or to determine the difficulty level, type, etc. of this test.

[0218] The second electroencephalogram (EEG) data may be data obtained from the EEG related to user 100's speech ability. The cognitive ability measurement module 430 can use the analysis results of the second EEG data, along with the analysis results of the speech data, to calculate an evaluation result regarding user 100's cognitive ability.

[0219] Thus, if the speech acquisition device 200 is implemented in a form that can simultaneously measure the brainwaves of the user 100, the brainwave data collected during the first test period T1 and the second test period T2 can be used to determine the test method or to calculate evaluation results regarding cognitive ability.

[0220] Furthermore, if a test for assessing cognitive ability is conducted during the first test period T1, the electroencephalogram (EEG) data acquired during the first test period T1 can also be used to calculate the cognitive ability assessment results for user 100. In this case, similar to the speech data, the EEG data obtained during the first test period T1 and the second test period T2 can be used to calculate the cognitive ability assessment results by applying different weights.

[0221] Alternatively, it can be used to measure different abilities of 100 users via electroencephalogram (EEG) data and to calculate evaluation results regarding the cognitive abilities of 100 users. As mentioned above, the cognitive ability assessment may be performed on the mobile terminal 400, or it may be performed on the cognitive ability assessment management server 500. If the cognitive ability assessment is performed on the cognitive ability assessment management server 500, the assessment results may be sent from the cognitive ability assessment management server 500 to the mobile terminal 400.

[0222] As an example, referring to Figure 11, the speech acquisition device 200 can acquire the voice produced by the user 100's speech and simultaneously acquire the user 100's brainwaves.

[0223] The speech acquisition device 200 can acquire multiple types of brainwaves simultaneously.

[0224] As an example, the speech acquisition device 200 can acquire electroencephalograms (EEGs) related to the user 100's auditory ability during at least a portion of the first test period T1 and the second test period T2.

[0225] Furthermore, the speech acquisition device 200 can acquire electroencephalograms (EEGs) related to the speech ability of user 100 during at least a portion of the first test period T1 and the second test period T2.

[0226] The mobile terminal 400 can calculate an evaluation result regarding user 100's cognitive ability using electroencephalogram (EEG) data related to auditory ability acquired during the first test period T1 and the second test period T2. The mobile terminal 400 can also calculate an evaluation result regarding user 100's cognitive ability using EEG data related to speech ability acquired during the first test period T1 and the second test period T2.

[0227] If the first test period T1 primarily involves auditory activity by user 100, and the second test period T2 primarily involves speech activity by user 100, then the electroencephalogram (EEG) data related to user 100's auditory ability acquired during the first test period T1 and the EEG data related to user 100's speech ability acquired during the second test period T2 can be used to calculate an evaluation result regarding user 100's cognitive ability.

[0228] Alternatively, the cognitive ability evaluation results for user 100 can be calculated using both the electroencephalogram (EEG) data related to auditory ability and the EEG data related to speech ability obtained during the first test period T1 and the second test period T2. In this case, depending on the circumstances, different weights may be applied to each EEG data for each test period to calculate the evaluation results.

[0229] The mobile terminal 400 can calculate an evaluation result regarding the cognitive abilities of user 100 by using speech data and electroencephalogram data acquired from user 100 in combination.

[0230] A decline in cognitive ability can lead to a decline in speech ability and a decline in hearing ability. Alternatively, both speech and hearing abilities may decline simultaneously. Therefore, a decline in the cognitive ability of user 100 can be measured by testing speech and hearing abilities.

[0231] Furthermore, in some cases, the degree of decline in speech ability and the degree of decline in hearing ability may differ. For example, in growing children, the rate of auditory development may be faster than the rate of speech development. On the other hand, in older adults, the rate of speech degeneration may be faster than the rate of hearing degeneration. Alternatively, unlike in the developmental process, the rates of degeneration of both abilities may be similar. Or, the rate of degeneration of hearing ability may be faster at the time of degeneration.

[0232] In this way, the state of cognitive ability can be determined based on whether or not there is a decline in auditory ability. Furthermore, in some cases, the state of cognitive ability can be classified into various levels based on the difference between auditory ability and speech ability, and an evaluation result regarding the user 100's cognitive ability can be calculated. In addition, when considering whether or not there is a decline in auditory ability when determining the state of cognitive ability, data corresponding to the level of hearing function provided by the speech acquisition device 200 can be used. An evaluation result regarding cognitive ability can also be calculated by reflecting data on the degree of hearing decline according to the level of hearing function.

[0233] Since electroencephalogram (EEG) data is used along with speech data, the accuracy of cognitive ability assessment by the cognitive ability assessment system according to the embodiment of this disclosure can be improved.

[0234] Figures 12 and 13 are flowcharts illustrating an example of the process of a cognitive ability assessment method according to an embodiment of the present disclosure.

[0235] Referring to Figure 12, the cognitive ability assessment system allows user 100 to select the tasks to be provided to user 100 (S1200). The difficulty level, type, and content of the questions included in the tasks can be selected by user 100.

[0236] The cognitive ability assessment system can be used to provide the task (S1210).

[0237] 100 user responses are generated for the task, and as a result, the speech data of user 100 can be obtained (S1220).

[0238] The cognitive ability assessment system can analyze the linguistic characteristics of speech data (S1230). Based on the analysis of the linguistic characteristics of the speech data, the cognitive ability assessment system can provide an assessment result of the cognitive abilities of user 100 (S1240).

[0239] Alternatively, in some cases, the cognitive ability assessment system can provide an assessment result regarding the cognitive abilities of user 100 based on the linguistic and acoustic characteristics of the speech data.

[0240] The linguistic characteristics of the speech data can represent the characteristics of the vocabulary, such as sentences and words, produced by user 100. The acoustic characteristics of the speech data can represent the characteristics related to the intensity, frequency, and timbre of the voice produced by user 100.

[0241] By using the linguistic and acoustic characteristics of speech from healthy individuals, patients with mild cognitive impairment, and patients with dementia, it is possible to calculate an evaluation result regarding user 100's cognitive ability. Furthermore, based on the acoustic characteristics of the speech data, it is also possible to distinguish between diseases other than mild cognitive impairment, such as depression, and calculate an evaluation result regarding user 100's cognitive ability.

[0242] By comparing the linguistic characteristics extracted from user 100's speech data with various already stored reference data, an evaluation result regarding user 100's cognitive abilities can be calculated.

[0243] Referring to Figure 13, the cognitive ability assessment system can extract linguistic characteristics from speech data (S1300). In some cases, as mentioned above, acoustic characteristics of speech data can also be used.

[0244] The cognitive ability assessment system can check whether past data exists for 100 users being tested (S1310).

[0245] The cognitive ability assessment system can calculate a first evaluation result by comparing the speech data of user 100, provided that past data for user 100 exists (S1320). By analyzing the speech data, changes in user 100's speech ability can be confirmed, and based on this, the first evaluation result can be calculated. The cognitive ability assessment system can then apply a first weight to the first evaluation result.

[0246] The cognitive ability assessment system can calculate a second assessment result after comparing the speech data of 100 users with the speech data of other users (S1330). The speech data of other users may be the speech data of other users who have taken the test, or it may be speech data collected from users with conditions such as mild cognitive impairment. The cognitive ability assessment system can apply a second weight to the second assessment result.

[0247] If no past data exists for user 100, the cognitive ability assessment system can calculate the assessment result after comparing the speech data of user 100 with that of other users (S1340).

[0248] The cognitive ability assessment system can provide assessment results regarding cognitive ability based on the results of comparing the individual's own speech data with that of others (S1350).

[0249] The first weight applied to the first evaluation result obtained by comparing the individual's own speech data, and the second weight applied to the second evaluation result obtained by comparing the individual's speech data with that of others, can be set in a variety of ways.

[0250] For example, based on user 100's age, the degree of change in cognitive ability, and the level of risk, it is possible to determine in more detail whether user 100 is at risk of mild cognitive impairment by prioritizing user 100's own rate of change in cognitive ability, or by prioritizing the difference between user 100 and others.

[0251] According to embodiments of this disclosure, when using a wearable device having noise reduction and selective amplification functions, language tests can be performed in everyday life or in public places, and the cognitive abilities of user 100 can be easily measured. In other words, according to embodiments of this disclosure, user 100 can easily access the tests and conduct them conveniently anytime, anywhere.

[0252] According to embodiments of this disclosure, the speech recognition effect can be enhanced by applying noise reduction technology.

[0253] According to the embodiments of this disclosure, detailed information about the speaker can be collected via the voice amplification function, thereby enhancing analytical capabilities.

[0254] According to embodiments of this disclosure, by selecting selective questions for each task, it is possible to select the optimal set of test questions, taking into account the individual subject's intellectual ability, verbal cognitive ability, etc.

[0255] According to embodiments of this disclosure, physical sensors and time-based question start and end information can be provided to enable the selection of a set of questions.

[0256] According to the embodiments of this disclosure, the examination time can be shortened by using a selective question examination method tailored to individual circumstances, and the level of fatigue in elderly individuals can be significantly improved, allowing for more effective examinations.

[0257] According to embodiments of this disclosure, utterance information for each question can be quantified. Multiple questions can be asked for each task, and the utterances for each question can be quantified and presented as an average value. All the information for each task can be combined.

[0258] The above description is merely illustrative of the technical concept of this disclosure, and any person with ordinary skill in the art to which this disclosure pertains may make various modifications and variations without departing from the essential characteristics of this disclosure. Furthermore, the embodiments shown in this disclosure are for illustrative purposes only and not to limit the technical concept of this disclosure; therefore, these embodiments do not limit the scope of the technical concept of this disclosure. The scope of protection of this disclosure should be interpreted by the following claims, and all technical concepts within an equivalent scope should be interpreted as being included in the scope of rights of this disclosure.

[0259] Cross-reference to related application

[0260] This patent application claims priority under Section 119(a) of the United States Patent Act (35 U.S.SC § 119(a)) to Patent Application No. 10-2022-0105435, filed in Korea on August 23, 2022, and all contents of that application are merged into this patent application as references. Furthermore, if this patent application claims priority in any country other than the United States for the same reasons as described above, all contents of that application will also be merged into this patent application as references.

Claims

1. An output module that outputs the questions provided to the user during the first period; A speech input module that collects the user's utterances in response to the aforementioned questions during a second period; An electroencephalogram (EEG) input module that acquires the user's brainwaves during the first period in which the aforementioned question is output; and A processing module that converts the utterances collected during the second period into digital speech data and transmits the digitized speech data and the electroencephalogram data based on the electroencephalograms collected during the first period to an external source. A speech acquisition device, including a speech acquisition device.

2. The aforementioned processing module is The speech acquisition device according to claim 1, which analyzes the frequency band of the utterance and selectively amplifies it according to the frequency.

3. The aforementioned processing module is The speech acquisition device according to claim 1, which stores the user's utterance in response to the aforementioned question, and collects speech data corresponding to the user's utterances in response to a plurality of questions and transmits it to the external device.

4. The aforementioned processing module is The speech acquisition device according to claim 1, which stores the speech data in response to the aforementioned question in synchronization with time information.

5. The output module is The speech acquisition device according to claim 1, which removes ambient noise while outputting the aforementioned question.

6. The speech acquisition device according to claim 5, wherein during a first test period, ambient noise of the user is collected, a noise frequency band is extracted by frequency band analysis of the ambient noise, and during a second test period after the first test period, the ambient noise is removed according to the noise frequency band.

7. The output module is The speech acquisition device according to claim 1, which amplifies and outputs the aforementioned question according to the user's hearing ability.

8. A speech acquisition device that, during a first test period and a second test period following the first test period, transmits information to the user corresponding to multiple tasks; adjusts the volume of at least a portion of the utterances corresponding to the user's responses to at least one of the multiple tasks acquired during the second test period, converts them into digital speech data; acquires the user's brainwaves during the first test period and converts them into digital brainwave data; and transmits the speech data and the brainwave data to an external source; and A cognitive ability evaluation device that provides the aforementioned multiple tasks and calculates an evaluation result regarding the user's cognitive ability based on at least one linguistic characteristic of the speech data and the electroencephalogram data. A cognitive ability assessment system, including the following.

9. The speech acquisition device is The cognitive ability evaluation system according to claim 8, comprising collecting ambient noise of the user during the first test period and extracting noise frequency bands through frequency band analysis of the ambient noise.

10. The speech acquisition device is The cognitive ability evaluation system according to claim 9, wherein noise based on the noise frequency band in the utterances collected during the second test period is removed, the utterances are amplified, and then transmitted to the cognitive ability evaluation device.

11. The speech acquisition device is The cognitive ability assessment system according to claim 8, comprising at least one directional microphone, and using the at least one directional microphone to collect the user's speech.

12. The speech acquisition device is The cognitive ability evaluation system according to claim 8, wherein sounds corresponding to the plurality of tasks are provided to the user, and the volume of the sounds corresponding to the plurality of tasks is adjusted according to the user's auditory ability.

13. The cognitive ability evaluation device, The cognitive ability evaluation system according to claim 12, which calculates the evaluation result regarding the user's cognitive ability based on the at least one linguistic characteristic of the speech data and the user's auditory ability.

14. During the first test period, electroencephalogram data relating to the user's auditory ability is collected, and during the second test period, speech data is collected. The cognitive ability evaluation system according to claim 13, wherein the evaluation result regarding the user's cognitive ability is calculated based on the electroencephalogram data and the speech data.

15. During the first test period, the user's first electroencephalogram (EEG) data is collected, and during the second test period, the user's second electroencephalogram (EEG) data is collected. The cognitive ability assessment system according to claim 8, wherein the first electroencephalogram data is used to determine whether or not to provide a task during the second test period, and the second electroencephalogram data is used to calculate the assessment result regarding the user's cognitive ability.

16. The cognitive ability evaluation system according to claim 8, wherein the user's utterances are collected during at least one of the first test period or the second test period, and the user's brainwaves are collected at the same time.

17. A task provisioning unit that provides a plurality of tasks to the user during a first test period and a second test period following the first test period; A speech data collection unit that converts the user's utterances for at least one of the multiple tasks provided during the second test period into digital speech data and amplifies at least a portion of the speech data; An electroencephalogram (EEG) data acquisition unit that acquires the user's brainwaves during the first test period and converts them into digital EEG data; and A cognitive ability measurement unit calculates an evaluation result regarding the user's cognitive ability based on at least one linguistic characteristic of the speech data acquired during the second test period and the electroencephalogram data acquired during the first test period. A cognitive ability assessment device, including...

18. The cognitive ability evaluation device according to claim 17, wherein ambient noise of the user is collected during the first test period, and a noise frequency band is extracted by frequency band analysis of the ambient noise.

19. The cognitive ability evaluation device according to claim 18, wherein, in the utterances collected during the second test period, at least a portion is amplified after noise based on the noise frequency band has been removed.

20. An output module that outputs the questions provided to the user during the first period; A speech input module that collects the user's utterances in response to the aforementioned questions during a second period; An electroencephalogram (EEG) input module for collecting electroencephalograms generated around the user's head or ears during the first period; and A processing module that converts the utterances collected during the second period into digital speech data, processes the digitized speech data and transmits it externally, and transmits the electroencephalogram data based on the electroencephalograms collected during the first period to the external source. A speech acquisition device, including a speech acquisition device.

21. The speech acquisition device according to claim 20, wherein the user's speech and the user's brainwaves are acquired in a time-synchronized manner.

22. The speech acquisition device according to claim 20, wherein the user's brainwaves are collected during both the period in which the question is output and the period in which the user's utterances are collected.

23. The speech acquisition device according to claim 20, wherein an evaluation result regarding the user's cognitive ability is provided using both information acquired during the period in which the aforementioned question is output and information acquired during the period in which the user's utterance is collected.

24. The speech acquisition device according to claim 20, wherein the user's hearing state is monitored using the user's brainwaves, and the voice corresponding to the question is amplified and provided according to the hearing state.

Images