A cognitive impairment assessment method based on speech recognition

Abstract

The application discloses a cognitive impairment evaluation method based on voice recognition, and relates to the technical field of cognitive impairment evaluation. The method comprises the following steps: acquiring voice data at the time of testing; analyzing the voice data by using a preprocessing engine to obtain target voice data; analyzing the target voice data by using a natural voice processing engine to obtain language feature data, and analyzing the target voice data by using a voice audio signal processing engine to obtain audio feature data; fusing the audio feature data and the voice feature data based on an attention mechanism to obtain voice fusion feature data; and analyzing the voice fusion feature data by using a cognitive voice analysis engine to obtain a cognitive ability detection result. The cognitive impairment evaluation method based on voice recognition can effectively evaluate the cognitive ability and the degree of cognitive impairment of a testee by analyzing and processing voice data and combining the fusion mode of the attention mechanism.

Description

Technical Field

[0001] This invention relates to the field of cognitive impairment assessment technology, and more specifically to a cognitive impairment assessment method based on speech recognition. Background Technology

[0002] Dementia, exemplified by Alzheimer's disease, remains a major challenge in the medical field. Once diagnosed, it is irreversible, causing patients to experience memory decline, drastic personality changes, and even incontinence, inflicting long-term suffering on their families. If the onset of AD can be delayed by 5 years through early screening, early detection, and early intervention, approximately 57% of AD patients will be eliminated, significantly reducing the economic burden of cognitive impairment on families and society. The rapidly expanding population at risk of cognitive impairment lays the necessary foundation for the widespread adoption of cognitive function screening.

[0003] Traditional methods for assessing cognitive impairment include neuropsychological scales, MRI, PET, cerebrospinal fluid, etc. However, these methods all have drawbacks to varying degrees. For example, the simplified neuropsychological scale is not sensitive enough to identify people with cognitive impairment, the full set of scales takes too long, MRI or PET is expensive, and cerebrospinal fluid puncture has practical problems such as being invasive and having low user acceptance, making it impossible to use for large-scale promotion and application. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a cognitive impairment assessment method based on speech recognition, comprising:

[0005] Acquire audio data during testing;

[0006] The speech data is analyzed using a preprocessing engine to obtain the target speech data;

[0007] The target speech data is analyzed using a natural speech processing engine to obtain language feature data, and the target speech data is analyzed using a speech audio signal processing engine to obtain audio feature data.

[0008] The audio feature data and the speech feature data are fused based on the attention mechanism to obtain speech fusion feature data. The speech fusion feature data is then analyzed using a cognitive speech analysis engine to obtain cognitive ability detection results.

[0009] Furthermore, the speech data is analyzed using a preprocessing engine to obtain target speech data, including:

[0010] Initiate environmental sound analysis to perform noise reduction processing on the environmental sounds in the speech data;

[0011] The voiceprint recognition model is used to identify the test subject in the voice data, and the voice data of the test subject is extracted from the voice data to obtain the target voice data.

[0012] Furthermore, voiceprint recognition models are used to identify test subjects in speech data, including:

[0013] The voiceprint recognition model is used to obtain the voiceprint information from the speech data;

[0014] The voiceprint information is compared and analyzed with the first user database. Based on the comparison and analysis results, the test subject and his / her corresponding voiceprint information are determined from the first user database, or;

[0015] The voiceprint information is compared and analyzed with the second user database. Based on the comparison and analysis results, the voiceprint information of the tester is determined from the second user database. Based on the voiceprint information of the tester, the voiceprint information of the test subject is determined, and the voiceprint information of the test subject is stored in the first user database.

[0016] Furthermore, the target speech data is analyzed using a natural speech processing engine to obtain language feature data, including:

[0017] Convert the target speech data into speech-text;

[0018] The text recognition model is used to identify text errors, which include at least one of grammatical errors, spelling errors, vocabulary errors, and repetition errors.

[0019] The speech text is corrected based on the text errors to obtain the target speech text;

[0020] Extract language feature data from the target speech text.

[0021] Furthermore, language feature data is extracted from the target speech text, including:

[0022] Perform part-of-speech and keyword analysis on the target language text to extract keywords;

[0023] The keywords are copied according to their importance to construct a language feature vector.

[0024] Furthermore, the keywords are copied according to their importance to construct a language feature vector, including:

[0025] The text error is evaluated based on the text error type, text error data, and the weight corresponding to the text error type to obtain a text error evaluation value;

[0026] The ratio of the difference between the text error evaluation value and the preset evaluation value to the error text evaluation value is obtained, wherein the preset evaluation value is determined according to the tester's age and occupation;

[0027] The ratio is used as a vector coefficient to correct the language feature vector, thus obtaining language feature data.

[0028] Furthermore, the target speech data is analyzed using a speech audio signal processing engine to obtain audio feature data, including:

[0029] First audio feature data is extracted from the target language data based on the language feature vector;

[0030] The second audio feature data is extracted according to preset parameters, including a pause duration threshold and a pitch threshold. The second audio feature data includes long pause speech features, long pitch speech features, and low pitch speech features.

[0031] Furthermore, based on an attention mechanism, the audio feature data and the speech feature data are fused to obtain speech fusion feature data, including:

[0032] The speech feature data and the first audio feature data are fused to obtain the first fused feature data;

[0033] The first fused feature data and the second audio feature data are fused together to obtain fused feature data.

[0034] Furthermore, the speech fusion feature data is analyzed using a cognitive speech analysis engine to obtain cognitive ability detection results, including:

[0035] The speech fusion data was analyzed using a cognitive impairment analysis model to determine whether the test subjects had cognitive impairment.

[0036] When test subjects have cognitive impairment, the cognitive speech analysis model is used to analyze the speech fusion data to determine the degree of cognitive impairment corresponding to the cognitive impairment test items.

[0037] The level of cognitive impairment is determined based on the cognitive impairment items and their corresponding degrees of cognitive impairment.

[0038] Furthermore, the cognitive speech analysis model is used to analyze the speech fusion data to determine the degree of cognitive impairment corresponding to the cognitive impairment test item. This includes inputting the fused speech feature data corresponding to the cognitive impairment test item and all fused speech feature data into the cognitive speech analysis model corresponding to the cognitive impairment test item to obtain the degree of cognitive impairment corresponding to the cognitive impairment test item.

[0039] The beneficial effects of this invention are as follows: the speech recognition-based cognitive impairment assessment method, through the analysis and processing of speech data and the fusion of attention mechanisms, can effectively assess the cognitive abilities and degree of cognitive impairment of test subjects. Simultaneously, this method utilizes preprocessing engines, natural language processing engines, and speech audio signal processing engines to perform noise reduction, text conversion, and feature extraction on the speech data, thereby improving the accuracy and reliability of the assessment. Attached Figure Description

[0040] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0041] Figure 1 A flowchart illustrating a speech recognition-based cognitive impairment assessment method provided in an embodiment of the present invention;

[0042] Figure 2 A sub-flowchart of a cognitive impairment assessment method based on speech recognition provided in an embodiment of the present invention;

[0043] Figure 3 Another sub-flowchart of a speech recognition-based cognitive impairment assessment method provided in an embodiment of the present invention. Detailed Implementation

[0044] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0045] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0046] like Figure 1 As shown, a cognitive impairment assessment method based on speech recognition includes:

[0047] Step 1: Obtain the voice data during the test;

[0048] Specifically, voice data is obtained through a front-end testing tool using a questionnaire. The voice data includes the voice data of the tester and the test subject. The tester can be an AI or a robot. The questions in the questionnaire correspond to various cognitive impairment test items, including picture recognition, short-term memory, attention, short-term memory, and long-term memory.

[0049] Step 2: Analyze the speech data using a preprocessing engine to obtain the target speech data;

[0050] like Figure 2 As shown, in this embodiment of the invention, the preprocessing engine is used to analyze the speech data to obtain target speech data, including: starting environmental sound analysis and performing noise reduction processing on the environmental sounds in the speech data; using a voiceprint recognition model to identify the test subject in the speech data, extracting the test subject's speech data from the speech data, and obtaining target speech data.

[0051] Specifically, environmental sound analysis methods include, but are not limited to, statistical feature methods and machine learning methods. Statistical feature-based methods calculate statistical characteristics such as frame energy, spectral average, and cepstral coefficients of speech data to determine the energy and distribution of environmental noise, thus enabling noise reduction. Machine learning-based methods use machine learning algorithms, such as Gaussian mixture models (GMMs) or hidden Markov models (HMMs), to model and classify environmental sounds to distinguish between target speech and environmental noise.

[0052] Specifically, noise reduction methods include statistical filtering and spectral subtraction. Statistical filtering, based on the distribution of statistical features, uses filtering algorithms (such as mean filtering and median filtering) to reduce environmental noise while preserving the target speech signal. Spectral subtraction estimates and subtracts the spectrum of the speech signal to suppress noise components, thereby reducing noise levels. Gaussian Mixture Model (GMM) models speech data and extracts speaker features; this model is based on modeling and classifying features such as the sound spectrum and formants.

[0053] In this embodiment of the invention, identifying a test subject in voice data using a voiceprint recognition model includes: acquiring voiceprint information from the voice data using the voiceprint recognition model; comparing and analyzing the voiceprint information with a first user database, and determining the test subject and its corresponding voiceprint information from the first user database based on the comparison and analysis results; or; comparing and analyzing the voiceprint information with a second user database, determining the test subject's voiceprint information from the second user database based on the comparison and analysis results, determining the test subject's voiceprint information based on the test subject's voiceprint information, and storing the test subject's voiceprint information in the first user database.

[0054] First, a voiceprint recognition model extracts the test subject's voiceprint information from speech data. This unique biometric feature can be used for individual identification, similar to fingerprints or iris scans. Second, by comparing and analyzing a first user database and a second user database, the test subject's voiceprint information can be accurately determined. The first user database contains information on known users; by comparing, matching voiceprint information can be found to identify the test subject. The second user database contains voiceprint information from multiple users; by comparing, the voiceprint information that best matches the test subject's voiceprint can be found, further confirming the test subject's identity. Finally, based on the results of the comparative analysis, the test subject's voiceprint information is stored in the first user database. This allows for faster and more accurate identification when encountering the same test subject again, improving the efficiency and accuracy of the identification process. Furthermore, this invention allows for the updating and expansion of the user database as needed to adapt to a constantly changing user base.

[0055] This embodiment achieves rapid and accurate identification of the test subject's identity in voice data through a voiceprint recognition model and comparative analysis technology, and can update and expand the user database, providing a convenient and reliable solution for individual identity identification.

[0056] Step 3: Analyze the target speech data using a natural speech processing engine to obtain language feature data, and analyze the target speech data using a speech audio signal processing engine to obtain audio feature data;

[0057] like Figure 3 As shown, in this embodiment of the invention, the target speech data is analyzed using a natural speech processing engine to obtain language feature data, including: converting the target speech data into speech text; identifying text errors using a text recognition model, wherein the text errors include at least one of grammatical errors, spelling errors, vocabulary errors, and repetition errors; correcting the speech text according to the text errors to obtain target speech text; and extracting language feature data from the target speech text.

[0058] The text errors provided in this embodiment are simple, correctable text errors. Correcting them allows for a better assessment of the impact of the spoken text itself on the cognitive impairment of the test subjects.

[0059] In this embodiment of the invention, extracting language feature data from the target speech text includes: performing part-of-speech analysis and keyword analysis on the target language text to extract keywords; and copying the keywords according to their importance to construct a language feature vector.

[0060] Specifically, the keywords include first-class keywords and second-class keywords. The first-class keywords are cognitive keywords, which include phonological keywords (memory, attention, comprehension, phonological ability), cognitive processing keywords (cognitive speed, attention shifting, conceptual driving, problem solving), and executive function keywords (planning and organizing, self-monitoring, implementation ability). The second-class keywords are affective keywords (anxiety, happiness, confidence level).

[0061] Specifically, the importance of keywords is preset according to the category, and the higher the importance, the more times they can be copied.

[0062] In this embodiment of the invention, the keywords are copied according to their importance to construct a language feature vector, including: evaluating the text error according to the text error type, text error data, and the weight corresponding to the text error type to obtain a text error evaluation value; obtaining the ratio of the difference between the text error evaluation value and a preset evaluation value to the error text evaluation value; and using the ratio as a vector coefficient to correct the language feature vector to obtain language feature data.

[0063] Specifically, the preset evaluation value is determined based on the tester's age and occupation.

[0064] Regardless of whether a person has or does not have cognitive impairment, they may make some common errors when answering questions. However, if the number of common errors exceeds a certain threshold, we believe it has a certain impact on cognitive impairment. Therefore, this embodiment provides a method to evaluate text errors and correct speech feature vectors based on the evaluation results, thereby improving the effectiveness of speech feature data.

[0065] In this embodiment of the invention, the target speech data is analyzed using a speech audio signal processing engine to obtain audio feature data, including: extracting first audio feature data from the target language data based on the language feature vector; and extracting second audio feature data according to preset parameters, the preset parameters including a pause duration threshold and a pitch threshold, the second audio feature data including long pause speech features, long pitch speech features and low pitch speech features.

[0066] Specifically, the target speech data is input into the speech audio signal processing engine, and the first audio feature data and the second speech feature data are extracted from the target speech data using language feature vectors. This can be achieved by using a suitable speech feature extraction algorithm, such as the Mel spectrum feature extraction algorithm or the linear predictive coding (LPC) algorithm.

[0067] Specifically, long pause speech features represent long pauses in the target speech data and can be used to detect sentence boundaries or extract speech units; long pitch speech features represent long pitch changes in the target speech data and can be used to detect stress or tone changes; low pitch speech features represent low pitch conditions in the target speech data and can be used to detect speech tone patterns.

[0068] Step 4: Based on the attention mechanism, fuse the audio feature data and the speech feature data to obtain speech fusion feature data. Analyze the speech fusion feature data using a cognitive speech analysis engine to obtain cognitive ability detection results.

[0069] In this embodiment of the invention, the audio feature data and the speech feature data are fused based on an attention mechanism to obtain speech fusion feature data, including: fusing the speech feature data and the first audio feature data to obtain first fusion feature data; and fusing the first fusion feature data and the second audio feature data to obtain fusion feature data.

[0070] This embodiment fuses speech feature data and first audio feature data through an attention mechanism to obtain first fused feature data, achieving multi-level feature fusion and improving the depth of speech feature data. Then, the first fused feature data is fused with second audio feature data to achieve multi-angle fusion, improving the perception of key features, reducing conflicts between features, and helping to improve evaluation accuracy.

[0071] In this embodiment of the invention, the cognitive speech analysis engine is used to analyze the speech fusion feature data to obtain cognitive ability detection results, including: using a cognitive impairment analysis model to analyze the speech fusion data to determine whether the test subject has a cognitive impairment; when the test subject has a cognitive impairment, using a cognitive speech analysis model to analyze the speech fusion data to determine the degree of cognitive impairment corresponding to the cognitive impairment test item; and determining the cognitive impairment level based on the cognitive impairment item and its corresponding degree of cognitive impairment.

[0072] Specifically, the cognitive impairment analysis model is trained on training data with and without cognitive impairment, while the cognitive speech analysis model is trained on training data corresponding to cognitive impairment test items.

[0073] In this embodiment of the invention, the cognitive speech analysis model is used to analyze speech fusion data to determine the degree of cognitive impairment corresponding to the cognitive impairment test item. This includes: inputting the fused speech feature data corresponding to the cognitive impairment test item and all fused speech feature data into the cognitive speech analysis model corresponding to the cognitive impairment test item to obtain the degree of cognitive impairment corresponding to the cognitive impairment test item.

[0074] In this embodiment, when assessing a single cognitive impairment, not only is attention paid to the speech fusion feature data corresponding to the single cognitive impairment, but also to all speech data and the speech fusion feature data corresponding to the single cognitive impairment, thereby improving recognition accuracy.

[0075] In summary, the speech recognition-based cognitive impairment assessment method provided in this invention can effectively assess the cognitive abilities and degree of cognitive impairment of test subjects by analyzing and processing speech data and combining it with an attention mechanism fusion approach. Furthermore, this method utilizes preprocessing engines, natural language processing engines, and speech audio signal processing engines to perform noise reduction, text conversion, and feature extraction on the speech data, thereby improving the accuracy and reliability of the assessment.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A cognitive impairment assessment method based on speech recognition, characterized by, The method comprises the following steps: acquiring voice data at the time of testing; analyzing the voice data by using a preprocessing engine to obtain target voice data; analyzing the target voice data by using a natural voice processing engine to obtain language feature data, and analyzing the target voice data by using a voice audio signal processing engine to obtain audio feature data; fusing the audio feature data and the voice feature data based on an attention mechanism to obtain voice fusion feature data, and analyzing the voice fusion feature data by using a cognitive voice analysis engine to obtain a cognitive ability detection result; analyzing the voice data by using the preprocessing engine to obtain target voice data, comprising: starting environmental sound analysis, and performing noise reduction processing on the environmental sound in the voice data; identifying a testee in the voice data by using a voiceprint recognition model, extracting voice data of the testee from the voice data, and obtaining target voice data; analyzing the target voice data by using the natural voice processing engine to obtain language feature data, comprising: converting the target voice data into voice text; identifying text errors by using a text recognition model, the text errors comprising at least one of syntax errors, spelling errors, vocabulary errors, and repetition errors; correcting the voice text according to the text errors to obtain target voice text; and extracting language feature data from the target voice text; extracting language feature data from the target voice text, comprising: performing part-of-speech analysis and keyword analysis on the target language text to extract keywords; and copying the keywords according to their importance to construct a language feature vector; copying the keywords according to their importance to construct a language feature vector, comprising: evaluating the text errors according to the type of text errors, text error data, and the weight corresponding to the type of text errors to obtain a text error evaluation value; obtaining the ratio of the difference between the text error evaluation value and a preset evaluation value to the error text evaluation value, the preset evaluation value being determined according to the age and occupation of the testee; and correcting the language feature vector by using the ratio as a vector coefficient to obtain language feature data; analyzing the target voice data by using the voice audio signal processing engine to obtain audio feature data, comprising: extracting first audio feature data from the target language data according to the language feature vector; and extracting second audio feature data according to a preset parameter, the preset parameter comprising a pause duration threshold and a pitch level threshold, and the second audio feature data comprising long pause voice features, long pitch voice features, and low pitch voice features.

2. The cognitive impairment assessment method based on speech recognition according to claim 1, wherein, identifying a testee in voice data by using a voiceprint recognition model, comprising: obtaining voiceprint information in the voice data by using the voiceprint recognition model; comparing and analyzing the voiceprint information with a first user database, and determining a testee and corresponding voiceprint information from the first user database according to the comparison and analysis result, or The voiceprint information is compared and analyzed with the second user database. Based on the comparison and analysis results, the voiceprint information of the tester is determined from the second user database. Based on the voiceprint information of the tester, the voiceprint information of the test subject is determined, and the voiceprint information of the test subject is stored in the first user database. 3.The cognitive impairment assessment method based on speech recognition of claim 1, wherein, Based on an attention mechanism, the audio feature data and the speech feature data are fused to obtain speech fusion feature data, including: The speech feature data and the first audio feature data are fused to obtain the first fused feature data; The first fused feature data and the second audio feature data are fused together to obtain fused feature data. 4.The cognitive impairment assessment method based on voice recognition according to claim 1, wherein, The cognitive speech analysis engine is used to analyze the speech fusion feature data to obtain cognitive ability detection results, including: The speech fusion data was analyzed using a cognitive impairment analysis model to determine whether the test subjects had cognitive impairment. When test subjects have cognitive impairment, the cognitive speech analysis model is used to analyze the speech fusion data to determine the degree of cognitive impairment corresponding to the cognitive impairment test items. The level of cognitive impairment is determined based on the cognitive impairment items and their corresponding degrees of cognitive impairment.

5. The cognitive impairment assessment method based on speech recognition according to claim 4, wherein, The cognitive speech analysis model is used to analyze speech fusion data to determine the degree of cognitive impairment corresponding to the cognitive impairment test item. This includes inputting the fused speech feature data corresponding to the cognitive impairment test item and all fused speech feature data into the cognitive speech analysis model corresponding to the cognitive impairment test item to obtain the degree of cognitive impairment corresponding to the cognitive impairment test item.

Images