A cognitive function assessment method, apparatus, device, medium and product

Abstract

This application discloses a method, apparatus, device, medium, and product for cognitive function assessment, relating to the fields of computer and speech processing technology. The method includes: acquiring target speech data; extracting features from the target speech data according to feature categories within a preferred subset of speech features corresponding to the target cognitive test task, obtaining a target speech feature set; inputting the target speech feature set into a cognitive function assessment model corresponding to the target cognitive test task, obtaining the cognitive function assessment result of the target test subject; wherein, by constructing a multimodal cognitive database, extracting wide-area speech features from each form of speech data in the database, and based on the wide-area speech features and corresponding cognitive function assessment results, selecting a preferred subset of speech features based on the correlation strength with cognitive function, and determining the cognitive function assessment model, machine learning and statistical methods are used. This application achieves accurate and efficient cognitive function assessment.

Description

Technical Field

[0001] This application relates to the field of computer and speech processing technology, and in particular to a method, apparatus, device, medium and product for assessing cognitive function. Background Technology

[0002] Currently, the mainstream techniques for cognitive function assessment globally still rely on traditional cognitive scales as their core tools. These tools have a long history of development, with the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA) being the most representative. Due to their relative ease of use and comprehensive assessment dimensions (covering orientation, memory, attention, and language abilities), they have become the most commonly used screening tools in clinical and research fields. However, the application of traditional scales consistently faces the following limitations.

[0003] From the perspective of current technological development, traditional scales rely on paper-based assessment processes, requiring subjects to complete 20-30 minutes of question answering under the guidance of the tester, which is time-consuming and inefficient. When scales are translated into non-native languages ​​such as English, semantic bias and dialect compatibility issues may occur, affecting the accuracy of the assessment.

[0004] From the perspective of application pain points, traditional scales are highly sensitive to the educational level and cultural background of the test takers: elderly people with lower levels of education may have normal cognition but abnormal scores because they cannot understand the test content; groups with large differences in cultural background may be unfamiliar with the test content, which cannot truly reflect their cognitive function level, resulting in assessment results that deviate from reality.

[0005] In summary, how to accurately and efficiently conduct cognitive function assessment has become an urgent problem to be solved. Summary of the Invention

[0006] The purpose of this application is to provide a method, apparatus, device, medium, and product for assessing cognitive function, which can accurately and efficiently achieve cognitive function assessment.

[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for assessing cognitive function, including: Acquire target speech data; the target speech data is the speech data of the target test subject when performing the target cognitive test task; Based on the feature categories in the preferred speech feature subset corresponding to the target cognitive test task, feature extraction is performed on the target speech data to obtain the target speech feature set; The target speech feature set is input into the cognitive function assessment model corresponding to the target cognitive test task to obtain the cognitive function assessment result of the target test subject; The method for determining the preferred subset of speech features and the cognitive function assessment model includes: Construct a multimodal cognitive database; the multimodal cognitive database includes: speech data of test subjects when performing different forms of cognitive test tasks and corresponding cognitive function assessment results; Extract wide-area speech features from each form of speech data in the multimodal cognitive database; For each type of wide-area speech feature, based on the wide-area speech feature and the corresponding cognitive function assessment results, machine learning and statistical methods are used to screen features with a stronger correlation to cognitive function than a set strength, to obtain the preferred speech feature subsets corresponding to different types of cognitive test tasks, and the classification model trained based on the preferred speech feature subsets corresponding to each type of cognitive test task is determined as the corresponding cognitive function assessment model.

[0008] In one embodiment, for each type of wide-area speech feature, based on the wide-area speech feature and the corresponding cognitive function assessment result, machine learning and statistical methods are used to screen features with a cognitive function correlation strength greater than a set strength, obtaining a preferred speech feature subset corresponding to different types of cognitive test tasks. The classification model trained based on the preferred speech feature subset corresponding to each type of cognitive test task is then determined as the corresponding cognitive function assessment model, specifically including: Based on the wide-area speech features and the corresponding cognitive function evaluation results, the first machine learning model is used to rank the importance of the wide-area speech features to obtain a feature importance sequence. Based on the feature importance sequence, features with importance less than a set value in the wide-area speech features are deleted to obtain a preliminary selected feature subset; Based on information theory, a feature selection algorithm is used to remove redundancy from the preliminary selected feature subset according to the correlation between features, resulting in a feature subset after redundancy removal. The features in the removed feature subset are input one by one into the second machine learning model. Cross-validation is used to select the optimal speech feature subset with the goal of achieving the best performance of the second machine learning model. The second machine learning model trained based on the optimal speech feature subset is then determined as the corresponding cognitive function assessment model.

[0009] In one implementation, based on information theory, a feature selection algorithm is used to remove redundancy from the initially selected feature subset according to the correlation between features, resulting in a feature subset after redundancy removal. Specifically, this includes: Based on information theory, the minimum redundancy maximum relevance algorithm is adopted. With the goal of maximizing the correlation between features and cognitive states and minimizing the correlation between features, the redundancy of the initially screened feature subset is eliminated to obtain the feature subset after elimination.

[0010] In one embodiment, the first machine learning model is a random forest model or an XGBoost model; the second machine learning model is a support vector machine or a logistic regression model.

[0011] In one embodiment, the wide-area speech features include: prosodic features, spectral features, and phonological features; The prosodic features include: fundamental frequency related indicators, energy related indicators, and duration related indicators; the fundamental frequency related indicators include: fundamental frequency mean, fundamental frequency median, fundamental frequency standard deviation, fundamental frequency maximum, fundamental frequency minimum, fundamental frequency variation range, and fundamental frequency jitter; the energy related indicators include: intensity mean, intensity standard deviation, intensity dynamic range, and intensity perturbation; the duration related indicators include: total speech rate, speech duration, silence duration, silence frequency, and speech-pause ratio; The spectral features include: Mel frequency cepstral coefficients, linear predictive coding coefficients, and formant correlation indices; The sound quality characteristics include: harmonic noise ratio, spectral slope, and energy distribution.

[0012] In one embodiment, different forms of cognitive testing tasks include: episodic memory testing tasks, language fluency testing tasks, executive function and processing speed testing tasks, and attention and working memory testing tasks.

[0013] Secondly, this application provides a cognitive function assessment device, comprising: The target speech data acquisition module is used to acquire target speech data; the target speech data is the speech data of the target test subject when performing the target cognitive test task. The feature extraction module is used to extract features from the target speech data according to the feature categories in the preferred speech feature subset corresponding to the target cognitive test task, so as to obtain the target speech feature set. The cognitive function assessment module is used to input the target speech feature set into the cognitive function assessment model corresponding to the target cognitive test task to obtain the cognitive function assessment result of the target test subject. The method for determining the preferred subset of speech features and the cognitive function assessment model includes: Construct a multimodal cognitive database; the multimodal cognitive database includes: speech data of test subjects when performing different forms of cognitive test tasks and corresponding cognitive function assessment results; Extract wide-area speech features from each form of speech data in the multimodal cognitive database; For each type of wide-area speech feature, based on the wide-area speech feature and the corresponding cognitive function assessment results, machine learning and statistical methods are used to screen features with a stronger correlation to cognitive function than a set strength, to obtain the preferred speech feature subsets corresponding to different types of cognitive test tasks, and the classification model trained based on the preferred speech feature subsets corresponding to each type of cognitive test task is determined as the corresponding cognitive function assessment model.

[0014] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the cognitive function assessment method described in any one of the above.

[0015] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cognitive function assessment method described in any one of the above descriptions.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the cognitive function assessment method described in any one of the above statements.

[0017] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method, apparatus, device, medium, and product for cognitive function assessment. First, a multimodal cognitive database is constructed, and wide-area speech features are extracted from each form of speech data in the database. Based on the wide-area speech features and corresponding cognitive function assessment results, and considering the strength of association with cognitive function, machine learning and statistical methods are used to select a preferred subset of speech features and determine a cognitive function assessment model. Then, according to the feature categories in the preferred subset of speech features corresponding to the target cognitive test task, features are extracted from the target speech data, and the corresponding cognitive function assessment model is used to obtain the cognitive function assessment results of the target test subject. This application achieves cognitive function screening based on speech analysis. By extracting a preferred subset of speech features during the cognitive testing process, it directly reflects the cognitive function state through the essential characteristics of sound, thereby solving the problems of traditional scales being time-consuming, having poor dialect adaptability, and being greatly influenced by educational and cultural backgrounds, thus achieving accurate and efficient cognitive function assessment. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a cognitive function assessment method provided in an embodiment of this application; Figure 2 A schematic flowchart illustrating the method for determining the preferred subset of speech features and the cognitive function assessment model provided in the embodiments of this application; Figure 3 An overall architecture diagram of a cognitive function assessment method provided in this application embodiment in practical application; Figure 4 A schematic diagram illustrating a preferred speech feature subset construction method provided in this application embodiment; Figure 5 A schematic diagram of the functional modules of a cognitive function assessment device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Given the current practical needs, there is an urgent need for a more efficient cognitive function assessment scheme. This application proposes a cognitive function assessment method that uses speech analysis to screen cognitive functions. By extracting the acoustic features of speech during the cognitive test (such as speech rate, pauses, and intonation), it bypasses the barriers of text comprehension and dialect semantics, and directly reflects the state of cognitive function through the essential features of sound. This solves the problems of traditional scales being time-consuming, having poor dialect adaptability, and being greatly affected by educational and cultural background.

[0023] In one exemplary embodiment, such as Figure 1 As shown, a cognitive function assessment method is provided, including: Step 101: Obtain target speech data; the target speech data is the speech data of the target test subject when performing the target cognitive test task.

[0024] Step 102: Extract features from the target speech data according to the feature categories in the preferred speech feature subset corresponding to the target cognitive test task to obtain the target speech feature set.

[0025] Step 103: Input the target speech feature set into the cognitive function assessment model corresponding to the target cognitive test task to obtain the cognitive function assessment result of the target test subject.

[0026] The cognitive function assessment result is a cognitive assessment score (e.g., 0-100 points) or a cognitive assessment risk level (e.g., "low risk", "medium risk", "high risk").

[0027] Among them, such as Figure 2 As shown, the method for determining the preferred subset of speech features and the cognitive function assessment model includes: Step 201: Construct a multimodal cognitive database.

[0028] The multimodal cognitive database includes: speech data of test subjects when performing different forms of cognitive test tasks and corresponding cognitive function assessment results.

[0029] Step 202: Extract wide-area speech features for each form of speech data in the multimodal cognitive database.

[0030] Step 203: For each type of wide-area speech feature, based on the wide-area speech feature and the corresponding cognitive function assessment results, machine learning and statistical methods are used to screen features with a stronger correlation to cognitive function than a set strength, to obtain a preferred speech feature subset corresponding to different types of cognitive test tasks, and the classification model trained based on the preferred speech feature subset corresponding to each type of cognitive test task is determined as the corresponding cognitive function assessment model.

[0031] In another exemplary embodiment of this application, in step 201, different forms of cognitive testing tasks include: episodic memory testing tasks, verbal fluency testing tasks, executive function and processing speed testing tasks, and attention and working memory testing tasks.

[0032] In another exemplary embodiment of this application, in step 202, the wide-area speech features are features that reflect the essential attributes of speech, including but not limited to pitch, intensity, speech rate, rhythm, voice quality, dynamic changes in volume, and intonation patterns. The wide-area speech features are the most comprehensive acoustic features possible extracted from speech data in a multimodal cognitive database.

[0033] For example, the wide-area speech features may include: prosodic features, spectral features, and voice quality features.

[0034] The prosodic features include: fundamental frequency (Pitch / F0) related indicators, energy (Energy / Intensity) related indicators, and duration (Temporal) related indicators. The fundamental frequency related indicators include: fundamental frequency mean, fundamental frequency median, fundamental frequency standard deviation, fundamental frequency maximum, fundamental frequency minimum, fundamental frequency range, and fundamental frequency jitter. The energy related indicators include: intensity mean, intensity standard deviation, intensity dynamic range, and intensity shimmer. The duration related indicators include: total speech rate, speech duration, silence duration, silence frequency, and speech-to-pause ratio.

[0035] The spectral features include: Mel frequency cepstral coefficients (MFCC), linear predictive coding (LPC) coefficients, and formants-related indices.

[0036] The sound quality characteristics include: harmonic-to-noise ratio (HNR), spectral slope, and energy distribution.

[0037] In another exemplary embodiment of this application, step 203 specifically includes: (1) Based on the wide-area speech features and the corresponding cognitive function evaluation results, the first machine learning model is used to rank the importance of the wide-area speech features to obtain the feature importance sequence.

[0038] (2) Based on the feature importance sequence, delete the features with importance less than the set value in the wide-area speech features to obtain a preliminary selected feature subset.

[0039] (3) Based on information theory, a feature selection algorithm is used to remove redundancy from the initially selected feature subset according to the correlation between features, resulting in a feature subset after redundancy removal. Specifically: Based on information theory, the Minimum Redundancy Maximum Relevance (mRMR) algorithm is used to remove redundancy from the preliminary selected feature subset with the goal of maximizing the correlation between features and cognitive states and minimizing the correlation between features.

[0040] (4) Input the features from the removed feature subset one by one into the second machine learning model, and use cross-validation to select the optimal speech feature subset with the goal of achieving the best performance of the second machine learning model. Then, determine the second machine learning model trained based on the optimal speech feature subset as the corresponding cognitive function assessment model. The optimal performance of the second machine learning model can be: the AUC (Area Under Curve) or accuracy of the second machine learning model reaches its peak or no longer significantly improves.

[0041] The first machine learning model is a Random Forest model or an XGBoost model; the second machine learning model is a Support Vector Machine or a Logistic Regression model.

[0042] The cognitive function assessment method in this embodiment extracts and processes the voice features of test subjects collected during various forms of cognitive tests, identifies and separates features reflecting the essential attributes of voice (including but not limited to pitch, speech rate, dynamic changes in volume, and intonation patterns), and obtains a set of voice features corresponding to each cognitive function category. Based on a dataset containing multiple sets of control data, the method determines the importance of each voice feature in relation to cognitive function. The dataset consists of voice data collected synchronously by each subject during various forms of cognitive tests, and the voice features with the highest correlation to cognitive function are selected to form a corresponding preferred subset of voice features. When screening a target test subject for cognitive function, the method collects their voice data in real time during the cognitive test, extracts key features from the preferred subset of voice features, and calculates feature indicators. Finally, based on the target test subject's voice feature indicators, the method completes the screening and assessment of their cognitive function status. This embodiment, by focusing on the analysis of the essential voice features in cognitive tests, eliminates the need for traditional question-and-answer processes, enabling convenient and rapid cognitive function screening.

[0043] This embodiment focuses on the underlying physical properties of speech and establishes an innovative process for systematically selecting the optimal feature subset from massive features, possessing significant technical advantages: (1) Detached from semantic dependence, focusing on the essential attributes of speech: One of the biggest innovations of this embodiment is that it completely bypasses the recognition and understanding of speech content. It focuses on analyzing acoustic features that reflect the physiological mechanism of vocalization, such as pitch, volume, speech rate, rhythm, and tone quality. This method not only avoids the inherent defects of ASR technology, but also gives it the potential for universality across languages ​​and cultures, and the evaluation results are more objective and stable.

[0044] (2) Systematic, data-driven feature selection mechanism: This embodiment does not simply select a set of preset features, but proposes a complete, data-driven method for constructing a "preferred subset of speech features". This method combines large-scale datasets, machine learning models and advanced feature selection algorithms, and can automatically discover feature combinations that are most closely related to changes in cognitive function and have the lowest information redundancy. This is a significant improvement over the simplification or absence of the feature selection process in existing technologies.

[0045] (3) Cognitive domain specificity analysis: This embodiment innovatively proposes that exclusive preferred subsets of speech features can be constructed for different cognitive test paradigms (such as memory test, fluency test, attention test, etc.). This means that this method can not only provide an overall cognitive state assessment, but also has the potential to distinguish which specific cognitive domain (e.g., memory, executive function, language) may have problems, providing more valuable clues for subsequent precise intervention.

[0046] (4) High efficiency and low cost of real-time screening capability: Since this method only needs to extract a subset of preferred features with low computational cost in the application stage, without the need for complex ASR transcription and natural language processing, it can be easily deployed on terminals such as smartphones, smart speakers or wearable devices to achieve real-time, seamless and high-frequency monitoring and screening of users' cognitive status, which greatly reduces the evaluation threshold.

[0047] The following is a detailed implementation process in a practical application, further illustrating the cognitive function assessment method of this application. The overall architecture of this process is as follows: Figure 3 As shown, it integrates data acquisition, feature extraction, model training, and real-time evaluation functions.

[0048] The cognitive function assessment method in this embodiment is divided into two stages: an offline training stage and an online assessment stage.

[0049] (a) Offline Training Phase: The main objectives are to build the core evaluation model and determine the "optimal subset of speech features." This phase includes: 1. Multimodal cognitive database construction: Collect a large-scale comparative dataset containing "cognitive test data" and synchronous "speech data". 2. Wide-domain speech feature extraction: Extract the most comprehensive possible acoustic features from the speech data. 3. Feature-cognitive function correlation modeling and selection: Through machine learning and statistical methods, determine the feature combinations most strongly correlated with the functions of each cognitive domain, forming an optimal subset of speech features.

[0050] (II) Online Assessment Phase: Real-time screening of new target test subjects. This phase includes: 1. Real-time speech acquisition: Recording is performed synchronously while the target test subject performs a cognitive task. 2. Optimal feature extraction and index calculation: Features are extracted only from a pre-determined optimal subset of speech features, and corresponding indices are calculated. 3. Cognitive state assessment and output: The feature indices are input into a pre-trained assessment model to generate the final screening report.

[0051] Specifically, for the offline training phase in Part (I): (1) Construct a control dataset to build a multimodal cognitive database. This step is the foundation of the entire method. A high-quality, large-scale dataset is the guarantee of model performance.

[0052] Participant Recruitment: A large number of participants were recruited, including cognitively healthy young and older adults (as a healthy control group) and patients with clinically diagnosed cognitive impairment. The dataset size can be set to hundreds to thousands of participants to ensure the model's generalization ability.

[0053] Cognitive Testing Protocol: To comprehensively assess cognitive function and lay the foundation for subsequently constructing cognitive domain-specific feature subsets, a series of standardized cognitive testing paradigms are required. These tests should cover multiple core cognitive domains, such as: Episodic memory test tasks include: picture description tasks (such as "cookie theft"), story retelling, and word list learning (such as Hopkins Verbal Learning Test - Revised, HVLT-R).

[0054] Language fluency test task: Category fluency test (e.g., say as many animal names as possible in one minute).

[0055] Execution function and processing speed test tasks: Trail Making Test A&B (TMT) and Digit Symbol Substitution Test (DSST).

[0056] Willpower and Working Memory Testing Task: Digit Span Test (forward and backward).

[0057] Data Collection: During each cognitive test performed by the subjects, their complete speech data was simultaneously recorded using a high-fidelity microphone (sampling rate ≥16kHz, quantization bit depth ≥16bit). At the same time, detailed demographic information (age, gender, years of education), clinical diagnostic results, and specific scores on each cognitive test (cognitive function assessment results) were recorded for each subject.

[0058] (2) Wide-area speech feature extraction and processing.

[0059] To avoid missing any potential biomarkers, the goal of this step is to extract a very broad and comprehensive set of acoustic features. This can be automated using open-source tools such as openSMILE, Praat, or Parselmouth. The extracted feature set may include, but is not limited to, the following categories: Prosodic Features: Fundamental frequency related indicators: mean, standard deviation, maximum / minimum value, range of variation, and jitter.

[0060] Energy-related indicators: mean intensity, standard deviation, dynamic range, and intensity perturbation.

[0061] Duration-related metrics: total speech rate (syllables per second), speech duration, silence duration, silence frequency, and speech-to-pause ratio.

[0062] Spectral characteristics: Mel-frequency cepstral coefficients: Extract 12-13 MFCC coefficients and their first-order difference (Delta) and second-order difference (Delta-Delta). MFCC is one of the most crucial features in speech analysis because it effectively simulates the characteristics of human hearing and has good robustness.

[0063] Linear predictive coding and others: LPC coefficients, linear predictive cepstral coefficients (LPCC), and perceptual linear prediction (PLP).

[0064] Resonance peak related indicators: mean and dynamic values ​​of each resonance peak frequency.

[0065] Sound quality characteristics: Harmonic noise ratio: reflects the hoarseness of the sound.

[0066] Spectral slope, energy distribution, etc.

[0067] Through the above process, for each speech sample, a high-dimensional feature vector containing hundreds or even thousands of features can be obtained (for example, the eGeMAPS feature set using openSMILE contains 88 features, and the ComParE feature set contains 6373 features).

[0068] (3) Determine the preferred subset of speech features.

[0069] This section aims to select the most predictive and interpretable "optimal subset of speech features" from the high-dimensional, redundant feature vectors extracted in the previous step. For example... Figure 4 As shown, this process can be broken down into the following three sub-steps: Step 1: Preliminary correlation ranking.

[0070] First, a machine learning model is used to rank all features by importance.

[0071] Model selection: Employ ensemble learning models, such as Random Forest or XGBoost. These models perform well when handling high-dimensional data, are less prone to overfitting, and can naturally output the contribution of each feature to the classification task (e.g., distinguishing between healthy individuals and MCI patients), i.e., "feature importance".

[0072] Implementation process: Using the subject's clinical diagnosis (e.g., "healthy" / "cognitive impairment") as the label, and the wide-area feature vector extracted in the above steps as input, a random forest classifier is trained. After training, the importance scores of each feature calculated by the model are extracted and sorted from high to low. This step can filter out a large number of noisy features that are completely irrelevant to cognitive state.

[0073] Step 2: Redundancy elimination based on information theory.

[0074] Simply selecting the most important features is insufficient, as top-ranked features may be highly correlated (e.g., mean and median fundamental frequencies). To obtain a more concise and efficient subset of features, feature selection algorithms are needed to remove redundant information.

[0075] Algorithm selection: An advanced feature selection algorithm, such as the Minimum Redundancy Maximum Relevance (mRMR) algorithm, is adopted. The goal of the mRMR algorithm is to find a subset of features that have the highest correlation with the target variable (cognitive state) while having the lowest cross-correlation between the features.

[0076] Implementation process: The features initially selected in step one (e.g., the top 200 in importance) are used as a candidate set, and the mRMR algorithm is applied. This algorithm iteratively selects the next feature, which must meet two conditions: maximum relevance to the target variable and minimum average mutual information with features already selected in the subset. Finally, a feature list sorted according to the mRMR criterion is obtained.

[0077] Step 3: Determine the final subset and build the evaluation model.

[0078] The optimal number of features is selected from the feature list sorted by mRMR through cross-validation.

[0079] Implementation process: Sequential Forward Selection (SFS) is employed. Starting with the feature ranked first in the mRMR (memory ratio), features are added sequentially, and a classification model (such as a Support Vector Machine (SVM) or Logistic Regression model) is trained. The model performance is evaluated using 10-fold cross-validation. When the model performance (such as AUC or accuracy) reaches its peak or no longer significantly improves, the feature combination at this point is the final "optimal subset of speech features".

[0080] Construct specific subsets for different cognitive domains: Repeat steps one through three above, but use different target labels each time. For example, to construct a "memory-specific feature subset," use the subjects' scores on memory tests (such as HVLT-R) as the target variable for training and selection. Similarly, scores on executive function tests can be used to construct an "executive function-specific feature subset."

[0081] Through this rigorous process, one or more highly optimized low-dimensional speech feature subsets closely related to specific cognitive functions are obtained, along with a corresponding, trained lightweight evaluation model.

[0082] Specifically, for the online assessment phase in Part (II), when conducting cognitive screening on a new target test subject, the following automated process is executed: (1) Guidance and voice acquisition: The system guides the user to complete a short cognitive task (such as repeating a sentence or conducting a 1-minute animal fluency test) through an interface (such as a mobile app). At the same time, the microphone on the device acquires the user's voice.

[0083] (2) Optimal feature extraction: The system no longer needs to calculate thousands of features, but directly calls the pre-stored list of "optimal speech feature subsets" and only calculates the features contained in this subset (which may only have 15-30 features). For example, the subset may include: fundamental frequency standard deviation, mean third formant frequency, variance of MFCC-2, average duration of silence, etc.

[0084] (3) Indicator calculation and input: The calculated feature values ​​are used as vectors and input into the lightweight evaluation model (such as the trained SVM model) that has been trained in the offline stage.

[0085] (4) Generation and presentation of assessment results: The model outputs a quantitative assessment score (e.g., 0-100 points) or a risk level (e.g., "low risk", "medium risk", "high risk"). The results can be displayed intuitively to the user or their guardian, and "high risk" users are advised to seek professional medical advice. The entire process can be completed in tens of seconds.

[0086] The advantages of the embodiments of this application will be illustrated below by comparing them with existing technical solutions.

[0087] In the field of cognitive function assessment, to achieve the goals of "convenience, speed, and reduced interference from dialects and educational / cultural backgrounds," various alternative solutions have been attempted in the prior art. However, each has its limitations and cannot fully cover the technical advantages of the embodiments in this application. The following analysis focuses on mainstream technical directions: (1) Electronic improvement scheme based on traditional scales: The traditional paper scales (such as MMSE, MoCA) are improved and the scores are automatically calculated by guiding the subjects to complete the questions through a digital interface. However, there are still some limitations: the time-consuming problem has not been solved and the efficiency improvement is limited; the interference of dialect and cultural background has not been eliminated.

[0088] (2) Assessment schemes based on physiological signals (such as EEG and eye-tracking): Physiological signals are collected through devices such as EEG and EOG to analyze the neural characteristics related to cognitive activities and indirectly reflect the state of cognitive function. However, there are still limitations: High equipment threshold: Professional-grade EEG caps or eye trackers are required, which are expensive and complicated to operate, making it difficult to popularize in community and primary healthcare scenarios; poor convenience; the correlation between neural characteristics and cognitive function depends on strict experimental paradigms, and the universality verification across cultural and dialect groups is insufficient.

[0089] (3) Behavioral video analysis-based approach: This approach involves collecting video data such as facial expressions and body movements of subjects through cameras, extracting behavioral features such as micro-expressions and gesture frequencies, and combining this with machine learning models to assess cognitive function. However, it still has limitations: privacy and scenario restrictions; high cultural sensitivity; and some behavioral features may be affected by physiological state, making it difficult to directly map to the cognitive function dimension.

[0090] Advantages of this application's embodiments: Compared to the above alternatives, this application's embodiments achieve cognitive function screening based on speech analysis, specifically through the following characteristics to achieve superior technical effects: Convenience: Only natural speech needs to be collected (such as dialogue and repetition tasks), without the need to wear devices or perform complex operations, meeting the needs of rapid assessment; Dialect and cultural adaptability: Focusing on the essential acoustic features of speech, avoiding dialectal semantic differences and text comprehension barriers, adapting to multi-ethnic and multi-dialect groups; Universality and interpretability: Speech features are directly driven by the physiological activities of the vocal organs (such as vocal cord vibration and respiratory control), and are not strongly correlated with education level or cultural background. Moreover, feature extraction and model inference can be achieved through general-purpose devices (such as mobile phones), facilitating large-scale promotion.

[0091] In summary, existing alternatives, due to insufficient efficiency, high equipment requirements, or cultural sensitivity, cannot simultaneously meet the core needs of "convenience, speed, and cross-dialect / cultural compatibility." The embodiments in this application, through a speech analysis technology approach, specifically address the key pain points of traditional methods, demonstrating significant practicality and innovation.

[0092] Based on the same inventive concept, this application also provides a cognitive function assessment device for implementing the cognitive function assessment method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the cognitive function assessment device provided below can be found in the limitations of the cognitive function assessment method described above, and will not be repeated here.

[0093] In one exemplary embodiment, such as Figure 5 As shown, a cognitive function assessment device is provided, comprising: The target speech data acquisition module 501 is used to acquire target speech data; the target speech data is the speech data of the target test subject when performing the target cognitive test task.

[0094] The feature extraction module 502 is used to extract features from the target speech data according to the feature categories in the preferred speech feature subset corresponding to the target cognitive test task, so as to obtain the target speech feature set.

[0095] The cognitive function assessment module 503 is used to input the target speech feature set into the cognitive function assessment model corresponding to the target cognitive test task to obtain the cognitive function assessment result of the target test subject.

[0096] The method for determining the preferred subset of speech features and the cognitive function assessment model includes: Construct a multimodal cognitive database; the multimodal cognitive database includes: speech data of test subjects when performing different forms of cognitive test tasks and corresponding cognitive function assessment results.

[0097] Extract wide-area speech features from each form of speech data in the multimodal cognitive database.

[0098] For each type of wide-area speech feature, based on the wide-area speech feature and the corresponding cognitive function assessment results, machine learning and statistical methods are used to screen features with a stronger correlation to cognitive function than a set strength, to obtain the preferred speech feature subsets corresponding to different types of cognitive test tasks, and the classification model trained based on the preferred speech feature subsets corresponding to each type of cognitive test task is determined as the corresponding cognitive function assessment model.

[0099] Compared to existing technologies (such as the MMSE / MoCA screening method, which relies on paper quality forms and requires 20-30 minutes to complete), the cognitive function assessment method or apparatus of the above embodiments has the following significant advantages: (1) The evaluation efficiency is high and the time consumption is significantly reduced.

[0100] Traditional scales (such as MMSE and MoCA) require subjects to complete 20-30 minutes of question answering and manual assessment. However, this application can reduce the time of a single cognitive screening to within 5 minutes (the specific duration depends on the efficiency of voice data collection) through automated processing of voice signals and rapid model reasoning, which greatly improves the assessment efficiency.

[0101] The core technologies of this application include a complete, data-driven method for constructing a "preferred subset of speech features", a speech feature extraction method (such as speech rate, pauses and other features can be quickly calculated from speech signals) and model inference (traditional machine learning or Transformer models can complete feature analysis and result output in milliseconds). The entire process does not require the subject to complete a large number of questions; they only need to naturally complete the speech interaction in the cognitive test.

[0102] (2) Dialects are more adaptable and reduce the impact of language barriers on the assessment results.

[0103] Traditional scales are mostly translated versions of non-native languages ​​(such as the Chinese translation of European and American scales). Dialect speakers (such as people from Cantonese and Minnan dialect regions) may have misunderstandings or errors in answering questions due to differences in language expression habits (such as vocabulary and grammar), affecting the accuracy of the assessment. However, this application can effectively avoid the interference of dialects on semantic content by analyzing the acoustic features of speech itself (such as pitch and speech rate), and adapt to the cognitive screening needs of people from multiple dialect regions.

[0104] This application directly extracts and analyzes the "essential features of speech" (such as intonation patterns, speech rate stability, and vocalization stability). These features are determined by the physiological activities of the vocal organs (such as vocal cord vibration frequency and respiratory airflow control) and are not strongly related to the semantic content of dialects. At the same time, the method of constructing "optimized subset of speech features" can further eliminate noise interference caused by differences in dialect pronunciation (such as differences in the number of tones in different dialects), ensuring the accurate extraction of core cognitive-related features (such as language fluency and reaction speed).

[0105] (3) Reduce the interference of education level and cultural background on the evaluation results.

[0106] Traditional scales rely on the subject's ability to understand the content of the questions and their cultural knowledge. Subjects with lower levels of education or different cultural backgrounds may have normal cognition but low scores due to difficulty in understanding the questions. This application reflects cognitive function indirectly through the acoustic characteristics of speech signals, without relying on the mastery of specific texts or cultural knowledge.

[0107] The core features of the model input in this application are the acoustic properties of speech (such as pause duration reflecting information processing speed and intonation changes reflecting attention concentration); at the same time, feature selection and fusion techniques (such as analyzing and screening speech features that are strongly related to cognitive labels and identifying and removing redundant speech features) can automatically filter out “false features” (such as lexical complexity) related to education level / cultural background, ensuring that the model only focuses on sound features directly related to cognitive function (such as speech performance of repeating accuracy).

[0108] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores the cognitive function assessment results of the target test subjects. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a cognitive function assessment method.

[0109] Those skilled in the art will understand that Figure 6 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0110] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0111] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0112] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0114] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0115] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0116] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for assessing cognitive function, characterized in that, The cognitive function assessment methods include: Acquire target speech data; the target speech data is the speech data of the target test subject when performing the target cognitive test task; Based on the feature categories in the preferred speech feature subset corresponding to the target cognitive test task, feature extraction is performed on the target speech data to obtain the target speech feature set; The target speech feature set is input into the cognitive function assessment model corresponding to the target cognitive test task to obtain the cognitive function assessment result of the target test subject; The method for determining the preferred subset of speech features and the cognitive function assessment model includes: Construct a multimodal cognitive database; the multimodal cognitive database includes: speech data of test subjects when performing different forms of cognitive test tasks and corresponding cognitive function assessment results; Extract wide-area speech features from each form of speech data in the multimodal cognitive database; For each type of wide-area speech feature, based on the wide-area speech feature and the corresponding cognitive function assessment results, machine learning and statistical methods are used to screen features with a stronger correlation to cognitive function than a set strength, to obtain the preferred speech feature subsets corresponding to different types of cognitive test tasks, and the classification model trained based on the preferred speech feature subsets corresponding to each type of cognitive test task is determined as the corresponding cognitive function assessment model.

2. The cognitive function assessment method according to claim 1, characterized in that, For each type of wide-area speech feature, based on the wide-area speech feature and the corresponding cognitive function assessment results, machine learning and statistical methods are used to screen features with a cognitive function correlation strength greater than a set strength, obtaining a preferred speech feature subset corresponding to different types of cognitive test tasks. The classification model trained based on the preferred speech feature subset corresponding to each type of cognitive test task is then determined as the corresponding cognitive function assessment model, specifically including: Based on the wide-area speech features and the corresponding cognitive function evaluation results, the first machine learning model is used to rank the importance of the wide-area speech features to obtain a feature importance sequence. Based on the feature importance sequence, features with importance less than a set value in the wide-area speech features are deleted to obtain a preliminary selected feature subset; Based on information theory, a feature selection algorithm is used to remove redundancy from the preliminary selected feature subset according to the correlation between features, resulting in a feature subset after redundancy removal. The features in the removed feature subset are input one by one into the second machine learning model. Cross-validation is used to select the optimal speech feature subset with the goal of achieving the best performance of the second machine learning model. The second machine learning model trained based on the optimal speech feature subset is then determined as the corresponding cognitive function assessment model.

3. The cognitive function assessment method according to claim 2, characterized in that, Based on information theory, a feature selection algorithm is used to remove redundancy from the initially screened feature subset according to the correlation between features, resulting in a feature subset after redundancy removal, specifically including: Based on information theory, the minimum redundancy maximum relevance algorithm is adopted. With the goal of maximizing the correlation between features and cognitive states and minimizing the correlation between features, the redundancy of the initially screened feature subset is eliminated to obtain the feature subset after elimination.

4. The cognitive function assessment method according to claim 2, characterized in that, The first machine learning model is a random forest model or an XGBoost model; the second machine learning model is a support vector machine or a logistic regression model.

5. The cognitive function assessment method according to claim 1, characterized in that, The wide-area speech features include: prosodic features, spectral features, and phonological features; The prosodic features include: fundamental frequency related indicators, energy related indicators, and duration related indicators; the fundamental frequency related indicators include: fundamental frequency mean, fundamental frequency median, fundamental frequency standard deviation, fundamental frequency maximum, fundamental frequency minimum, fundamental frequency variation range, and fundamental frequency jitter; the energy related indicators include: intensity mean, intensity standard deviation, intensity dynamic range, and intensity perturbation; the duration related indicators include: total speech rate, speech duration, silence duration, silence frequency, and speech-pause ratio; The spectral features include: Mel frequency cepstral coefficients, linear predictive coding coefficients, and formant correlation indices; The sound quality characteristics include: harmonic noise ratio, spectral slope, and energy distribution.

6. The cognitive function evaluation method according to claim 1, wherein, Different forms of cognitive testing tasks include: episodic memory testing tasks, language fluency testing tasks, executive function and processing speed testing tasks, and attention and working memory testing tasks.

7. A cognitive function assessment device, characterized by, The cognitive function assessment device includes: The target speech data acquisition module is used to acquire target speech data; the target speech data is the speech data of the target test subject when performing the target cognitive test task. The feature extraction module is used to extract features from the target speech data according to the feature categories in the preferred speech feature subset corresponding to the target cognitive test task, so as to obtain the target speech feature set. The cognitive function assessment module is used to input the target speech feature set into the cognitive function assessment model corresponding to the target cognitive test task to obtain the cognitive function assessment result of the target test subject. The method for determining the preferred subset of speech features and the cognitive function assessment model includes: Construct a multimodal cognitive database; the multimodal cognitive database includes: speech data of test subjects when performing different forms of cognitive test tasks and corresponding cognitive function assessment results; Extract wide-area speech features from each form of speech data in the multimodal cognitive database; For each type of wide-area speech feature, based on the wide-area speech feature and the corresponding cognitive function assessment results, machine learning and statistical methods are used to screen features with a stronger correlation to cognitive function than a set strength, to obtain the preferred speech feature subsets corresponding to different types of cognitive test tasks, and the classification model trained based on the preferred speech feature subsets corresponding to each type of cognitive test task is determined as the corresponding cognitive function assessment model.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the cognitive function assessment method according to any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, When executed by a processor, the computer program implements the cognitive function assessment method according to any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the cognitive function assessment method according to any one of claims 1-6.

Images