Cognitive disorder recognition method and device, electronic equipment and storage medium

Abstract

The invention provides a cognitive disorder recognition method and device, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring multi-modal data of a target; performing feature extraction on the multi-modal data to obtain a demographic feature and a plurality of acoustic features of the target; each acoustic feature corresponds to the category of the voice data; respectively calculating a category score corresponding to each category based on each acoustic feature; the category score is used for indicating the confidence value of the voice data belonging to the corresponding category; and performing cognitive impairment prediction on the target by using a cognitive prediction model according to the demographic characteristics of the target and the category scores to obtain a prediction result. According to the method and the device, the problem of low accuracy of cognitive impairment recognition in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a method, device, electronic device and storage medium for identifying cognitive impairment. Background Art

[0002] Mild cognitive impairment (MCI) is a cognitive function state between normal aging and dementia, which is mainly manifested by a mild decline in memory, attention or executive function, but has not yet seriously affected the ability to live in daily life. As an early stage of cognitive decline, MCI has important clinical significance. Studies have shown that the risk of MCI patients developing more serious cognitive impairments such as Alzheimer's disease is significantly higher than that of people with normal cognition. Therefore, early identification and intervention of MCI are of great value in delaying the progression of the disease.

[0003] Traditional MCI diagnostic methods usually rely on neuropsychological tests or imaging examinations, but these methods have problems such as high time cost, complex operation or high cost, which is not conducive to large-scale screening and long-term monitoring. In recent years, with the development of artificial intelligence technology, intelligent prediction models have provided new solutions for early screening and precise intervention of MCI.

[0004] However, current intelligent prediction models lack sufficient generalization capabilities and cannot adapt to the real screening needs of large-scale populations, resulting in poor classification results.

[0005] From the above, we can see that how to improve the accuracy of identifying cognitive impairment remains to be solved. Summary of the invention

[0006] The present application provides a method, device, electronic device and storage medium for identifying cognitive impairment, which can solve the problem of low accuracy of cognitive impairment identification in related technologies. The technical solution is as follows:

[0007] According to one aspect of the present application, a method for identifying cognitive impairment includes: acquiring multimodal data of a target; the multimodal data reflects speech data of the target related to cognitive impairment and demographic data related to individual health; performing feature extraction on the multimodal data to obtain demographic features and multiple acoustic features of the target; each acoustic feature corresponds to a category of the speech data; calculating a category score corresponding to each category based on each acoustic feature; the category score is used to indicate a confidence value that the speech data belongs to the corresponding category; using a cognitive prediction model to predict cognitive impairment of the target according to the demographic features and each category score of the target to obtain a prediction result; the cognitive prediction model is obtained based on the initial logistic regression model training.

[0008] According to one aspect of the present application, a cognitive impairment identification device includes: a data acquisition module, which is used to acquire multimodal data of a target; the multimodal data reflects the target's speech data related to cognitive impairment and demographic data related to individual health; a feature processing module, which is used to extract features from the multimodal data to obtain the target's demographic features and multiple acoustic features; each of the acoustic features corresponds to a category of the speech data; a score calculation module, which is used to calculate the category score corresponding to each of the categories based on each of the acoustic features; the category score is used to indicate the confidence value that the speech data belongs to the corresponding category; a cognitive prediction module, which is used to predict cognitive impairment of the target according to the target's demographic features and each of the category scores using a cognitive prediction model to obtain a prediction result; the cognitive prediction model is obtained based on the initial logistic regression model training.

[0009] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method for identifying cognitive impairment as described above is implemented.

[0010] According to one aspect of the present application, a storage medium stores a computer program thereon, and when the computer program is executed by one or more processors, the method for identifying cognitive impairment as described above is implemented.

[0011] According to one aspect of the present application, a computer program product includes a computer program, and when the computer program is executed by one or more processors, the method for identifying cognitive impairment as described above is implemented.

[0012] The beneficial effects of the technical solution provided by this application are:

[0013] In the above technical solution, the classification accuracy of MCI is effectively improved by combining the output of the audio learning model with demographic data. At the same time, the cognitive recognition model based on the logistic regression model is used to process multimodal data, which significantly reduces the computational complexity and makes it suitable for large-scale screening applications.

[0014] In addition, through the automated feature extraction and classification process, the need for manual intervention is reduced, the screening efficiency is improved, and it has good generalization ability and can adapt to the needs of different populations and scenarios. Finally, the re-classification takes into account demographic factors, further improving the accuracy and practicality of MCI prediction and increasing the scope of application of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0016] Figure 1 is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0017] Figure 2 is a flow chart of a method for identifying cognitive impairment according to an exemplary embodiment;

[0018] Figure 3 yes Figure 2 A flowchart of step 330 in one embodiment corresponding to the embodiment;

[0019] Figure 4 yes Figure 3 A flowchart of step 333 in one embodiment corresponding to the embodiment;

[0020] Figure 5 yes Figure 2 A flowchart of one embodiment of the steps of training the audio learning model in the corresponding embodiment;

[0021] Figure 6 It is a schematic diagram of a specific implementation of a cognitive impairment identification method in an application scenario;

[0022] Figure 7 yes Figure 6 A schematic diagram of a specific implementation of the speech data processing flow involved in the corresponding embodiment;

[0023] Figure 8 is a structural block diagram of a cognitive impairment identification device according to an exemplary embodiment;

[0024] Fig. 9 It is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0025] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0026] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0027] As mentioned earlier, intelligent prediction models provide new solutions for early screening and precise intervention of MCI.

[0028] Intelligent prediction models based on data such as speech, behavior, and EEG provide new solutions for early screening and precise intervention of MCI. Among them, speech, as a non-invasive and easily accessible biological signal, can effectively reflect the changing trend of cognitive function by extracting the acoustic features of the patient's speech, and has become one of the important research directions for identifying MCI.

[0029] However, on the one hand, current technologies mostly rely on small sample data for model training, lack sufficient generalization capabilities, and cannot adapt to the real screening needs of large-scale populations, resulting in poor classification results; on the other hand, they are highly dependent on manual engineering in the feature extraction and cleaning process, which is inefficient and difficult to meet the automation and rapidity requirements of large-scale screening. In addition, the features related to health status are not fully considered during model training, which limits the accuracy and applicability of the model.

[0030] From the above, it can be seen that the relevant technology still has the defect of low accuracy in identifying cognitive impairment.

[0031] To this end, the cognitive impairment identification method provided in the present application can effectively improve the accuracy of cognitive impairment identification. Accordingly, the cognitive impairment identification method is suitable for a cognitive impairment identification device, which can be deployed in an electronic device. The electronic device can be a computer device configured with a von Neumann architecture, for example, the computer device includes a desktop computer, a laptop computer, a server, etc.

[0032] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0033] See also Figure 1 , Figure 1 The figure is a hardware structure diagram of a server according to an exemplary embodiment.

[0034] It should be noted that the server is only an example adapted to the present application and cannot be considered to provide any limitation on the scope of use of the present application. The server cannot be interpreted as requiring dependence on or having Figure 1 One or more components of exemplary server 200 are shown.

[0035] The hardware structure of the server 200 may vary greatly due to differences in configuration or performance, such as Figure 1 As shown, the server 200 includes: a power supply 210 , an interface 230 , at least one memory 250 , and at least one central processing unit (CPU) 270 .

[0036] Specifically, the power supply 210 is used to provide operating voltage for each hardware device on the server 200 .

[0037] The interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices.

[0038] Of course, in other examples adapted by the present application, the interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input-output interface 235, and at least one USB interface 237, etc. Figure 1 As shown, this is not intended to be a specific limitation.

[0039] The memory 250 is a carrier for storing resources, which may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon include an operating system 251, an application 253 and data 255, etc. The storage method may be temporary storage or permanent storage.

[0040] Among them, the operating system 251 is used to manage and control the hardware devices and application programs 253 on the server 200 to enable the central processor 270 to calculate and process the massive data 255 in the memory 250. It can be WindowsServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0041] The application program 253 is a computer program formed by computer-readable instructions based on the operating system 251 to complete at least one specific task, and may include at least one module ( Figure 1 Each module may include corresponding computer-readable instructions. For example, the cognitive impairment identification device may be regarded as an application 253 deployed on the server 200.

[0042] The data 255 may be photos, pictures, etc. stored in a disk, or may be multimodal data, etc., stored in the memory 250 .

[0043] The central processor 270 may include one or more processors and is configured to communicate with the memory 250 through at least one communication bus to read the computer program stored in the memory 250, thereby realizing the operation and processing of the mass data 255 in the memory 250. For example, the cognitive impairment identification method is completed in the form of the central processor 270 reading the application program 253 stored in the memory 250.

[0044] In addition, the present application can also be implemented through hardware circuits or hardware circuits combined with software. Therefore, the implementation of the present application is not limited to any specific hardware circuits, software, or a combination of the two.

[0045] See also Figure 2 The embodiment of the present application provides a method for identifying cognitive impairment, which is applicable to an electronic device, and the hardware structure of the electronic device can be as follows Figure 1 shown.

[0046] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0047] like Figure 2 As shown, the method may include the following steps:

[0048] Step 310: Acquire multimodal data of the target.

[0049] The target is the object of cognitive impairment identification, which can be a patient or a healthy ordinary person, and there is no limitation here.

[0050] Regarding multimodal data, it reflects the target's voice data related to cognitive impairment and demographic data related to individual health. Among them, voice data can reflect the target's cognitive function status in terms of voice expression, thinking fluency and memory ability; demographic data can include the target's basic health information, such as age, gender, education level, living habits and medical history, which can provide important background basis for risk assessment of cognitive impairment. Some demographic information has a strong correlation with the occurrence of cognitive impairment.

[0051] Step 330 , extracting features from the multimodal data to obtain demographic features and multiple acoustic features of the target.

[0052] Among them, each acoustic feature corresponds to a category of speech data.

[0053] First of all, it should be noted that multimodal data can be classified using machine learning methods, such as random forests and support vector machines, without limitation here.

[0054] In one possible implementation, such as Figure 3 As shown, step 330 includes the following steps:

[0055] Step 331, structurally process the demographic data in the multimodal data to obtain demographic features.

[0056] Specifically, structured processing includes cleaning, standardizing and encoding the target's demographic data (such as age, gender, education level, medical history, etc.); through feature selection and transformation, the demographic data is converted into demographic features that can be used by the model, such as numerical, categorical or embedded feature representations, to form clear demographic features for input into the model for analysis.

[0057] In a possible implementation, the demographic characteristics include at least one of years of education, gender, and age.

[0058] Step 333: confirm the category according to the voice data to obtain at least one category.

[0059] Among them, acoustic features can describe the physical properties of sound, such as frequency, amplitude, power spectrum, etc., and can also describe the human ear's perception and understanding of sound, which is in line with human auditory characteristics, such as speaking speed, pause duration and distribution, and speech clarity.

[0060] Through acoustic features, we can explore the subtle sound properties from the physical dimension and capture the potential cognitive and emotional clues in the sound from the perceptual dimension. The dual descriptive capabilities of physics and perceptualness enable acoustic features to play an important role in the assessment and screening of cognitive disorders.

[0061] The categories include at least one of speech fluency test speech, logical memory test speech, and Hopkins language learning test speech.

[0062] Regarding the Speech Fluency Test, the Speech Fluency Test assesses the target's ability to express words fluently within a limited time, such as saying related words at the beginning of a specific theme or letter. The Logical Memory Test assesses the speaker's memory ability, such as retelling the sentences or stories heard, and recording the accuracy and completeness; the Hopkins Language Learning Test assesses language learning ability, such as identifying similar words or repeating certain phrases.

[0063] It can be understood that the above categories reflect the target's cognitive level in different aspects and can provide important basis for subsequent cognitive identification.

[0064] In another possible implementation, a classification model may be used to automatically perform category determination based on the context and task characteristics of the speech data.

[0065] Step 335 , based on each category, feature extraction processing is performed on the speech data to obtain acoustic features corresponding to each category.

[0066] First, it is noted that the feature extraction process may include performing feature extraction on speech data using an audio learning model to capture the time domain and frequency domain characteristics of the speech data, such as a Mel-spectrogram.

[0067] Then, for each category, the audio learning model can extract relevant acoustic features. Specifically, for the speech fluency test speech, the acoustic features such as speaking rate, pause distribution, and pitch smoothness are extracted; for the logical memory test speech: the acoustic features such as key content vocabulary and semantic consistency of the speech are extracted; for the Hopkins Language Learning Test speech: the acoustic features such as speech clarity and speech repetition quality are extracted.

[0068] Different acoustic features can help analyze the target's potential obstacles in different cognitive functions. For example, the distribution of pauses reflected in the speech fluency test may indicate language production difficulties or memory retrieval problems if the frequent and long pauses are frequent; the speech clarity reflected in the Hopkins Language Learning Test may be related to the target's decreased control of the pronunciation organs or language expression ability; the semantic consistency reflected in the logical memory test may indicate that the target's language comprehension or logical reasoning function is impaired if the target's expression is incoherent or logically confusing, etc., which are not specifically limited here.

[0069] In one possible implementation, step 3333 also includes the following steps: performing power spectrum calculation on the speech data based on each category to obtain power spectrum information corresponding to each category; extracting Mel-spectrograms of the power spectrum information to obtain acoustic features corresponding to each category.

[0070] First, it should be noted that before calculating the power spectrum of speech data based on each category, the speech data needs to be classified to ensure that the subsequent acoustic feature calculation can correspond to the category. Specifically, according to the above categories, speech data corresponding to different categories can be obtained.

[0071] Then, the power spectrum calculation can be performed on the speech data corresponding to different categories respectively. Specifically, the following steps can be included: the original, unprocessed speech data is pre-emphasized, frame-divided and windowed to obtain processed speech data; each frame of speech data is converted from the time domain to the frequency domain by fast Fourier transform to obtain the frequency characteristics corresponding to the speech data; the frequency domain characteristics are processed by absolute value square to obtain the power spectrum.

[0072] Furthermore, after obtaining the power spectrum, the power spectrum can be processed by Mel filters to generate spectrum information that conforms to human auditory characteristics, the spectrum information can be processed by absolute value square, and the processed spectrum information can be output as a Mel spectrum graph, so as to mark the corresponding Mel spectrum graph for each category, thereby obtaining the acoustic features corresponding to different categories.

[0073] In addition, a deep learning model may be selected for extracting acoustic features, such as LSTM, GRU, etc., which is not limited here.

[0074] Through the above process, the multimodal data of speech data can be comprehensively utilized to extract key acoustic features reflecting the individual's cognitive state from the speech data, thereby improving the model's ability to recognize and predict the individual's cognitive state.

[0075] Step 350: Calculate the category score corresponding to each category based on each of the acoustic features.

[0076] The category score is used to indicate the confidence value that the speech data belongs to the corresponding category. The confidence value refers to the confidence level used by the classification model to measure the judgment of a certain category during the prediction process. The category score can be a logits score, which is not limited here.

[0077] In one possible implementation, each acoustic feature can be processed by a multi-layer neural network of a deep learning model, namely an audio learning model, the network structure of which is a RESNET18 network, and finally a category score is output.

[0078] Step 370: Use the cognitive prediction model to predict cognitive impairment of the target based on the target's demographic characteristics and scores of each category to obtain a prediction result.

[0079] The cognitive prediction model is trained based on the initial logistic regression model. It should be noted that the logistic regression model is selected as the basic model of the cognitive prediction model because it can effectively process high-dimensional features and provide clear classification results. At the same time, the logistic regression model has low computational complexity and is suitable for processing multimodal data.

[0080] Since there are multiple category scores and demographic characteristics, feature fusion is required to make full use of the above information.

[0081] Regarding feature fusion, one possible implementation method includes fusing the scores of each category with demographic characteristics to obtain fused features; and predicting cognitive impairment of the target based on the fused features through a cognitive prediction model.

[0082] Specifically, demographic characteristics and category scores can be unified into a comprehensive representation, namely, fused features, through weighted averaging, concatenation or other feature aggregation methods.

[0083] For example, demographic features and category scores can be directly concatenated into a long vector, and the concatenated fusion features can be used in cognitive prediction models, which can process high-dimensional data and automatically learn complex relationships between features.

[0084] Figure 4 A specific implementation of a cognitive prediction model is shown in FIG. Figure 4 As shown in the figure, the demographic characteristics include the target's gender, age and years of education. The category features corresponding to the speech fluency test speech, the logical memory test speech and the Hopkins language learning test speech are respectively calculated by the audio learning model RESNET18 network to obtain the corresponding logits scores. The logits scores corresponding to the speech fluency test speech, the logical memory test speech and the Hopkins language learning test speech are fused with the gender, age and years of education to obtain the fused features. The logistic regression method is used to predict cognitive impairment based on the fused features.

[0085] In one possible implementation, the cognitive impairment prediction process of the cognitive prediction model may include: predicting the cognitive state of the target using a supervised binary classification method based on the fusion features of the target, wherein the cognitive state may include a normal state and mild cognitive impairment, which are not limited here.

[0086] Based on this, by using the cognitive prediction model to make cognitive predictions on the target, the corresponding prediction results can be obtained, and the prediction results can indicate whether the target suffers from cognitive impairment.

[0087] In one possible implementation, the training process of the cognitive prediction model may include the following steps: obtaining a large-scale sample data set, dividing the sample data set into a validation set, a training set, and a test set; constructing a cognitive prediction model based on an initial logistic regression model; training the constructed cognitive prediction model using the training set, adjusting the parameters of the cognitive prediction model based on the validation set, and evaluating the prediction performance of the cognitive prediction model using the test set until the prediction performance indicates that the cognitive prediction model has completed training.

[0088] First of all, it should be noted that large-scale sample datasets include multimodal data of multiple samples, such as voice data, demographic data, medical records, etc. These data can correspond to different cognitive states, such as healthy people, mild cognitive impairment (MCI) patients, Alzheimer's disease (AD) patients, etc., thereby ensuring the diversity and scale of the dataset and improving the generalization ability of the cognitive prediction model.

[0089] Regarding the division of the sample data set, it can be divided into 60% training set, 20% validation set and 20% test set, and the specific proportion is determined according to the actual situation.

[0090] Furthermore, the validation set is used to adjust the parameters of the cognitive prediction model, such as learning rate, batch size, etc., so that according to the results of the validation set, it may be necessary to adjust the parameters of the cognitive prediction model, increase or decrease features, change the model structure, etc., to improve the generalization ability of the cognitive prediction model.

[0091] Then, use the test set to evaluate the predictive performance of the cognitive prediction model, calculate various indicators on the test set, such as accuracy, recognition rate, F1 score, etc., and evaluate the predictive performance of the model to determine whether the cognitive prediction model has completed training. For example, if the accuracy of the cognitive prediction model on the test set is 95%, it is considered to have completed training.

[0092] Through the above process, by including multimodal data and covering different cognitive states, the diversity of large-scale data sets helps to improve the generalization ability of the model, so that it can better adapt to different practical application scenarios. The logistic regression model is selected as the basic model, which can effectively process high-dimensional features and provide clear classification results. At the same time, the model has low computational complexity and is suitable for processing multimodal data.

[0093] In combination with the above embodiments, the classification accuracy of MCI is effectively improved by combining the category scores and demographic data. At the same time, the cognitive recognition model based on the logistic regression model is used to process multimodal data, which significantly reduces the computational complexity and makes it suitable for large-scale screening applications.

[0094] In addition, through the automated feature extraction and classification process, the need for manual intervention is reduced, the screening efficiency is improved, and it has good generalization ability and can adapt to the needs of different populations and scenarios. Finally, the re-classification takes into account demographic factors, further improving the accuracy and practicality of MCI prediction and increasing the scope of application of the model.

[0095] See also Figure 5 In an exemplary embodiment, the training process of the audio deep learning model may include the following steps:

[0096] Step 410, obtaining sample data.

[0097] The sample data includes sample categories and corresponding sample acoustic features;

[0098] First of all, it is explained that a pre-trained residual network model is selected as a basic network to construct an audio learning model, so as to use the audio learning model to calculate the category scores corresponding to each of the categories based on each of the acoustic features.

[0099] In one possible implementation, a residual network (ResNet) pre-trained on a large-scale dataset is selected as the base network. ResNet alleviates the gradient vanishing problem through a residual block structure and is suitable for efficiently processing Mel-spectrograms (i.e., acoustic features).

[0100] In one possible implementation, an audio learning model can be constructed through transfer learning, which can include the following steps: processing tasks according to sample acoustic features, adjusting input layers, adjusting output layers, freezing some layers, fine-tuning the model, and evaluating and optimizing the model.

[0101] Step 430 , selecting a cross entropy loss function as an optimization target for the constructed audio learning model.

[0102] It should be noted that the cross entropy loss function is chosen as the optimization objective because it is suitable for classification problems and can effectively measure the difference between model predictions and actual labels.

[0103] The optimization goal is to train the model to improve the accuracy of classification or prediction by minimizing the cross entropy loss function.

[0104] Step 450 , training the audio learning model using the sample acoustic features according to the optimization goal.

[0105] During the training process, the sample acoustic features are input into the audio learning model, the difference between the predicted output and the true label is calculated using the selected optimization objective (cross entropy loss function), and the model parameters are adjusted through back-propagation.

[0106] In addition, an appropriate learning rate can be initialized to control the step size of each model parameter update to balance the model convergence speed and stability.

[0107] Step 470, during the training process, using an adaptive moment estimation optimization method to accelerate the gradient descent process and improve the convergence performance of the audio learning model, until the convergence performance indicates that the trained audio learning model is obtained.

[0108] It should be noted that in one possible implementation, a dynamic learning rate adjustment strategy is introduced. Specifically, the learning rate is dynamically adjusted according to the gradient changes during training in combination with an adaptive moment estimation optimization method (such as the Adam optimizer), ensuring that the loss value can be quickly reduced in the early stage of training, and that fine-tuned adjustments can be made in the later stage of training to improve the final performance of the model.

[0109] Under the effect of the above embodiment, the residual network is fine-tuned through transfer learning to adapt to the characteristics of speech data. The cross entropy loss function is selected as the optimization target, and the training process is accelerated through the adaptive optimization method to ensure that the model can converge quickly and obtain optimized results.

[0110] Figure 6 This is a schematic diagram of a specific implementation of a cognitive impairment identification method in an application scenario. In this application scenario, multimodal data is collected from multiple hospitals with unified standards, with no significant differences in recording instruments and quality, and no differences in adaptation among participants; the positive sample label is "MCI", the positive sample size is 173, and the negative sample label is "NC", and the negative sample size is 239.

[0111] Now combined Figure 6 The following describes the recognition of cognitive impairment in this application scenario:

[0112] Through step 801, feature processing is performed on the voice data and the demographic data respectively.

[0113] Through step 803, the demographic data is preprocessed.

[0114] Among them, preprocessing includes the processing of outliers, missing values, and duplicate values. The distribution of demographic data is as follows.

[0115]

[0116] Through step 805, the demographic data is structured to obtain demographic characteristics.

[0117] Demographic characteristics included gender, age, and years of education.

[0118] Through step 807, three different types of speech data are input, and each type of speech data is processed using the deep learning model ResNet18 to obtain the corresponding logits score.

[0119] Specifically, regarding the processing of voice data, Figure 7 A specific implementation diagram of a voice data processing flow is shown, such as Figure 7 As shown, the steps of processing voice data may include:

[0120] (1) Pre-emphasis, framing and windowing: The original audio signal is pre-emphasized to enhance the high-frequency components. The original audio signal is then framed and windowed to reduce the impact of signal truncation on the spectrum.

[0121] (2) Fast Fourier Transform (FFFT): The audio signal of each frame is converted from the time domain to the frequency domain through the fast Fourier transform to obtain the frequency signal.

[0122] (3) Signal absolute value square: The transformed frequency signal is processed by absolute value square to obtain the power spectrum, which is further used for feature extraction.

[0123] (4) Mel filter: The power spectrum is passed through a Mel filter to generate a spectrum graph based on human auditory characteristics.

[0124] (5) Squaring the absolute value of the signal: Perform an absolute value square operation on the spectrum graph passing through the Mel filter to calculate the energy.

[0125] (6) Output Mel-spectrogram: The processed signal is output as a Mel-spectrogram, which is the acoustic feature.

[0126] (7) Use the deep learning model to calculate based on each acoustic feature and obtain the corresponding logits score.

[0127] Through step 809, the logits score and the demographic feature are fused by splicing to form a fused feature.

[0128] Through step 813, the fused features are classified using a logistic regression model to output a prediction result of mild cognitive impairment.

[0129] In this application scenario, an end-to-end deep learning algorithm can be used to directly classify the MCI of the population to be tested, thereby eliminating the manual dependence on the intermediate feature extraction link, realizing an automated and rapid screening process, and comprehensively considering demographic characteristics (age, gender, and years of education) to improve the accuracy of MCI prediction, thereby enhancing the practicality and reliability of the model. In addition, this application scenario can also be extended to other fields such as cognitive impairment screening and mental health assessment, further enhancing its clinical application value.

[0130] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0131] The following is an embodiment of the device of the present application, which can be used to execute the cognitive impairment identification method involved in the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of the cognitive impairment identification method involved in the present application.

[0132] See also Figure 8 In an embodiment of the present application, a cognitive impairment identification device 900 is provided, including but not limited to: a data acquisition module 910, a feature processing module 930, a score calculation module 950, and a cognitive prediction module 950.

[0133] The data acquisition module 910 is used to acquire multimodal data of the target; the multimodal data reflects the target's speech data related to cognitive impairment and demographic data related to individual health;

[0134] The feature processing module 930 is used to extract features from the multimodal data to obtain demographic features and multiple acoustic features of the target; each acoustic feature corresponds to a category of the speech data;

[0135] A score calculation module 950 is used to calculate the category score corresponding to each category based on each acoustic feature; the category score is used to indicate the confidence value that the speech data belongs to the corresponding category;

[0136] The cognitive prediction module 950 is used to predict cognitive impairment of the target according to the demographic characteristics and scores of each category of the target using a cognitive prediction model to obtain a prediction result; the cognitive prediction model is obtained based on the initial logistic regression model training.

[0137] It should be noted that the cognitive impairment recognition device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing cognitive impairment recognition. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the cognitive impairment recognition device will be divided into different functional modules to complete all or part of the functions described above.

[0138] In addition, the cognitive impairment identification device and the cognitive impairment identification method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs operations has been described in detail in the method embodiment and will not be repeated here.

[0139] See also Fig. 9 In an embodiment of the present application, an electronic device 4000 is provided, and the electronic device 4000 may include: a desktop computer, a laptop computer, a server, etc.

[0140] exist Fig. 9 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0141] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0142] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0143] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0144] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store a computer program in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to this.

[0145] The memory 4003 stores a computer program, and the processor 4001 can read the computer program stored in the memory 4003 through the communication bus 4002 .

[0146] The computer program is executed by one or more processors 4001 to implement the cognitive impairment identification method in the above-mentioned embodiments.

[0147] In addition, an embodiment of the present application provides a storage medium on which a computer program is stored. The computer program is executed by one or more processors to implement the cognitive impairment identification method as described above.

[0148] A computer program product is provided in an embodiment of the present application, including a computer program, which is executed by one or more processors to implement the cognitive impairment identification method as described above.

[0149] Compared with related technologies, the classification accuracy of MCI is effectively improved by combining the output of the audio learning model with demographic data. At the same time, the recognition model based on the logistic regression model is used to process multimodal data, which significantly reduces the computational complexity and makes it suitable for large-scale screening applications.

[0150] In addition, through the automated feature extraction and classification process, the need for manual intervention is reduced, the screening efficiency is improved, and it has good generalization ability and can adapt to the needs of different populations and scenarios. Finally, the re-classification takes into account demographic factors, further improving the accuracy and practicality of MCI prediction and increasing the scope of application of the model.

[0151] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying cognitive impairment, characterized in that: The method comprises: Acquiring multimodal data of a target; the multimodal data reflecting speech data of the target related to cognitive impairment and demographic data related to individual health; Performing feature extraction on the multimodal data to obtain demographic features and a plurality of acoustic features of the target; each of the acoustic features corresponds to a category of the speech data; Calculating a category score corresponding to each of the categories based on each of the acoustic features; the category score is used to indicate a confidence value that the speech data belongs to the corresponding category; The cognitive prediction model is used to predict cognitive impairment of the target according to the demographic characteristics of the target and the scores of each category to obtain a prediction result; the cognitive prediction model is obtained based on the initial logistic regression model training.

2. The method according to claim 1, characterized in that The feature extraction of the multimodal data to obtain demographic features and multiple acoustic features of the target includes: Performing structural processing on the demographic data in the multimodal data to obtain demographic characteristics; Perform category confirmation according to the speech data to obtain at least one category; the category includes at least one of speech fluency test speech, logical memory test speech, and Hopkins language learning test speech; Based on each of the categories, feature extraction processing is performed on the speech data to obtain acoustic features corresponding to each of the categories.

3. The method according to claim 2, characterized in that The feature extraction process is performed on the speech data based on each of the categories to obtain acoustic features corresponding to each of the categories, including: Based on each of the categories, power spectrum calculation is performed on the speech data to obtain power spectrum information corresponding to each of the categories; The mel-spectrograms of the power spectrum information are extracted respectively to obtain the acoustic features corresponding to the categories.

4. The method according to claim 1, characterized in that The using a cognitive prediction model to predict cognitive impairment of the target according to the demographic characteristics of the target and each of the category scores comprises: Performing feature fusion on each of the category scores and the demographic features to obtain fused features; The cognitive prediction model is used to predict cognitive impairment of the target according to the fusion features.

5. The method according to claim 1, characterized in that Selecting a pre-trained residual network model as a basic network to construct an audio learning model, so as to use the audio learning model to calculate the category scores corresponding to each of the categories based on each of the acoustic features; The training process of the audio deep learning model includes: Acquiring sample data, wherein the sample data includes sample categories and corresponding sample acoustic features; Selecting a cross entropy loss function as an optimization target for the constructed audio learning model; According to the optimization goal, the audio learning model is trained using the sample acoustic features; During the training process, an adaptive moment estimation optimization method is used to accelerate the gradient descent process and improve the convergence performance of the audio learning model until the convergence performance indicates that a trained audio learning model is obtained.

6. The method according to claim 1, characterized in that The training process of the cognitive prediction model includes: Obtain a large-scale sample data set, and divide the sample data set into a validation set, a training set, and a test set; Based on the initial logistic regression model, a cognitive prediction model was constructed; The constructed cognitive prediction model is trained using the training set, and the parameters of the cognitive prediction model are adjusted according to the validation set, and the prediction performance of the cognitive prediction model is evaluated using the test set until the prediction performance indicates that the cognitive prediction model has completed training.

7. The method according to any one of claims 1 to 6, characterized in that: The demographic data includes at least one of years of education, gender, and age.

8. A cognitive impairment identification device, characterized in that: include: A data acquisition module, used to acquire multimodal data of the target; The multimodal data reflects the target's speech data related to cognitive impairment and demographic data related to individual health; A feature processing module, used for extracting features from the multimodal data to obtain demographic features and multiple acoustic features of the target; Each of the acoustic features corresponds to a category of the speech data; A score calculation module, used to calculate the category score corresponding to each of the categories based on each of the acoustic features; the category score is used to indicate the confidence value that the speech data belongs to the corresponding category; The cognitive prediction module is used to predict cognitive impairment of the target according to the demographic characteristics of the target and the scores of each category using a cognitive prediction model to obtain a prediction result; the cognitive prediction model is obtained based on the initial logistic regression model training.

9. An electronic device comprising at least one processor and at least one memory, wherein: The memory stores a computer program, wherein when the computer program is executed by the processor, the method for identifying cognitive impairment according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by one or more processors, the method for identifying cognitive impairment according to any one of claims 1 to 7 is implemented.

Images