A cognitive function assessment and training method and device for the elderly

Abstract

The present invention discloses a method and device for cognitive function assessment and training of the elderly. The method includes: when receiving a nuclear magnetic resonance image and user data uploaded by an elderly user on a question-and-answer webpage, performing a cognitive initial examination, generating an initial examination assessment result and a cognitive function type, if the cognitive function type is any cognitive impairment type, extracting multiple resonance image features from the nuclear magnetic resonance image, convolving the initial examination assessment result with all the resonance image features, generating a fusion feature, performing a cognitive re-examination based on the fusion feature, generating a re-examination assessment result and displaying corresponding cognitive training questions on the question-and-answer webpage, responding to answer information input for the cognitive training questions, updating the cognitive training questions, and performing cognitive function training on the elderly user. Thus, the cognitive function of the elderly is comprehensively assessed by means of multimodal data, and suitable cognitive training questions are selected based on the assessment results to conduct multiple rounds of related question-and-answer training.

Description

Technical Field

[0001] The present invention relates to the technical field of cognitive function assessment, and in particular to a cognitive function assessment training method and device for the elderly. Background Art

[0002] Currently, the clinical diagnosis of cognitive impairment primarily relies on physician judgment, cognitive function scale tests, and brain scans. However, these tests are typically performed only when symptoms appear, often missing the optimal time for intervention. Furthermore, traditional diagnostic methods are often influenced by the physician's subjective experience and judgment, making misdiagnosis a common occurrence. Research indicates that trained deep learning models have an accuracy rate of 90.2% in identifying Alzheimer's disease, while the clinical detection rate achieved by humans is only 77%. Furthermore, research indicates that cognitive digital therapy can effectively improve cognitive impairment caused by various causes.

[0003] However, due to the recent development of cognitive digital therapy, many issues remain to be addressed. For example, cognitive digital therapy is often only available in medical institutions and is often only available to patients with confirmed cognitive impairment. Furthermore, current cognitive digital therapy often takes the form of a single round of dialogue or question-and-answer sessions, allowing users to guess or guess at the answers, which reduces the accuracy of cognitive impairment assessments. Summary of the Invention

[0004] This invention provides a method and device for cognitive function assessment and training in the elderly. This addresses the technical issue that digital cognitive therapy is often only available in medical institutions and is often only available to patients with confirmed cognitive impairment. Furthermore, current digital cognitive therapy programs often involve single-round conversations or question-and-answer sessions, allowing users to guess or guess at questions, which can reduce the accuracy of cognitive impairment assessments.

[0005] The present invention provides a cognitive function assessment and training method for the elderly, comprising:

[0006] When the MRI image and user data uploaded by the elderly user are received on the Q&A webpage, a preliminary cognitive examination is performed to generate preliminary assessment results and cognitive function types;

[0007] If the cognitive function type is any cognitive impairment type, extracting multiple resonance image features from the nuclear magnetic resonance image and concatenating them with the convolution result of the initial examination assessment result to generate a fusion feature;

[0008] Perform cognitive re-examination based on the fusion features, generate re-examination evaluation results, and display corresponding cognitive training questions on the question-and-answer webpage;

[0009] In response to the answer information input for the cognitive training question, the cognitive training question is updated using a preset target question and answer data set to perform cognitive function training on the elderly user;

[0010] If the cognitive function type is a normal cognitive type, the process jumps to the step of displaying corresponding cognitive training questions on the question-and-answer webpage.

[0011] Optionally, when the MRI image and user data uploaded by the elderly user are received on the Q&A webpage, a preliminary cognitive examination is performed to generate preliminary assessment results and cognitive function types, including:

[0012] When receiving an MRI image and user data uploaded by an elderly user on the Q&A webpage, performing a cognitive function assessment on the MRI image using a preset image assessment model to determine a first preliminary examination probability vector corresponding to the elderly user;

[0013] Performing vectorization conversion on the user data to generate a first statistical feature;

[0014] Using the first statistical feature to perform cognitive function assessment using multiple preset feature assessment models, and generating second preliminary examination probability vectors corresponding to the elderly users respectively;

[0015] After calculating the sum of the first initial detection probability vector and all the second initial detection probability vectors, calculating the mean probability vector as the cognitive initial detection result corresponding to the elderly user;

[0016] The corresponding cognitive type is matched according to the highest element probability in the mean probability vector as the cognitive function type corresponding to the elderly user.

[0017] Optionally, the user data includes income data, occupation data, educational background data, and multiple feature data; and the vectorizing the user data to generate the first statistical feature includes:

[0018] Performing feature conversion on each of the feature data according to preset rules to obtain initial features;

[0019] According to a preset conversion table, the income data, the occupation data, and the educational background data are respectively mapped into specific values, accumulated, and normalized to obtain socioeconomic status characteristics;

[0020] The initial features and the socioeconomic status features are combined to generate first statistical features.

[0021] Optionally, when the MRI image and user data uploaded by the elderly user are received on the Q&A webpage, a preliminary cognitive examination is performed to generate preliminary assessment results and cognitive function types, including:

[0022] When receiving the MRI image and user data uploaded by the elderly user on the Q&A webpage, encoding is performed according to the variable type in the user data to generate a second statistical feature;

[0023] Dividing the nuclear magnetic resonance image into blocks according to a preset size, and flattening each block into image block vectors;

[0024] Performing linear mapping on each of the image block vectors and concatenating them to obtain an image embedding vector;

[0025] Concatenating the image embedding vector and the second statistical feature to obtain an overall embedding vector;

[0026] Inputting the overall embedding vector into a plurality of encoder modules respectively to obtain a plurality of encoding vectors respectively;

[0027] Passing all the encoding vectors through the fully connected layer and the classification layer in sequence to generate a preliminary inspection evaluation result consisting of multiple detection probabilities;

[0028] The cognitive type corresponding to the match with the highest detection probability is selected as the cognitive function type corresponding to the elderly user.

[0029] Optionally, the encoder module comprises a word embedding layer, a first normalization layer, a kernel attention layer, a first multiplier layer, a second normalization layer, a multilayer perceptron layer, and a second multiplier layer connected in sequence;

[0030] There is a short-circuit connection between the output position of the word embedding layer and the input position of the first multiplier layer, and there is a short-circuit connection between the output position of the first multiplier layer and the input position of the second multiplier layer.

[0031] Optionally, if the cognitive function type is any cognitive impairment type, multiple resonance image features are extracted from the nuclear magnetic resonance image and concatenated with the convolution result of the initial examination assessment result to generate a fusion feature, including:

[0032] If the cognitive function type is any cognitive impairment type, calling multiple CNN network models with different resolutions to extract corresponding initial feature maps from the magnetic resonance image respectively;

[0033] Performing bilinear interpolation on each of the initial feature maps to generate an intermediate feature map with the same resolution;

[0034] Splicing and convolving the intermediate feature maps to generate a multi-scale feature map;

[0035] The multi-scale feature map is spliced ​​with each of the intermediate feature maps and then input into a self-attention layer to generate a resonant image feature;

[0036] The resonance image feature is concatenated with the convolution result of the initial inspection evaluation result to generate a fusion feature.

[0037] Optionally, performing cognitive re-examination based on the fusion features, generating re-examination evaluation results and displaying corresponding cognitive training questions on the question-and-answer webpage includes:

[0038] Classify the fused features in sequence through the fully connected layer to obtain a classification vector corresponding to the fused features;

[0039] Performing batch normalization on the classification vector through a batch normalization layer to obtain a batch normalized vector;

[0040] Mapping the batch normalized vector into output features through an activation function;

[0041] The outputtable feature is used as a new fusion feature, and the step of sequentially classifying the fusion feature through the fully connected layer to obtain a fusion probability vector corresponding to the fusion feature is skipped until the number of skips reaches a preset threshold;

[0042] The classification layer classifies the output features at the current moment and calculates the corresponding probability values, generating a probability vector as the re-inspection evaluation result;

[0043] According to the re-examination and evaluation results, the corresponding cognitive type is matched and the corresponding cognitive training questions are selected and displayed on the question-and-answer webpage.

[0044] Optionally, in response to the answer information input for the cognitive training question, updating the cognitive training question using a preset target question and answer data set includes:

[0045] In response to the answer information input for the cognitive training question, calculating the answer similarity between the answer information and the standard answer corresponding to the cognitive training question;

[0046] Comparing the answer similarity with a preset similarity threshold;

[0047] If the answer similarity exceeds the similarity threshold, a new cognitive training question is reselected from the preset target question and answer data set and displayed on the question and answer webpage;

[0048] If the answer similarity does not exceed the similarity threshold, a strongly related question is generated by combining the cognitive training question with the question generation model and displayed on the question and answer webpage.

[0049] Optionally, if the answer similarity does not exceed the similarity threshold, a question generation model is used to combine the cognitive training questions to generate strongly related questions, including:

[0050] If the answer similarity does not exceed the similarity threshold, then locate the corresponding paragraph text from the preset target question and answer data set according to the cognitive training question and divide it into multiple text words;

[0051] Convert each of the text words into a vector and extract the corresponding text features;

[0052] Inputting the text features and the explanatory text corresponding to the paragraph text into a preset machine learning model in sequence to determine whether the text features meet the preset answer conditions;

[0053] If so, then the text words belonging to the text feature are hollowed out from the paragraph text until the number of hollowed-out words reaches a preset hollowing threshold, and the hollowed-out paragraph text is used to generate a strongly related question, and the explanatory text corresponding to the paragraph text is used as the standard answer;

[0054] If not, new text features are selected according to the order of the text words and input into the machine learning model.

[0055] The present invention also provides a cognitive function assessment and training device for the elderly, comprising:

[0056] The cognitive initial examination module is used to perform a cognitive initial examination upon receiving the MRI images and user data uploaded by the elderly user on the Q&A webpage, and generate the initial examination assessment results and cognitive function type;

[0057] a feature fusion module, configured to extract a plurality of resonance image features from the nuclear magnetic resonance image and concatenate the features with the convolution result of the initial examination assessment result to generate a fusion feature if the cognitive function type is any cognitive impairment type;

[0058] A cognitive review module, configured to perform a cognitive review based on the fusion features, generate a review evaluation result, and display corresponding cognitive training questions on the question-and-answer webpage;

[0059] a training topic updating module, configured to respond to the answer information inputted for the cognitive training topic and update the cognitive training topic using a preset target question and answer data set, so as to perform cognitive function training on the elderly user;

[0060] The normal training module is used to jump to the step of displaying the corresponding cognitive training questions on the question-and-answer webpage if the cognitive function type is the normal cognitive type.

[0061] It can be seen from the above technical solutions that the present invention has the following advantages:

[0062] When the question-and-answer webpage receives the MRI images and user data uploaded by the elderly user, it performs an initial cognitive examination, generates the initial examination assessment results and the cognitive function type. If the cognitive function type is any type of cognitive impairment, it extracts multiple resonance image features from the MRI image, convolves the initial examination assessment results with all the resonance image features, and generates a fusion feature. Based on the fusion feature, it performs a cognitive re-examination, generates the re-examination assessment results, and displays the corresponding cognitive training questions on the question-and-answer webpage. In response to the answer information entered for the cognitive training questions, it updates the cognitive training questions to train the elderly user's cognitive function. In this way, the cognitive function of the elderly is comprehensively assessed through multimodal data such as MRI images and user data. Further, based on the assessment results, suitable cognitive training questions are selected to conduct multiple rounds of related question-and-answer training, thereby more accurately assessing and training the cognitive function of the elderly. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 A flowchart of the steps of a cognitive function assessment and training method for the elderly provided by an embodiment of the present invention;

[0065] Figure 2 A cognitive initial inspection network framework diagram provided by an embodiment of the present invention;

[0066] Figure 3 A schematic structural diagram of an encoder module provided in an embodiment of the present invention;

[0067] Figure 4 A schematic structural diagram of a core attention layer provided in an embodiment of the present invention.

[0068] Figure 5 A flowchart of the steps of a cognitive function assessment and training method for the elderly provided by an embodiment of the present invention;

[0069] Figure 6 Another cognitive initial inspection network framework diagram provided by an embodiment of the present invention;

[0070] Figure 7 A cognitive review network framework diagram provided by an embodiment of the present invention;

[0071] Figure 8 This is a structural block diagram of a cognitive function assessment and training device for the elderly provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The present invention provides a method and device for cognitive function assessment and training in the elderly. This approach addresses the problem that digital cognitive therapy is often only available in medical institutions and is often only available to patients with confirmed cognitive impairment. Furthermore, current digital cognitive therapy often involves a single round of dialogue or question-and-answer sessions, allowing users to guess or guess at the answers, which can reduce the accuracy of cognitive impairment assessments.

[0073] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0074] See also Figure 1 , Figure 1 This is a flowchart of the steps of a cognitive function assessment and training method for the elderly provided by an embodiment of the present invention.

[0075] The present invention provides a cognitive function assessment and training method for the elderly, comprising:

[0076] Step 101: When receiving the MRI image and user data uploaded by the elderly user on the Q&A webpage, perform an initial cognitive examination to generate an initial examination assessment result and a cognitive function type;

[0077] A question-and-answer webpage refers to an Internet page with multiple operable controls. The operable controls can trigger a jump to a secondary page within the page. Users can upload MRI images and user data on the secondary page, and return to the initial Internet page after receiving a confirmation instruction, displaying the corresponding assessment results and questions used for cognitive training in its content area.

[0078] Nuclear magnetic resonance imaging refers to the use of the principle of nuclear magnetic resonance (NMR). Based on the different attenuation of the released energy in different structural environments within the material, the emitted electromagnetic waves are detected by applying an external gradient magnetic field. The position and type of atomic nuclei that make up the object can be determined, and based on this, a structural image of the object's interior can be drawn.

[0079] User data refers to data entered or uploaded by users via the various data entry boxes pre-loaded on the Q&A website. This includes, but is not limited to, income data, occupation data, educational background data, and various characteristic data. Characteristic data includes, but is not limited to, gender, socioeconomic status, education level, MMSE (Minimum Mental State Examination) scores, age, and total and normalized intracranial volumes (NCVs) corresponding to MRI images.

[0080] In an embodiment of the present invention, when magnetic resonance images and user data uploaded by elderly users are received on a question-and-answer webpage, the corresponding network model can be called based on the magnetic resonance images and user data as data sources respectively to perform an initial cognitive examination, so as to perform image analysis through the content of the magnetic resonance images to determine the probability of each cognitive function type to which the elderly user belongs, and at the same time, perform feature analysis through the user data to determine the probability of each cognitive function type to which the elderly user belongs, and evaluate the cognitive function type of the elderly user by combining the two probabilities in the form of a voting method, and the corresponding probability vector is used as the initial examination evaluation result.

[0081] It should be noted that after a user logs in to the cognitive function assessment and training device for the elderly, the Mini-Mental State Examination (MMSE) questions are pre-displayed on the Q&A webpage in the form of images and text. The user can then answer in text and audio (via a voice processing interface). In each round of Q&A, the similarity between the user's answer and the standard answer is calculated. Correct answers are determined based on whether the similarity exceeds a threshold set based on experience. Correct answers are awarded one point, while incorrect answers are awarded no points. After traversing all questions in the checklist, the user's total score is calculated as the MMSE score.

[0082] Among them, the cognitive function type is one of the cognitive types, and the cognitive types can include normal cognitive types and multiple cognitive impairment types. Taking the degree of dementia as an example, cognitive failure types can include but are not limited to "mild dementia", "moderate dementia" and "severe dementia", and the normal cognitive type can be "no dementia".

[0083] See also Figure 2 In one example of the present application, step 101 may include the following sub-steps S11-S17:

[0084] S11. When receiving the MRI image and user data uploaded by the elderly user on the Q&A webpage, encoding is performed according to the variable type in the user data to generate a second statistical feature;

[0085] In this embodiment, after receiving the MRI image and user data uploaded by the elderly user on the Q&A webpage, since the user data includes continuous variables such as age, socioeconomic status, and MMSE score, as well as categorical variables such as sex and handedness, different variable types can be encoded separately to generate the second statistical feature.

[0086] Specifically, continuous variables can be directly processed through the Number Embedding layer to generate corresponding embedded features. For categorical variables, one-hot encoding is first performed and then input into the Category Embedding layer to map the categorical variables to a continuous vector space to obtain the corresponding embedded features, as shown below:

[0087]

[0088] in, The age value entered by the user, is a learnable mapping vector, is the one-hot encoding vector of gender, 、 ( Other embedding features can be obtained in the same way.

[0089] After obtaining all the embedded features, concatenate the embedded features to form the embedding vector of the second statistical feature , as shown below:

[0090]

[0091] in, Represents vertical series connection, .

[0092] The purpose of performing the above processing on statistical features is that, compared to one-hot encoding, Category Embedding not only reduces memory usage and accelerates neural networks, but more importantly, it reveals the intrinsic properties of categorical variables, helping to increase the robustness of neural networks. Compared to raw numerical values, Number Embedding can capture the inherent laws and patterns of distributions, thereby determining the weight and importance of the current data in the embedding space, which helps neural networks learn features. Furthermore, to achieve the fusion of categorical data, numerical data, and MRI image data, the above operations are required to map them into the same feature space, making them learnable inputs for the neural network.

[0093] S12, dividing the nuclear magnetic resonance image into blocks according to a preset size, and flattening them into image block vectors respectively;

[0094] S13, linearly map and concatenate each image block vector to obtain an image embedding vector;

[0095] In this embodiment, the input MRI image and the above-mentioned user data belong to data of different modalities and contain different semantics and feature information. Therefore, vector processing needs to be performed in different ways to facilitate subsequent multimodal data fusion.

[0096] Specifically, the preset size is ,against size of the MRI image, by Block operations and flattening into vectors .in, .

[0097] For each block, perform Linear Projection and splice it to get , as shown in the following formula:

[0098]

[0099] in, is a learnable mapping matrix, Given.

[0100] S14, concatenating the image embedding vector and the second statistical feature to obtain an overall embedding vector;

[0101] After obtaining the image embedding vector and the second statistical feature, concatenate the two to obtain the overall embedding vector:

[0102]

[0103] in, This concatenation enables the fusion of multimodal data. Data from different modalities complement each other and contain different semantic and feature information. Fusing this information can improve the model's generalization performance and make it better adaptable to unknown data.

[0104] S15, inputting the overall embedding vector into multiple encoder modules respectively to obtain multiple encoding vectors respectively;

[0105] The same tissue can appear differently in images for different elderly users, different modalities, and different imaging devices. Even different frames within the same modality can differ. This means that there is intra-class variability. The Transformer's self-attention and multi-head mechanisms apply multiple attention heads in parallel, allowing the model to learn different attention patterns from different representation subspaces. This better captures dependencies between different positions in the input sequence and improves the ability to model intra-class variability.

[0106] like Figure 3 As shown, the encoder module includes a word embedding layer, a first normalization layer, a kernel attention layer, a first multiplier layer, a second normalization layer, a multilayer perceptron layer, and a second multiplier layer connected in sequence;

[0107] There is a short-circuit connection between the output position of the word embedding layer and the input position of the first multiplier layer, and there is a short-circuit connection between the output position of the first multiplier layer and the input position of the second multiplier layer.

[0108] In this embodiment, the overall embedding vector is input into L transformer encoder modules to obtain multiple encoding vectors. Specifically, directly applying the Transformer model to this task may not necessarily achieve the best performance because the attention module in the Transformer has a large number of parameters and the training samples are usually limited. To address this problem, the original dot product attention module in the Transformer is replaced by kernel attention, that is, Figure 3 The Multi-Head Attention mechanism in

[15] is replaced by the core attention layer. The specific structure of the core attention layer is as follows: Figure 4 This significantly reduces the number of parameters that need to be trained, thereby alleviating the problem of limited number of samples and introducing a nonlinear attention mechanism to model complex functional connections.

[0109] In specific implementations, the calculation formula of the original dot product attention module is usually:

[0110]

[0111] in, The geometric meaning of vector dot product is to reflect the similarity of vectors. The calculation satisfies:

[0112]

[0113] are all trainable matrices, which introduce a large number of parameters into the model. The kernel function can be used directly to achieve the same effect, and the number of parameters that need to be trained can be greatly reduced, such as Figure 4 As shown, is a kernel function, which can be a Gaussian radial basis function. Middle two lines , use the following formula to calculate its similarity:

[0114]

[0115] Among them, the lth encoder module for , calculate any two rows and combine them into a matrix in sequence, as a kernel attention mechanism Part, and then calculate its multiplication value with V, we get .

[0116] S16. Pass all the encoding vectors through the fully connected layer and the classification layer in sequence to generate a preliminary inspection evaluation result consisting of multiple detection probabilities;

[0117] S17. Select the cognitive type corresponding to the match with the highest detection probability as the cognitive function type corresponding to the elderly user.

[0118] like Figure 2 As shown, in this embodiment, after L transformer encodings, a fully connected layer and a softmax layer are passed to obtain an initial inspection evaluation result consisting of multiple detection probabilities.

[0119] Among them, class 0, 1, 2, and 3 represent the cognitive function types of "no dementia", "mild dementia", "moderate dementia", and "severe dementia", respectively. The cognitive type with the highest detection probability is selected as the cognitive function type corresponding to the elderly user.

[0120] Step 102: If the cognitive function type is any cognitive impairment type, multiple resonance image features are extracted from the MRI image and concatenated with the convolution result of the initial examination assessment result to generate a fusion feature;

[0121] In this embodiment, if the cognitive function type is determined to be any type of cognitive impairment, further feature extraction is performed on the magnetic resonance image from multiple resolutions and bilinear interpolation is performed to unify the size and then perform splicing convolution. After splicing with the feature maps of the uniform size, the self-attention module analyzes and extracts the corresponding resonance image features.

[0122] It should be noted that resonance image features refer to image features extracted from the same MRI image using image processing models at different scales, such as CNN models, and then fused with the feature maps to unify the scale. These features can be represented as vector matrices. They are used to characterize the local characteristics of MRI images at different resolutions.

[0123] After obtaining the initial assessment results, they can be convolved to obtain the feature information of each probability within the initial assessment results, and further spliced ​​with all the resonance image features to generate fusion features as the data basis for cognitive re-examination.

[0124] Step 103: Perform cognitive re-examination based on the fusion features, generate re-examination evaluation results, and display corresponding cognitive training questions on the question-and-answer webpage;

[0125] In this embodiment, the fusion features are sequentially input into the fully connected layer, the batch normalization layer, and the activation function, and then classified by the classification function after multiple cycles, thereby completing the cognitive review and generating a review evaluation result.

[0126] At the same time, after obtaining the re-examination and evaluation results, the probability vectors of each type of cognitive impairment are obtained, so that the specific type of cognitive impairment can be accurately determined, and cognitive training questions of different levels can be matched according to the type of cognitive impairment and displayed on the question-and-answer webpage to facilitate further cognitive function training for elderly users.

[0127] Step 104 , in response to the answer information input for the cognitive training questions, the cognitive training questions are updated using a preset target question and answer data set to perform cognitive function training on the elderly user.

[0128] After the cognitive training questions are generated on the question-and-answer webpage, the elderly user can read the displayed cognitive training questions and enter corresponding answer information on the question-and-answer webpage to answer the cognitive training questions.

[0129] At the same time, the question-and-answer webpage can respond to the answer information and judge whether it is correct based on its similarity to the standard answer. The cognitive training questions can be updated based on the judgment results, thereby constructing multiple rounds of conversations with strong correlation and effectively preventing elderly users from guessing questions, etc., and providing more effective cognitive function training for elderly users.

[0130] Step 105: If the cognitive function type is the normal cognitive type, the process jumps to the step of displaying the corresponding cognitive training questions on the question-and-answer webpage.

[0131] In an embodiment of the present invention, when an MRI image and user data uploaded by an elderly user are received on a question-and-answer webpage, a cognitive initial examination is performed to generate an initial examination evaluation result and a cognitive function type. If the cognitive function type is any type of cognitive impairment, multiple resonance image features are extracted from the MRI image, the initial examination evaluation result is convolved with all the resonance image features, and a fusion feature is generated. A cognitive re-examination is performed based on the fusion feature, and a re-examination evaluation result is generated and the corresponding cognitive training questions are displayed on the question-and-answer webpage. In response to the answer information input for the cognitive training questions, the cognitive training questions are updated to train the cognitive function of the elderly user. Thus, the cognitive function of the elderly is comprehensively evaluated by multimodal data such as MRI images and user data, and further, suitable cognitive training questions are selected based on the evaluation results to conduct multiple rounds of related question-and-answer training, thereby more accurately evaluating and training the cognitive function of the elderly.

[0132] See also Figure 5 , Figure 5 This is a flowchart of the steps of a cognitive function assessment and training method for the elderly provided by an embodiment of the present invention.

[0133] The present invention provides a cognitive function assessment and training method for the elderly, comprising:

[0134] Step 501: When receiving an MRI image and user data uploaded by an elderly user on a Q&A webpage, a cognitive function assessment is performed on the MRI image using a preset image assessment model to determine a first initial examination probability vector corresponding to the elderly user.

[0135] In an embodiment of the present invention, when an MRI image and user data uploaded by an elderly user are received on a question-and-answer webpage, a preset image evaluation model is called to perform cognitive function evaluation on the MRI image, thereby determining a first initial examination probability vector corresponding to the elderly user.

[0136] In a specific implementation, when evaluating cognitive function, the results must be accurate, especially when the user is believed to have cognitive dysfunction. For this purpose, the initial and re-examination of cognitive function can be performed through a network of recursive cascade mechanisms. When an MRI image is obtained, it can be processed by calling a CNN or other model to determine the corresponding category probability vector as the first initial inspection probability vector, where the number of elements of the first primary probability vector can be set according to the category, for example, the elements are sorted in the order of "no dementia", "mild dementia", "moderate dementia" and "severe dementia" to generate the first initial inspection probability vector. :

[0137]

[0138] In a specific implementation, the user's MRI image and user data can be obtained by displaying prompt information on the Q&A webpage, namely:

[0139]

[0140] MRI is a magnetic resonance imaging image. a first statistical feature generated for the user data, Refers to the processing operations performed on the input data of the question and answer web page.

[0141] Convolutional Neural Networks (CNNs) are deep learning models or multilayer perceptrons similar to artificial neural networks, commonly used to analyze visual images. They primarily consist of an input layer, a convolutional computation layer, a ReLU layer, a pooling layer, and a fully connected layer. In this embodiment, training can be performed using historically acquired representative MRI images of various cognitive function types, and the model parameters are adjusted to generate the model. Furthermore, user age data can be used to determine user group membership.

[0142] Step 502: performing vectorization conversion on the user data to generate a first statistical feature;

[0143] Optionally, the user data includes income data, occupation data, educational background data, and various feature data; step 502 may include the following sub-steps:

[0144] Perform feature conversion on various feature data according to preset rules to obtain initial features;

[0145] According to the preset conversion table, income data, occupation data, and educational background data are mapped to specific values, accumulated, and normalized to obtain socioeconomic status characteristics;

[0146] The initial features and socioeconomic status features are spliced ​​together to generate the first statistical features.

[0147] In an embodiment of the present invention, after obtaining user data, the feature data therein may have different dimensions and belong to different categories. Various feature data can be characterized and converted separately according to preset rules, and converted into a unified dimension to facilitate subsequent model processing and obtain initial features.

[0148] The process of characterizing feature data can be as follows:

[0149] Gender is set to 0 for female and 1 for male; education level is the number of years of education; MMSE score is the actual test score of the user in the question-answering system; age is the actual user input in the question-answering system; estimated total intracranial volume is the automatic estimate of total intracranial volume derived from the ASF raw space. This is a continuous measure between 1100 and 2000. Normalized intracranial volume is the brain volume estimate based on automatic tissue segmentation (grayscale plus white matter). It is normalized to a percentage based on the atlas target mask. This measure is continuous and ranges from 0 to 1.

[0150] For socioeconomic status characteristics, specific numerical values ​​can be mapped in the following ways:

[0151] Individual monthly income (Z1) is: 1 = "less than 1,000" yuan; 2 = "1,001-1,500" yuan; 3 = "1,501-2,000" yuan; 4 = "2,001-3,000" yuan; 5 = "3,001-5,000" yuan; 6 = "5,001-8,000" yuan; 7 = "8,001-10,000" yuan; 8 = "10,001-20,000" yuan; 9 = "more than 20,000" yuan; Occupation treatment method (Z2); Individual occupation is: 1 = "student"; 2 = "more than 20,000" yuan; 3 = "less than 20,000" yuan; 4 = "less than 20,000" yuan; 5 = "less than 20,000" yuan; 6 = "less than 20,000" yuan; 7 = "less than 20,000" yuan; 8 = "less than 20,000" yuan; 9 = "less than 20,000" yuan; Occupation treatment method (Z2); Individual occupation is: 1 = "student"; 2 = "less than 20,000" yuan; 3 = "less than 20,000" yuan; 4 = "less than 20,000" yuan; 5 = "less than 20,000" yuan; 6 = "less than 20,000" yuan; 7 = "less than 20,000" yuan; 8 = "less than 20,000" yuan; 9 = "less than 20,000" yuan; 10 = "less than 20,000" yuan; 11 = "less than 20,000" yuan; 12 = "less = "Government / Institution Cadre Civil Servant"; 3 = "Enterprise Manager (including grassroots and middle and senior management)"; 4 = "Ordinary Staff (Office / Office Building Staff)"; 5 = "Professionals (such as journalists / teachers, etc.)"; 6 = "Ordinary Workers (such as factory workers / manual laborers, etc.)"; 7 = "Commercial Service Workers"; 8 = "Self-employed / Contractor"; 9 = "Freelancer"; 10 = "Agricultural, Forestry, Animal Husbandry and Fishery Workers"; 11 = "Retired"; 12 = "No occupation yet"; Combine occupation options: 11, 12, 131 Others; 12 students; 8, 9, 103 self-employed, freelance, agriculture, forestry, animal husbandry and fishery; 6, 74 ordinary blue-collar workers; 4, 55 ordinary white-collar workers; 2, 36 senior white-collar workers; Education processing method (Z3); So far, the highest educational level of an individual is:; 1 = "junior high school and below"; 2 = "senior high school / technical secondary school / technical school"; 3 = "junior college"; 4 = "bachelor's degree"; 5 = "master's degree and above"; Economic status SES = Z1 + Z2 + Z3, and it is normalized to between 1 and 5 to obtain socioeconomic status characteristics.

[0152] User data refers to data entered or uploaded by users via the various data entry boxes preloaded on the Q&A website. This includes, but is not limited to, income data, occupation data, educational background data, and various characteristic data. Characteristic data includes, but is not limited to, gender, education level, MMSE (Minimum Mental State Examination) scores, age, and total and normalized intracranial volumes (CTVs) corresponding to MRI images.

[0153] Step 503: Perform cognitive function assessment using a preset feature assessment model combined with the first statistical feature to determine a second initial detection probability vector corresponding to the elderly user;

[0154] In the field of medical image processing, single-modality medical images cannot comprehensively provide the anatomical and detailed features of lesions, which is not conducive to rapid and accurate diagnosis and treatment of diseases. To address the shortcomings of single-modality medical images, the first statistical feature data is used as a supplement and assistance.

[0155] In this embodiment, the cognitive function is evaluated by combining the preset feature evaluation model with the first statistical feature, thereby determining the second preliminary detection probability vector representing the probability of various cognitive types. , which can be specifically expressed as:

[0156]

[0157] Among them, ML includes machine learning methods such as support vector machines, decision trees, random forests, and logistic regression.

[0158] Step 504: After calculating the sum of the first initial detection probability vector and the second initial detection probability vector, the mean probability vector is calculated as the cognitive initial detection result corresponding to the elderly user;

[0159] After obtaining the first initial inspection probability vector and the second initial inspection probability vector, in order to further improve the accuracy of the initial inspection, the sum of each element between the first initial inspection probability vector and the second initial inspection probability vector can be calculated, and the mean probability corresponding to each cognitive type can be calculated respectively, so as to obtain the probability vector after voting as the cognitive initial inspection result corresponding to the elderly user.

[0160] See Figure 6 , Figure 6 This is a network block diagram of a cognitive initial inspection in an embodiment of the present invention.

[0161] In this embodiment, by obtaining the nuclear magnetic resonance imaging and the first statistical feature, the CNN model is used to classify the nuclear magnetic resonance image to obtain the corresponding category probability vector, that is, the first preliminary detection probability vector, in which each element corresponds to the probability of a cognitive type, that is, At the same time, the first statistical feature is analyzed through machine learning models such as support vector machine, decision tree, random forest, logistic regression and other machine learning methods, that is, , thereby obtaining the corresponding second initial inspection probability vector, and using the voting method to determine the initial inspection result corresponding to the elderly user.

[0162] It should be noted that the cognitive function type of the elderly user may also be directly determined by taking the cognitive type corresponding to the highest value of the first preliminary detection probability vector and the second preliminary detection probability vector as the cognitive function type of the elderly user.

[0163] Step 505: Match the corresponding cognitive type according to the highest element probability in the mean probability vector as the cognitive function type corresponding to the elderly user;

[0164] After obtaining the mean probability vector, the highest probability of the element in the mean probability vector may be used to match the corresponding cognitive type as the cognitive function type corresponding to the elderly user.

[0165] In this embodiment, steps 501-205 are the initial examination process, which provides the initial examination results of the user's cognitive function by inputting the MRI image and user data. and their cognitive function types, namely:

[0166]

[0167] MRI is a magnetic resonance imaging image. The first statistical feature is obtained by converting the user data.

[0168] Step 506 , if the cognitive function type is any cognitive impairment type, multiple resonance image features are extracted from the MRI image and concatenated with the convolution result of the initial assessment result to generate a fusion feature;

[0169] In one example of the present invention, step 506 may include the following sub-steps:

[0170] If the cognitive function type is any type of cognitive impairment, multiple CNN network models with different resolutions are called to extract the corresponding initial feature maps from the magnetic resonance imaging.

[0171] Perform bilinear interpolation on each initial feature map to generate an intermediate feature map with the same resolution;

[0172] Concatenate and convolve the intermediate feature maps to generate multi-scale feature maps;

[0173] The multi-scale feature maps are concatenated with each intermediate feature map and then input into the self-attention layer to generate the resonant image features.

[0174] The convolution results of the resonance image features and the initial inspection evaluation results are spliced ​​to generate fusion features.

[0175] Medical images are unevenly distributed within classes, resulting in intra-class imbalance. Furthermore, the images of the same tissue may vary between different patients, modalities, and imaging devices. Even different frames within the same modality may differ. This means that there is intra-class variability. Based on these characteristics, we have adopted a solution that uses multi-scale features. At different stages of the CNN network, semantic information is gradually acquired, from shallow to deep layers. This allows the network to fully learn from each MRI image, maximizing data utilization and improving the network's cognitive capabilities.

[0176] To this end, if the cognitive function type is determined to be any type of cognitive impairment, such as "mild dementia", "moderate dementia", or "severe dementia", multiple CNN network models of different resolutions can be called to extract the initial feature maps that can be extracted at each resolution from the MRI image to help encode the global and local context. The initial feature maps extracted at multiple resolutions are represented as .in, Represents the network hierarchy. Since each level The features of have different resolutions, so bilinear interpolation is used to upsample them to a common resolution to generate an intermediate feature map with the same resolution Then, all scales Concatenate to form a tensor and perform convolution to create a general multi-scale feature map: .then, It encodes shallow low-level information and deep learned high-level semantics.

[0177] CNN has a local receptive field, which leads to the generation of local feature representation. When the context information is not correctly encoded, the local feature representation may lead to potential differences between the features corresponding to pixels with the same label, which may introduce intra-class inconsistency and ultimately affect the recognition performance. Therefore, an attention mechanism can be introduced to establish connections between different feature maps. In addition, the attention mechanism can help make the feature maps The multi-scale feature map is refined and the noise is gradually eliminated, giving more weight to the truly important areas. and feature maps at different resolutions After splicing, they are input into the self-attention layer for feature refinement to generate multiple resonant image features. :

[0178]

[0179] in, refers to the s-th self-attention layer, is the sth resonance image feature.

[0180] In the above steps, the multimodal features of MRI and the first statistical feature are integrated to generate the initial inspection evaluation result, thereby improving the problem of incompleteness of a single modality.

[0181] In the embodiment of the present invention, the initial inspection evaluation result can be convolved to generate , by and Splicing to generate fusion features .

[0182] Step 507: Perform cognitive re-examination based on the fusion features, generate re-examination evaluation results, and display corresponding cognitive training questions on the question-and-answer webpage;

[0183] In one example of the present invention, step 507 may include the following sub-steps:

[0184] Classify the fused features in sequence through the fully connected layer to obtain the classification vector corresponding to the fused features;

[0185] The classification vector is batch normalized through the batch normalization layer to obtain a batch normalized vector;

[0186] Mapping the batch normalized vector into output features through the activation function;

[0187] The output feature is used as a new fusion feature, and the fusion feature is sequentially classified through the fully connected layer to obtain the fusion probability vector corresponding to the fusion feature until the number of jumps reaches a preset threshold;

[0188] The classification layer classifies the output features at the current moment and calculates the corresponding probability values, generating a probability vector as the re-inspection evaluation result;

[0189] Match the corresponding cognitive type according to the re-examination and assessment results and select the corresponding cognitive training questions to be displayed on the question and answer webpage.

[0190] In this embodiment, the common features are processed in sequence through the fully connected layer, batch normalization layer, and RELU activation function, and after four cycles, the probability vector is output as the re-inspection evaluation result.

[0191] Specifically, the processing of the fully connected layer is as follows:

[0192]

[0193] The batch normalization layer processes as follows:

[0194]

[0195] The processing of the RELU activation function is as follows:

[0196]

[0197] Among them, the initial value of i is 0. After the activation function is activated, i=i+1, and then As a new fusion feature . is the batch mean, is the batch variance, is the trainable weight of the fully connected layer, is the trainable bias of the fully connected layer, is the trainable scaling factor of the batch normalization layer, is the trainable shift factor of the batch normalization layer, For a given very small value (close to 0. To prevent the denominator = 0, add a ), is the i-th output feature, .

[0198] After 4 cycles, the output features are obtained through the softmax function Classify and calculate the corresponding probability values ​​to construct a probability vector as the re-examination evaluation result :

[0199]

[0200] After receiving the re-evaluation results, the highest probability value within the re-evaluation results is selected and matched to the corresponding cognitive type, completing a precise re-examination of the MRI images and user data to determine the exact type of cognitive impairment. At the same time, to provide cognitive training for elderly users, cognitive training questions corresponding to the cognitive impairment type are selected and displayed on the Q&A webpage, facilitating cognitive training for elderly users and slowing their cognitive decline.

[0201] See also Figure 7 , Figure 7 This is a framework diagram of a review network in an embodiment of the present invention.

[0202] In this embodiment, the acquired MRI images are input into CNN-1 to CNN-4 with different resolutions (scales), and feature extraction is performed through each CNN model to obtain initial feature maps. , and further upsample each initial feature map to an intermediate feature map of the same resolution through bilinear interpolation , further for each intermediate feature map After splicing, convolution is performed to generate multi-scale feature maps , the multi-scale feature map With each intermediate feature map The splicing is sent to the self-attention module for feature extraction to extract features of different scales , and compare it with the initial assessment results The convolution result of After splicing, generate fusion features After 4 cycles of fully connected layers, batch normalization layers, and activation functions, the softmax function is used for classification, thus generating the re-inspection evaluation results in the form of probability vectors. Indicates the probability corresponding to each cognitive type.

[0203] In one example of the present invention, the method may further include the following steps:

[0204] Translate multiple preset question-and-answer datasets into the required languages ​​set on the question-and-answer webpage, and generate corresponding translation datasets. The question-and-answer datasets include paragraph text, multiple question-and-answer pairs, and explanatory text.

[0205] Each translation dataset is localized according to the preset list of sensitive elements to generate the target question-answering dataset.

[0206] Question-and-answer datasets are composed of multiple paragraphs, question-and-answer pairs, and corresponding explanatory text. These datasets include, but are not limited to, the Science Q&A dataset (ScienceQA) and the Fairytale Q&A dataset (FairytaleQA). The Science Q&A dataset is a multimodal dataset covering three disciplines: natural sciences, linguistics, and social sciences. The Fairytale Q&A dataset covers 278 fairy tale books, each with multiple paragraphs. For each paragraph, there are several pairs of educational question-and-answer pairs annotated by educational experts, enabling multi-round conversations.

[0207] In this embodiment, since the question-and-answer dataset may be described in only a single language, it may not be suitable for users speaking different languages. Therefore, when the question-and-answer webpage is accessed, the required language set on the question-and-answer webpage can be detected in real time, and the multiple preset question-and-answer datasets can be translated according to the required language to generate corresponding types of translation datasets.

[0208] At the same time, due to language or regional differences, the data set may contain some sensitive elements. Therefore, each translation data set can be localized according to the preset list of sensitive elements. According to the topic, subject and other information provided by the data set, inappropriate elements can be deleted and appropriate elements can be retained, thereby generating a target question-and-answer data set as the data basis for subsequent cognitive training questions.

[0209] Step 508 , in response to the answer information input for the cognitive training question, the cognitive training question is updated using the preset target question and answer data set to perform cognitive function training on the elderly user.

[0210] In one example of the present invention, step 508 may include the following sub-steps:

[0211] In response to the answer information input for the cognitive training question, calculating the answer similarity between the answer information and the standard answer corresponding to the cognitive training question;

[0212] Compare the answer similarity with the preset similarity threshold;

[0213] If the answer similarity exceeds the similarity threshold, a new cognitive training question will be reselected from the preset target question and answer dataset and displayed on the question and answer webpage;

[0214] If the answer similarity does not exceed the similarity threshold, a strongly related question is generated through the question generation model combined with the cognitive training questions and displayed on the question and answer webpage.

[0215] In this embodiment, questions are printed in the form of text and images on the web page, and users answer the questions in the form of text and audio. By responding to the answer information input by the elderly for the cognitive training questions, the answer similarity between the answer information and the standard answer to the cognitive training questions is calculated. The difference between the answer similarity and the preset similarity threshold is further compared. If the answer similarity exceeds the similarity threshold, it indicates that the user's answer is correct. At this time, the next question is randomly selected from any topic in the database, and a new cognitive training question is reselected from the preset target question and answer data set and displayed on the question and answer web page.

[0216] It should be noted that the answer similarity can be calculated by converting the answer information and the standard answer into vectors and then calling an existing similarity algorithm.

[0217] If the answer similarity does not exceed the similarity threshold, the current cognitive training question and the corresponding explanatory text are used as input to the question generation model to generate questions that are strongly related to the current cognitive training question and display them on the question-and-answer webpage, thereby achieving multi-round dialogue training.

[0218] Furthermore, if the answer similarity does not exceed the similarity threshold, the question generation model is combined with the cognitive training questions to generate highly relevant questions, including:

[0219] If the answer similarity does not exceed the similarity threshold, the corresponding paragraph text is located from the preset target question and answer dataset according to the cognitive training question and divided into multiple text words;

[0220] Convert each text word into a vector and extract the corresponding text features;

[0221] The text features and the explanatory text corresponding to the paragraph text are input into the preset machine learning model in turn to determine whether the text features meet the preset answer conditions;

[0222] If so, the text words belonging to the text feature are hollowed out from the paragraph text until the number of hollowed-out words reaches the preset hollowing threshold, and the hollowed-out paragraph text is used to generate strongly related questions, and the explanatory text corresponding to the paragraph text is used as the standard answer;

[0223] If not, new text features are selected according to the order of text words and input into the machine learning model.

[0224] In this embodiment, if the answer similarity does not exceed the similarity threshold, it indicates that the user's answer is wrong. At this time, the corresponding paragraph text can be located in the preset target question and answer data set according to the careful training questions, and the paragraph text can be divided into multiple text words. The pre-trained word2vec model is used to convert each text word into a vector and extract the corresponding text features. The text features and the explanatory text corresponding to the paragraph text are input into the preset machine learning model such as the naive Bayes classifier, support vector machine, etc. in turn to determine whether the text features meet the preset answer conditions and whether they are suitable as answers to the questions. If the answer conditions are met, the text words to which the text features belong can be hollowed out from the paragraph text until the number of hollowing reaches the preset hollowing threshold, and the hollowed paragraph text is used to generate strongly related questions, and the explanatory text corresponding to the paragraph text is used as the standard answer. If the answer conditions are not met, new text features can be selected according to the order of the text words and input into the machine learning model for further judgment.

[0225] It should be noted that the answer condition may be that the similarity between the text feature and the explanatory text is greater than a preset threshold, or that the similarity between the text feature and features such as a specific noun, verb, or the subject of a paragraph text is greater than a preset threshold. The embodiments of the present invention do not impose any restrictions on this.

[0226] Step 509: If the cognitive function type is the normal cognitive type, the process jumps to the step of displaying the corresponding cognitive training questions on the question-and-answer webpage.

[0227] In this embodiment, if the cognitive function type is determined to be the normal cognitive type, it indicates that the user currently does not have cognitive impairment. At this time, in order to avoid the deterioration of his cognitive function, you can jump to step 507 and display cognitive training questions matching the normal cognitive type on the question-and-answer webpage.

[0228] In an embodiment of the present invention, when an MRI image and user data uploaded by an elderly user are received on a question-and-answer webpage, a cognitive initial examination is performed to generate an initial examination evaluation result and a cognitive function type. If the cognitive function type is any type of cognitive impairment, multiple resonance image features are extracted from the MRI image, the initial examination evaluation result is convolved with all the resonance image features, and a fusion feature is generated. A cognitive re-examination is performed based on the fusion feature, and a re-examination evaluation result is generated and the corresponding cognitive training questions are displayed on the question-and-answer webpage. In response to the answer information input for the cognitive training questions, the cognitive training questions are updated to train the cognitive function of the elderly user. Thus, the cognitive function of the elderly is comprehensively evaluated by multimodal data such as MRI images and user data, and further, suitable cognitive training questions are selected based on the evaluation results to conduct multiple rounds of related question-and-answer training, thereby more accurately evaluating and training the cognitive function of the elderly.

[0229] See also Figure 8 , Figure 8 This is a structural block diagram of a cognitive function assessment and training device for the elderly provided by an embodiment of the present invention.

[0230] The present invention also provides a cognitive function assessment and training device for the elderly, comprising:

[0231] The cognitive initial examination module 801 is used to perform a cognitive initial examination upon receiving an MRI image and user data uploaded by an elderly user on the Q&A webpage, and generate an initial examination assessment result and a cognitive function type;

[0232] A feature fusion module 802 is configured to extract multiple resonance image features from the MRI image if the cognitive function type is any cognitive impairment type, and concatenate the features with the convolution result of the initial examination assessment result to generate a fusion feature;

[0233] The cognitive review module 803 is used to perform cognitive review based on the fusion features, generate review evaluation results and display corresponding cognitive training questions on the question-and-answer webpage;

[0234] The training topic updating module 804 is used to respond to the answer information input for the cognitive training topic and update the cognitive training topic using the preset target question and answer data set to train the cognitive function of the elderly user;

[0235] The normal training module 805 is used to jump to the step of displaying corresponding cognitive training questions on the question-and-answer webpage if the cognitive function type is the normal cognitive type.

[0236] Optionally, the cognitive initial inspection module 801 includes:

[0237] A first preliminary examination probability vector generation submodule is configured to, upon receiving an MRI image and user data uploaded by an elderly user on the Q&A webpage, perform a cognitive function assessment on the MRI image using a preset image assessment model to determine a first preliminary examination probability vector corresponding to the elderly user;

[0238] A first statistical feature generation submodule, configured to perform vectorization conversion on user data to generate a first statistical feature;

[0239] A second initial detection probability vector generation submodule is configured to perform cognitive function assessment using a preset feature evaluation model combined with the first statistical feature to determine a second initial detection probability vector corresponding to the elderly user;

[0240] The initial inspection generation submodule is used to calculate the sum of the first initial inspection probability vector and the second initial inspection probability vector, and then calculate the mean probability vector as the cognitive initial inspection result corresponding to the elderly user;

[0241] The cognitive function type determination submodule is used to match the corresponding cognitive type according to the highest element probability in the mean probability vector as the cognitive function type corresponding to the elderly user.

[0242] Optionally, the user data includes income data, occupation data, educational background data, and multiple feature data; the first statistical feature generation submodule is specifically used to:

[0243] Perform feature conversion on various feature data according to preset rules to obtain initial features;

[0244] According to the preset conversion table, income data, occupation data, and educational background data are mapped to specific values, accumulated, and normalized to obtain socioeconomic status characteristics;

[0245] The initial features and socioeconomic status features are spliced ​​together to generate the first statistical features.

[0246] Optionally, the cognitive initial inspection module 801 is specifically configured to:

[0247] When receiving the MRI image and user data uploaded by the elderly user on the Q&A webpage, encoding is performed according to the variable type in the user data to generate a second statistical feature;

[0248] Divide the MRI image into blocks according to a preset size and flatten them into image block vectors;

[0249] Perform linear mapping and concatenation on each image block vector to obtain the image embedding vector;

[0250] Concatenate the image embedding vector and the second statistical feature to obtain the overall embedding vector;

[0251] Input the overall embedding vector into multiple encoder modules respectively to obtain multiple encoding vectors;

[0252] All encoded vectors are passed through the fully connected layer and the classification layer in sequence to generate a preliminary inspection evaluation result consisting of multiple detection probabilities;

[0253] The cognitive type corresponding to the match with the highest detection probability is selected as the cognitive function type corresponding to the elderly user.

[0254] Optionally, the encoder module includes a word embedding layer, a first normalization layer, a kernel attention layer, a first multiplier layer, a second normalization layer, a multilayer perceptron layer, and a second multiplier layer connected in sequence;

[0255] There is a short-circuit connection between the output position of the word embedding layer and the input position of the first multiplier layer, and there is a short-circuit connection between the output position of the first multiplier layer and the input position of the second multiplier layer.

[0256] Optionally, the feature fusion module 802 is specifically configured to:

[0257] If the cognitive function type is any type of cognitive impairment, multiple CNN network models with different resolutions are called to extract the corresponding initial feature maps from the magnetic resonance imaging.

[0258] Perform bilinear interpolation on each initial feature map to generate an intermediate feature map with the same resolution;

[0259] Concatenate and convolve the intermediate feature maps to generate multi-scale feature maps;

[0260] The multi-scale feature maps are concatenated with each intermediate feature map and then input into the self-attention layer to generate the resonant image features.

[0261] The convolution results of the resonance image features and the initial inspection evaluation results are spliced ​​to generate fusion features.

[0262] Optionally, the cognitive review module 804 is specifically configured to:

[0263] Classify the fused features in sequence through the fully connected layer to obtain the classification vector corresponding to the fused features;

[0264] The classification vector is batch normalized through the batch normalization layer to obtain a batch normalized vector;

[0265] Mapping the batch normalized vector into output features through the activation function;

[0266] The output feature is used as a new fusion feature, and the fusion feature is sequentially classified through the fully connected layer to obtain the fusion probability vector corresponding to the fusion feature until the number of jumps reaches a preset threshold;

[0267] The classification layer classifies the output features at the current moment and calculates the corresponding probability values, generating a probability vector as the re-inspection evaluation result;

[0268] Match the corresponding cognitive type according to the re-examination and assessment results and select the corresponding cognitive training questions to be displayed on the question and answer webpage.

[0269] Optionally, the training topic updating module 805 includes:

[0270] A similarity calculation submodule is used to respond to the answer information input for the cognitive training question and calculate the answer similarity between the answer information and the standard answer corresponding to the cognitive training question;

[0271] Similarity comparison submodule, used to compare the answer similarity with the preset similarity threshold;

[0272] The first judgment submodule is used to reselect new cognitive training questions from the preset target question and answer dataset if the answer similarity exceeds the similarity threshold, and display them on the question and answer webpage;

[0273] The second judgment submodule is used to generate strongly related questions by combining the question generation model with the cognitive training questions if the answer similarity does not exceed the similarity threshold, and display them on the question and answer webpage.

[0274] Optionally, the second determination submodule is specifically configured to:

[0275] If the answer similarity does not exceed the similarity threshold, the corresponding paragraph text is located from the preset target question and answer dataset according to the cognitive training question and divided into multiple text words;

[0276] Convert each text word into a vector and extract the corresponding text features;

[0277] The text features and the explanatory text corresponding to the paragraph text are input into the preset machine learning model in turn to determine whether the text features meet the preset answer conditions;

[0278] If so, the text words belonging to the text feature are hollowed out from the paragraph text until the number of hollowed-out words reaches the preset hollowing threshold, and the hollowed-out paragraph text is used to generate strongly related questions, and the explanatory text corresponding to the paragraph text is used as the standard answer;

[0279] If not, new text features are selected according to the order of text words and input into the machine learning model.

[0280] Optionally, the device further comprises:

[0281] The Q&A dataset translation module is used to translate multiple preset Q&A datasets into the required languages ​​set by the Q&A webpage, generating corresponding translation datasets. The Q&A datasets include paragraph text, multiple sets of Q&A pairs, and explanatory text.

[0282] The dataset localization module is used to localize each translation dataset according to the preset sensitive element list to generate the target question-answering dataset.

[0283] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and sub-modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0284] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0285] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected to achieve the purpose of the present embodiment according to actual needs.

[0286] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cognitive function assessment and training method for the elderly, characterized in that: include: When the MRI image and user data uploaded by the elderly user are received on the Q&A webpage, a preliminary cognitive examination is performed to generate preliminary assessment results and cognitive function types; If the cognitive function type is any cognitive impairment type, extracting multiple resonance image features from the nuclear magnetic resonance image and concatenating them with the convolution result of the initial examination assessment result to generate a fusion feature; Perform cognitive re-examination based on the fusion features, generate re-examination evaluation results, and display corresponding cognitive training questions on the question-and-answer webpage; In response to the answer information input for the cognitive training question, the cognitive training question is updated using a preset target question and answer data set to perform cognitive function training on the elderly user; If the cognitive function type is a normal cognitive type, jump to the step of displaying the corresponding cognitive training questions on the question-and-answer webpage; If the cognitive function type is any cognitive impairment type, multiple resonance image features are extracted from the nuclear magnetic resonance image and concatenated with the convolution result of the initial examination assessment result to generate a fusion feature, including: If the cognitive function type is any cognitive impairment type, calling multiple CNN network models with different resolutions to extract corresponding initial feature maps from the magnetic resonance image respectively; Performing bilinear interpolation on each of the initial feature maps to generate an intermediate feature map with the same resolution; Splicing and convolving the intermediate feature maps to generate a multi-scale feature map; The multi-scale feature map is spliced ​​with each of the intermediate feature maps and then input into a self-attention layer to generate a resonant image feature; Concatenating the resonance image feature with the convolution result of the initial inspection evaluation result to generate a fusion feature; The cognitive re-examination is performed based on the fusion features, and the re-examination evaluation results are generated and the corresponding cognitive training questions are displayed on the question-and-answer webpage, including: Classify the fused features in sequence through the fully connected layer to obtain a classification vector corresponding to the fused features; Performing batch normalization on the classification vector through a batch normalization layer to obtain a batch normalized vector; Mapping the batch normalized vector into output features through an activation function; The outputtable feature is used as a new fusion feature, and the step of sequentially classifying the fusion feature through the fully connected layer to obtain a fusion probability vector corresponding to the fusion feature is skipped until the number of skips reaches a preset threshold; The classification layer classifies the output features at the current moment and calculates the corresponding probability values, generating a probability vector as the re-inspection evaluation result; According to the re-examination and evaluation results, the corresponding cognitive type is matched and the corresponding cognitive training questions are selected and displayed on the question-and-answer webpage.

2. The method according to claim 1, characterized in that When the MRI image and user data uploaded by the elderly user are received on the Q&A webpage, a preliminary cognitive examination is performed to generate preliminary assessment results and cognitive function types, including: When receiving an MRI image and user data uploaded by an elderly user on the Q&A webpage, performing a cognitive function assessment on the MRI image using a preset image assessment model to determine a first preliminary examination probability vector corresponding to the elderly user; Performing vectorization conversion on the user data to generate a first statistical feature; Using the first statistical feature to perform cognitive function assessment using multiple preset feature assessment models, and generating second preliminary detection probability vectors corresponding to the elderly users respectively; After calculating the sum of the first initial detection probability vector and all the second initial detection probability vectors, calculating the mean probability vector as the cognitive initial detection result corresponding to the elderly user; The corresponding cognitive type is matched according to the highest element probability in the mean probability vector as the cognitive function type corresponding to the elderly user.

3. The method according to claim 2, characterized in that The user data includes income data, occupation data, educational background data, and multiple feature data; the vectorization conversion of the user data to generate the first statistical feature includes: Performing feature conversion on each of the feature data according to preset rules to obtain initial features; According to a preset conversion table, the income data, the occupation data, and the educational background data are respectively mapped into specific values, accumulated, and normalized to obtain socioeconomic status characteristics; The initial features and the socioeconomic status features are combined to generate first statistical features.

4. The method according to claim 1, wherein When the MRI image and user data uploaded by the elderly user are received on the Q&A webpage, a preliminary cognitive examination is performed to generate preliminary assessment results and cognitive function types, including: When receiving the MRI image and user data uploaded by the elderly user on the Q&A webpage, encoding is performed according to the variable type in the user data to generate a second statistical feature; Dividing the nuclear magnetic resonance image into blocks according to a preset size, and flattening each block into image block vectors; Performing linear mapping on each of the image block vectors and concatenating them to obtain an image embedding vector; Concatenating the image embedding vector and the second statistical feature to obtain an overall embedding vector; Inputting the overall embedding vector into a plurality of encoder modules respectively to obtain a plurality of encoding vectors respectively; Passing all the encoding vectors through the fully connected layer and the classification layer in sequence to generate a preliminary inspection evaluation result consisting of multiple detection probabilities; The cognitive type corresponding to the match with the highest detection probability is selected as the cognitive function type corresponding to the elderly user.

5. The method according to claim 4, characterized in that The encoder module includes a word embedding layer, a first normalization layer, a kernel attention layer, a first multiplier layer, a second normalization layer, a multilayer perceptron layer, and a second multiplier layer connected in sequence; There is a short-circuit connection between the output position of the word embedding layer and the input position of the first multiplier layer, and there is a short-circuit connection between the output position of the first multiplier layer and the input position of the second multiplier layer.

6. The method according to claim 1, characterized in that The step of responding to the answer information inputted for the cognitive training question and updating the cognitive training question using a preset target question and answer data set includes: In response to the answer information input for the cognitive training question, calculating the answer similarity between the answer information and the standard answer corresponding to the cognitive training question; Comparing the answer similarity with a preset similarity threshold; If the answer similarity exceeds the similarity threshold, a new cognitive training question is reselected from the preset target question and answer data set and displayed on the question and answer webpage; If the answer similarity does not exceed the similarity threshold, a strongly related question is generated by combining the cognitive training question with the question generation model and displayed on the question and answer webpage.

7. The method according to claim 6, characterized in that If the answer similarity does not exceed the similarity threshold, a question generation model is used to combine the cognitive training questions to generate strongly related questions, including: If the answer similarity does not exceed the similarity threshold, then locate the corresponding paragraph text from the preset target question and answer data set according to the cognitive training question and divide it into multiple text words; Convert each of the text words into a vector and extract the corresponding text features; Inputting the text features and the explanatory text corresponding to the paragraph text into a preset machine learning model in sequence to determine whether the text features meet the preset answer conditions; If so, then the text words belonging to the text feature are hollowed out from the paragraph text until the number of hollowed-out words reaches a preset hollowing threshold, and the hollowed-out paragraph text is used to generate a strongly related question, and the explanatory text corresponding to the paragraph text is used as the standard answer; If not, new text features are selected according to the order of the text words and input into the machine learning model.

8. A cognitive function assessment and training device for the elderly, characterized in that: include: The cognitive initial examination module is used to perform a cognitive initial examination upon receiving the MRI images and user data uploaded by the elderly user on the Q&A webpage, and generate the initial examination assessment results and cognitive function type; a feature fusion module, configured to extract a plurality of resonance image features from the nuclear magnetic resonance image and concatenate the features with the convolution result of the initial examination assessment result to generate a fusion feature if the cognitive function type is any cognitive impairment type; A cognitive review module, configured to perform a cognitive review based on the fusion features, generate a review evaluation result, and display corresponding cognitive training questions on the question-and-answer webpage; a training topic updating module, configured to respond to the answer information inputted for the cognitive training topic and update the cognitive training topic using a preset target question and answer data set, so as to perform cognitive function training on the elderly user; A normal training module, configured to jump to the step of displaying corresponding cognitive training questions on the question-and-answer webpage if the cognitive function type is a normal cognitive type; The feature fusion module is specifically used for: If the cognitive function type is any cognitive impairment type, calling multiple CNN network models with different resolutions to extract corresponding initial feature maps from the magnetic resonance image respectively; Performing bilinear interpolation on each of the initial feature maps to generate an intermediate feature map with the same resolution; Splicing and convolving the intermediate feature maps to generate a multi-scale feature map; The multi-scale feature map is spliced ​​with each of the intermediate feature maps and then input into a self-attention layer to generate a resonant image feature; Concatenating the resonance image feature with the convolution result of the initial inspection evaluation result to generate a fusion feature; The cognitive review module is specifically used for: Classify the fused features in sequence through the fully connected layer to obtain a classification vector corresponding to the fused features; Performing batch normalization on the classification vector through a batch normalization layer to obtain a batch normalized vector; Mapping the batch normalized vector into output features through an activation function; The outputtable feature is used as a new fusion feature, and the step of sequentially classifying the fusion feature through the fully connected layer to obtain a fusion probability vector corresponding to the fusion feature is skipped until the number of skips reaches a preset threshold; The classification layer classifies the output features at the current moment and calculates the corresponding probability values, generating a probability vector as the re-inspection evaluation result; According to the re-examination and evaluation results, the corresponding cognitive type is matched and the corresponding cognitive training questions are selected and displayed on the question-and-answer webpage.

Images