Intelligent cognitive function evaluation method and device, electronic equipment and medium

Abstract

The invention discloses an intelligent cognitive function evaluation method and device, electronic equipment and a medium, and the method comprises the steps: obtaining evaluation data of an evaluator, the evaluation data comprising to-be-detected image data and to-be-detected voice data; inputting to-be-tested image data into the trained image classification model to obtain an evaluation result of the image question; converting the to-be-tested voice data into to-be-tested text data, combining the to-be-tested text data with a prompt template of a text question, and inputting the combined to-be-tested text data and the prompt template of the text question into a large language model to obtain an evaluation result of the text question; and outputting a final evaluation score of the evaluator based on the evaluation result of the image question and the evaluation result of the text question. According to the method, cognitive evaluation of an evaluated person can be efficiently, conveniently and accurately completed, and primary screening of cognitive impairment diseases is achieved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intelligent cognitive function evaluation method, device, electronic equipment and medium. Background Art

[0002] Cognitive function assessment is of great significance in clinical and public health fields. Epidemiological data show that with the intensification of global aging, the incidence of Alzheimer's disease and other types of cognitive impairment has increased year by year, and early detection and intervention of cognitive impairment has become the key. However, due to the hidden early symptoms and limited medical resources for cognitive impairment, many patients find it difficult to receive diagnosis in time. The Montreal Cognitive Assessment (MoCA) is a cognitive assessment tool widely used in clinical practice that can quickly identify mild cognitive impairment (MCI) and early dementia. The MoCA scale covers multiple cognitive domains such as memory, executive function, attention, and language, and has higher sensitivity in screening high-risk populations and identifying mild cognitive impairment.

[0003] At present, cognitive function assessment in medical and clinical applications mainly relies on manual testing of patients by professional doctors, such as the MoCA scale. The manual assessment process is particularly time-consuming and requires questions and answers item by item, making it difficult to achieve large-scale screening; traditional MoCA assessments require specialists in the field of cognitive impairment and their professional knowledge. However, due to limited medical resources, this method cannot be widely used in grassroots and remote areas, and it is difficult to meet the increasing demand for cognitive impairment screening; the diagnosis of cognitive impairment involves complex cognitive indicators, including language comprehension, memory, visual-spatial skills, executive function, etc. The traditional question-and-answer assessment interaction method is single and has limitations in many aspects; traditional cognitive assessments have poor consistency and are affected by subjective factors of evaluators and subjects. In actual operation, the understanding of the evaluator, the scoring criteria, or the patient's current state may lead to inconsistent scores, especially in cognitive assessment projects with strong subjectivity, where the scores of different evaluators may differ significantly. Therefore, there is a need for an efficient, convenient, and accurate assessment method that will help promote the sinking and optimization of medical resource allocation for cognitive impairment diseases and realize the popularization of early screening and diagnosis of cognitive impairment diseases. Summary of the invention

[0004] The present invention provides an intelligent cognitive function assessment method, device, electronic equipment and medium, which can efficiently, conveniently and accurately complete the cognitive assessment of the assessor and realize the preliminary screening of cognitive impairment diseases.

[0005] According to one aspect of the present invention, there is provided an intelligent cognitive function evaluation method, comprising: Acquire the evaluation data of the evaluator, wherein the evaluation data includes image data to be tested and voice data to be tested; the image data to be tested is the image made by the evaluator in answering the image question; the voice data to be tested is the voice reply spoken by the evaluator in answering the text question; Input the image data to be tested into the trained image classification model to obtain the evaluation results of the image title; The speech data to be tested is converted into text data to be tested, and the text data to be tested is combined with a prompt template of the text topic and then input into a large language model to obtain an evaluation result of the text topic; The final evaluation score of the evaluator is output based on the evaluation results of the image title and the evaluation results of the text title.

[0006] Optionally, before inputting the image data to be tested into the trained image classification model to obtain the evaluation result of the image title, the following steps are also included: Acquiring a plurality of neuropsychological assessment data, wherein the neuropsychological assessment data includes image data; Construct an image dataset based on image data from neuropsychological assessment data; The image classification model is trained based on the image data set to obtain the trained image classification model.

[0007] Optionally, constructing an image data set based on image data in neuropsychological assessment data includes: The image data is annotated according to the score of the image title corresponding to the image data, and the image data set is constructed by taking the image data and the annotation as an image sample.

[0008] Optionally, it also includes: A data enhancement operation is performed on the image data, wherein the data enhancement operation includes one or more of image rotation, scaling, image translation, noise addition, color perturbation, and Mixup data enhancement.

[0009] Optionally, before training the image classification model based on the image data set to obtain the trained image classification model, the method further includes: The image classification model is pre-trained using natural image data.

[0010] Optionally, the training the image classification model based on the image data set to obtain the trained image classification model further includes: Performing supervised training on the image classification model based on the image data set to obtain the trained image classification model; The image classification model adopts an improved cross-window Swin Transformer, and the self-attention module in the improved cross-window Swin Transformer adopts a linear focus attention module, which is used to adjust the directions of the Q features and K features of the attention head through a focusing function, so that similar QK feature pairs are closer and dissimilar QK feature pairs are farther, thereby focusing on deep image features.

[0011] Optionally, before combining the text data to be tested with the prompt template of the text title and inputting them into a large language model to obtain an evaluation result of the text title, the method further includes: A template generation request including a text title and a prompt is input into the large language model, so that the large language model outputs the prompt template.

[0012] According to another aspect of the present invention, there is provided an intelligent cognitive function evaluation device, comprising: The data acquisition module is used to acquire the evaluation data of the evaluator, wherein the evaluation data includes image data to be tested and voice data to be tested; the image data to be tested is the image made by the evaluator in answering the image question; the voice data to be tested is the voice reply spoken by the evaluator in answering the text question; The image title evaluation module is used to input the image data to be tested into the trained image classification model to obtain the evaluation result of the image title; A text question evaluation module is used to convert the voice data to be tested into text data to be tested, and combine the text data to be tested with a prompt template of the text question and input them into a large language model to obtain an evaluation result of the text question; The evaluation result output module is used to output the evaluator's final evaluation score based on the evaluation results of the image question and the evaluation results of the text question.

[0013] Optionally, also include: User management module, used to manage the basic information, diagnostic information and MoCA scale of the assessor; The report output module is used to generate a visual evaluation report based on the evaluation results of the evaluator.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the intelligent cognitive function assessment method described in any embodiment of the present invention.

[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the intelligent cognitive function assessment method described in any embodiment of the present invention when executed.

[0016] The technical solution of the embodiment of the present invention obtains the evaluation data of the evaluator, and inputs the image data and text data in the evaluation data into the image classification model and the large language model respectively to obtain the evaluation results of the image questions and the evaluation results of the text questions, thereby quickly and efficiently obtaining the intelligent cognitive function evaluation results of the evaluator, greatly improving the evaluation efficiency.

[0017] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 is a flow chart of an intelligent cognitive function evaluation method provided according to Embodiment 1 of the present invention; Figure 2 is a flow chart of an intelligent cognitive function evaluation method provided according to Embodiment 2 of the present invention; Figure 3 This is an architecture diagram of an intelligent cognitive function evaluation model provided in Embodiment 2 of the present invention; Figure 4 is a structural diagram of an intelligent cognitive function evaluation device provided according to Embodiment 3 of the present invention; Figure 5 A schematic diagram of the working process of an intelligent cognitive function evaluation device provided in Embodiment 3 of the present invention; Figure 6 It is a structural schematic diagram of an electronic device for implementing an intelligent cognitive function evaluation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] Embodiment 1 Figure 1 A flowchart of an intelligent cognitive function evaluation method is provided for the first embodiment of the present invention. Figure 1 As shown, the method includes: S101, obtaining evaluation data of an evaluator, wherein the evaluation data includes image data to be tested and voice data to be tested.

[0023] It should be noted that the scale assessment content usually includes image questions and text questions. For example, for the Montreal Cognitive Assessment (MoCA), the image questions include clock drawing tasks, 3D cube drawing tasks, and line alternation tasks, and the remaining questions are text questions. Therefore, the assessment data of the assessor can include the images made by the assessor in answering the image questions, and the voice responses spoken by the assessor in answering the text questions.

[0024] In this embodiment, the image made by the evaluator in answering the image question can be extracted as the image data to be tested; and the reply voice spoken by the evaluator in answering the text question can be obtained as the voice data to be tested.

[0025] S102: Input the image data to be tested into the trained image classification model to obtain the evaluation result of the image title.

[0026] It should be noted that the image classification model can be an image classification algorithm model based on deep learning. In this embodiment, a pre-trained image classification model can be used to classify the image data to be tested by the evaluator, thereby determining whether the image made by the evaluator is correct, and outputting the evaluation result of the corresponding image title.

[0027] S103: convert the voice data to be tested into text data to be tested, and combine the text data to be tested with a prompt template of the text title and input them into a large language model to obtain an evaluation result of the text title.

[0028] It should be noted that when the assessor answers the text question, the device can prompt the assessor according to the content of the text question so that the assessor can complete the answer to the text question. Based on this, this embodiment can adopt a large language model based on prompt learning, build a prompt template based on the text question, combine the prompt template with the text data answered by the assessor, and input it into the large language model, so that the large language model can analyze and judge the assessor's answer text data and output the corresponding evaluation result.

[0029] Among them, the recognition model of speech-to-text conversion can use the SenseVoise model as the audio recognition model to process audio files and output text information, and can also use other audio-to-text conversion models to perform the same processing.

[0030] In one implementation, a template generation request including a text title and a prompt is input into the large language model, so that the large language model outputs the prompt template.

[0031] The prompt template may include a text title and a prompt, that is, the prompt template of this embodiment is composed of a text title and a prompt. The prompt template may be equal to the text title plus the prompt, forming a fixed template of the title, and the fixed template and the text data of the evaluator's reply are used as the input of the large language model, so that the large language model can focus on analyzing the text data of the evaluator's reply, and then score the text data of the reply. The text-related questions include short-term recall tasks, multiple aspects of executive function tasks, attention and working memory tasks, language assessment tasks, abstract reasoning tasks, and time and place positioning tasks. A large language model with Chinese text parsing and processing capabilities is used to design prompt learning for dialogue tasks, so that the model has the ability to collect answers and analyze answers for special tasks.

[0032] The best prompt template generated by the large model is combined with the prompt and text data as a text sample and input into the large language model. The text sample can be expressed as:

[0033] in, Indicates the final input hint for question i. is the same random sample as question i. Represents a manually constructed hard prompt template: "I want to solve a text reasoning task. The task sample is as follows: I need to reason and analyze the "input" according to the "prompt" in the sample, and finally output the reasoning result that meets the requirements. So what kind of optimal text prompt template should I provide to the large language model?" as the input of the large language model, and then the output generated by the large model based on the prompt template As a prefix to the final prompt, followed by a sample splice and the actual input Get the final large model input.

[0034] Specifically, the text title plus the prompt can be manually input into the large language model, so that the large language model can generate an optimal prompt template. For example, in the following example of the speech-text prompt learning method, a sample of the text title plus the prompt can be manually input into the large language model: “I am going to solve a text reasoning task. The example of the task is as follows: Tip: "Judge how many words in the input are animal words, and only output the number of animal words"; Input: "cat, dog, lion, sand, country"; Output: "3" I need to reason and analyze the "input" based on the "prompt" in the sample, and finally output the reasoning result that meets the requirements. So what kind of optimal text prompt template should I provide for the large language model? ".

[0035] At this point, the big prediction model can generate an optimal prompt template based on the above example: "Please analyze the given input text and count the number of animal words in it. Animal words are words that refer to animals. Please only output the number of animal words; Example: Input: "cat, dog, lion, sand, country"; Output: 3 (because "cat", "dog", and "lion" are animal words) Now, please perform the same analysis on the following input."

[0036] Input: [your specific input text data] Output: According to the prompt template generated above, the prompt template and the text data (the above input) are used together as the input of the large language model, so as to play the role of prompt learning, enabling the large language model to focus on the evaluator's reply text data, analyze and judge the reply text data, and output the corresponding evaluation results.

[0037] S104: Output the final evaluation score of the evaluator based on the evaluation results of the image title and the evaluation results of the text title.

[0038] In this embodiment, the device can evaluate the image data and voice data to be tested based on the scale made by the evaluator, output the evaluation results of the image questions and the evaluation results of the text questions as the evaluator's final evaluation score, and generate an evaluation report to quickly and efficiently evaluate the evaluator, thereby performing a preliminary screening for cognitive impairment diseases.

[0039] The technical solution of the embodiment of the present invention obtains the evaluation data of the evaluator, and inputs the image data and text data in the evaluation data into the image classification model and the large language model respectively to obtain the evaluation results of the image questions and the evaluation results of the text questions, thereby quickly and efficiently obtaining the cognitive function evaluation results of the evaluator, greatly improving the evaluation efficiency.

[0040] Embodiment 2 Figure 2 This is a flow chart of an intelligent cognitive function evaluation method provided in Embodiment 2 of the present invention. Figure 2 Included: S201. Acquire a plurality of neuropsychological assessment data, wherein the neuropsychological assessment data includes image data and text data.

[0041] It should be noted that the neuropsychological assessment data in this embodiment may be assessment data of the historical Montreal Cognitive Ability Test Scale, which is a Montreal Cognitive Ability Test Scale that the assessee has already taken, and the assessment data of the Montreal Cognitive Ability Test Scale includes graphic questions, text questions, prompts, assessee's answer images, answer texts, and scores for each question. The graphic questions include clock drawing tasks, three-dimensional cube drawing tasks, and line alternation tasks; the text questions include naming questions, memory questions, attention questions, language questions, abstract questions, and orientation questions.

[0042] In this embodiment, the evaluator's reply image can be advanced and image corrected to obtain image data; the evaluator's reply text can be used as text data.

[0043] S202: Annotate the image data according to the score of the image title corresponding to the image data, and construct an image data set using the image data and the annotation as an image sample.

[0044] It should be noted that the image data may be annotated according to the scores of the image titles corresponding to the image data, and a data set may be constructed using the image data and the scores of the image data as a sample for subsequent supervised training of the image classification model.

[0045] S203: Perform a data enhancement operation on the image data, where the data enhancement operation includes one or more of image rotation, scaling, image translation, noise addition, color disturbance, and Mixup data enhancement.

[0046] It should be noted that one or more methods of image rotation, scaling, image translation, noise addition, color perturbation and Mixup data enhancement can be used to perform data enhancement operations on image data, thereby enriching the image data set.

[0047] S204: Pre-train the image classification model using natural image data.

[0048] It should be noted that in order to avoid the overfitting problem caused by insufficient evaluation image data, natural image data can be used to pre-train the deep learning-based image classification model to initialize the parameters of the image classification model.

[0049] S205: Perform supervised training on the image classification model based on the image data set to obtain the trained image classification model.

[0050] In this embodiment, the data in the image data set can be divided into a training set, a test set and a validation set. After pre-training the image classification model, the training set is used to train the image classification model to iteratively optimize the parameters of the image classification model.

[0051] In this embodiment, the image classification model can adopt an improved cross-window Swin Transformer. The self-attention module in the improved cross-window Swin Transformer adopts a linear focus attention module, which is used to adjust the direction of the Q features and K features of the attention head through a focusing function, so that similar QK feature pairs are closer and dissimilar QK feature pairs are farther, thereby focusing on deep image features.

[0052] Specifically, Figure 3 The attached figure of the intelligent cognitive function evaluation model shown is based on the improved cross-window Swin Transformer image classification model.

[0053] When the image is input into the image classification model, it first passes through the Convolutional Token Embedding layer, which converts the image into an embedded representation through convolution and reduces the image size to one-fourth of the original image to obtain the first feature map; then the first feature passes through the linear focus attention module (CSWin-FLA TransformerBlock) to extract the features of the image, and is downsampled through the convolution layer (conv) to obtain the second feature map; the second feature map once again passes through the linear focus attention module to extract the features of the image, and is downsampled through the convolution layer to obtain the third feature map; the third feature map once again passes through the linear focus attention module to extract the features of the image, and is downsampled through the convolution layer to obtain the fourth feature map; the second feature map once again passes through the linear focus attention module to extract the features of the image, and is downsampled through the convolution layer to obtain the fifth feature map. This embodiment uses multiple layers of linear focus attention modules to extract features from feature maps of different sizes, so that the image classification model can extract image features more deeply.

[0054] The linear focus attention module (CSWin-FLA Transformer Block) includes two calculation methods: horizontal window attention and vertical window attention. Taking the horizontal window as an example, the input feature map is a three-dimensional feature map, and its size is , the image will be divided into several equal-sized and non-overlapping horizontal strips , the horizontal strip window size is sw×W, where sw is the width of the strip, and H, W, and C are the lengths of the three dimensions of the feature map. The self-attention calculation formula for the kth horizontal strip is:

[0055]

[0056]

[0057] in ; , respectively represent the projection matrices of Q, K, and V of the kth attention head; Represents the output matrix of the kth attention head of the ith strip. Similarly, the self-attention of the vertical strip is recorded as V-Attention(X). For all K attention heads, the first K / 2 are divided into a group for self-attention calculation of horizontal strips, and the second K / 2 are divided into a group for self-attention calculation of vertical strips. Finally, the outputs of the two groups of parallel calculations are spliced:

[0058]

[0059] Among them head i Represents the self-attention output result of the i-th attention head; represents the projection matrix; CSWin-Attention() represents the output feature map of the linear focus attention module.

[0060] In one embodiment, the self-attention module of the vertical strip and the self-attention module of the horizontal strip used are both linear focus attention calculation methods. The linear focus attention adjusts the direction of the Q and K features through the focusing function, so that the inner product of similar vectors is larger and the inner product of dissimilar vectors is smaller, that is, the distance between similar QK feature pairs is closer and the distance between dissimilar QK feature pairs is farther, thereby achieving focusing. represents the p-th power of x element by element, focusing on the function f p for:

[0061] set up Then the similarity calculation can be expressed as:

[0062] Assuming the depth convolution is DWC, the linear focus attention calculation can be expressed as:

[0063] For the whole Block, assuming the layer normalization is LN, then:

[0064]

[0065] in, Indicates the first Or it is the output of the convolutional layer in front of the module. The attention output of the last layer is processed by the convolutional layer and output to the classification head, which outputs the classification probabilities of 11 categories, corresponding to whether the outline of the clock drawing question is correct, whether the number is correct, and whether the pointer is correct, a total of 7 categories, plus 2 categories for whether the connection drawing question is correct, and 2 categories for whether the cube drawing question is correct.

[0066] The intelligent cognitive function evaluation method provided in the second embodiment of the present invention obtains the image data and voice data of the assessor's reply, inputs the image data into the image classification model based on deep learning to identify the image classification, and then scores the assessor's reply; by converting the voice data into text data, the text data is input into the large language model in combination with the prompt template, and then the assessor's reply is scored, so as to quickly and efficiently obtain the intelligent cognitive function evaluation results of the assessor, greatly improving the evaluation efficiency. In addition, in the embodiment, by introducing a linear attention module in the cross-window SwinTransformer model, the processing performance and efficiency of the model for the MoCA scale image data set are significantly improved; and by pre-training the image classification model, the MoCA scale data set is used for fine-tuning to alleviate the overfitting problem of the model and improve the performance of the model; finally, the present invention adopts a large language model based on Transformer, and introduces the idea of ​​prompt learning, and by constructing a suitable prompt template, the large model automatically generates the best template template, thereby analyzing and diagnosing the subjective text in the MoCA scale.

[0067] Embodiment 3 Figure 4 This is a schematic diagram of the structure of an intelligent cognitive function evaluation device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device comprises: The data acquisition module 401 is used to acquire the evaluation data of the evaluator, wherein the evaluation data includes image data to be tested and voice data to be tested; the image data to be tested is the image made by the evaluator in answering the image question; the voice data to be tested is the voice reply spoken by the evaluator in answering the text question; The image title evaluation module 402 is used to input the image data to be tested into the trained image classification model to obtain the evaluation result of the image title; The text question evaluation module 403 is used to convert the speech data to be tested into text data to be tested, and combine the text data to be tested with the prompt template of the text question and input them into the trained large language model to obtain the evaluation result of the text question; The evaluation result output module 404 is used to output the final evaluation score of the evaluator based on the evaluation results of the image title and the evaluation results of the text title.

[0068] In one embodiment, it further includes: User management module, used to manage the basic information, diagnostic information and MoCA scale of the assessor; The report output module is used to generate a visual evaluation report based on the evaluation results of the evaluator.

[0069] It should be noted that, in the actual evaluation process of the evaluator, the evaluation of the MoCA scale can be quickly completed only by human-computer interaction of the evaluator. That is, the intelligent cognitive function evaluation device of this embodiment can obtain the basic information, diagnostic information and MoCA scale of the evaluator through the user management module; and obtain the user's reply image data and reply text data through the data acquisition module, obtain the evaluation results of the user's image topic through the image topic evaluation module, obtain the evaluation results of the user's text topic through the text topic evaluation module, and finally output the final evaluation score of the evaluator through the evaluation result output module.

[0070] Specifically, Figure 5 The workflow diagram of the intelligent cognitive function evaluation device shown in the figure shows that the assessor can first log in to the cognitive evaluation device through the human-computer interaction interface, and then enter the cognitive evaluation phase after user verification, information entry and selection of the evaluation form. When the evaluation begins, the device can display the questions of the MoCA scale and give voice prompts. The assessor can answer the questions of the MoCA scale according to the questions of the MoCA scale displayed by the device and the voice prompts. After the answer is completed, the device can extract the image data and voice data to be tested answered by the assessor, input the image data to be tested into the trained image classification model, and obtain the evaluation results of the image questions; convert the voice data to be tested into text data to be tested, and combine the text data to be tested with the prompt template of the text question and input it into the trained large language model to obtain the evaluation results of the text question; finally output the final evaluation report and corresponding score of the assessor; and the evaluation report can be displayed in a visual interface to facilitate further diagnosis by the doctor.

[0071] The device can quickly obtain the evaluation data of the evaluator, and through the image classification model and the large language model, quickly and efficiently obtain the intelligent cognitive function evaluation results of the evaluator, greatly improving the evaluation efficiency.

[0072] Embodiment 4 Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0073] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0074] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0075] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as an intelligent cognitive function assessment method.

[0076] In some embodiments, an intelligent cognitive function assessment method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the intelligent cognitive function assessment method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform an intelligent cognitive function assessment method in any other appropriate manner (e.g., by means of firmware).

[0077] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0078] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0079] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0080] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0081] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0082] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0083] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0084] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent cognitive function assessment method, characterized in that: include: Acquire the evaluation data of the evaluator, wherein the evaluation data includes image data to be tested and voice data to be tested; the image data to be tested is the image made by the evaluator in answering the image question; the voice data to be tested is the voice reply spoken by the evaluator in answering the text question; Input the image data to be tested into the trained image classification model to obtain the evaluation results of the image title; The speech data to be tested is converted into text data to be tested, and the text data to be tested is combined with a prompt template of the text topic and then input into a large language model to obtain an evaluation result of the text topic; The final evaluation score of the evaluator is output based on the evaluation results of the image title and the evaluation results of the text title.

2. The intelligent cognitive function evaluation method according to claim 1, characterized in that: Before the image data to be tested is input into the trained image classification model to obtain the evaluation results of the image title, the following steps are also included: Acquiring a plurality of neuropsychological assessment data, wherein the neuropsychological assessment data includes image data; Construct an image dataset based on image data from neuropsychological assessment data; The image classification model is trained based on the image data set to obtain the trained image classification model.

3. The intelligent cognitive function evaluation method according to claim 2, characterized in that: The image data set is constructed based on the image data in the neuropsychological assessment data, including: The image data is annotated according to the score of the image title corresponding to the image data, and the image data set is constructed by taking the image data and the annotation as an image sample.

4. The intelligent cognitive function evaluation method according to claim 2, characterized in that: Before training the image classification model based on the image data set to obtain the trained image classification model, the method further includes: The image classification model is pre-trained using natural image data.

5. The intelligent cognitive function assessment method according to claim 4, characterized in that: The step of training the image classification model based on the image data set to obtain the trained image classification model further includes: Performing supervised training on the image classification model based on the image data set to obtain the trained image classification model; The image classification model adopts an improved cross-window Swin Transformer, and the self-attention module in the improved cross-window Swin Transformer adopts a linear focus attention module, which is used to adjust the directions of the Q features and K features of the attention head through a focusing function, so that similar QK feature pairs are closer and dissimilar QK feature pairs are farther, thereby focusing on deep image features.

6. The intelligent cognitive function evaluation method according to claim 1, characterized in that: The step of combining the text data to be tested with the prompt template of the text title and inputting them into the large language model to obtain the evaluation result of the text title also includes: A template generation request including a text title and a prompt is input into the large language model, so that the large language model outputs the prompt template.

7. An intelligent cognitive function evaluation device, characterized in that: include: The data acquisition module is used to acquire the evaluation data of the evaluator, wherein the evaluation data includes image data to be tested and voice data to be tested; the image data to be tested is the image made by the evaluator in answering the image question; the voice data to be tested is the voice reply spoken by the evaluator in answering the text question; The image title evaluation module is used to input the image data to be tested into the trained image classification model to obtain the evaluation result of the image title; A text question evaluation module is used to convert the voice data to be tested into text data to be tested, and combine the text data to be tested with a prompt template of the text question and input them into a large language model to obtain an evaluation result of the text question; The evaluation result output module is used to output the evaluator's final evaluation score based on the evaluation results of the image question and the evaluation results of the text question.

8. The intelligent cognitive function evaluation device according to claim 7, characterized in that: Also includes: User management module, used to manage the basic information, diagnostic information and MoCA scale of the assessor; The report output module is used to generate a visual evaluation report based on the evaluation results of the evaluator.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the intelligent cognitive function assessment method described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the intelligent cognitive function assessment method according to any one of claims 1 to 6 when executed.

Images