Digital cognitive evaluation system and method based on virtual human

Abstract

The invention discloses a digital cognitive evaluation system and method based on a virtual human. The cognitive evaluation system comprises a system end, a doctor end and a user end, the doctor end and the user end are connected with the system end, the user end interacts with a user based on a virtual human, and then, answers of the user are judged or scored according to a preset model to obtain scale evaluation data of the user; and after the user completes the cognitive evaluation process, the evaluation personnel can correct the judgment or score of the preset model at the doctor end, and finally, the final cognitive evaluation result of the user is generated after confirmation. Therefore, through the application of the virtual human, an auxiliary person does not need to accompany the user for a long time to carry out cognitive evaluation, the demand for human resources can be reduced, and cognitive evaluation can be carried out on multiple users at the same time in a single time.

Description

Technical Field

[0001] The present invention relates to a digital cognitive assessment system based on virtual humans, and also to a corresponding digital cognitive assessment method, belonging to the technical field of cognitive assessment. Background Art

[0002] In existing technologies, cognitive assessments are typically performed using the Mini-Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA), the Alzheimer's Disease Screening Scale (AD8), and other tools. Currently, digital assessment scales are commercially available that allow subjects to perform screenings with the assistance of an assessor, with the results recorded online. Compared to traditional paper-based assessment scales, digital assessment scales are more efficient and make the results easier to store and disseminate.

[0003] However, digital assessment scales often require the full presence of assessors to judge and record the subjects' responses, a process that consumes significant human resources. In the absence of adequate assessors, subjects often face long wait times. Furthermore, different assessors may make different judgments on certain responses, leading to inconsistent assessment results.

[0004] In Chinese patent application number 202310995715.4, a cognitive assessment method and system based on intelligent guidance and algorithmic analysis are disclosed. This cognitive assessment method includes the following steps: entering patient information and establishing an initial cognitive assessment task in the cognitive assessment client subsystem; conducting a cognitive assessment to obtain the patient's assessment data; the cognitive assessment algorithm model subsystem performs data calculations based on the assessment data to obtain assessment scores; and calibrating and reviewing the assessment scores to generate a cognitive assessment report. This cognitive assessment method avoids scoring discrepancies caused by differences in the cognition and professionalism of professionals, making the assessment results more objective and accurate, and achieving the unification of scale assessment results. Summary of the Invention

[0005] The primary technical problem to be solved by the present invention is to provide a digital cognitive assessment system based on virtual humans.

[0006] Another technical problem to be solved by the present invention is to provide a digital cognitive assessment method based on virtual humans.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, there is provided a digital cognitive assessment system based on a virtual human, comprising a system terminal, a doctor terminal, and a user terminal connected to each other;

[0009] The system side includes:

[0010] The scale management module has a variety of pre-set evaluation scales for user cognitive evaluation;

[0011] Data storage module, used to store user's scale evaluation data;

[0012] Intent recognition module, used to identify the intent of the user's voice data;

[0013] an intelligent judgment module, connected to the intention recognition module, to generate a preliminary judgment result based on the user's intention recognition result;

[0014] The doctor side includes:

[0015] A scale publishing module, connected to the scale management module, is used to send the scale to be evaluated to the user end;

[0016] A scale result display module, connected to the data storage module, for displaying the user's scale evaluation data;

[0017] A review module, connected to the intelligent judgment module, for manually reviewing the preliminary judgment result;

[0018] The user terminal includes:

[0019] A virtual human display module is connected to the scale publishing module to explain the scale questions to the user through the virtual human; the virtual human display module is also connected to the intention recognition module to provide interactive feedback to the user through the virtual human based on the intention recognition result;

[0020] The voice collection module is used to collect the user's voice data when conducting cognitive assessment;

[0021] A speech recognition module, connected to the speech acquisition module, for performing speech recognition on the speech data; the speech recognition module is also connected to the intention recognition module, for transmitting speech recognition content to the intention recognition module;

[0022] A scale logic module, connected to the virtual human display module, to output the user's scale evaluation data for the scale to be evaluated based on a preset model;

[0023] The scale result processing module is connected to the scale logic module and the data storage module to receive the scale evaluation data and upload it to the data storage module.

[0024] Preferably, each of the doctor terminals corresponds to multiple user terminals.

[0025] Preferably, there are multiple user terminals, and each user terminal is connected to the system terminal and the doctor terminal through an independent data channel.

[0026] Preferably, the system end also includes a comparison module, which is connected to the review module to obtain the review result; the comparison module is also connected to the scale logic module to feed back the review result to the scale logic module and iteratively update the preset model.

[0027] Preferably, the user terminal is a mobile phone or computer with a preset application downloaded thereon.

[0028] According to a second aspect of an embodiment of the present invention, a digital cognitive assessment method using the digital cognitive assessment system is provided, comprising the following steps:

[0029] Send the evaluation scale to the user end through the doctor end;

[0030] Open the to-be-assessed scale through the user terminal, and explain the questions in the to-be-assessed scale based on the virtual person;

[0031] Obtain the user's answer to the current question and switch to the next question through the virtual person until the user completes all questions in the assessment scale;

[0032] Obtaining the assessed scale of the user after completing the cognitive assessment through the system end, and sending the assessed scale to the doctor end;

[0033] Opening the evaluated scale through the doctor's terminal and manually reviewing the answer results in the evaluated scale;

[0034] The manual review result of the user is saved as the cognitive assessment result of the user this time.

[0035] Preferably, the step of sending the evaluation scale to be measured from the doctor end to the user end specifically includes:

[0036] Obtaining personal information input by the user through the user terminal, wherein the personal information includes at least age, name, education level, and type of disease;

[0037] Based on the user's personal information, select a required scale from the scale management module on the system side as the scale to be evaluated;

[0038] The evaluation scale is sent to the user terminal through the doctor terminal.

[0039] Preferably, during the cognitive assessment process of the user, if the answer to the current question is correct, the virtual person provides positive feedback to the user; if the answer to the current question is wrong, the virtual person provides active encouragement to the user.

[0040] Preferably, the answer result at least includes the user's voice data, drawing data, and click data.

[0041] Preferably, during the manual review process, the doctor can modify the manual review result multiple times to save the final manual review result and upload it to the system.

[0042] Compared with the prior art, the present invention has the following technical effects:

[0043] 1. The use of virtual humans eliminates the need for assistants to accompany subjects for extended periods of time during assessments, reducing the demand on medical personnel and enabling the simultaneous assessment of multiple subjects within a single timeframe. Furthermore, deep learning-based intent recognition ensures consistent assessments.

[0044] 2. Through network testing and multi-terminal deployment of services, the best service plan is dynamically selected according to network speed to ensure smooth user experience.

[0045] 3. All functional modules on the system, physician, and user sides are written in Go, which features simple development, low resource usage, and high scalability for cluster deployment. Furthermore, Go applications are compact in size, making them particularly suitable for cloud computing scenarios like cluster deployment, maximizing the utilization of software and hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A structural diagram of a digital cognitive assessment system based on a virtual human provided by the first embodiment of the present invention;

[0047] Figure 2 This is a structural diagram of the doctor's terminal in the first embodiment of the present invention;

[0048] Figure 3 This is a structural diagram of the user terminal in the first embodiment of the present invention;

[0049] Figure 4 This is an overall flow chart of a digital cognitive assessment method based on a virtual human provided by the second embodiment of the present invention;

[0050] Figure 5 This is a detailed flow chart of a digital cognitive assessment method based on a virtual human provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0051] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The main concept of the embodiments of the present invention is to provide a digital cognitive assessment system based on virtual humans, which interacts with users and then judges or scores the user's answers based on a deep learning model. After the user completes the interaction process, the evaluator (for example, a doctor) can modify the judgment or score of the deep learning model on the doctor's side, and finally generate the user's final cognitive assessment results after confirmation. Therefore, through the use of virtual humans, auxiliary personnel do not need to accompany users for a long time during the assessment, which can reduce the demand for medical human resources and realize the simultaneous cognitive assessment of multiple users in a single time.

[0053] First embodiment

[0054] like Figure 1 As shown, the first embodiment of the present invention provides a digital cognitive assessment system based on a virtual human, including a system end 10, a doctor end 20 and a user end 30, wherein the doctor end 20 and the user end 30 are respectively connected to the system end 10 to achieve data synchronization, so as to complete the digital cognitive assessment of the user using the user end 30, and review the cognitive assessment results of the user using the doctor end 20 to ensure the accuracy of the cognitive assessment.

[0055] Specifically, such as Figure 1 As shown, the system end 10 includes a scale management module 11, a data storage module 12, an intention recognition module 13 and an intelligent judgment module 14. Among them, the scale management module 11 is preset with a variety of assessment scales, such as the Mini-Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA), the Alzheimer's Disease Screening Scale (AD8), etc., which are used to perform cognitive assessments on users. The data storage module 12 is used to store the user's scale assessment data. It can be understood that the scale assessment data includes the user's screen click data, voice data, and drawing data during the cognitive assessment process. The intention recognition module 13 is used to perform intent recognition on the user's voice data for interaction with the user. The intelligent judgment module 14 is connected to the intention recognition module 13 to generate a preliminary judgment result based on the user's intention recognition result, so as to be used for subsequent manual review by the doctor end 20.

[0056] like Figure 2As shown, in one embodiment of the present invention, the doctor end 20 includes a scale publishing module 21, a scale result display module 22 and a review module 23. Among them, the scale publishing module 21 is connected to the scale management module 11, and is used to send the scale to be evaluated to the user end. The scale to be evaluated is one or more of the above-mentioned multiple evaluation scales, and can be adaptively selected according to the user's personal information. The scale result display module 22 is connected to the data storage module 12, and is used to display the user's scale evaluation data for the doctor to review. The review module 23 is connected to the intelligent judgment module 14, and is used to manually review the user's preliminary judgment results. It can be understood that during the manual review process, the doctor can modify the review results multiple times to avoid misjudgment.

[0057] like Figure 3 As shown, in one embodiment of the present invention, the user terminal 30 includes a virtual human display module 31, a voice collection module 32, a voice recognition module 33, a scale logic module 34, and a scale result processing module 35. The virtual human display module 31 is connected to the scale publishing module 21 to explain the scale questions to the user through a virtual human. Furthermore, the virtual human display module 31 is also connected to the intention recognition module 13 to provide interactive feedback to the user through the virtual human based on the intention recognition results. The voice collection module 32 is used to collect voice data from the user when performing cognitive assessment. The voice recognition module 33 is connected to the voice collection module 32 to perform voice recognition on the voice data. Furthermore, the voice recognition module 33 is also connected to the intention recognition module 13 to transmit the voice recognition content to the intention recognition module so that the user's intention can be identified based on the voice recognition content. The scale logic module 34 is connected to the virtual human display module 31 to output the user's scale evaluation data for the scale to be evaluated based on a preset model. In a preferred embodiment of the present invention, the preset model is a deep learning model built based on historical data, which is used to determine whether the user's answer is correct based on preset logic. The scale result processing module 35 is connected to the scale logic module 34 and the data storage module 12 to receive the scale assessment data and upload it to the data storage module 12.

[0058] In addition, as shown in FIG3 , in the above embodiment, preferably, the user terminal 30 also includes a resource management module 36 for controlling the operation of other modules of the user terminal 30 and storing corresponding cognitive assessment data for subsequent research and investigation.

[0059] In the above embodiment, each doctor terminal 20 preferably corresponds to multiple user terminals 30. This allows a single doctor to simultaneously correspond to multiple users, optimizing the allocation of medical human resources through a one-to-many matching approach. Furthermore, the user terminal 30 can be a mobile phone or computer with a pre-installed APP (application program) downloaded. Furthermore, when there are multiple user terminals 30, each user terminal 30 is connected to the system terminal 10 and the doctor terminal 20 via an independent data channel, thereby avoiding data interference and reducing the possibility of misjudgment by the doctor.

[0060] In the above embodiment, preferably, the system end 10 further includes a comparison module (not shown in the figure), which is connected to the review module 23 to obtain the review results. In addition, the comparison module is also connected to the scale logic module 34 to feed the review results back to the scale logic module, thereby iteratively updating the model parameters of the preset model. It can be understood that as the deep learning model is continuously iteratively updated, the prediction accuracy of the model will continue to improve, which will help reduce the review work of doctors.

[0061] In the above embodiment, the optimal service option can be dynamically selected based on network speed to ensure smooth user experience. Specifically, the dynamic service option selection includes: before entering the evaluation scale, the user terminal will check the network speed of the connected backend service. If the network speed falls below a preset threshold, it will switch to another backend service. The switching order is: cloud data center service -> edge computing node service -> user terminal local service.

[0062] In the above embodiment, each functional module in the system end 10, the doctor end 20, and the user end 30 is preferably written in Go, which features simple development, low resource usage, and high scalability for cluster deployment. Furthermore, at the deployment level, Go applications are compact in size and are particularly suitable for cloud computing scenarios such as cluster deployment, maximizing the utilization of software and hardware resources.

[0063] Second embodiment

[0064] like Figure 4 and Figure 5 As shown, based on the above first embodiment, the second embodiment of the present invention further provides a digital cognitive assessment method based on a virtual human, which specifically includes steps S1 to S6:

[0065] S1: Send the evaluation scale to be measured to the user terminal 30 via the doctor terminal 20.

[0066] Specifically, it includes steps S11 to S13:

[0067] S11: Obtaining personal information input by the user through the user terminal 30, the personal information at least including age, name, education level, and disease type.

[0068] S12: Based on the user's personal information, a required scale is selected from the scale management module on the system side as the scale to be evaluated.

[0069] S13: The evaluation scale to be measured is sent to the user terminal 30 via the doctor terminal 20 .

[0070] S2: The user terminal 30 opens the scale to be evaluated, and the virtual person explains the questions in the scale to be evaluated.

[0071] In this embodiment, when the user opens the evaluation scale on the user terminal 30, a virtual person image (for example, a female nurse image) will appear on the display interface of the user terminal 30. Then, the virtual person will explain the key points of each question in the evaluation scale to the user one by one.

[0072] S3: Obtain the user's answer to the current question and switch to the next question through the virtual person until the user completes all questions in the assessment scale.

[0073] It is understood that after the virtual person finishes explaining the question to the user, the user is required to answer the current question. If the current question is a voice question, the user terminal 30 will collect the user's voice data and perform voice recognition and intent recognition in sequence to identify the user's intent. The virtual person will then respond accordingly to complete the interaction with the user. If the current question is not a voice question, other user response data, such as screen click data and drawing data, will be collected. The virtual person will then respond accordingly based on the user's response data to complete the interaction with the user.

[0074] After completing the current question, the virtual person automatically switches to the next question and explains the question to the user again until the user has completed all questions in the assessment scale. It is understood that by continuously looping through steps S2 to S3, when the user has completed all questions in the assessment scale, the user terminal 30 will save the user's voice data and other response data during the cognitive assessment process, ultimately forming an assessed scale that is sent to the system terminal 10.

[0075] Furthermore, in this embodiment, preferably, during the user's cognitive assessment process, if the answer to the current question is correct, the virtual person will provide positive feedback to the user (e.g., "You are awesome!"). If the answer to the current question is incorrect, the virtual person will provide positive encouragement to the user (e.g., "Don't lose heart, you will do better!"). This makes the entire cognitive assessment process more engaging and improves user compliance with the cognitive assessment.

[0076] S4: Obtain the evaluated scale after the user completes the cognitive assessment through the system end 10, and send the evaluated scale to the doctor end 20.

[0077] Specifically, the evaluated scale in step S3 is stored in the data storage module 12 of the system terminal 10. Furthermore, the system terminal 10 sends the evaluated scale to the result display module 22 of the doctor terminal 20. At this time, the doctor can log in to the doctor terminal 20 to open the evaluated scale.

[0078] S5: The doctor terminal 20 opens the evaluated scale and manually reviews the answer results in the evaluated scale.

[0079] Specifically, after the doctor opens the user's completed evaluation scale, he or she can manually review the answer results in the evaluation scale. During this process, the manual review results can be modified multiple times to save the final manual review results and upload them to the system terminal 10.

[0080] S6: Save the user's manual review result as the user's cognitive assessment result.

[0081] In summary, the embodiments of the present invention provide a digital cognitive assessment system and method based on a virtual human, which has the following beneficial effects:

[0082] 1. The use of virtual humans eliminates the need for assistants to accompany subjects for extended periods of time during assessments, reducing the demand on medical personnel and enabling the simultaneous assessment of multiple subjects within a single timeframe. Furthermore, deep learning-based intent recognition ensures consistent assessments.

[0083] 2. Through network testing and multi-terminal deployment of services, the best service plan is dynamically selected according to network speed to ensure smooth user experience.

[0084] 3. All functional modules in the system side 10, the doctor side 20, and the user side 30 are written in the Go language, which features simple development, low resource usage, and high scalability for cluster deployment. Furthermore, Go applications are compact in size, making them particularly suitable for cloud computing scenarios such as cluster deployment, maximizing the utilization of software and hardware resources.

[0085] It should be noted that the above embodiments are only examples, and the technical solutions of the various embodiments can be combined and are all within the protection scope of the present invention.

[0086] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0087] The above describes in detail the virtual human-based digital cognitive assessment system and method provided by the present invention. For those skilled in the art, any obvious modification to the present invention without departing from its essence would constitute an infringement of the present invention's patent rights and would result in corresponding legal liability.

Claims

1. A digital cognitive assessment system based on virtual humans, characterized by Including the interconnected system side, doctor side and user side; The system side includes: The scale management module has a variety of pre-set evaluation scales for user cognitive evaluation; Data storage module, used to store user's scale evaluation data; Intent recognition module, used to identify the intent of the user's voice data; an intelligent judgment module, connected to the intention recognition module, to generate a preliminary judgment result based on the user's intention recognition result; The doctor side includes: A scale publishing module, connected to the scale management module, is used to send the scale to be evaluated to the user end; A scale result display module, connected to the data storage module, for displaying the user's scale evaluation data; A review module, connected to the intelligent judgment module, for manually reviewing the preliminary judgment result; The user terminal includes: A virtual human display module is connected to the scale publishing module to explain the scale questions to the user through the virtual human; the virtual human display module is also connected to the intention recognition module to provide interactive feedback to the user through the virtual human based on the intention recognition result; The voice collection module is used to collect the user's voice data when conducting cognitive assessment; A speech recognition module, connected to the speech acquisition module, for performing speech recognition on the speech data; the speech recognition module is also connected to the intention recognition module, for transmitting speech recognition content to the intention recognition module; A scale logic module, connected to the virtual human display module, to output the user's scale evaluation data for the scale to be evaluated based on a preset model; The scale result processing module is connected to the scale logic module and the data storage module to receive the scale evaluation data and upload it to the data storage module.

2. The digital cognitive assessment system according to claim 1, wherein: Each of the doctor terminals corresponds to multiple user terminals.

3. The digital cognitive assessment system according to claim 1, wherein: There are multiple user terminals, and each user terminal is connected to the system terminal and the doctor terminal through an independent data channel.

4. The digital cognitive assessment system according to claim 1, wherein: The system end also includes a comparison module, which is connected to the review module to obtain a review result; the comparison module is also connected to the scale logic module to feed back the review result to the scale logic module and iteratively update the preset model.

5. The digital cognitive assessment system according to claim 1, wherein: The user terminal is a mobile phone or computer with a preset application downloaded.

6. A digital cognitive assessment method based on a virtual human, implemented based on the digital cognitive assessment system according to any one of claims 1 to 5, characterized in that The steps include: Send the evaluation scale to the user end through the doctor end; Open the to-be-assessed scale through the user terminal, and explain the questions in the to-be-assessed scale based on the virtual person; Obtain the user's answer to the current question and switch to the next question through the virtual person until the user completes all questions in the assessment scale; Obtaining the assessed scale of the user after completing the cognitive assessment through the system end, and sending the assessed scale to the doctor end; Opening the evaluated scale through the doctor's terminal and manually reviewing the answer results in the evaluated scale; The manual review result of the user is saved as the cognitive assessment result of the user this time.

7. The digital cognitive assessment method according to claim 6, characterized in that The step of sending the evaluation scale to be measured to the user terminal through the doctor terminal specifically includes: Obtaining personal information input by the user through the user terminal, wherein the personal information includes at least age, name, education level, and type of disease; Based on the user's personal information, select a required scale from the scale management module on the system side as the scale to be evaluated; The evaluation scale is sent to the user terminal through the doctor terminal.

8. The digital cognitive assessment method according to claim 6, wherein: During the cognitive assessment process of the user, if the answer result of the current question is correct, the virtual person will provide positive feedback to the user; if the answer result of the current question is wrong, the virtual person will actively encourage the user.

9. The digital cognitive assessment method according to claim 6, wherein: The answer result includes at least the user's voice data, drawing data, and click data.

10. The digital cognitive assessment method according to claim 6, wherein: During the manual review process, the doctor side modifies the manual review result multiple times to save the final manual review result and upload it to the system side.

Images