Intelligent auxiliary cognitive training system based on deep learning and virtual reality

Abstract

The invention discloses an intelligent auxiliary cognitive training system based on deep learning and virtual reality, which comprises a user interface layer, a business logic layer, a data processing layer and a resource management layer, and relates to the technical field of cognitive training. According to the intelligent auxiliary cognitive training system based on deep learning and virtual reality, cognitive ability data of a cognitive impairment patient is classified by combining a deep learning technology, a personalized cognitive training scheme is automatically recommended according to the course of disease of the cognitive impairment patient, a user is helped to carry out cognitive training in a virtual reality training form, and the cognitive ability of the cognitive impairment patient is improved. The method has good attraction, improves the cognitive training effect, and can realize long-term tracking and objective evaluation according to the training result, so as to improve the cognitive ability, promote the improvement of the cognitive ability and further improve the life quality.

Description

Technical Field

[0001] The present invention relates to the technical field of cognitive training, and specifically to an intelligent assisted cognitive training system based on deep learning and virtual reality. Background Art

[0002] Cognitive impairment is a group of cognitive dysfunctions caused by damage to the nervous system, including aspects such as memory, attention, thinking ability, language ability, and spatial perception. These impairments can have a serious impact on daily life, reduce the quality of life, and also increase the risk of psychological problems such as depression and anxiety. In existing assisted cognitive training systems, most rely on the guidance and training of medical staff, with low efficiency and difficulty in meeting the rehabilitation needs of patients. Although there are some medications and behavioral therapies that can relieve these symptoms, many patients still have difficulty performing daily activities.

[0003] Current cognitive training systems are often general and based on a mass sample, often monotonous, boring, and unappealing. Moreover, different cognitive impairment patients have different cognitive deficiencies. The current cognitive training systems do not fully consider the specific situation of each patient and cannot provide a more personalized training plan, having the defect of lacking personalized customization. In addition, some current cognitive training systems only provide short-term training, and the training results lack long-term tracking and evaluation, resulting in subjectivity and uncertainty in the judgment of the cognitive training effect and being unable to provide a more scientific and comprehensive evaluation method. Therefore, an intelligent assisted cognitive training system based on deep learning and virtual reality is specifically proposed, which automatically recommends a personalized cognitive training plan according to the course of the cognitive impairment patient, helps users carry out cognitive training in the form of virtual reality training, and realizes long-term tracking and objective evaluation according to the training results to improve their cognitive ability and promote the improvement of cognitive ability. Summary of the Invention

[0004] In view of the deficiencies of the prior art, the present invention provides an intelligent assisted cognitive training system based on deep learning and virtual reality, which solves the problems of the current cognitive training system such as lacking personalized customization, lacking long-term tracking and evaluation, and being monotonous, boring, and unappealing.

[0005] To achieve the above object, the present invention is realized through the following technical solutions: An intelligent assisted cognitive training system based on deep learning and virtual reality, including:

[0006] User interface layer: The user interface layer is used for virtual reality interaction with the user and storing user information;

[0007] Business logic layer: The business logic layer is used for detecting whether the user has cognitive impairment, generating an assisted cognitive training task when the user has cognitive impairment, and providing feedback on the cognitive training effect;

[0008] Data processing layer: The data processing layer is used to read user information and store the user information in a database.

[0009] Resource management layer: The resource management layer is used to construct a virtual reality scene and encapsulate and manage the BP neural network algorithm and the personalized recommendation algorithm.

[0010] The present invention is further configured that: the user interface layer includes a virtual reality interaction module, a virtual reality cognitive training module, and a patient data storage module;

[0011] The virtual reality interaction module is used to transmit the input information of the user to the business logic layer and display the output of the business logic layer to the user;

[0012] The virtual reality cognitive training module is used to render the assisted cognitive training tasks in a virtual reality manner to the user side;

[0013] The patient data storage module is used to store user information, wherein the user information includes personal information and learning progress.

[0014] The present invention is further configured that: the business logic layer includes a cognitive impairment detection module, a cognitive training task generation module, and a cognitive training effect feedback module;

[0015] The cognitive impairment detection module is used to detect whether the user has cognitive impairment by using the BP neural network algorithm according to the input information of the user;

[0016] The cognitive training task generation module is used to generate corresponding assisted cognitive training tasks through the personalized recommendation algorithm when it is detected that the user has cognitive impairment, as the output of the business logic layer;

[0017] The cognitive training effect feedback module is used to obtain the performance of the user in the assisted cognitive training tasks and perform learning progress feedback.

[0018] The present invention is further configured that: the data processing layer includes a data reading module, a data storage module, and a data analysis module;

[0019] The data reading module is used to read the data input by the user;

[0020] The data storage module is used to store the user data in the database;

[0021] The data analysis module is used to perform statistics on the data stored in the database for the business logic layer to make decisions.

[0022] The present invention is further configured such that: the resource management layer includes a virtual reality engine, a BP neural network algorithm, and a personalized recommendation algorithm;

[0023] The virtual reality engine is used for establishing, running, and managing a virtual reality environment;

[0024] The BP neural network algorithm is used for detecting cognitive impairment and performing qualitative and quantitative analyses of cognitive impairment;

[0025] The personalized recommendation algorithm is used for generating auxiliary cognitive training tasks related to the user according to the qualitative and quantitative analyses of cognitive impairment, wherein the auxiliary cognitive training tasks include several types of virtual reality environments.

[0026] The present invention is further configured such that: the training data of the BP neural network algorithm includes: collecting test data of cognitive impairment patients in a virtual reality environment, converting and processing the test data into a cognitive ability report list, and after assigning values to the items of the cognitive ability report list, constituting the training data, wherein the cognitive impairment patients are determined according to the evaluation criteria of cognitive impairment by the MMSE scale;

[0027] The types of the cognitive ability report list include memory, language, orientation, planning, organization, and concentration.

[0028] The present invention is further configured such that: the output results of the BP neural network algorithm include qualitative analysis data and quantitative analysis data;

[0029] The qualitative analysis data includes mild cognitive impairment, moderate cognitive impairment, and severe cognitive impairment;

[0030] The quantitative analysis data includes a cognitive ability list and assigned values.

[0031] The present invention is further configured such that: the personalized recommendation algorithm is one of a collaborative filtering algorithm, a content filtering algorithm, and a hybrid algorithm of collaborative filtering and content filtering.

[0032] The present invention provides an intelligent assisted cognitive training system based on deep learning and virtual reality. It has the following beneficial effects:

[0033] By combining deep learning technology, the present invention classifies the cognitive ability data of cognitive impairment patients, automatically recommends personalized cognitive training programs according to the disease course of cognitive impairment patients, and helps users conduct cognitive training in the form of virtual reality training. It has good attraction, can improve the cognitive training effect, and can achieve long-term tracking and objective evaluation according to the training results to improve their cognitive ability, promote the improvement of cognitive ability, and further improve the quality of life. Description of the Drawings

[0034] Figure 1 Schematic diagram of the system architecture of the present invention;

[0035] Figure 2 Schematic diagram of the cognitive ability report list and assignment in the present invention. Specific implementation manners

[0036] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention.

[0037] Please refer to Figure 1-2 , the embodiments of the present invention provide the following technical solutions: an intelligent assisted cognitive training system based on deep learning and virtual reality, which consists of a user interface layer, a business logic layer, a data processing layer, and a resource management layer. Among them, the user interface layer is used to perform virtual reality interaction with the user and store user information. Specifically, the user interface layer includes a virtual reality interaction module, a virtual reality cognitive training module, and a patient data storage module;

[0038] The virtual reality interaction module is used to transmit the input information of the user to the business logic layer and display the output of the business logic layer to the user. This process can be realized through virtual reality input devices such as head-mounted devices and handles;

[0039] The virtual reality cognitive training module is used to render the assisted cognitive training tasks to the user side in a virtual reality manner;

[0040] The patient data storage module is used to store user information, where the user information includes personal information and learning progress.

[0041] As a preferred solution, the business logic layer is used to detect whether the user has cognitive impairment. When the user has cognitive impairment, it generates assisted cognitive training tasks and performs feedback on the cognitive training effect. Specifically, the business logic layer includes a cognitive impairment detection module, a module for generating cognitive training tasks, and a module for feedback on the cognitive training effect;

[0042] The cognitive impairment detection module is used to detect whether the user has cognitive impairment according to the input information of the user by using the BP neural network algorithm;

[0043] The module for generating cognitive training tasks is used to generate corresponding assisted cognitive training tasks through a personalized recommendation algorithm when it is detected that the user has cognitive impairment, as the output of the business logic layer;

[0044] The module for feedback on the cognitive training effect is used to obtain the performance of the user in the assisted cognitive training tasks and perform feedback on the learning progress.

[0045] As a preferred solution, the data processing layer is used to read user information and store the user information in the database. Specifically, the data processing layer includes a data reading module, a data storage module, and a data analysis module;

[0046] The data reading module is used to read the data input by the user;

[0047] The data storage module is used to store the user data in the database;

[0048] The data analysis module is used to perform statistics on the data stored in the database for the business logic layer to make decisions.

[0049] As a preferred solution, the resource management layer is used to construct a virtual reality scene and perform encapsulation and management of the BP neural network algorithm and the personalized recommendation algorithm. Specifically, the resource management layer includes a virtual reality engine, the BP neural network algorithm, and the personalized recommendation algorithm. Among them, the virtual reality engine is used to establish, run, and manage the virtual reality environment, including but not limited to scene rendering, physical effect simulation, and sound processing.

[0050] The BP neural network algorithm is used to perform cognitive impairment detection and qualitative and quantitative analysis of cognitive impairment. The output results of the BP neural network algorithm include qualitative analysis data and quantitative analysis data;

[0051] The qualitative analysis data includes mild cognitive impairment, moderate cognitive impairment, and severe cognitive impairment;

[0052] The quantitative analysis data includes a cognitive ability list and assignments.

[0053] The training data of the BP neural network algorithm includes: collecting test data of cognitive impairment patients in the virtual reality environment. Among them, the cognitive impairment patients are determined according to the evaluation criteria of the MMSE scale for cognitive impairment. After converting the test data into a cognitive ability report list and assigning values to each item of the cognitive ability report list, the training data is formed. The training data is divided into a training set and a validation set according to a ratio of 7:3. The cognitive ability report list includes six categories: memory, language, orientation, planning, organization, and concentration, with a total of forty items, as shown in the appendix Figure 2 as shown.

[0054] According to the evaluation criteria of the MMSE scale for cognitive impairment, data is collected from hospital cognitive impairment patients, and the data is converted into a cognitive ability report list and assigned values. This cognitive ability report list corresponds to the MMSE scale, and thus the degree of cognitive impairment of the patient can be judged.

[0055] For further explanation, the BP neural network includes an input layer, a hidden layer, and an output layer. Among them, the input layer contains all the feature vectors in the dataset; the hidden layer contains several neurons, and the output value of each neuron is obtained by calculating the sum of all the feature vectors in the input layer and their respective weights and adding a bias; the output layer contains only one neuron, and its output value represents the classification result.

[0056] For further explanation, the training of the BP neural network: The training process of the BP neural network is realized by the backpropagation algorithm, which mainly includes:

[0057] Randomly initialize the weights: Before starting the training, all weights need to be randomly initialized;

[0058] Forward propagation: Transfer the training set from the input layer to the hidden layer and the output layer;

[0059] Calculate the error: Calculate the error of the output layer and convert it into the error of the hidden layer;

[0060] Backward propagation: Use the error to update each weight to reduce the error;

[0061] Repeat the above steps until the model converges.

[0062] Furthermore, after completing the training of the BP neural network, use metrics to evaluate it. The metrics include but are not limited to accuracy, recall, and precision.

[0063] The personalized recommendation algorithm is one of the collaborative filtering algorithm, the content filtering algorithm, and the hybrid algorithm of collaborative filtering and content filtering, and is used to generate user-related assisted cognitive training tasks according to the qualitative and quantitative analysis of cognitive impairment. Among them, the assisted cognitive training tasks include several types of virtual reality environments.

[0064] When in use, obtain the MMSE scale evaluation data of the cognitive impairment patient, import it into the BP neural network, obtain the cognitive ability report list of the patient. When it is determined that the patient has one of mild cognitive impairment, moderate cognitive impairment, and severe cognitive impairment, generate personalized assisted cognitive training tasks for the patient through the personalized recommendation algorithm, and render the virtual reality scene on the user side for assisted cognitive training.

[0065] Furthermore, targeted enhanced training is carried out on items with a lower proportion in the patient's cognitive ability report list, that is, the proportion of this item in the auxiliary cognitive training task is adjusted. In this way, feedback on the results of auxiliary cognitive training can provide support for the next step of training suggestions and intelligent adjustment of tasks. In a long-term tracking manner, personalized feedback and assistance can be provided for the patient's cognitive training. For example, if the patient makes an error in a certain item, more training can be provided according to his or her specific situation, and corresponding prompts or suggestions can be provided to help the patient complete the task of the corresponding item in a targeted manner, so as to enhance the effect of auxiliary cognitive training.

[0066] A virtual training environment for improving various aspects of cognitive ability is developed based on the MMSE scale. Sufficient virtual training scenarios are created based on six categories and forty items. Virtual training scenarios for different patients are selected for personalized recommendation algorithms to generate auxiliary cognitive training tasks. By creating various virtual reality scenarios, the auxiliary training scenarios are made more interesting, vivid and effective, effectively improving their ability to help patients exercise and improve cognition, helping patients better cope with cognitive challenges and improve cognitive abilities.

[0067] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent assisted cognitive training system based on deep learning and virtual reality, characterized in that: It includes: User interface layer: The user interface layer is used to perform virtual reality interaction with users and store user information; Business logic layer: The business logic layer is used to detect whether a user has cognitive impairment. When a user has cognitive impairment, it generates an assisted cognitive training task and provides feedback on the cognitive training effect; Data processing layer: The data processing layer is used to read user information and store the user information in a database; Resource management layer: The resource management layer is used to construct a virtual reality scene and encapsulate and manage the BP neural network algorithm and the personalized recommendation algorithm.

2. The intelligent assisted cognitive training system based on deep learning and virtual reality according to claim 1, characterized in that: The user interface layer includes a virtual reality interaction module, a virtual reality cognitive training module, and a patient data storage module; The virtual reality interaction module is used to transmit the input information of the user to the business logic layer and display the output of the business logic layer to the user; The virtual reality cognitive training module is used to render the assisted cognitive training task to the user side in a virtual reality manner; The patient data storage module is used to store user information, where the user information includes personal information and learning progress.

3. The intelligent assisted cognitive training system based on deep learning and virtual reality according to claim 1, characterized in that: The business logic layer includes a cognitive impairment detection module, a cognitive training task generation module, and a cognitive training effect feedback module; The cognitive impairment detection module is used to detect whether a user has cognitive impairment according to the input information of the user by using the BP neural network algorithm; The cognitive training task generation module is used to generate a corresponding assisted cognitive training task through the personalized recommendation algorithm when it is detected that the user has cognitive impairment, as the output of the business logic layer; The cognitive training effect feedback module is used to obtain the performance of the user in the assisted cognitive training task and provide feedback on the learning progress.

4. The intelligent assisted cognitive training system based on deep learning and virtual reality according to claim 1, characterized in that: The data processing layer includes a data reading module, a data storage module, and a data analysis module; The data reading module is used to read the data input by the user; The data storage module is used to store the user data in the database; The data analysis module is used to statistically analyze the data stored in the database for the business logic layer to make decisions.

5. The intelligent assisted cognitive training system based on deep learning and virtual reality according to claim 1, wherein: The resource management layer includes a virtual reality engine, a BP neural network algorithm, and a personalized recommendation algorithm; The virtual reality engine is used to establish, run, and manage a virtual reality environment; The BP neural network algorithm is used to perform cognitive impairment detection and qualitative and quantitative analysis of cognitive impairment; The personalized recommendation algorithm is used to generate an assisted cognitive training task related to the user according to the qualitative and quantitative analysis of cognitive impairment, where the assisted cognitive training task includes several types of virtual reality environments.

6. The intelligent assisted cognitive training system based on deep learning and virtual reality according to claim 5, characterized in that: The training data of the BP neural network algorithm includes: collecting test data of cognitive impairment patients in a virtual reality environment, converting the test data into a cognitive ability report list, and after assigning values to the items of the cognitive ability report list, forming the training data; The types of the cognitive ability report list include memory, language, location, planning, organization, and concentration.

7. The intelligent assisted cognitive training system based on deep learning and virtual reality according to claim 6, wherein: The output results of the BP neural network algorithm include qualitative analysis data and quantitative analysis data; The qualitative analysis data includes mild cognitive impairment, moderate cognitive impairment, and severe cognitive impairment; The quantitative analysis data includes a cognitive ability list and assigned values.

8. The intelligent assisted cognitive training system based on deep learning and virtual reality according to claim 7, characterized in that: The personalized recommendation algorithm is one of a collaborative filtering algorithm, a content filtering algorithm, and a hybrid algorithm of collaborative filtering and content filtering.

Images