Intelligent evaluation system and method of spatial cognitive ability based on virtual reality
Through virtual reality technology and machine learning algorithms, combined with virtual water maze testing tasks, the environmental and equipment limitations of traditional evaluation methods are solved, efficient and accurate spatial cognitive ability assessment is achieved, and more detailed data and automation results are provided.
Patent Information
- Application Number
- CN202210186467.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-28
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2042-02-28
AI Technical Summary
Traditional spatial cognitive ability assessment methods are limited by environmental and equipment requirements, difficult to provide high immersion and accurate data, and rely on professionals to make full use of behavioral data.
Develop an intelligent evaluation system for spatial cognitive ability based on virtual reality. Through virtual water maze testing tasks, combined with machine learning algorithms, users use task performance data and behavioral data under different difficulties and modes to achieve intelligent evaluation. The system includes a virtual reality module, a data recording module, a data processing module and an intelligent evaluation module, and uses a machine learning regression model for automatic evaluation.
Improves the implementability and ecological effectiveness of assessments, provides more detailed and accurate data, reduces dependence on professionals, enables assessments in multiple locations, and provides rapid assessment results through automated results.
Smart Images

Figure CN114664442B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of virtual reality technology, and in particular to a system and method for intelligently evaluating spatial cognitive ability based on virtual reality. Background Art
[0002] Cognitive function refers to the ability of the human brain to process, store and extract information, that is, people's ability to grasp the composition, performance, relationship with other things, development momentum, development direction and basic laws of objective things, including memory, attention, language, visual space, processing speed, etc. Spatial cognitive ability is a high-level cognitive function, which is responsible for recording information about the environment and spatial direction, and is closely related to the visual space ability and memory ability in the basic cognitive function. Spatial cognitive ability plays an important role in human daily life, such as finding routes, locating places and targets, using maps for navigation, etc., which is a necessary condition for people to move independently in the environment. At the same time, a variety of developmental disorders, neurological diseases and cerebrovascular diseases can cause spatial cognitive dysfunction. Therefore, accurate and effective assessment of spatial cognitive ability is of great significance for the study of human brain development, cognitive aging and auxiliary diagnosis and treatment of diseases.
[0003] Traditional methods often use scale tests to assess spatial cognitive ability. Scale tests generally need to be conducted under the guidance of personnel with professional backgrounds, are greatly affected by the test environment, and take a long time to evaluate. In addition to the inconvenience of personnel and environmental factors, the results provided by scale tests are usually limited, and some objective and accurate information is difficult to obtain. In recent years, computerized cognitive tests have also been widely used. Although they can achieve a certain degree of automatic evaluation, the test tasks are limited by the presentation format and still lack relevance to real-life scenarios. It is difficult to provide a good sense of environmental immersion, and the behavioral data of the subjects is rarely taken into account.
[0004] Virtual reality technology has developed rapidly in recent years. Its high immersion, multi-sensory and strong interactivity provide a means of reproducing and expanding cognitive function assessment methods. Virtual reality-based systems can flexibly control the appearance and content of stimuli, the complexity of tasks and the response status of subjects. At the same time, they can directly record the rich and accurate data information of subjects in tasks, which provides a basis for further processing using artificial intelligence algorithms, thus providing a feasible new means to improve the reliability and ecological validity of spatial cognitive ability assessment. Summary of the invention
[0005] In order to solve the problems existing in the existing spatial cognition assessment, the present invention provides a spatial cognition ability intelligent assessment system and method based on virtual reality, develops a set of virtual water maze test tasks, and uses the user's task performance data and behavior data in different difficulty and different mode task states, and uses a machine learning algorithm to train a regression model to achieve intelligent assessment of spatial cognition ability.
[0006] In order to achieve the above-mentioned purpose, the system of the present invention adopts the following technical solutions: a spatial cognitive ability intelligent evaluation system based on virtual reality, including a virtual reality module, a data recording module, a data processing module, an intelligent evaluation module and a database;
[0007] The virtual reality module includes a task software unit and a main control interaction unit, wherein:
[0008] The task software unit is used to provide a task login interface, an immersive virtual reality scene and a water maze cognitive test task. The water maze cognitive test task includes three levels of difficulty and three modes of test tasks that can be selected. In the task, the operation input of the main control interaction unit generates result feedback including perspective conversion under the first person, position movement and platform lifting in the scene;
[0009] The main control interaction unit includes a situation display component and an interactive input component. It presents the immersive environment scene and cognitive task content to the user through hardware peripherals, transmits the user's operation input through the hardware peripherals to the task software unit, triggers the corresponding feedback to be presented again, and realizes the interaction between the user and the virtual reality scene;
[0010] The data recording module is connected to the virtual reality module and is used to collect the user's operation input and task feedback information in the task as the synchronous task performance data and behavior data in the task state, and store the corresponding data and the user's personal information entered in the login interface in the database in the form of a file and transmit it to the data processing module;
[0011] The data processing module includes data preprocessing and feature extraction, uses the original behavior data obtained by the data recording module to perform behavior analysis, and extracts effective task performance features and behavior data features;
[0012] The intelligent assessment module uses the task performance characteristics and behavioral data characteristics obtained by the data processing module as input and uses the trained machine learning regression model to automatically predict the spatial cognitive ability score;
[0013] The database is used to store user information, original task performance data and behavior data, extracted task performance features and behavior data features, and intelligent evaluation results.
[0014] The evaluation method of the present invention is based on the above evaluation system, which comprises the following steps:
[0015] Step 1: Enter the system login interface, enter the user's personal information, including number, age, gender and medical history, and introduce the test process, operation guide and precautions;
[0016] Step 2: Select the practice mode and enter the virtual water maze task scene. The scene environment and interaction method are similar to those in the test mode, but there are no specific task objectives. Users practice in it to familiarize themselves with the operation methods.
[0017] Step 3: Select water maze cognitive test tasks of different difficulty and mode in turn, and the user enters the scene to complete the corresponding task according to the current test task requirements. The user's task performance data and behavior data during the task are collected and stored in the database;
[0018] Step 4: preprocess the stored original task performance data and behavior data, and extract corresponding task performance features and behavior data features;
[0019] Step 5: Using the extracted task performance characteristics and behavioral data characteristics as input, a machine learning regression model is used to predict the spatial cognitive ability score and store it in a database to achieve intelligent assessment of spatial cognitive ability.
[0020] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0021] 1. The present invention provides a set of spatial cognitive ability test tasks including setting and recording functions through a virtual reality module and a data recording module. The present invention virtualizes the traditional water maze task based on its ideas and principles. On the one hand, it greatly reduces the requirements for the site, equipment and environmental layout required for testing and evaluation, and enhances the feasibility and convenience of the evaluation; on the other hand, compared with the traditional two-dimensional test form, the virtual reality three-dimensional environment is closer to the scene in real life, which helps to improve the sense of immersion and the ecological validity of the evaluation. At the same time, the effective connection of the cognitive task module, the interactive control module and the data acquisition module provides more comprehensive and accurate data for further analysis.
[0022] 2. The present invention uses artificial intelligence to evaluate spatial cognitive ability using machine learning algorithms. Existing evaluations generally use traditional statistical analysis methods, and the feature information used is limited, especially for behavioral data, which cannot be fully utilized. The present invention collects task performance data and behavioral data of a large number of people, extracts multiple features, trains the machine learning regression model, uses genetic algorithms for feature selection, and uses the leave-one-out cross-validation method to improve the generalization ability of the model, thereby automatically obtaining accurate evaluation results.
[0023] 3. The evaluation system of the present invention is easy to operate. The test process, operation guidance, etc. are all presented in a virtual reality scene. The evaluation results are automatically given by the machine learning regression model. Users can operate independently, reducing the degree of dependence on professionals. The hardware equipment used in the system is easy to wear and has low environmental requirements. Evaluations can be carried out in homes, communities, schools, hospitals and other places. It can be used for screening or long-term monitoring of personal cognitive functions and health conditions, and can also provide auxiliary means for selecting personnel for specific industries and units. At the same time, since the detailed original data and evaluation results obtained by the task test are stored in the database, it is convenient for query management, and professional medical staff and researchers can also use the relevant data to carry out further analysis and research. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 This is a block diagram of the spatial cognitive ability intelligent assessment system based on virtual reality of the present invention;
[0025] Figure 2 This is a schematic diagram of the task login interface of the present invention;
[0026] Figure 3 It is a schematic diagram of an immersive virtual reality scene of the present invention;
[0027] Figure 4 This is a flow chart of the spatial cognitive ability assessment method based on virtual reality of the present invention. DETAILED DESCRIPTION
[0028] The present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application, and all terms used herein are only for describing specific implementations and are not intended to limit the exemplary implementations according to the present application.
[0029] Example
[0030] like Figures 1 to 3 As shown, the spatial cognitive ability intelligent assessment system based on virtual reality in this embodiment includes a virtual reality module, a data recording module, a data processing module, an intelligent assessment module and a database. Users can obtain task performance data, behavior data and the specific scoring results of the final spatial cognitive function by completing tasks, thereby realizing automatic quantitative assessment of spatial cognitive ability.
[0031] The virtual reality module includes a task software unit and a main control interaction unit, wherein:
[0032] The task software unit is used to provide a task login interface, an immersive virtual reality scene and a water maze cognitive test task. The task login interface includes user information entry and task selection functions. The water maze task is a classic experimental paradigm in the field of spatial cognition, which is used to evaluate spatial learning and memory ability. The present invention constructs a set of virtual water maze cognitive tasks of different difficulty and mode based on the ideas and principles of traditional water maze tasks. The immersive virtual reality scene is an indoor swimming pool scene, including a central circular pool (radius 20m), a water surface, surrounding walls, poolside reference objects (vases, trees, tires, columns) and a square platform (side length 2m) at the same height as the water surface, and also includes animation and sound effects that match the scene. The difficulty of the water maze cognitive test task is set according to the number of visible poolside reference objects. As the number of reference objects decreases, the difficulty of the task increases. The configuration file is used to set parameters. In this embodiment, three difficulty levels are set. The configuration file can also be used to set the user's starting position and starting orientation, the position and visibility of the underwater platform, and the task duration limit. Select the task through the drop-down list on the task login interface. After selecting, enter the corresponding task. First, the task content and operation instructions are presented. After reading and confirming, you will enter the task scene and start the test from a first-person perspective. Each difficulty of the water maze cognitive test task includes three modes:
[0033] Visible platform mode: The user enters the task scene at a random position and orientation in the pool. The pool platform is visible. The user needs to memorize the platform position based on the reference objects around the pool within a 2-minute limit, and operate through the task control module to move to the platform position. When the user moves to the platform position or reaches the 2-minute limit, the platform is triggered to automatically rise, and the task ends and returns to the login interface;
[0034] Hidden platform mode: The user needs to complete 5 tests in a row. The initial positions of entering the task scene are 5 positions in the pool with equal distances to the platform (20m). The task scene is consistent with the visible platform mode, but the platform is set to be hidden and invisible to the user. The user needs to use the reference objects around the pool as remote clues according to memory and operate through the task control module to reach the platform position in the shortest possible time. When the user moves to the platform position, the platform is triggered to rise automatically, and enters the next test after 20 seconds, until all 5 tests are completed, and the task is completed and returns to the login interface;
[0035] Exploration Platform Mode: 3 minutes after completing the hidden platform mode task, the user enters this mode task. The task scene is consistent with the hidden platform mode, but the platform is removed and cannot be triggered to rise. The user needs to operate through the task control module according to memory and reference clues within the 2-minute limit, and try to move and stay at the original platform position. When the 2-minute limit is reached, the task automatically ends and returns to the login interface.
[0036] At the same time, the cognitive task unit in this embodiment will provide corresponding animation and sound effects, including water surface fluctuations in the pool, the sound of water moving in the pool, the mechanical operation sound of the platform rising, etc., which greatly enhances the realism of the scene and the feedback of the task, allowing users to gain a better sense of immersion and help improve evaluation accuracy.
[0037] The main control interaction unit includes a situational display component and an interactive input component, which is used to present immersive environment scenes and cognitive task content to users through hardware peripherals, send the user's operation input through hardware peripherals to the task software unit, trigger the corresponding feedback to be presented again, and realize the interaction between the user and the virtual reality scene. The user controls the operation through the HTC Vive device, and the virtual reality scene is presented to the user through the head display (head-mounted display device). When the user presses the trigger button on the handle, a displacement will occur in the virtual scene. Calculated from the head display's viewing direction and preset speed:
[0038]
[0039] Among them, x forward and z forward are the unit vectors of the x-axis and z-axis of the head display's perspective in the three-dimensional space coordinate system, and v is the preset velocity scalar.
[0040] In the visible platform mode and hidden platform mode tasks, when the user moves to the platform position, the collision detection is triggered, the user's perspective rises with the platform, and the corresponding sound effect is heard.
[0041] The data recording module is connected to the virtual reality module and is used to collect the user's operation input and task feedback information in the task as the synchronous task performance data and behavior data under the task state. The data collected in this embodiment include the navigation duration HTime and navigation distance HRoute of each test in the hidden platform mode, and the navigation distance PRoute of each test in the exploration platform mode, the distance PPlatRoute and the duration PPlatTime within the 1 / 4 pool range where the platform is located, and also include the position coordinates (positionX, positionY) of each frame of the user in the two modes of navigation, the direction coordinates (rotationX, rotationY, rotationZ) and the synchronization timestamp (yyyy-MM-dd_HH-mm-ss), and the sampling rate is 50Hz. The data recording module stores the above-mentioned data collected and the user personal information entered in the login interface into the CSV (Comma-Separated Values, character separated value) file in the database, and transmits it to the data processing module.
[0042] The data processing module includes data preprocessing and feature extraction. Preprocessing mainly includes denoising and normalization. The abnormal behavior data that may be generated by the user during the test task will form noise, so the denoising algorithm is used to identify and delete the abnormal behavior data, and then the data is normalized. After preprocessing, feature extraction is performed on the data to obtain task performance features and behavior data features. In this embodiment, the task performance features include the maximum value HtMax, minimum value HtMin, and average value HtMean of the navigation duration in the hidden platform mode test, the maximum value HrMax, minimum value HrMin, and average value HrMean of the navigation distance, the number of tests HNumber with a navigation duration of less than 2 minutes, and the maximum value PrMax, minimum value PrMin, and average value PrMean of the navigation distance in the exploration platform mode, the platform area distance ratio PlatRc, the duration ratio PlatTc, and the minimum distance to the platform PrDis; the behavioral data features are mainly extracted from the user's motion trajectory, including the heading error StartDeviation, the maximum value VMax, minimum value VMin, average value VMean, standard deviation VSDV, and normalized variability NVV of the speed, the maximum value SMax, minimum value SMin, average value SMean, standard deviation SSDV, and normalized variability NSV of the speed, the signal entropy SE of the coordinates, and the average relative tremor MRT. The description and calculation formula of some of the above features are as follows:
[0043] PlatRc and PlatTc are the platform area distance ratio and time ratio in the exploration platform mode test, respectively. The two are extracted with reference to the classic water maze task paradigm and calculated as shown in formulas (2) and (3), respectively:
[0044]
[0045]
[0046] Among them, PRoute is the total navigation distance, PPlatRoute is the distance within the 1 / 4 range of the pool where the platform is located, PPlatTime is the time within the 1 / 4 range of the pool where the platform is located, and TotalTime is the total time of each test. In this embodiment, TotalTime=2min.
[0047] StartDeviation is the heading error, calculated as shown in formula (4):
[0048] StartDeviation=|d start -d plat | (4)
[0049] Among them, d start is the unit vector of the initial displacement direction, d platIt is the unit vector of the line from the initial position to the platform position.
[0050] NVV and NSV are normalized velocity variability and normalized rate variability, respectively, which can reflect whether the motion speed is constant and stable, as shown in formulas (5) and (6):
[0051]
[0052]
[0053] Where T is the test time, N is the number of recorded frames, VMean and SMean are the average speed and average rate respectively. SE is the information entropy of the trajectory, which is an estimate of the probability density function of the trajectory information, calculated as shown in formulas (7) and (8):
[0054]
[0055]
[0056] Among them, x i ,y i is the position coordinate of the motion trajectory point.
[0057] MRT is the mean relative tremor, which measures the tremor of the trajectory by the coordinate position of each point, as shown in formula (9):
[0058]
[0059] Among them, x i ,y i is the position coordinate of the motion trajectory point.
[0060] The intelligent evaluation module uses the task performance characteristics and behavior data characteristics extracted by the data processing module as input, and uses the machine learning regression model to score and predict the spatial cognitive ability. First, the regression model is trained, and the system of the present invention is applied to collect task state data of a large number of people and extract corresponding features as samples. The spatial cognitive ability score results given by the authoritative evaluation scale with good validity of the same population are used as labels for model training. The regression model is trained using the LASSO algorithm, and the optimal value of the LASSO regularization term coefficient is selected through cross-validation within the training set.
[0061] Genetic algorithms are used for feature selection to obtain the optimal feature subset; leave-one-out cross-validation is used to verify model performance and improve the generalization ability of the model. The optimal feature subset after selection is used as input, and the trained LASSO regression model is used to predict the spatial cognitive ability score to obtain the final intelligent evaluation result. Similar to the original data, the intelligent evaluation result will be saved in the database together with the user information.
[0062] The database is used to store user information, original task performance data and behavior data, extracted task performance features and behavior data features, and intelligent evaluation results. The data files in the database are stored for a long time, and users and relevant professionals can query and manage the content stored in the database under different conditions.
[0063] This embodiment also provides a method for evaluating spatial cognition ability based on virtual reality, such as Figure 4 As shown, the method is implemented on the basis of the spatial cognition ability evaluation system in this embodiment, and specifically includes the following steps:
[0064] Step 1: Enter the system login interface, enter the user's personal information, including number, age, gender and medical history, and introduce the test process, operation guide and precautions;
[0065] Step 2: Select the practice mode and enter the virtual water maze task scene. The scene environment and interaction method are similar to those in the test mode, but there are no specific task objectives. Users practice in it to familiarize themselves with the operation methods.
[0066] Step 3: Select test tasks in order of difficulty from low to high, and in the order of "visible platform mode - hidden platform mode - exploration platform mode" for each difficulty. The user enters the scene and completes the corresponding task according to the requirements of the current test task. The user's task performance data and behavior data during the task are collected and stored in the database;
[0067] Step 4: preprocess the stored original task performance data and behavior data, and extract corresponding task performance features and behavior data features;
[0068] Step 5: Using the extracted task performance characteristics and behavioral data characteristics as input, a machine learning regression model is used to predict the spatial cognitive ability score and store it in the database to achieve intelligent assessment of spatial cognitive ability.
[0069] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments, which are merely illustrative and not restrictive. All other embodiments obtained by a person skilled in the art under the guidance of the present invention without departing from the scope of protection of the present invention and the claims shall fall within the scope of protection of the present invention.
Claims
1. Intelligent evaluation system of spatial cognitive ability based on virtual reality, It is characterized in that It includes virtual reality module, data recording module, data processing module, intelligent evaluation module and database; The virtual reality module includes a task software unit and a main control interaction unit, wherein: The task software unit is used to provide a task login interface, an immersive virtual reality scene and a water maze cognitive test task. The water maze cognitive test task includes three levels of difficulty and three modes of test tasks that can be selected. In the task, the operation input of the main control interaction unit generates result feedback including perspective conversion under the first person, position movement and platform lifting in the scene; The main control interaction unit includes a situation display component and an interactive input component. It presents the immersive environment scene and cognitive task content to the user through hardware peripherals, transmits the user's operation input through the hardware peripherals to the task software unit, triggers the corresponding feedback to be presented again, and realizes the interaction between the user and the virtual reality scene; The data recording module is connected to the virtual reality module and is used to collect the user's operation input and task feedback information in the task as the synchronous task performance data and behavior data in the task state, and store the corresponding data and the user's personal information entered in the login interface in the database in the form of a file and transmit it to the data processing module; The data processing module includes data preprocessing and feature extraction, uses the original behavior data obtained by the data recording module to perform behavior analysis, and extracts effective task performance features and behavior data features; The intelligent assessment module uses the task performance characteristics and behavioral data characteristics obtained by the data processing module as input and uses the trained machine learning regression model to automatically predict the spatial cognitive ability score; The database is used to store user information, original task performance data and behavior data, extracted task performance features and behavior data features, and intelligent evaluation results; The task login interface of the task software unit includes user information entry and task selection functions. The user information includes user name, number, age, gender and previous medical history. The task selection is selected in a drop-down list. The immersive virtual reality scene is an indoor swimming pool scene, including a central circular pool, a water surface, surrounding walls, poolside reference objects and a square platform at the same height as the water surface, and also includes animations and sound effects that match the scene. In the water maze cognitive test task of the task software unit, the task difficulty is set according to the number of visible poolside reference objects, and the three task modes include: Visible platform mode: The user enters the task scene at a random position and orientation in the pool. The pool platform is visible. The user needs to memorize the platform position based on the reference objects around the pool within the preset time, and operate through the task control module to move to the platform position; when the user moves to the platform position or reaches the preset time, the platform is triggered to automatically rise, and the task ends and returns to the login interface; Hidden platform mode: users need to complete multiple tests in succession. The initial positions of entering the task scene are multiple positions in the pool with equal distances to the platform. The task scene is consistent with the visible platform mode, but the platform is set to be hidden and invisible to the user. Users need to use their memory and the reference objects around the pool as remote clues to operate through the task control module to reach the platform position in the shortest possible time. When the user moves to the platform position, the platform is triggered to automatically rise, and then enter the next test, until all multiple tests are completed, and the task is completed and returns to the login interface; Exploration Platform Mode: After completing the hidden platform mode task, the user enters this mode task. The task scene is consistent with the hidden platform mode, but the platform is removed and cannot be triggered to rise. The user needs to operate through the task control module according to memory and reference clues within the preset time, and try to move and stay at the original platform position as much as possible. When the preset time is reached, the task automatically ends and returns to the login interface. The data collected by the data recording module include the navigation duration and navigation distance of each test in the hidden platform mode, the navigation distance of each test in the exploration platform mode, the distance and duration within the 1 / 4 pool where the platform is located, and the position coordinates, direction coordinates and synchronization timestamp of each frame of the user during the navigation process in the two modes; Task performance characteristics include the maximum, minimum, and average values of navigation time in hidden platform mode tests, the maximum, minimum, and average values of navigation distance, the number of tests with navigation time less than 2 minutes, and the maximum, minimum, and average values of navigation distance in exploration platform mode, platform area to distance ratio, time ratio, and minimum distance to the platform; Behavioral data features are extracted from the user's motion trajectory, including heading error, maximum, minimum, average, standard deviation, and normalized variability of speed, maximum, minimum, average, standard deviation, and normalized variability of velocity, signal entropy of coordinates, and average relative tremor; The intelligent evaluation module adopts LASSO algorithm to train the regression model, selects the optimal value of LASSO regularization coefficient through cross-validation within the training set, uses genetic algorithm for feature selection, and adopts leave-one-out cross-validation method to verify the model performance.
2. The spatial cognition ability intelligent assessment system according to claim 1, It is characterized in that Users use VR external devices to perform operations and controls, and virtual reality scenes are presented to users through head-mounted displays.
3. A method for evaluating spatial cognitive ability based on the spatial cognitive ability intelligent evaluation system of claim 1, It is characterized in that The following steps are involved: Step 1: Enter the system login interface, enter the user's personal information, including number, age, gender and medical history, and introduce the test process, operation guide and precautions; Step 2: Select the practice mode and enter the virtual water maze task scene. The scene environment and interaction method are similar to those in the test mode, but there are no specific task objectives. Users practice in it to familiarize themselves with the operation methods. Step 3: Select water maze cognitive test tasks of different difficulty and mode in turn, and the user enters the scene to complete the corresponding task according to the current test task requirements. The user's task performance data and behavior data during the task are collected and stored in the database; Step 4: preprocess the stored original task performance data and behavior data, and extract corresponding task performance features and behavior data features; Step 5: Using the extracted task performance characteristics and behavioral data characteristics as input, a machine learning regression model is used to predict the spatial cognitive ability score and store it in the database to achieve intelligent assessment of spatial cognitive ability.
