ICU delirium evaluation and preventive intervention system based on VR eye movement tracking
Through the ICU delirium assessment and preventive intervention system based on VR eye tracking, combined with virtual reality technology and multi-module content, the objectivity and accuracy of ICU delirium assessment and prevention are solved, and the intelligent diagnosis and management of ICU delirium is realized. Improve the efficiency of patients and medical staff.
Patent Information
- Application Number
- CN202411255122.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art lacks VR technology focusing on preventive intervention and evaluation of ICU delirium, and the evaluation method of ICU delirium lacks objectivity and accuracy. It depends on the subjective judgment of medical staff, making it difficult to achieve intelligent diagnosis and management.
Using ICU delirium assessment and preventive intervention system based on VR eye movement tracking, testing tasks are constructed through virtual reality technology, patients' eye movement characteristics are collected in real time, and pre-trained models are used for screening and prediction, combining information education and cognitive training. , early exercise, mood regulation and sleep optimization and other multi-module contents to achieve systematic implementation of prevention and evaluation.
It improves the evaluation accuracy and preventive effect of ICU delirium, enhances the subjective initiative of patients, improves the accuracy of disease management and the work efficiency of medical staff, and realizes the intelligent diagnosis and management of ICU delirium.
Smart Images

Figure CN119028588A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of intelligent medical technology, and in particular to an ICU delirium assessment and preventive intervention system based on VR eye tracking, and an application method and electronic device thereof. Background Art
[0002] Delirium is an acute brain dysfunction syndrome characterized by acute and fluctuating changes in attention, consciousness, and cognition. Its incidence in ICU patients can be as high as 60% to 80%. Delirium not only prolongs the length of ICU stay and total hospital stay, increases in-hospital mortality, but may also cause long-term functional impairment, medical expenditure, and social burden on patients. Therefore, the guidelines call for early screening of ICU delirium and implementation of effective preventive interventions.
[0003] Currently, non-drug intervention measures such as ABCDEF bundled care are considered to be the first-line method for preventing delirium. However, the preventive effect of this type of practical non-drug multi-component intervention on delirium is still uncertain, and its implementation in clinical practice is still highly dependent on medical staff. Therefore, the evaluation method of ICU delirium is greatly affected by the subjectivity of the evaluator and lacks objective indicators. At the same time, the delirium assessment indicators are relatively single and the accuracy of the assessment results is not high.
[0004] Virtual Reality (VR) technology is a technology that uses computers to create and experience a virtual world. Through specific equipment and platforms, the experiencer can be "invested" in the environment, enabling the research subject to directly conduct a variety of realistic and natural real-time interactions in the artificially constructed computer virtual environment.
[0005] However, the existing VR intervention for ICU delirium mainly focuses on the treatment and rehabilitation stages after the occurrence of delirium, while it is important to move the threshold of preventive intervention for ICU delirium forward. There is currently a lack of VR technology designed and implemented to prevent delirium in ICU patients, and there is no technical equipment that can simultaneously integrate the prevention and evaluation functions of ICU delirium, and there is a lack of systematic implementation of intelligent diagnosis and management of clinical ICU delirium. Summary of the invention
[0006] In order to solve the above problems, the present application proposes an ICU delirium assessment and preventive intervention system based on VR eye tracking, and its application method and electronic device.
[0007] On the one hand, this application proposes an ICU delirium assessment and preventive intervention system based on VR eye tracking, which is implemented based on a matching virtual reality system and VR virtual equipment, including:
[0008] A VR-based ICU preventive intervention system to build different types of test tasks for ICU delirium patients;
[0009] An ICU delirium screening system based on VR eye tracking is used to collect the actual eye movement features of the eye positions of ICU delirium patients in different test tasks in real time, and input the actual eye movement features into a pre-trained ICU delirium screening model, output the predicted type of the actual eye movement features and the conversion risk probability corresponding to the predicted category through the ICU delirium screening model, and send the predicted type and the corresponding conversion risk probability to the VR virtual device;
[0010] The VR technology-based ICU preventive intervention system and the VR eye tracking-based ICU delirium screening system are communicatively connected.
[0011] As an optional implementation scheme of the present application, optionally, the application method of the VR technology-based ICU preventive intervention system includes:
[0012] Get attribute information of patients with delirium in ICU;
[0013] Build a virtual environment platform, and create test tasks including information education, cognitive training, early exercise, emotion regulation and sleep optimization for ICU delirium patients in the virtual environment, and regularly send each test task to the VR virtual device for playback testing.
[0014] As an optional embodiment of the present application, optionally, the ICU delirium screening system based on VR eye tracking includes:
[0015] A display module, used to display different types of test tasks in sequence at the eye positions of ICU delirium patients through the VR virtual device;
[0016] A tracking module, used to track the pupil movement trajectory and pupil change curve of the eye position of the ICU delirium patient, and obtain the eye movement trajectory of the eye position of the ICU delirium patient in different test tasks;
[0017] A first determination module, used to determine the focus of the test task;
[0018] A first feature extraction module, configured to extract a corresponding first eye movement feature from the eye movement trajectory corresponding to the test task according to the focus point;
[0019] a first evaluation and screening module, configured to input the first eye movement feature corresponding to the test task into a pre-trained ICU delirium screening model, and output a predicted type of the actual eye movement feature through the ICU delirium screening model, wherein the predicted type is divided into normal, ICU sub-delirium and delirium;
[0020] The display module, tracking module, first determination module, first feature extraction module and first evaluation and screening module are communicatively connected in sequence.
[0021] As an optional embodiment of the present application, optionally, the ICU delirium screening system based on VR eye tracking further includes:
[0022] A second determination module is used to determine, when the predicted category is normal or ICU sub-delirium, the target test task corresponding to the predicted category of normal or ICU sub-delirium from a plurality of preset target test tasks, and determine the corresponding target eye movement trajectory;
[0023] A second extraction module, used to extract a second eye movement feature in the target eye movement trajectory;
[0024] A second assessment and screening module is used to input the second eye movement feature and the corresponding predicted category into a preset risk conversion assessment model, and the risk conversion assessment model outputs the conversion risk probability under the corresponding predicted category; and send the conversion risk probability under the predicted category to the VR virtual device;
[0025] The first evaluation and screening module, the second determination module, the second extraction module and the second evaluation and screening module are communicatively connected in sequence.
[0026] As an optional implementation scheme of the present application, optionally, the risk conversion assessment model outputs the conversion risk probability corresponding to the predicted category, including:
[0027] If the predicted category is normal, the result output by the risk conversion assessment model is the first risk conversion probability of normal conversion to ICU sub-delirium;
[0028] If the predicted category is ICU subdelirium, the result output by the risk conversion assessment model is the second risk conversion probability of ICU subdelirium converting to ICU delirium;
[0029] The first risk conversion probability or the second risk conversion probability is calculated based on the risk conversion prediction result of normal or ICU sub-delirium by the risk conversion assessment model.
[0030] As an optional embodiment of the present application, optionally, the method for generating the ICU delirium screening model comprises:
[0031] Collect eye movement feature data of normal, ICU subdelirium and delirium ICU delirium patients in different test tasks;
[0032] Preprocessing the eye movement feature data of different test tasks and dividing them into a training set and a validation set according to a preset ratio;
[0033] Inputting the training set into the CNN model, performing model training, and generating a sub-screening model for each of the test tasks;
[0034] Fusion of the sub-screening models of the various test tasks to obtain the ICU delirium screening model;
[0035] Importing the verification set into the ICU delirium screening model to verify the model performance of the ICU delirium screening model;
[0036] If the verification is qualified, the ICU delirium screening model will be deployed in the first assessment and screening module.
[0037] On the other hand, the present application proposes an application method of an ICU delirium assessment and preventive intervention system based on VR eye tracking, comprising the following steps:
[0038] The VR-based ICU preventive intervention system builds different types of test tasks for ICU delirium patients;
[0039] An ICU delirium screening system based on VR eye tracking collects the actual eye movement characteristics of the eye positions of ICU delirium patients in different test tasks in real time, and inputs the actual eye movement characteristics into a pre-trained ICU delirium screening model;
[0040] Outputting the predicted type of the actual eye movement feature and the conversion risk probability corresponding to the predicted category through the ICU delirium screening model;
[0041] The prediction type and the corresponding conversion risk probability are sent to the VR virtual device.
[0042] In another aspect, the present application further provides an electronic device, comprising:
[0043] processor;
[0044] a memory for storing processor-executable instructions;
[0045] Wherein, the processor is configured to implement the application method of the ICU delirium assessment and preventive intervention system based on VR eye tracking when executing the executable instructions.
[0046] Technical effects of the present invention:
[0047] This application achieves the effect of accurately assessing ICU delirium by collecting and processing the eye movement signals of ICU delirium patients in real time based on the dimensional characteristics of ICU delirium. Combined with dynamic collections, the collection includes information education, cognitive training, early exercise, emotion regulation, sleep optimization and other multi-module content to achieve the effect of preventing ICU delirium. The clinical intelligent technology equipment that integrates the prevention and screening evaluation functions of ICU delirium is portable and practical, so as to systematically realize the intelligent diagnosis and management of ICU delirium.
[0048] For ICU delirium patients, VR technology has great potential in ICU delirium assessment and preventive intervention. It is not only conducive to improving the accuracy of ICU delirium assessment, but also conducive to mobilizing the subjective initiative of ICU delirium patients to better practice non-drug prevention. Based on the good effects of VR technology in improving emotions, cognition, sleep, etc., the implementation of ICU delirium assessment and preventive intervention programs with the help of VR technology will help improve the precise management of the disease and the intelligent control of the ICU environment. Therefore, by focusing on the preventive intervention of ICU delirium and designing and implementing VR technology, the prevention and screening assessment functions of ICU delirium are dynamically aggregated, which will not only help improve the clinical outcomes of ICU delirium patients, but also improve the work efficiency of medical staff.
[0049] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.
[0051] Figure 1 Shown is a schematic diagram of the system composition structure of the present invention;
[0052] Figure 2 The figure shows an application logic schematic diagram of the ICU preventive intervention system based on VR technology of the present invention;
[0053] Figure 3 The figure shows a schematic diagram of the system composition structure of the ICU delirium screening system based on VR eye tracking of the present invention;
[0054] Figure 4 It is a schematic diagram showing the application of the electronic device of the present invention. DETAILED DESCRIPTION
[0055] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0056] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0057] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.
[0058] Example 1
[0059] like Figure 1 As shown, on the one hand, the present application proposes an ICU delirium assessment and preventive intervention system based on VR eye tracking, which is implemented based on a matching virtual reality system and VR virtual equipment, including:
[0060] The VR-based ICU preventive intervention system (referred to as the task building system) is used to build different types of test tasks for ICU delirium patients;
[0061] An ICU delirium screening system based on VR eye tracking (hereinafter referred to as the screening system) is used to collect the actual eye movement characteristics of the eye positions of ICU delirium patients in different test tasks in real time, and input the actual eye movement characteristics into a pre-trained ICU delirium screening model, output the predicted type of the actual eye movement characteristics and the conversion risk probability corresponding to the predicted category through the ICU delirium screening model, and send the predicted type and the corresponding conversion risk probability to the VR virtual device;
[0062] The VR technology-based ICU preventive intervention system and the VR eye tracking-based ICU delirium screening system are communicatively connected.
[0063] The present invention designs and implements VR technology by focusing on preventive intervention for ICU delirium, dynamically aggregating the prevention, screening and evaluation functions of ICU delirium, and aggregating the prevention, screening and evaluation functions of ICU delirium into a portable and highly practical clinical intelligent technology device to systematically realize intelligent diagnosis and management of ICU delirium.
[0064] For patients with ICU delirium, VR technology is used to conduct ICU delirium assessment and preventive intervention in ICU delirium, so as to mobilize the subjective initiative of patients with ICU delirium.
[0065] Based on the good effects of VR technology in improving mood, cognition, sleep, etc., VR technology is used to implement ICU delirium assessment and preventive intervention plans.
[0066] Combined with Figure 1 As shown, the task construction system and screening system of the present invention need to be implemented in conjunction with a virtual reality system and a VR virtual device. Therefore, the information display, interaction and VR control involved therein can be implemented and understood in conjunction with the user's matching virtual reality system and VR virtual device.
[0067] VR virtual devices include VR headset main control module, VR headset somatosensory recognition module, VR motion interaction module, and VR function quantitative analysis module. Among them, the VR headset main control module needs to issue corresponding test tasks, and other modules will cooperate and participate.
[0068] The application and composition of the two systems will be further described below.
[0069] As attached Figure 2 As shown, as an optional implementation scheme of the present application, optionally, the application method of the VR technology-based ICU preventive intervention system includes:
[0070] Get attribute information of patients with delirium in ICU;
[0071] Build a virtual environment platform, and create test tasks including information education, cognitive training, early exercise, emotion regulation and sleep optimization for ICU delirium patients in the virtual environment, and regularly send each test task to the VR virtual device for playback testing.
[0072] The ICU preventive intervention system can send corresponding interactive information (text or voice) to the VR virtual device worn by the patient through the background system to collect the patient's basic information, such as attributes. It can also respond by interactive operations through the NR virtual device.
[0073] The ICU preventive intervention system mainly constructs test tasks in different virtual links for patients. The steps include:
[0074] (1) Obtain attribute information of the ICU delirium patients to be tested, including case number, gender, age, etc. of the ICU delirium patients to be tested.
[0075] (2) Construct a virtual environment platform to collect virtual data of each module of ICU delirium patients in a virtual environment according to the pre-designed clustered multi-module ICU delirium prevention link (five types of training in the virtual environment platform) and output it to the system. The virtual environment platform contains multiple modules such as information education, cognitive training, early exercise, emotion regulation, and sleep optimization. Starting from the time the ICU delirium patients enter the ICU, each program module is routinely cycled once a day, and the playback terminal is uniformly controlled by the clinical nurse station;
[0076] (3) During the playback test, pupil changes and movement information of the ICU delirium patients are collected in real time, and the intervention program of interest to the ICU delirium patients is intelligently adjusted based on this information.
[0077] The above specific methods include:
[0078] (1) Obtain the attribute information of the ICU delirium patients to be intervened. The tester inputs the attribute information of the ICU delirium patients to be intervened, puts on the VR virtual device for the ICU delirium patients to be intervened, and starts the intervention program in sequence. Because most ICU delirium patients are in a supine position (including semi-recumbent position), the attribute information of the ICU delirium patients that affects the sense of space, such as height and weight, is not collected. However, in the design stage, scenes that meet the requirements of supine position (including semi-recumbent position) and sitting position (including screen display ratio, etc.) are set in advance to meet the requirements of the spatial experience. Considering that different genders may have different levels of interest in different categories of scenes, female ICU delirium patients may prefer fresh theme scenes, while male ICU delirium patients may like passionate scenes. Selecting scenes that ICU delirium patients are more likely to be interested in based on different attributes to carry out the program will help ICU delirium patients immerse themselves in the experience in a shorter period of time.
[0079] (2) During the playback test, pupil changes and movement information of the ICU delirium patients are collected in real time to further determine the VR program scenes that the ICU delirium patients may be interested in. Pupil change information includes: changes in pupil size and changes in pupil movement trajectory. First, the initial pupil value of the ICU delirium patients to be tested is determined. After playing a short scene of each module program, the coordinates and pupil values of the pupils of the ICU delirium patients to be tested are recorded at each time point to obtain changes in pupil movement trajectory and pupil size. In the early movement module program, the type of virtual distractors that attract the attention of the ICU delirium patients to be tested is obtained by comparing the time when the virtual distractors appear in the short scene with the corresponding pupil change information. By comparing the pupil change information in brief scenes of each module program, the training scenes that the ICU delirium patients are interested in are obtained. According to the types of virtual distractors that attract the attention of the ICU delirium patients in the early motor test and the VR program scenes that the ICU delirium patients are interested in, virtual training scenes that are more in line with the personalized needs of ICU delirium patients are generated, which is conducive to improving the prevention effect of ICU delirium.
[0080] (3) Virtual environment platform: The VR head display main control terminal generates a virtual scene and presents it to the user, and plays the corresponding program. The virtual environment platform contains multiple program modules such as information education, cognitive training, early exercise, emotion regulation, and sleep optimization. This part of the system is completed by VR hardware related to computer program instructions. The program is stored in a non-volatile computer-readable storage medium and is executed by the processor when in use.
[0081] (3.1) Information education
[0082] The information education program includes a VR headset main control module. Through the VR headset, a 720-degree immersive ICU simulation nursing experience is implemented: experience the ICU ward space and understand the treatment equipment from the first-person perspective of the ICU delirium patient; the information education voice-over related to the ICU and delirium in the virtual scene is synchronized with the experience perspective of the ICU delirium patient.
[0083] (3.2) Cognitive training
[0084] The cognitive training program includes a VR headset main control module, a VR headset somatosensory recognition module, a VR interaction module, and a VR function quantitative analysis module. The VR headset somatosensory recognition module generates a virtual body model of the ICU delirium patient by capturing the head position of the subject ICU delirium patient, and presents it in front of the field of vision of the subject ICU delirium patient in the virtual cognitive training scene. The virtual cognitive training scene is a supermarket, and the context is supermarket shopping. Through VR, ICU delirium patients are trained in spatial orientation, attention, execution, thinking ability, and memory, including targeted and interesting training of the cognitive ability of the subject ICU delirium patients, such as digital verification, commodity list memory, commodity function recognition, and shopping account settlement. The quantification results of cognitive function are judged simultaneously by the accuracy and rate of task completion of the subject ICU delirium patients.
[0085] (3.3) Early Movement
[0086] The early activity program includes a VR headset main control module, a VR headset somatosensory recognition module, a VR motion interaction module, and a VR function quantitative analysis module. The VR headset somatosensory recognition module generates a virtual body model of the ICU delirium patient by capturing the head position of the ICU delirium patient, and presents it in front of the field of vision of the ICU delirium patient in the virtual early movement scene. The early movement scene is a game-like dynamic space. Virtual distractors are generated according to the early movement plan. The determination of virtual distractors in the early movement plan is mainly based on the pupil curve of the ICU delirium patient in the test. By comparing the pupil change curve corresponding to the appearance time and position of the virtual distractors in the test scene, the type of virtual distractors that can arouse the interest of the ICU delirium patient in the test is determined. The ICU delirium patient in the test controls the position change of the virtual body model through movement to avoid the virtual distractors, thereby achieving the effect of early movement to prevent the occurrence of ICU delirium.
[0087] This system synchronously records the somatosensory signals of the ICU delirium patients to evaluate the quantitative results of early motor training. Considering the restricted activities of ICU delirium patients, the training mainly focuses on the head, neck and upper limbs.
[0088] The present invention realizes movement and positioning in virtual reality space through VR technology and somatosensory recognition technology, realizes training mainly based on head, neck and upper limb movements through motion interaction, digitally collects motion data of ICU delirium patients, and systematically records it in the terminal machine. The resulting report can be used by the clinical research team to further explore delirium prevention measures, and improve the training program of early motion modules according to the results of motor function quantification, such as changing the size and position of virtual interference objects. The functional quantification results are judged by the avoidance action and avoidance result of the ICU delirium patients. The avoidance action mainly refers to whether the avoidance angle is consistent with the preset, the speed and amplitude of the action, etc.
[0089] (3.4) Emotion Regulation
[0090] The emotion regulation program includes a VR headset control module. Through the VR headset, a 720-degree immersive virtual scene is presented directly in front of the ICU delirium patient. The virtual scene content is a natural scene, integrated with mindfulness meditation, and accompanied by tranquil music.
[0091] (3.5) Sleep optimization
[0092] The sleep optimization program includes a VR headset control module. Through the VR headset, a 720-degree immersive virtual scene is presented directly in front of the ICU delirium patient. The virtual scene content changes from an ICU scene with changing light to a natural scene dominated by dark colors such as night, accompanied by hypnotic music.
[0093] The system can play each test task according to the preset playback information and perform playback test at a fixed time. It can also be triggered actively by the patient or user.
[0094] As attached Figure 3 As shown, as an optional embodiment of the present application, optionally, the ICU delirium screening system based on VR eye tracking includes:
[0095] A display module, used to display different types of test tasks in sequence at the eye positions of ICU delirium patients through the VR virtual device;
[0096] A tracking module, used to track the pupil movement trajectory and pupil change curve of the eye position of the ICU delirium patient, and obtain the eye movement trajectory of the eye position of the ICU delirium patient in different test tasks;
[0097] A first determination module, used to determine the focus of the test task;
[0098] A first feature extraction module, configured to extract a corresponding first eye movement feature from the eye movement trajectory corresponding to the test task according to the focus point;
[0099] a first evaluation and screening module, configured to input the first eye movement feature corresponding to the test task into a pre-trained ICU delirium screening model, and output a predicted type of the actual eye movement feature through the ICU delirium screening model, wherein the predicted type is divided into normal, ICU sub-delirium and delirium;
[0100] The display module, tracking module, first determination module, first feature extraction module and first evaluation and screening module are communicatively connected in sequence.
[0101] In response to the above-mentioned problems that the current ICU delirium assessment lacks evaluation indicators and the results are not accurate, this application proposes an ICU delirium screening system based on VR eye tracking. The system realizes a simple, efficient, objective and accurate ICU delirium assessment, and can predict the risk of ICU delirium patients who have not yet developed delirium. A VR screening tool combining delirium assessment tasks with eye tracking technology was designed and developed, including: ① displaying different types of test tasks in sequence at the eye positions of the ICU delirium patients through a VR head display device; ② tracking the eye movement trajectory of the ICU delirium patients within a preset time after being instructed to identify and fixate on a target image; ③ based on the focus point of each test task; ④ extracting the actual eye movement features corresponding to the test task from the eye movement trajectory corresponding to the test task according to the focus point of the test task; ⑤ using the actual eye movement features corresponding to each test task as the input of the ICU delirium screening model to obtain the predicted screening results output by the relevant ICU delirium screening model. The delirium screening model includes multiple sub-screening models and a fusion output layer. The fusion output layer is used to determine the predicted screening results based on multiple sub-prediction results, thereby achieving more accurate early screening of ICU delirium.
[0102] By applying VR and eye tracking technology, through setting up five modules of display, tracking, determination, extraction and evaluation screening, key eye movement information when delirium occurs can be accurately measured, such as recording and analyzing the number of gaze points, percentage of gaze duration, eye movement trajectory within a preset time, etc. of the ICU delirium patients, so as to realize real-time collection, communication and processing of eye movement signals in non-invasive delirium assessment tasks.
[0103] Display module: Different types of test tasks are displayed in sequence at the eye positions of the ICU delirium patients through VR virtual devices. The test tasks include target images and non-target images. The ICU delirium patients recognize and look at the target images.
[0104] Tracking module: Track the eye movement trajectory of the ICU delirium patient within a preset time (e.g., 15 seconds) after being instructed to identify and stare at the target image, determine the light sensitivity of the ICU delirium patient according to the pupil size of the ICU delirium patient, adjust the light of the VR virtual device according to the light sensitivity to match the appropriate brightness, and determine the initial pupil value of the ICU delirium patient; record the pupil coordinates at each time point and the pupil value at each time point; generate the pupil movement trajectory according to the pupil coordinates of the ICU delirium patient at each time point, determine the pupil change curve according to the pupil value, and obtain the eye movement trajectory according to the pupil movement trajectory and the pupil change curve; after being instructed to identify and stare at the target image, the ICU delirium patient needs to track and record the pupil coordinates at each time point within the preset time to identify the target option. The eye movement trajectory of the ICU delirium patient within the preset time is captured to obtain the real reaction of the ICU delirium patient and generate the pupil movement trajectory.
[0105] The first determination module: Determine the focus of the corresponding test task according to the task type of each test task. Among them, in the process of delirium assessment and screening of ICU delirium patients, it is necessary to set test tasks to examine different aspects of the ICU delirium patients, and different task types correspond to different abilities that need to be examined. The diagnostic points of ICU delirium include test tasks for examining "visual-spatial ability", "attention", "level of consciousness", "thinking ability", etc. Different test tasks have different focus points corresponding to different test tasks. The focus points include: the duration of fixation on the target image, the effective duration, the total duration, the last stop position, etc. For example, when examining "thinking ability", the focus point can be set to the last stop position, and when examining "attention", the focus point is set to the effective duration of fixation on the target image.
[0106] The first extraction module can extract the corresponding first eye movement feature from the eye movement trajectory corresponding to the task according to the focus point of the test task. The eye movement feature focused on by the test task can be extracted from the eye movement trajectory by setting the focus point.
[0107] The first evaluation and screening module uses the first eye movement feature corresponding to each test task as the input of the ICU delirium screening model to obtain the predicted category output by the ICU delirium screening model. The ICU delirium screening model includes multiple sub-screening models and a fusion output layer. Each test task corresponds to a sub-screening model predicted according to the first eye movement feature. The fusion output layer determines the prediction category through multiple sub-prediction results. The prediction category is divided into normal, ICU sub-delirium, and delirium. For those whose screening status is normal or ICU sub-delirium, it is necessary to predict the conversion risk and intervene in advance when there is a conversion risk. Different test tasks examine different abilities, so multiple sub-screening models are set to improve the accuracy of the prediction. The system integrates multiple sub-models to achieve more accurate screening, evaluation and risk prediction of ICU delirium.
[0108] As an optional embodiment of the present application, optionally, the ICU delirium screening system based on VR eye tracking further includes:
[0109] A second determination module is used to determine, when the predicted category is normal or ICU sub-delirium, the target test task corresponding to the predicted category of normal or ICU sub-delirium from a plurality of preset target test tasks, and determine the corresponding target eye movement trajectory;
[0110] A second extraction module, used to extract a second eye movement feature in the target eye movement trajectory;
[0111] A second assessment and screening module is used to input the second eye movement feature and the corresponding predicted category into a preset risk conversion assessment model, and the risk conversion assessment model outputs the conversion risk probability under the corresponding predicted category; and send the conversion risk probability under the predicted category to the VR virtual device;
[0112] The first evaluation and screening module, the second determination module, the second extraction module and the second evaluation and screening module are communicatively connected in sequence.
[0113] As an optional implementation scheme of the present application, optionally, the risk conversion assessment model outputs the conversion risk probability corresponding to the predicted category, including:
[0114] If the predicted category is normal, the result output by the risk conversion assessment model is the first risk conversion probability of normal conversion to ICU sub-delirium;
[0115] If the predicted category is ICU subdelirium, the result output by the risk conversion assessment model is the second risk conversion probability of ICU subdelirium converting to ICU delirium;
[0116] The first risk conversion probability or the second risk conversion probability is calculated based on the risk conversion prediction result of normal or ICU sub-delirium by the risk conversion assessment model.
[0117] The second determination module, when the predicted category is normal or ICU sub-delirium, determines the target test task corresponding to the predicted category and determines the corresponding target eye movement trajectory. The target test tasks corresponding to the normal category and the ICU sub-delirium category are set in advance, and the test tasks are different. In risk prediction, the target eye movement trajectory extracted during the previous ICU delirium screening can be obtained according to the target test task, or the ICU delirium patient can be retested after the target test task is determined and a new target eye movement trajectory can be extracted.
[0118] The second extraction module determines a second eye movement feature based on the extracted target eye movement trajectory. The second eye movement feature has a different focus from the first eye movement feature. The first eye movement feature focuses on all aspects, while the second eye movement feature focuses on certain features.
[0119] The second evaluation and screening module uses the predicted category and the second eye movement feature as inputs to the risk conversion evaluation model (screening system) to obtain the conversion risk probability under the corresponding predicted category. If the predicted category is normal, the result output by the risk conversion evaluation model is the risk conversion probability of normal conversion to ICU subdelirium. If the predicted category is ICU subdelirium, the result output by the risk conversion evaluation model is the risk conversion probability of ICU subdelirium to ICU delirium. The risk assessment module is conducive to preventing conversion risks.
[0120] The risk conversion assessment model can be trained by referring to the generation method of the ICU delirium screening model, except that the training object is replaced with a data set of conversion risk probabilities corresponding to different prediction categories. Feature engineering and model training can also be performed.
[0121] The risk conversion probability can be specifically calculated in combination with clinical conversion data, and can be calculated and configured by the user in combination with clinical conversion data.
[0122] As an optional embodiment of the present application, optionally, the method for generating the ICU delirium screening model comprises:
[0123] Collect eye movement feature data of normal, ICU subdelirium and delirium ICU delirium patients in different test tasks;
[0124] Preprocessing the eye movement feature data of different test tasks and dividing them into a training set and a validation set according to a preset ratio;
[0125] Inputting the training set into the CNN model, performing model training, and generating a sub-screening model for each of the test tasks;
[0126] Fusion of the sub-screening models of the various test tasks to obtain the ICU delirium screening model;
[0127] Importing the verification set into the ICU delirium screening model to verify the model performance of the ICU delirium screening model;
[0128] If the verification is qualified, the ICU delirium screening model will be deployed in the first assessment and screening module.
[0129] ICU delirium screening model training generation method:
[0130] 1. Data preparation
[0131] Data collection: Collect eye movement feature data of normal, ICU sub-delirium, and ICU delirium patients. The data should include pupil coordinates, pupil size, number of fixations, percentage of fixation duration, eye movement trajectory, etc.
[0132] 2. Data preprocessing: Preprocess the eye movement feature data, including normalization, data enhancement (such as rotation, scaling, etc.) and data set division (training set, validation set and test set).
[0133] 3. Model Construction
[0134] Define the CNN model structure:
[0135] Input layer: accepts preprocessed eye movement feature data.
[0136] Convolutional layer: extract features, using multiple convolutional layers and pooling layers.
[0137] Fully connected layer: maps the extracted features to the classification space.
[0138] Output layer: Use the Softmax activation function to output three types of predictions (normal, ICU subdelirium, delirium).
[0139] 4. Model Training
[0140] Compile the model: select an appropriate loss function (such as cross entropy loss) and optimizer (such as Adam).
[0141] Train the model: Use the training set data to train the model and use the validation set data to monitor the model performance.
[0142] 5. Model Validation and Evaluation
[0143] Validate model: Evaluate the performance of the model on the validation set and adjust hyperparameters (such as learning rate, batch size, etc.) to optimize the model.
[0144] Test model: Evaluate the final performance of the model on the test set to ensure the generalization ability of the model.
[0145] 6. Model Application
[0146] Prediction category: Use the trained model to predict new data and output the predicted category (normal, ICU subdelirium, delirium).
[0147] 7. Risk prediction: For ICU delirium patients whose screening status is normal or ICU sub-delirium, further predict their conversion risk. If there is a risk of conversion, intervene in advance.
[0148] 8. Fusion of multiple sub-models
[0149] Sub-screening model: Each test task corresponds to a sub-screening model, which makes predictions based on the first eye movement features.
[0150] Fusion output layer: Fusion the prediction results of multiple sub-screening models to determine the final prediction category.
[0151] Please understand the above model training in conjunction with the conventional training and verification steps of the CNN model, which will not be described in detail in this embodiment.
[0152] Obviously, those skilled in the art should understand that the implementation of all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.
[0153] Example 2
[0154] Based on the implementation principle of Example 1, on the other hand, this application proposes an application method of an ICU delirium assessment and preventive intervention system based on VR eye tracking, comprising the following steps:
[0155] The VR-based ICU preventive intervention system builds different types of test tasks for ICU delirium patients;
[0156] An ICU delirium screening system based on VR eye tracking collects the actual eye movement characteristics of the eye positions of ICU delirium patients in different test tasks in real time, and inputs the actual eye movement characteristics into a pre-trained ICU delirium screening model;
[0157] Outputting the predicted type of the actual eye movement feature and the conversion risk probability corresponding to the predicted category through the ICU delirium screening model;
[0158] The prediction type and the corresponding conversion risk probability are sent to the VR virtual device.
[0159] The above method steps should be understood in conjunction with the system functions and interactive services of Example 1, and will not be described in detail in this example.
[0160] The modules or steps of the present invention described above can be implemented by a general-purpose computing system, they can be concentrated on a single computing system, or distributed on a network composed of multiple computing systems, and optionally, they can be implemented by a program code executable by a computing system, so that they can be stored in a storage system and executed by the computing system, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0161] Example 3
[0162] Furthermore, in another aspect, the present application also proposes an electronic device, comprising:
[0163] processor;
[0164] a memory for storing processor-executable instructions;
[0165] Among them, the processor is configured to implement the application method of the ICU delirium assessment and preventive intervention system based on VR eye tracking described in Example 2 when executing the executable instructions.
[0166] The electronic device of the embodiment of the present disclosure includes a processor and a memory for storing instructions executable by the processor, wherein the processor is configured to implement the application method described in the above embodiment 2 when executing the executable instructions.
[0167] Here, it should be noted that the number of processors can be one or more. At the same time, the electronic device of the embodiment of the present disclosure may also include an input system and an output system. Among them, the processor, memory, input system and output system may be connected through a bus or in other ways, which are not specifically limited here.
[0168] The memory is a computer-readable storage medium that can be used to store software programs, computer executable programs, and various modules, such as the program or module corresponding to the application method of the embodiment of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running the software programs or modules stored in the memory.
[0169] The input system can be used to receive input numbers or signals. The signal can be a key signal related to user settings and function control of the device / terminal / server. The output system can include display devices such as display screens.
[0170] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. An ICU delirium assessment and preventive intervention system based on VR eye tracking, implemented based on a matching virtual reality system and VR virtual equipment. It is characterized in that include: A VR-based ICU preventive intervention system to build different types of test tasks for ICU delirium patients; An ICU delirium screening system based on VR eye tracking is used to collect the actual eye movement features of the eye positions of ICU delirium patients in different test tasks in real time, and input the actual eye movement features into a pre-trained ICU delirium screening model, output the predicted type of the actual eye movement features and the conversion risk probability corresponding to the predicted category through the ICU delirium screening model, and send the predicted type and the corresponding conversion risk probability to the VR virtual device; The VR technology-based ICU preventive intervention system and the VR eye tracking-based ICU delirium screening system are communicatively connected.
2. According to claim 1, the ICU delirium assessment and preventive intervention system based on VR eye tracking, It is characterized in that The application method of the VR technology-based ICU preventive intervention system includes: Get attribute information of patients with delirium in ICU; Build a virtual environment platform, and create test tasks including information education, cognitive training, early exercise, emotion regulation and sleep optimization for ICU delirium patients in the virtual environment, and regularly send each test task to the VR virtual device for playback testing.
3. According to claim 1, the ICU delirium assessment and preventive intervention system based on VR eye tracking, It is characterized in that The ICU delirium screening system based on VR eye tracking includes: A display module, used to display different types of test tasks in sequence at the eye positions of ICU delirium patients through the VR virtual device; A tracking module, used to track the pupil movement trajectory and pupil change curve of the eye position of the ICU delirium patient, and obtain the eye movement trajectory of the eye position of the ICU delirium patient in different test tasks; A first determination module, used to determine the focus of the test task; A first feature extraction module, configured to extract a corresponding first eye movement feature from the eye movement trajectory corresponding to the test task according to the focus point; a first evaluation and screening module, configured to input the first eye movement feature corresponding to the test task into a pre-trained ICU delirium screening model, and output a predicted type of the actual eye movement feature through the ICU delirium screening model, wherein the predicted type is divided into normal, ICU sub-delirium and delirium; The display module, tracking module, first determination module, first feature extraction module and first evaluation and screening module are communicatively connected in sequence.
4. According to claim 3, the ICU delirium assessment and preventive intervention system based on VR eye tracking, It is characterized in that The ICU delirium screening system based on VR eye tracking also includes: A second determination module is used to determine, when the predicted category is normal or ICU sub-delirium, the target test task corresponding to the predicted category of normal or ICU sub-delirium from a plurality of preset target test tasks, and determine the corresponding target eye movement trajectory; A second extraction module, used to extract a second eye movement feature in the target eye movement trajectory; A second assessment and screening module is used to input the second eye movement feature and the corresponding predicted category into a preset risk conversion assessment model, and the risk conversion assessment model outputs the conversion risk probability under the corresponding predicted category; and send the conversion risk probability under the predicted category to the VR virtual device; The first evaluation and screening module, the second determination module, the second extraction module and the second evaluation and screening module are communicatively connected in sequence.
5. According to claim 4, the ICU delirium assessment and preventive intervention system based on VR eye tracking, It is characterized in that The risk conversion assessment model outputs the conversion risk probability corresponding to the predicted category, including: If the predicted category is normal, the result output by the risk conversion assessment model is the first risk conversion probability of normal conversion to ICU sub-delirium; If the predicted category is ICU subdelirium, the result output by the risk conversion assessment model is the second risk conversion probability of ICU subdelirium converting to ICU delirium; The first risk conversion probability or the second risk conversion probability is calculated based on the risk conversion prediction result of normal or ICU sub-delirium by the risk conversion assessment model.
6. According to claim 3, the ICU delirium assessment and preventive intervention system based on VR eye tracking, It is characterized in that The method for generating the ICU delirium screening model comprises: Collect eye movement feature data of normal, ICU subdelirium and delirium ICU delirium patients in different test tasks; Preprocessing the eye movement feature data of different test tasks and dividing them into a training set and a validation set according to a preset ratio; Inputting the training set into the CNN model, performing model training, and generating a sub-screening model for each of the test tasks; Fusion of the sub-screening models of the various test tasks to obtain the ICU delirium screening model; Importing the verification set into the ICU delirium screening model to verify the model performance of the ICU delirium screening model; If the verification is qualified, the ICU delirium screening model will be deployed in the first assessment and screening module.
7. An application method of the ICU delirium assessment and preventive intervention system based on VR eye tracking according to any one of claims 1 to 6, It is characterized in that The steps include: The VR-based ICU preventive intervention system builds different types of test tasks for ICU delirium patients; An ICU delirium screening system based on VR eye tracking collects the actual eye movement characteristics of the eye positions of ICU delirium patients in different test tasks in real time, and inputs the actual eye movement characteristics into a pre-trained ICU delirium screening model; Outputting the predicted type of the actual eye movement feature and the conversion risk probability corresponding to the predicted category through the ICU delirium screening model; The prediction type and the corresponding conversion risk probability are sent to the VR virtual device.
8. Electronic equipment, It is characterized in that include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the application method of the ICU delirium assessment and preventive intervention system based on VR eye tracking as described in claim 7 when executing the executable instructions.
