Cognitive rehabilitation system based on digital twin interaction and music therapy

Abstract

The utility model provides a cognitive rehabilitation system based on digital twin interaction and music therapy, and the system comprises a positioning module which locates the position of a user in real time; the acquisition module is used for acquiring user physiological index parameters in real time; the display module is used for displaying the digital person, the virtual digital body of the user and the virtual scene to the user; the audio module is used for playing output voice and treatment music of the digital human to a user; and the processing module is connected with the positioning module, the acquisition module, the display module and the audio module, and is used for adjusting the content of the virtual scene and / or the treatment music. By using the system provided by the invention, CBT and MT can be implemented on the user in a virtual situation, so that the user is helped to identify and change bad thinking modes and behavioral habits, and deal with negative emotions and thinking, thereby improving gradually decreasing cognitive functions and emotional disorder symptoms of the user.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of virtual reality, and in particular, relates to a cognitive rehabilitation system based on digital twin interaction and music therapy. BACKGROUND

[0002] Music therapy (MT) is a professional treatment method that applies music and music elements to promote physical and mental health, improve emotional state, enhance social skills, and improve quality of life. It utilizes the psychological, physiological, and social functions of music to guide individuals to express emotions and self-adjust through listening, creating, playing, and singing. For people with mild cognitive impairment (MCI), music therapy can help them alleviate emotional disorders such as anxiety and depression caused directly or indirectly by cognitive decline through the unique effects of music, thereby improving their quality of life. Music therapy can activate brain regions related to emotional regulation, such as the limbic system, thereby improving the emotional state of MCI patients. By adjusting music suitable for the emotional state of patients, it helps to regulate the autonomic nervous system of patients, balance the activities of sympathetic and parasympathetic nerves, and promote the release of neurotransmitters such as dopamine and endorphins in the brain, thereby relieving tension and stress.

[0003] Virtual reality (VR) cognitive rehabilitation can create a realistic virtual environment, making patients feel as if they are in a real scene during treatment. This immersive experience helps to stimulate patients' interest and participation. Based on VR, digital humans can be personalized 3D modeled according to the specific circumstances of patients, including the appearance, behavior characteristics, etc. of patients or designated characters, and digital humans can be used as guides and emotional value providers for VR-based music therapy, making the treatment process more closely related to the actual situation of patients and helping patients with emotional regulation and coping training.

[0004] In the early stages of the development of VR-based MCI emotional disorder treatment, Optale et al. (Neurorehabil Neural Repair 2010) designed a VR memory stimulus based on auditory stimulation and music therapy, and the results showed that compared with classic music therapy, participants' memory tests and other aspects of cognition improved significantly. However, Meta analysis showed that VR research on cognitive function subcategories of emotional disorders in patients with mild cognitive impairment was still insufficient; there is currently no research on the therapeutic effect of VR combined with MT on MCI emotional disorders.

[0005] At present, there are different treatment effects of different patients and therapists are difficult to quantitatively perceive the treatment effect when treating patients with low cognitive impairment by MT. This individual difference makes it difficult to predict and standardize the efficacy of music therapy, resulting in non-uniform evaluation of efficacy standards. There are disadvantages such as insufficient content development and non-uniform evaluation standards when treating patients with low cognitive impairment through VR technology, which specifically manifest in that it is difficult to achieve individualization and diversification for different low cognitive impairment patient needs, lack of standardized evaluation tools, etc. Utility model content

[0006] In order to solve the problems in the prior art, the purpose of the present application is to provide a cognitive rehabilitation system based on digital twin interaction and music therapy, which aims to provide CBT treatment and MT treatment experience with high immersion and high comfort for MCI patients, so as to alleviate cognitive impairment and emotional disorders caused thereby.

[0007] Specifically, the present application relates to the following aspects:

[0008] The present application provides a cognitive rehabilitation system based on digital twin interaction and music therapy, comprising:

[0009] A positioning module is arranged in the user's activity range to locate the user's position in real time. The acquisition module includes a pulse oxygen device, a heart detection device and a skin electricity reaction analysis device, which are arranged at multiple body parts of the user to acquire physiological index parameters of the user in real time. The display module is used to display digital people, virtual digital avatars of the user and virtual scenes to the user. The audio module is used to play output speech of the digital people and treatment music to the user. The processing module is connected with the positioning module, the acquisition module, the display module and the audio module. The processing module adjusts the content of the output speech of the digital people, the virtual scene and / or the treatment music based on the physiological index parameters of the user acquired by the acquisition module in real time.

[0010] Preferably, the user's position includes the spatial positions of the user's head, torso and both hands. The positioning module sends the user's position to the processing module, and the processing module generates a virtual digital avatar of the user based on the user's position and transmits it to the display module.

[0011] Preferably, the user's physiological index parameters include blood oxygen saturation, heart rate, blood pressure and skin electricity. The acquisition module sends the user's physiological index parameters to the processing module, and the processing module calculates user emotional state parameters based on the user's physiological index parameters and classifies the user's emotional state. The processing module sends control instructions to the display module and the audio module based on the user's emotional state classification result, respectively, to adjust the virtual scene and the treatment music to the virtual scene and the treatment music matched with the user's emotional state classification result, respectively.

[0012] Preferably, the acquisition module further comprises: an image acquisition assembly comprising a photographic device and a first data interface, which are respectively connected with the processing module, the photographic device is used for acquiring the face and / or body image, and / or the first data interface is used for obtaining the face and / or body image from the data storage device and sending to the processing module; the processing module generates the digital person according to the face and / or body image and transmits to the display module.

[0013] Preferably, the acquisition module further comprises: an audio acquisition assembly comprising a microphone device and a second data interface, which are respectively connected with the processing module, the microphone device is used for acquiring the user voice and / or other individual voice, and / or the second data interface is used for obtaining the other individual voice and / or treatment text from the data storage device and sending to the processing module; the processing module generates the output voice of the digital person according to the other individual voice and / or treatment text; and the processing module adjusts the output voice of the digital person according to the user voice.

[0014] Preferably, the display module comprises: a first display device and a digital controller; wherein the first display device is a VR glasses, a VR helmet, an AR helmet, an MR helmet or a naked eye 3D imaging device, which is used for displaying the digital person, the user virtual digital avatar and the virtual scene; the digital controller is connected with the positioning module and the audio module respectively, acquires the position of the user's hands positioned by the positioning module and recognizes the user's gestures, so as to adjust the content of the virtual scene and / or send the audio adjustment instruction to the audio module.

[0015] Preferably, the audio module comprises: an audio playing device and an audio control assembly; wherein the audio playing device is a sound, a microphone or a headset, which is used for playing the user guiding voice of the digital person and the treatment music; the audio control assembly is used for receiving the audio adjustment instruction sent by the digital controller, so as to adjust the content of the treatment music.

[0016] Preferably, the acquisition module further comprises: a management platform, which is arranged on one side of the user's body, and comprises an upper layer and a lower layer, wherein the upper layer is arranged at the same height as the user's arm; the galvanic skin response analysis device comprises a galvanic skin sensor and a galvanic skin analyzer, wherein the galvanic skin analyzer is arranged on the upper layer and is connected with the galvanic skin sensor by wire; the lower layer is arranged with the processing module, and the lower layer is optionally arranged with a storage part, when the first display device is a VR helmet, an AR helmet or an MR helmet, the first display device is placed in the storage part.

[0017] Preferably, the cognitive rehabilitation system based on digital twinning further comprises a medical monitoring module, the medical monitoring module comprises: a second display device and a communication assembly, the communication assembly is connected with the processing module, the processing module converts the user physiological index parameters collected by the acquisition module in real time into a digital user physiological index signal set and transmits to the communication assembly; the communication assembly sends the digital user physiological index signal set to the second display device and displays to the medical staff.

[0018] The application uses AI technology to analyze the pictures of the user's communication objects to obtain a virtual 3D digital human image, and uses text-to-speech and voice cloning technology to form voice interaction with the user to achieve communication simulation with the digital human in VR. Through natural language processing and emotion recognition technology, the digital human can understand and respond to the emotional state in the user's voice information, so as to realize more natural and temperature emotional interaction in the interaction process, and enhance the immersion and satisfaction of the user in the treatment process.

[0019] The system proposed in the application combines digital humans in VR scenarios, so as to facilitate the implementation of cognitive behavioral therapy through digital humans, help patients identify and change bad thinking patterns and behavior habits, and thus improve their gradually declining cognitive function; by guiding patients to identify and cope with negative emotions and thoughts, the symptoms of cognitive impairment and emotional disorders caused thereby can be effectively alleviated; by helping patients establish positive life attitudes and coping strategies, the social function and mental health level of the patients can be significantly improved; by helping patients identify and change bad thinking and behavior patterns, the patients can be guided to reshape healthy and positive personality.

[0020] In the system proposed in the application, the patient can freely browse multiple scenes by wearing a VR display device, and actively or passively experience selective scene experience to naturally immerse in it. The scene constructed according to the patient's personal album can provide a more personalized and customizable rehabilitation program for the patient, as an external stimulus to promote the regeneration of neurons and the formation of synapses, and thus possibly promote the formation of neural plasticity by inducing neural regeneration and remodeling; reviewing the scene in the virtual environment avoids the risks that may occur in actual operation, such as emotional fluctuations or physical fatigue caused by actual visits. At the same time, the music therapy provided by the system is integrated with the scene, and music that can cause resonance and emotional resonance can be selected according to the individual characteristics of the patient and according to the professional knowledge and experience of the system, the characteristics and effects of the music are objectively evaluated, and the patient is more likely to have an emotional connection with the music. According to the heart rate, blood pressure, and skin electricity and other indicators reflecting the emotional state of the individual transmitted in real time by various sensor devices coupled with the VR device data, the type and soothing rhythm of the music can be adjusted in time according to the specific emotional state of the user to reduce the excitation of the sympathetic nerves of the user, so as to achieve the best treatment effect.

[0021] Finally, the system proposed in the application can better control the safety of the treatment process, quantify the outcome indicators and evaluate the improvement degree of the patient's condition according to the collected blood pressure, skin electricity, heart rate, HAMA and HAMD scores and other evaluation indicators, and obtain a standardized rehabilitation treatment evaluation tool. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1Fig. 1 shows a user use schematic diagram of a cognitive rehabilitation system based on digital twin interaction and music therapy according to an embodiment of the present application;

[0023] Figure 2 Fig. 1 shows a user use schematic diagram of a cognitive rehabilitation system based on digital twin interaction and music therapy according to an embodiment of the present application;

[0024] Figure 3 Fig. 1 shows a user use schematic diagram of a cognitive rehabilitation system based on digital twin interaction and music therapy according to an embodiment of the present application; DETAILED DESCRIPTION

[0025] The present application will be further described below with reference to the embodiments. It should be understood that the embodiments are only used to further illustrate and explain the present application, and are not used to limit the present application.

[0026] Unless otherwise defined, technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of the present application, methods and materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and not intended to be limiting. The present application will be further described below with reference to the embodiments. It should be understood that the embodiments are only used to further illustrate and explain the present application, and are not used to limit the present application.

[0027] The present application proposes a cognitive rehabilitation system based on digital twin interaction and music therapy, which can include:

[0028] A positioning module is arranged in the user's activity range to locate the user's position in real time. The acquisition module includes a pulse oxygen device, a heart detection device, and a skin electricity reaction analysis device, as shown in Figure 1 which are arranged at multiple body parts of the user to collect physiological index parameters of the user in real time. A display module is used to display a digital person, a user virtual digital avatar, and a virtual scene to the user. An audio module is used to play output speech of the digital person and therapeutic music to the user. A processing module is connected with the positioning module, the acquisition module, the display module, and the audio module. The processing module adjusts the content of the output speech of the digital person, the virtual scene, and / or the therapeutic music through the physiological index parameters of the user collected by the acquisition module in real time.

[0029] In the present application, the pulse oxygen device is used to collect the user's blood oxygen saturation in real time, which can be exemplified as a finger clip pulse oximeter, etc.; the heart detection device is used to collect the user's heart rate and blood pressure in real time, which can be exemplified as a smart monitoring bracelet, a smart watch, a wearable electrocardiogram device, etc.; the skin electricity reaction analysis device is used to collect the user's skin electricity signal in real time, which can be exemplified as a skin electricity collector, a skin electricity recording analysis system, etc. By analyzing physiological index parameters such as blood oxygen saturation, heart rate, blood pressure and skin electricity, the user's real-time emotional state can be effectively grasped, and data support can be provided for the processing module to adjust the treatment scene or music.

[0030] In the present application, the user position includes the spatial position of the user's head, torso, and both hands; the positioning module sends the user position to the processing module, and the processing module generates a user virtual digital avatar according to the user position and transmits it to the display module. The user virtual digital avatar can be a self-defined digital virtual person image, which can include one or more of the head, torso, and limbs. The processing module can customize rendering according to user needs and synchronize the digital virtual person image with user behavior to improve the user's immersion in the digital treatment scene.

[0031] In the present application, the user physiological index parameters include blood oxygen saturation, heart rate, blood pressure, and skin electricity; as shown in Figure 3 The acquisition module sends the physiological index parameters of the user when performing multiple tasks to the processing module, and the processing module can calculate the user emotional state parameters based on the user physiological index parameters and evaluate and classify the user emotional state. The processing module sends control instructions to the display module and the audio module based on the classification result of the user emotional state, respectively, to adjust the virtual scene and the treatment music to match the virtual scene and the treatment music corresponding to the classification result of the user emotional state.

[0032] In one embodiment, the classification result of the user emotional state can be happy, calm, nervous, or depressed, etc. The processing module has a storage component that stores a plurality of virtual scene data sets and treatment music data sets corresponding to a plurality of emotional state types. The processing module sends a call control instruction to the display module and the audio module according to the classification result, and the display module and the audio module call the virtual scene and the treatment music in the corresponding virtual scene data set and the treatment music data set from the storage component after receiving the control instruction, to replace the initial virtual scene and the treatment music. The present application mobilizes the user's positive emotions through vision and hearing. Through long-term use of the system of the present application, the user can significantly improve his emotional state affected by cognitive impairment.

[0033] In an embodiment, the degree of emotional disorder of the user can also be evaluated by HAMA score and HAMD score. Specifically, the digital person conducts a joint HAMA examination and a joint HAMD examination on the user by voice questioning, and a professional evaluator determines a first HAMA score and a first HAMD score according to the answers of the user; the user uses the cognitive rehabilitation system described in the present application for training, and after a certain training period (for example, once a day for two weeks), the second HAMA score and the second HAMD score are determined again by the same way, and the improvement degree of the training on the emotional disorder of the user is determined by the difference between the two scores.

[0034] In the present application, the acquisition module further comprises an image acquisition component, the image acquisition component comprising a photographic device and a first data interface, respectively connected with the processing module, acquiring the face and / or body image by the photographic device and / or obtaining the face and / or body image from the data storage device by the first data interface, and sending to the processing module; the processing module generates a digital person according to the face and / or body image and transmits to the display module. The digital person provides CBT treatment by voice interaction with the user, thereby correcting the user's bad thinking and helping to improve the user's memory and attention disorder. For example, a digital person of a doctor image can be generated by using the face and body image of a doctor, and the user can be trained for memory by voice interaction with the digital person, and the user can be actively guided to complete the training task; or a digital person of a user relative image is generated, and the digital person is encouraged by voice to relieve negative emotions. In this way, the user saves the medical cost invested in training while conducting rehabilitation training through high-immersion CBT; in addition, the rehabilitation training is also convenient for home performance.

[0035] In the present application, the acquisition module further comprises an audio acquisition component, the audio acquisition component comprising a microphone device and a second data interface, respectively connected with the processing module. The user voice and / or other individual voice and / or treatment text can be acquired by the microphone device and / or obtained from the data storage device by the second data interface, and sent to the processing module. The processing module can generate the output voice of the digital person according to the other individual voice and / or treatment text; the processing module can also adjust the output voice of the digital person according to the user voice.

[0036] Specifically, the audio acquisition component acquires treatment text for helping the user to identify and change bad thinking mode and behavior habit to generate a first text set; the acquisition module further converts the treatment text in the first text set into user guide voice by using text-to-speech technology to generate a first voice set, and the digital person calls the user guide voice in the first voice set through interaction with the virtual digital avatar of the user, and plays the user guide voice to the user through the audio module.

[0037] In one embodiment, the audio collection component of the collection module can also be used to receive sound samples to generate a second speech set, train and learn the timbre, intonation, speech rate, pause, accent features of the sound samples in the second speech set, convert the therapy text in the first text set into user guidance speech with the timbre, intonation, speech rate, pause, accent features of the sound samples in the second speech set, to generate the first speech set with more realistic user guidance speech.

[0038] In one embodiment, the audio collection component of the collection module can also be used to receive sound samples to generate a third speech set, train and learn the emotional features of the sound samples in the third speech set, further adjust the user guidance speech to also have the emotional features of the sound samples in the third speech set, to generate the first speech set with more emotionally rich user guidance speech.

[0039] In particular, the processing module of the cognitive rehabilitation system of the present application can also send control instructions to the audio module based on the classification result of the user's emotional state to adjust the user guidance speech output by the digital person. Specifically, the user guidance speech in the first speech set includes multiple emotional states, and the processing module controls the digital person to output the user guidance speech corresponding to the emotional state based on the classification result of the user's emotional state, to provide emotional value and emotional relief to the user through the digital person. In one embodiment, the processing module identifies that the user's emotional state is depressed, and accordingly controls the digital person to output continuous and positive guidance speech to help the user overcome negative emotions.

[0040] In the present application, the display module comprises: a first display device and a digital controller; wherein the first display device is a VR glasses, a VR helmet, an AR helmet, an MR helmet or a naked-eye 3d imaging device, used to display a digital person, a user virtual digital avatar and a virtual scene; the digital controller is connected with the positioning module and the audio module respectively, to obtain the positions of the user's two hands positioned by the positioning module and to identify the user's gestures, to adjust the content of the virtual scene and / or to send audio adjustment instructions to the audio module.

[0041] Specifically, the digital controller is configured with a digital user interface in the display module, which displays all preset information of the digital therapy scene, including thumbnail display icons of the virtual scene, therapy music, etc., and the user can interact with the digital user interface through the user virtual digital avatar in the first display device to manually adjust the content of the virtual scene and / or therapy music of the digital therapy scene. The virtual scene is used to help the user maintain a positive emotion and reduce negative emotions, which can be exemplified by a virtual photo album scene, which includes 3D modeling or mapping of image content and image associated scenery in the user's photo album.

[0042] In the present application, the audio module includes: an audio playback device and an audio control component; wherein the audio playback device is a sound, a microphone, a headset, and is used to play the user guidance voice and the therapeutic music of the digital person. The audio control component is used to receive the audio adjustment instruction sent by the digital controller to adjust the content of the therapeutic music, and the audio control component is connected with the processing module in a wired or wireless connection mode.

[0043] In this way, by combining the digital person CBT treatment and the virtual scene MT treatment, as shown in the cognitive rehabilitation system of the present application, the cognitive impairment treatment method and the emotional disorder treatment method can be organically combined in the virtual reality treatment environment, helping the user to correct the wrong cognitive behavior in a positive emotional state, and achieving a comprehensive rehabilitation effect. Figure 2

[0044] In the present application, the medical monitoring module includes: a second display device and a communication component, and the communication component is connected with the processing module. The processing module converts the user physiological index parameters collected by the collection module in real time into a digital user physiological index signal set, and transmits it to the communication component. The communication component sends the digital user physiological index signal set to the second display device and displays it to the medical personnel.

[0045] Specifically, the second display device can be: a flat panel display, a VR eye, a VR helmet, an AR helmet, an MR helmet, or a naked-eye 3D imaging device, etc. The digital user physiological index signal set can include: a user real-time blood oxygen change fold line graph, a user real-time heart rate change fold line graph, a user real-time blood pressure change fold line graph, a user skin electricity graph, a user emotional state change graph, and variable record table data of the above visualization graphs, etc. The medical personnel can manually adjust the scene and music content in the digital treatment scene according to the digital user physiological index signal set, and adjust the rehabilitation treatment time, course, etc. of the user basic application system, or provide more professional opinions to improve the treatment content.

[0046] In the present application, the collection module can also include: a management platform, which is arranged on one side of the user's body and includes an upper layer and a lower layer, wherein the upper layer is arranged at the same height as the user's arm. The skin electricity response analysis device includes a skin electricity sensor and a skin electricity analyzer, wherein the skin electricity analyzer is configured to be fixed to the upper layer and is connected with the skin electricity sensor in a wired manner. In this way, when the skin electricity sensor is attached to the user's arm, the skin electricity analyzer is at the same height as the arm, which can effectively reduce the introduction of noise and the attenuation of the skin electricity signal.

[0047] ​The lower layer of the management platform is configured with a processing module, specifically, the lower layer is used for fixing the processing module and is provided with a communication channel, the communication channel can accommodate the connection data line of the processing module and the positioning module, the acquisition module, the display module and the audio module, and avoids entanglement of multiple lines. The lower layer is optionally provided with a storage part, when the first display device is a VR helmet, an AR helmet or an MR helmet, additional space is needed for placement, therefore the first display device can be placed in the storage part and taken out by the user when in use.

[0048] In the present application, the processing module can be exemplified as a personal computer, a server, a workstation, etc.

[0049] The cognitive rehabilitation system based on digital twinning of the present application can accelerate the recovery of cognitive function by forming a double stimulation effect from vision and hearing through the combination of digital human CBT treatment, MT treatment and virtual scenes; the virtual scene can effectively arouse the memory of the patient and promote memory reconstruction and consolidation by being constructed based on the individual characteristics of the user and being matched with the familiar voice or music of the user; the digitization of rehabilitation training can make the user more immersed, focused and concentrated in the treatment process, reduce distraction and resistance psychology, enhance the rehabilitation effect, and at the same time reduce the consumption of medical hospitals; according to the multi-aspect evaluation indexes of blood pressure, skin electricity, heart rate, HAMA and HAMD scores, the safety of the treatment process can be better controlled, the outcome indicators can be quantified, and the improvement degree of the patient's condition can be evaluated, to help the user control the progression of the disease.

Claims

1. A cognitive rehabilitation system based on digital twin interaction and music therapy, characterized in that, The application relates to a digital human system, which comprises the following modules: a positioning module arranged in a user activity range and used for positioning a user position in real time; a collection module, which comprises pulse oxygen equipment, heart detection equipment and skin electricity reaction analysis equipment arranged at multiple body parts of the user and used for collecting physiological index parameters of the user in real time; the collection module further comprises an image collection assembly, which comprises a photography equipment connected with a processing module, is used for collecting human face and / or human body images through the photography equipment and sends the images to the processing module; the collection module further comprises an audio collection assembly, which comprises a microphone equipment connected with the processing module, is used for collecting user voice and / or other individual voice through the microphone equipment and sends the voice to the processing module; the collection module further comprises a management platform arranged on one side of the user body, which comprises an upper layer and a lower layer, wherein the upper layer is arranged at the same height as the user's arm, the skin electricity reaction analysis equipment comprises a skin electricity sensor and a skin electricity analyzer, and the skin electricity analyzer is arranged in the upper layer and is connected with the skin electricity sensor through a wire; a display module used for displaying a digital human, a user virtual digital avatar and a virtual scene to the user; the display module comprises a first display equipment, which is a VR glasses, a VR helmet, an AR helmet, an MR helmet or naked-eye 3D imaging equipment and is used for displaying the digital human, the user virtual digital avatar and the virtual scene; the lower layer is provided with a storage part, and when the first display equipment is the VR helmet, the AR helmet or the MR helmet, the first display equipment is placed in the storage part; an audio module used for playing output voice of the digital human and treatment music to the user; the audio module comprises an audio playing equipment, which is a sound box, a microphone or earphones and is used for playing user guide voice of the digital human and the treatment music; a processing module connected with the positioning module, the collection module, the display module and the audio module; the processing module is a personal computer, a server or a workstation and has a storage assembly, and the storage assembly stores a plurality of virtual scene data sets and treatment music data sets; wherein the processing module adjusts content of the output voice of the digital human, the virtual scene and / or the treatment music according to the physiological index parameters of the user collected by the collection module in real time.

2. The system of claim 1, wherein, Wherein, the user position comprises spatial positions of a user head, a trunk and two hands; the positioning module sends the user position to the processing module, and the processing module generates the user virtual digital avatar according to the user position and transmits the user virtual digital avatar to the display module.

3. The system of claim 1, wherein, Wherein, the physiological index parameters of the user comprise blood oxygen saturation, heart rate, blood pressure and skin electricity; the collection module sends the physiological index parameters of the user to the processing module, and the processing module calculates user emotional state parameters based on the physiological index parameters of the user and classifies user emotional states. The processing module sends control instructions to the display module and the audio module based on the user emotional state classification result, so as to respectively adjust the virtual scene and the therapeutic music to the virtual scene and the therapeutic music matched with the user emotional state classification result.

4. The system of claim 1, wherein, Wherein, The image acquisition component includes a first data interface connected with the processing module, which acquires the face and / or body image from the data storage device and sends it to the processing module by the first data interface; The processing module generates the digital person according to the face and / or body image and transmits it to the display module.

5. The system of claim 1, wherein, Wherein, The audio acquisition component includes a second data interface connected with the processing module, which acquires other individual voice and / or therapeutic text from the data storage device and sends it to the processing module by the second data interface; The processing module generates the output voice of the digital person according to the other individual voice and / or therapeutic text; and The processing module adjusts the output voice of the digital person according to the user voice.

6. The system of claim 1, wherein, Wherein, The display module includes a digital controller; wherein the digital controller is connected with the positioning module and the audio module respectively, acquires the user's hand position positioned by the positioning module and identifies the user's gesture, so as to adjust the content of the virtual scene and / or send the audio adjustment instruction to the audio module.

7. The system of claim 6, wherein, Wherein, The audio module includes an audio control component; wherein the audio control component is used to receive the audio adjustment instruction sent by the digital controller, so as to adjust the content of the therapeutic music.

8. The system of claim 1, wherein, Wherein, The lower layer is configured with the processing module.

9. The system of any one of claims 1-8, wherein, Also includes: A medical monitoring module, the medical monitoring module includes a second display device and a communication component, the communication component is connected with the processing module, the processing module converts the user physiological index parameter collected by the acquisition module in real time into a digital user physiological index signal set, and transmits it to the communication component; The communication component sends the digital user physiological index signal set to the second display device and displays it to the medical staff.

Images