Psychotherapy system and method based on virtual reality

Abstract

The invention discloses a psychotherapy system and method based on virtual reality, the psychotherapy system comprises a VR platform, a monitoring module and a control module, the VR platform comprises an interaction module and an optimization module; the VR platform is connected with the monitoring module and the control module, and the monitoring module is connected with the control module; the monitoring module comprises at least one heart rate detection unit, at least one skin electricity detection unit, at least one face recognition unit and at least one pickup unit. According to the invention, outside the preset scene, multi-modal signal collection and emotional model establishment are carried out through the monitoring module, and psychotherapy or intervention is carried out according to the prediction result of the emotional model, so that the limitation of psychotherapy or intervention in the preset scene is avoided, and the flexibility and individuation degree are improved; in the process of establishing the preset scene and the emotion model and judging the emotion, the monitoring module continuously collects the multi-modal information of the user in real time, and the treatment or intervention means can be adjusted in time.

Description

Technical Field

[0001] The present invention relates to a VR-based psychotherapy device, and in particular to a virtual reality-based psychotherapy system and method, belonging to the technical field of VR psychotherapy. Background Art

[0002] Virtual reality (VR) technology can create an immersive environment that makes users feel like they are in a completely different space. In the field of psychotherapy, VR is used to simulate situations that may trigger anxiety or fear in patients in real life, such as heights, public speaking, or specific social environments. In this way, VR helps patients face and overcome their fears under controlled and safe conditions. Virtual reality is mainly used in psychotherapy for exposure therapy, which has been shown to be effective in treating various phobias and anxiety disorders. In addition, AI technology has been explored in providing guidance and feedback in cognitive behavioral therapy, such as through chatbots to assist treatment.

[0003] Although virtual reality and AI technologies have made progress in the field of psychotherapy, there are still some obvious shortcomings, such as the lack of real-time personalization: existing VR psychotherapy methods are usually preset and non-personalized, generally with multiple typical scenarios preset for psychotherapy or intervention. This means that the treatment or intervention plan cannot be adjusted according to the patient's immediate response and specific needs. Therefore, the effect of the treatment may be limited by generalized plans, resulting in a lack of flexibility and personalization. Another example is limited interactivity: current virtual reality psychotherapy or intervention lacks deep emotional understanding and complex conversation capabilities, and cannot conduct multi-modal intervention or treatment for detected psychological states or information. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a psychological treatment system and method based on virtual reality, which solves the problems such as lack of personalization of the existing virtual reality treatment system.

[0005] The technical problem to be solved by the present invention is achieved by adopting the following technical solutions:

[0006] The present invention provides a virtual reality-based psychotherapy system, comprising a VR platform, a monitoring module and a control module, wherein:

[0007] The VR platform includes an interaction module and an optimization module;

[0008] The VR platform is connected to the monitoring module and the control module, wherein:

[0009] The monitoring module is connected to the control module.

[0010] As a preferred technical solution of the present invention, the monitoring module includes at least one heart rate detection unit, at least one skin conduction detection unit, at least one facial recognition unit and at least one sound pickup unit.

[0011] As a preferred technical solution of the present invention, the interaction module includes a display unit and a speaker unit that are independently arranged or integrated into the VR platform.

[0012] As a preferred technical solution of the present invention, the optimization module includes a computing unit for performing terminal-side computing processing.

[0013] The present invention also provides a virtual reality-based psychotherapy method, which is applied to the aforementioned virtual reality-based psychotherapy system, and includes the following steps:

[0014] Step S1, after the user enters the VR platform, the interaction module prompts the user to select a preset scene;

[0015] Step S2, executing the preset scene selected by the user in step S1;

[0016] Step S3, performing feature extraction during the execution of step S2;

[0017] Step S4, constructing a sentiment model based on the features in step S3;

[0018] Step S5, the monitoring module collects data and sends it to the optimization module;

[0019] Step S6, matching the data in step S5 with the emotion model in step S4 to perform emotion prediction;

[0020] Step S7, the interactive module performs treatment or intervention according to the prediction result in step S6;

[0021] Step S8, the feature extraction in step S3 is cyclically executed together with steps S2 to S7.

[0022] As a preferred technical solution of the present invention, the feature extraction in step S3 is performed by the monitoring module, wherein the extracted features at least include facial expression features, voice features and heart rate features.

[0023] As a preferred technical solution of the present invention, in step S4, the way of constructing the emotion model includes at least one of vector machine (SVM), random forest, and deep neural network.

[0024] As a preferred technical solution of the present invention, after step S7, the monitoring module continues to perform feature extraction until the user finishes using the VR platform.

[0025] As a preferred technical solution of the present invention, the treatment or intervention performed by the interactive module in step S7 includes adjusting at least one of lighting, music, and color, and / or conducting a conversation.

[0026] Preferably, the control module is used to control the VR platform and the monitoring module to transmit data to the optimization module.

[0027] The beneficial effects of the present invention are: in addition to the preset scenarios, the present invention collects multimodal signals through the monitoring module, establishes an emotional model, and performs psychotherapy or intervention based on the prediction results of the emotional model, thereby avoiding the limitations of performing psychotherapy or intervention in preset scenarios and improving flexibility and personalization; in the process of establishing preset scenarios and emotional models and judging emotions, the monitoring module continuously collects the user's multimodal information in real time and can adjust the treatment or intervention methods in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a structural schematic diagram of the present invention;

[0029] In the figure: 1. VR platform; 2. Interaction module; 21. Optimization module; 3. Monitoring module; 4. Control module. DETAILED DESCRIPTION

[0030] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0031] Example 1

[0032] like Figure 1 As shown, this embodiment provides a virtual reality-based psychotherapy system, including a VR platform 1, a monitoring module 3 and a control module 4, wherein the VR platform 1 includes an interaction module 2 and an optimization module 21, and the interaction module 2 includes a display unit and a speaker unit independently set or integrated in the VR platform 1, so as to realize psychotherapy or intervention after judging the emotional state, such as adjusting at least one of lighting, music and color. The optimization module 21 includes a computing unit, which performs terminal-side computing and processing, and can perform emotional monitoring and psychotherapy or intervention more flexibly. The optimization module 21 includes the functions of receiving data from the monitoring module 3, establishing an emotional model, performing emotional judgment, and issuing intervention instructions to the interaction module 2;

[0033] The VR platform 1 is connected to the monitoring module 3 and the control module 4 , wherein the monitoring module 3 is connected to the control module 4 , and the control module 4 is used to control the VR platform 1 and the monitoring module 3 to transmit data to the optimization module 21 .

[0034] The monitoring module 3 includes at least one heart rate detection unit, at least one skin conduction detection unit, at least one facial recognition unit and at least one sound pickup unit, and sends the monitored signal to the optimization module 21. The optimization module 21 establishes an emotion model and performs emotion judgment based on the monitoring signal data, and issues intervention instructions to the interaction module 2.

[0035] The specific process is as in Example 2.

[0036] Example 2

[0037] This embodiment provides a virtual reality-based psychotherapy method, which is used in a virtual reality-based psychotherapy system in Embodiment 1, and includes the following steps:

[0038] Step S1, after the user enters the VR platform 1, the interactive module 2 prompts the user to select a preset scene, or before starting treatment, the psychotherapist will conduct a preliminary assessment of the patient to understand his or her specific anxiety trigger points and symptom severity, and select a specific preset scene for the user (such as a simulated social gathering or conference room);

[0039] Step S2, executing the preset scene selected by the user in step S1, such as creating a simulated public speech scene, which includes virtual audiences. Specific elements in the scene, such as the number of audiences, reactions (positive or negative), etc., can also be adjusted to match the needs of the initial treatment. In this process, the preset scene can be set to fluctuate within a certain range (such as the number of people, lighting, scene, etc.) to obtain user feedback on element changes more quickly, so that the monitoring module 3 can collect personalized data for the user; it can also be applied to flight training or PTSD intervention or treatment;

[0040] Step S3, performing feature extraction during the execution of step S2;

[0041] In this embodiment, the feature extraction in step S3 is performed by the monitoring module 3, wherein the extracted features include at least facial expression features, speech features and heart rate features. Specifically, facial recognition software, such as OpenCV or Dlib, can be used to detect facial feature points from video frames, extract expression features such as eyebrow height, eye opening degree, etc., and perform facial expression recognition; voice processing tools, such as Python's librosa library, can be used to extract Mel frequency spectrum coefficients (MFCCs), pitch, volume and other acoustic features of speech to perform sound feature recognition, and features can be extracted from physiological signals, such as heart rate, heart rate variability (HRV), peak value of skin electrical response, etc.

[0042] Step S4, constructing a sentiment model based on the features in step S3;

[0043] In this embodiment, in step S4, the emotional model is constructed by at least one of a vector machine (SVM), a random forest (SVM), and a deep neural network (such as CNN, LSTM). Optimization can also be performed, typically using a labeled emotional data set for training, the data set contains predefined emotional categories (such as happiness, sadness, anger, etc.), and the optimization can use cross-validation and hyperparameter tuning techniques (such as grid search) to optimize model parameters to ensure that the model has the best performance.

[0044] Step S5, the monitoring module 3 collects data and sends it to the optimization module 21;

[0045] Step S6, matching the data in step S5 with the emotion model in step S4 to perform emotion prediction, the model integrates facial expressions, voice and physiological data, and outputs the most likely current emotional state;

[0046] Steps S4 to S6 are as follows: a typical calculation process, assuming that a simple fusion model is used to merge the feature vectors of facial, voice and physiological data into a large feature vector, which is input into the model for emotional state classification, such as using the Softmax function to assign probabilities between multiple emotional states (happy, sad, angry, etc.). The emotional state with the highest probability is output as the current emotional prediction.

[0047] Step S7, based on the prediction result in step S6, the interactive module 2 performs treatment or intervention. In this embodiment, the treatment or intervention performed by the interactive module 2 in step S7 includes adjusting at least one of lighting, music, and color, and / or conducting a conversation to match or adjust the patient's emotions, such as providing a comforting conversation when sadness is detected; or, if the model identifies the patient's emotion as "anxiety", the system may lower the ambient volume, play soothing music, soften the color tone of the virtual environment, and make suggestions for relaxing breathing, or guide the patient to perform simple relaxation exercises;

[0048] Step S8, the feature extraction in step S3 is executed in a loop together with steps S2 to S7. Preferably, after step S7, the monitoring module 3 continues to perform feature extraction until the user finishes using the VR platform, thereby realizing multimodal signal monitoring, model building, model optimization, and psychotherapy or intervention based on personalized emotional models during the entire process of the user using virtual reality psychotherapy.

[0049] Regarding model building, it is preferably carried out in interactive module 4. The following is a typical example, specifically, an example of a specific calculation process for a moderately sad patient, showing the entire process of data collection, feature extraction, emotion model construction, emotion prediction and intervention:

[0050] 1. Data Collection and Feature Extraction

[0051] 1. Facial expression features

[0052] Tools: OpenCV and Dlib

[0053] Detect facial landmarks:

[0054] Eyebrow height: BrowHeight

[0055] Eye opening degree: EyeOpen

[0056] Example data:

[0057] BrowHeight=30

[0058] EyeOpen=40

[0059] Calculation formula:

[0060]

[0061]

[0062] 2. Voice Features

[0063] Tool: librosa

[0064] Extract features:

[0065] Mel Spectral Coefficients (MFCCs): MFCCs

[0066] Pitch

[0067] Volume: Volume

[0068] Example data:

[0069] MFCCsMean=0.25

[0070] Pitch=120HzVolume=0.3

[0071] Calculation formula:

[0072]

[0073]

[0074]

[0075] 3. Heart rate characteristics

[0076] Tools: Heart rate monitoring device

[0077] Extract features:

[0078] Heart rate: HeartRate

[0079] Heart Rate Variability (HRV): HRV

[0080] Example data:

[0081] HeartRate=70bpm

[0082] HRV=0.08

[0083] Calculation formula:

[0084]

[0085]

[0086] 2. Feature Fusion

[0087] Combine the facial, voice, and heart rate feature vectors into one large feature vector:

[0088] FeatureVector=[0.1667,0.4286,0.25,0.28,0.3,0.25,0.12]

[0089] 3. Emotion Model Construction and Training

[0090] 1. Model selection

[0091] Use Long Short-Term Memory (LSTM) network for sentiment classification,

[0092] Network structure: input layer, LSTM layer, fully connected layer, output layer.

[0093] 2. Model training

[0094] Dataset: A multimodal dataset with predefined emotion labels such as happy, sad, anxious, etc. Optimization method: The Adam optimizer is used for training and the loss function is the cross entropy loss.

[0095] 3. Model Optimization

[0096] Hyperparameter tuning: Optimize the model’s hyperparameters, such as learning rate, number of LSTM units, etc., through grid search.

[0097] 4. Training Examples

[0098] Assume that the training set has 10,000 samples, and after multiple rounds of training, the accuracy of the model reaches 95%.

[0099] 4. Sentiment Classification and Prediction

[0100] 1. Feature vector input model

[0101] Input the fused feature vector [0.1667, 0.4286, 0.25, 0.28, 0.3, 0.25, 0.12] to the sentiment model.

[0102] 2. Model outputs emotional state

[0103] Use the Softmax function to assign probabilities between multiple categories of emotional states.

[0104] formula:

[0105]

[0106] Among them, z i is the raw score of the i-th emotional state, K is the total number of emotional states, and the example output (assuming the raw score z i ):

[0107] Happy: z 快乐 =0.2

[0108] Sadness: z 悲伤 =1.0

[0109] Anxiety: z 焦虑 =0.5

[0110] Angry: z 愤怒 =0.1

[0111] Calculate Softmax probability:

[0112]

[0113] Calculate the index of each emotional state:

[0114] e 0.2 ≈1.2214,e 1.0 ≈2.7183,e 0.5 ≈1.6487,e 0.1 ≈1.1052

[0115] Substitute the above values ​​into the formula for calculation:

[0116]

[0117]

[0118] The following is the probability calculation for other emotions:

[0119]

[0120]

[0121]

[0122] 3. Sentiment prediction

[0123] The model outputs the highest probability emotional state: sad (0.406);

[0124] V. Dynamic Adjustment and Intervention

[0125] 1. Sentiment prediction results

[0126] Emotional state: sad.

[0127] 2. Therapeutic interventions

[0128] Preset dialogue: Based on the sadness state, a soothing dialogue is played: "It's normal for you to feel sad now. We can get through this together."

[0129] Music adjustment: Play warm, soft music, such as string music.

[0130] Lighting adjustment: Use soft light to create a warm atmosphere.

[0131] 6. Feedback and Optimization

[0132] 1. Continuous monitoring

[0133] The monitoring module continuously collects the user's multimodal information (facial expressions, voice, heart rate, etc.) and dynamically adjusts the treatment plan until the treatment is completed.

[0134] 2. User Feedback

[0135] Collect users' emotional feedback and physiological data to optimize emotional models and treatment plans.

[0136] The following is the specific implementation process:

[0137] Regarding 1. Continuous monitoring, the monitoring module continuously collects multimodal information of users, including physiological and emotional data such as facial expressions, voice, heart rate, etc. The following is the specific data processing and feature extraction formula:

[0138] Facial Expression Recognition:

[0139]

[0140]

[0141] Speech feature extraction:

[0142]

[0143]

[0144] Heart rate and galvanic skin response:

[0145]

[0146]

[0147] Regarding 2. User feedback, we collect users’ emotional feedback and physiological data, and optimize the emotional model and treatment plan to ensure that the system can make the best adjustment according to the user’s immediate state. The following are the specific implementation methods and algorithms:

[0148] Regarding the construction and optimization of the sentiment model, we use multimodal data to build and optimize the sentiment model, and use more complex high-level formulas for feature fusion and sentiment prediction. Follow the steps below:

[0149] 1. Feature vector fusion:

[0150] F=[f1,f2,…,f n ]

[0151] Among them, f i is the i-th feature, such as facial expression, voice and heart rate features.

[0152] 2. Sentiment prediction model:

[0153] Use deep neural networks (such as LSTM or CNN) for sentiment classification, and the model output is the probability distribution of different emotional states:

[0154]

[0155] Among them, z i is the original score of the i-th emotional state, and K is the total number of emotional states.

[0156] 3. Emotional state prediction:

[0157] Input the feature vector into the LSTM model, calculate the Softmax probability, and output the most likely emotional state:

[0158]

[0159] About dynamic adjustment algorithm, improved dynamic adjustment algorithm, combining adaptive control theory and deep reinforcement

[0160] Learning technology is optimized and adjusted based on real-time data, as follows:

[0161] 1. Dynamically adjust the control equation:

[0162] Use nonlinear adaptive control equations to adjust based on emotion model predictions and real-time physiological data:

[0164]

[0165] Among them, u(t) is the adjustment variable, e(t) is the error, K p , K i and K d is the control gain coefficient, α i is the adjustment parameter, φ i is the basis function,

[0166] 2. Optimize and adjust strategies:

[0167] Using deep reinforcement learning algorithms (such as Deep Q-Learning or Proximal Policy

[0168] Optimization) optimizes treatment plans through continuous experimentation and feedback:

[0169]

[0170] Among them, Q(s,a) is the state action value function, α is the learning rate, γ is the discount factor, r is the immediate reward, s and a are the current

[0171] s ′ For the next state.

[0172] Regarding the improved Pareto optimal solution algorithm, the improved Pareto optimal solution algorithm can better

[0173] Optimize multiple treatment goals such as emotional stability, treatment effectiveness, and user comfort.

[0174] 1. Multi-objective optimization:

[0175] Using a multi-objective optimization algorithm (such as NSGA-II), the Pareto optimal solution set is searched in the decision space.

[0176] minF(x)=[f1(x),f2(x),…,f m (x)]

[0177] Among them, F(x) is the objective function vector and x is the decision variable vector.

[0178] 2. Weight distribution and fitness function:

[0179] By introducing the weight coefficient, the multi-objective optimization is transformed into a single-objective optimization problem:

[0180]

[0181] Among them, w i (t) is the weight coefficient of the i-th target at time t, satisfying since

[0182] The adaptation weight can be dynamically adjusted by the following formula: Among them, β is the adjustment coefficient, which controls the speed of change of the weight.

[0183] 3. Fitness function evaluation:

[0184] Evaluate the quality of each solution by calculating the fitness function value:

[0185]

[0186] The specific implementation steps are as follows:

[0187] 1. Data collection and feature extraction:

[0188] Use facial recognition software to detect facial feature points from video frames and extract facial expression features such as eyebrow height and eye opening degree;

[0189] The speech processing tool is used to extract the Mel Spectral Coefficients (MFCCs), pitch, volume and other acoustic features of the speech.

[0190] Extract features such as heart rate, heart rate variability (HRV), and skin galvanic response from physiological signals.

[0191] 2. Emotion model training:

[0192] Use multimodal datasets to train the LSTM model and optimize model parameters to ensure high-accuracy sentiment prediction.

[0193] Optimize model performance using cross-validation and hyperparameter tuning techniques such as grid search.

[0194] 3. Emotional prediction and intervention:

[0195] The fused feature vector is input into the emotion model, the Softmax probability distribution is calculated, and the current emotion state is predicted.

[0196] Based on the emotion prediction results, the system automatically adjusts the lighting, music and color, and provides comforting conversations or other intervention measures.

[0197] 4. Continuous monitoring and dynamic adjustment:

[0198] The monitoring module continuously collects the user's multimodal information and adjusts the treatment plan in real time until the treatment is completed.

[0199] Collect the user's emotional feedback and physiological data, optimize the emotional model and treatment plan, and ensure that the system always provides the best personalized treatment. The control module 4 is used to control the VR platform 1 and the monitoring module 3 to transmit data to the optimization module 21.

[0200] Regarding the comforting dialogue, this embodiment adopts a combination of preset sentences and dynamically generated dialogues. The following is a specific implementation method or process:

[0201] First, preset sentences are associated with the emotion model, and the preset sentences can be selected and played according to the judgment results of the emotion model, thereby realizing the association of the dialogue content with the user's current emotional state.

[0202] Typical treatment methods:

[0203] 1. Preset sentence library construction: Establish a broad preset sentence library that covers various emotional states (such as anxiety, depression, anger, calmness, etc.), that is, each emotional state has a corresponding sentence in the sentence library.

[0204] 2. Emotional state classification: Use the emotion model to classify the user's emotional state in real time.

[0205] 3. Sentence selection logic: According to the classification results of the emotional state, the most suitable comforting dialogue content (i.e. preset sentences) is selected from the preset sentence library.

[0206] 4. Dynamic adjustment: During the conversation, continuously monitor changes in the user's emotional state and dynamically adjust the content and tone of preset sentences.

[0207] The advantage of using preset statements is that they can achieve quick response. Preset statements can provide feedback quickly and reduce user waiting time. The technology is easy to implement and has low implementation difficulty, making it suitable for rapid deployment.

[0208] Correspondingly, in this embodiment, the comforting dialogue can also adopt a separately developed innovative dynamic generation dialogue, using natural language processing (NLP) and machine learning technology to generate personalized comforting dialogue according to the user's real-time emotional state and dialogue context.

[0209] Treatment method:

[0210] 1. Data collection and model training:

[0211] Collect a large amount of psychological consultation and conversation data for training NLP models.

[0212] Use sentiment analysis technology to label the emotional state in the data.

[0213] 2. Sentiment model optimization:

[0214] Multimodal data fusion (such as heart rate, skin charge, facial expression, voice, etc.) improves the accuracy of emotional state recognition.

[0215] Use deep learning models (such as LSTM, Transformer) for emotional state prediction.

[0216] 3. Dialogue Generation Model:

[0217] Use a pre-trained language model (such as GPT-4) to generate personalized dialogue content. Specifically, generate personalized dialogue content or dialogue sentences based on the structure predicted by the emotional state.

[0218] Generate comforting conversations in real time based on the user's current emotional state and conversation context.

[0219] 4. Feedback and optimization:

[0220] Collect user feedback and continuously optimize the model for generating conversations to ensure the effectiveness and applicability of the conversation content.

[0221] The advantages are high personalization, the generated dialogue can be adjusted according to the specific situation of the user, providing more tailored comfort; and strong flexibility. It can handle more diverse emotional states and dialogue scenarios to improve user experience.

[0222] Preferably, a comforting dialogue can be conducted in a hybrid mode, that is, a combination of preset sentences and dynamically generated dialogues, taking advantage of both to provide the best user experience:

[0223] Implementation steps:

[0224] 1. Build a basic preset sentence library:

[0225] According to common emotional states, prepare a set of basic comforting dialogue sentences, that is, match the preset sentences with the emotional states.

[0226] 2. Train the NLP generation model:

[0227] Using large-scale psychological consultation data to train generative models.

[0228] 3. Integration and deployment:

[0229] Integrate the preset sentences and NLP generation model into the system, that is, into the interaction module;

[0230] Based on the output of the sentiment model, preset sentences are prioritized to provide quick feedback, and the NLP generation model is called when necessary to provide personalized conversations.

[0231] 4. User feedback and optimization:

[0232] Collect user feedback during use and continuously optimize the effects of preset statements and generated models.

[0233] The following is a typical example:

[0234] 1. Basic comfort: When the emotional model detects that the user is slightly anxious, a preset comforting dialogue can be quickly played, such as "Please relax, take a deep breath, and we will overcome this difficulty together."

[0235] 2. Deep comfort: When the emotion model detects that the user is seriously anxious, the system can call the NLP model to generate a more personalized dialogue, such as "I know you are anxious now, which is normal. Let's talk about the specific situation together, and maybe we can find a solution."

[0236] In addition to the preset scenes, the present invention collects multimodal signals through the monitoring module, establishes an emotion model, and performs psychotherapy or intervention according to the predicted results of the emotion model, avoiding the limitations of psychotherapy or intervention in the preset scenes and improving flexibility and personalization. In the process of establishing the preset scenes and emotion models and judging emotions, the monitoring module continuously collects the multimodal information of the user in real time, and can adjust the treatment or intervention methods in a timely manner. The emotion analysis model is trained with a large amount of annotated data to ensure that different psychological states can be accurately understood. By monitoring the physiological and psychological reactions of patients in real time, the system can dynamically adjust the virtual environment and treatment strategies to respond to the immediate needs of patients in the best way. This adaptability not only makes the treatment process more personalized, but also improves the pertinence and efficiency of treatment.

[0237] The basic principles, main features and advantages of the present invention are shown and described above. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of the present invention is defined by the attached claims and their equivalents.

Claims

1. A virtual reality-based psychotherapy system, characterized in that: It comprises a VR platform (1), a monitoring module (3) and a control module (4), wherein: The VR platform (1) comprises an interaction module (2) and an optimization module (21); The VR platform (1) is connected to the monitoring module (3) and the control module (4), wherein: The monitoring module (3) is connected to the control module (4).

2. A virtual reality based psychotherapy system according to claim 1, characterized in that: The monitoring module (3) comprises at least one heart rate detection unit, at least one skin conductance detection unit, at least one facial recognition unit and at least one sound pickup unit.

3. A virtual reality based psychotherapy system according to claim 1, characterized in that: The interaction module (2) comprises a display unit and a speaker unit which are independently arranged or integrated in the VR platform (1).

4. A virtual reality based psychotherapy system according to claim 1, characterized in that: The optimization module (21) includes a computing unit for performing terminal-side computing processing.

5. A virtual reality-based psychotherapy method, applied to a virtual reality-based psychotherapy method as claimed in any one of claims 1 to 4, characterized in that: The steps include: Step S1, after the user enters the VR platform (1), the interaction module (2) prompts the user to select a preset scene; Step S2, executing the preset scene selected by the user in step S1; Step S3, performing feature extraction during the execution of step S2; Step S4, constructing a sentiment model based on the features in step S3; Step S5, the monitoring module (3) collects data and sends it to the optimization module (21); Step S6, matching the data in step S5 with the emotion model in step S4 to perform emotion prediction; Step S7, the interactive module (2) performs treatment or intervention according to the prediction result in step S6; Step S8, the feature extraction in step S3 is cyclically executed together with steps S2 to S7.

6. A virtual reality based psychotherapy method according to claim 5, characterized in that: The feature extraction in step S3 is performed by the monitoring module (3), wherein the extracted features at least include facial expression features, speech features and heart rate features.

7. A virtual reality based psychotherapy method according to claim 5, characterized in that: In step S4, the emotional model is constructed by at least one of support vector machine (SVM), random forest, deep neural network, and / or conducting a conversation.

8. A virtual reality based psychotherapy method according to claim 5, characterized in that: After step S7, the monitoring module (3) continues to perform feature extraction until the user finishes using the VR platform.

9. A virtual reality based psychotherapy method according to claim 5, characterized in that: The treatment or intervention performed by the interactive module (2) in step S7 includes adjusting at least one of light, music, and color.

10. A virtual reality based psychotherapy method according to claim 5, characterized in that: The control module (4) is used to control the VR platform (1) and the monitoring module (3) to transmit data to the optimization module (21).

Images