Emotion regulation and control system based on adaptive virtual reality scene

Abstract

The invention relates to the technical field of emotion regulation and control, in particular to an emotion regulation and control system based on a self-adaptive virtual reality scene, which comprises an emotion monitoring module, a virtual reality device, a self-adaptive regulation and control module, a feedback module, an emotion guiding module and an effect evaluation module, the emotion category and the emotion intensity are analyzed and judged; the virtual reality equipment is used for providing virtual reality experience for a user; the self-adaptive regulation and control module is used for adjusting elements and parameters of the virtual scene; the feedback module is used for collecting evaluation on the emotion regulation effect, analyzing the evaluation and generating a feedback report; the emotion guiding module is used for providing emotion guiding suggestions; and the effect evaluation module is used for quantifying the effect of emotion regulation and control and evaluating the effect of emotion regulation and control. According to the invention, real-time monitoring and dynamic regulation and control of the emotion of the user can be realized, personalized virtual reality experience is provided, and more effective mental health support is provided for the user.

Description

Technical Field

[0001] The present invention relates to the technical field of emotion regulation, and particularly to an emotion regulation system based on an adaptive virtual reality scenario. Background Art

[0002] In the field of mental health, emotion regulation is of great significance for improving emotional disorders and enhancing the quality of life. Traditional emotion regulation methods mainly include psychological counseling, drug treatment, and training of self-regulation skills. However, these methods have certain limitations. Psychological counseling requires professional psychiatrists and a long time investment, and the effects vary from person to person; drug treatment may have side effects and cannot fundamentally solve emotional problems; although the training of self-regulation skills is effective, it requires users to have a high level of self-awareness and self-discipline ability, and it is difficult to sustain in practical applications.

[0003] In recent years, the application of virtual reality (VR) technology in the field of mental health has gradually attracted attention. VR technology can provide users with immersive experiences and help users regulate their emotions by simulating different environments and scenarios. However, most of the existing VR-based emotion regulation systems adopt fixed scenarios and preset regulation strategies, lacking adaptability. These systems cannot adjust the elements and parameters of the virtual scenario in real time according to the user's emotional state, resulting in limited regulation effects. In addition, the existing systems also have deficiencies in emotion monitoring, usually relying only on a single physiological signal or behavioral data, and cannot comprehensively and accurately judge the user's emotional state.

[0004] Deep learning algorithms have significant advantages in emotion monitoring and can more accurately judge the user's emotional state by analyzing multi-modal data. However, most of the existing deep learning-based emotion monitoring systems have problems such as complex models, large training data requirements, and poor real-time performance. In addition, the existing systems are also relatively simple in effect evaluation, lacking quantitative indicators and long-term effect evaluation mechanisms, and cannot comprehensively evaluate the effect of emotion regulation.

[0005] Therefore, the present invention proposes an emotion regulation system based on an adaptive virtual reality scenario. Summary of the Invention

[0006] The purpose of the present invention is to propose an emotion regulation system based on an adaptive virtual reality scenario to solve the problems of lack of real-time performance, personalization, and multi-modal monitoring ability in the prior art.

[0007] To achieve the above purpose, the present invention adopts the following technical solutions: An emotion regulation system based on an adaptive virtual reality scenario, comprising: An emotion monitoring module, which is used to collect the physiological signals, facial expressions and speech intonations of users, and analyze and judge the current emotion category and emotion intensity of users through an improved deep learning neural network algorithm; A virtual reality device, which is used to provide a virtual reality experience for users and can present a preset virtual scene according to the emotion category and emotion intensity of users; An adaptive regulation module, which is used to adjust the elements and parameters of the virtual scene according to the emotion category and emotion intensity of users; A feedback module, which is used to collect users' evaluations of the emotion regulation effect through various interaction methods, and analyze the feedback information through an emotion analysis algorithm to generate a feedback report; An emotion guidance module, which is used to provide emotion guidance suggestions according to the emotion category of users, including relaxation exercise methods, mindfulness meditation exercises, emotion expression assistance and emotion release; An effect evaluation module, which is used to quantify the effect of emotion regulation and evaluate the short-term and long-term effects of emotion regulation according to the quantification results.

[0008] Compared with the prior art, the beneficial effects of the present invention are as follows: Through the improved deep learning neural network algorithm and combined with the multi-modal data fusion technology, the present invention can more accurately monitor the emotion state of users, and realizes the real-time and accurate judgment of the emotion category and intensity of users.

[0009] The adaptive regulation module of the present invention can dynamically adjust the elements and parameters of the virtual scene according to the emotion state of users, which can not only better meet the emotion needs of users, but also improve the effect of emotion regulation and the user experience.

[0010] The feedback module of the present invention collects users' evaluations of the emotion regulation effect through various interaction methods and generates a detailed feedback report, which can not only timely understand the emotion state of users and the satisfaction with the regulation effect, but also further optimize the regulation strategy according to the users' feedback.

[0011] The effect evaluation module of the present invention comprehensively evaluates the effect of emotion regulation through quantitative indicators, can more comprehensively and objectively evaluate the effect of emotion regulation, and provides more scientific regulation suggestions and evaluation reports for users. Description of the Drawings

[0012] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0013] Figure 1 Schematic diagram of the system provided by the embodiment of the present invention. Detailed implementation manners

[0014] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the intended invention purpose, the following combines the accompanying drawings and preferred embodiments to detail the specific implementation manners, structures, features and effects of an emotion regulation system based on an adaptive virtual reality scenario proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0015] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0016] The following embodiments are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0017] The following specifically describes the specific solution of an emotion regulation system based on an adaptive virtual reality scenario provided by the present invention with reference to the accompanying drawings.

[0018] Embodiment In the embodiment of the present invention, an emotion regulation system based on an adaptive virtual reality scenario is provided, including: An emotion monitoring module, configured to collect the physiological signals, facial expressions and speech intonations of a user, and analyze and judge the current emotion category and emotion intensity of the user through an improved deep learning neural network algorithm; A virtual reality device, configured to provide a virtual reality experience for the user, and capable of presenting a preset virtual scene according to the emotion category and emotion intensity of the user; An adaptive regulation module, configured to adjust the elements and parameters of the virtual scene according to the emotion category and emotion intensity of the user; A feedback module, configured to collect the evaluation of the emotion regulation effect by the user through various interaction methods, and analyze the feedback information through an emotion analysis algorithm to generate a feedback report; An emotion guidance module, configured to provide emotion guidance suggestions according to the emotion category of the user, including relaxation exercise methods, mindfulness meditation practice, emotion expression assistance and emotion release; An effect evaluation module, configured to quantify the effect of emotion regulation, and evaluate the short-term and long-term effects of emotion regulation according to the quantization result.

[0019] Please refer to Figure 1 , the schematic diagram of the system provided by the embodiment of the present invention.

[0020] I. Emotion monitoring module The emotion monitoring module includes a physiological sensor, an expression recognition camera, a voice analysis device, and an analysis unit; The physiological sensor includes a heart rate sensor, a skin conductance sensor, a respiration sensor, a heart rate variability sensor, a blood pressure sensor, and a body temperature sensor, and is used to collect the physiological signals of the user, so as to achieve real-time monitoring of the user's physiological state. The physiological signals include heart rate, skin conductance, respiration rate, heart rate variability, blood pressure, and body temperature; The expression recognition camera is used to capture the facial expressions of the user, and analyze them using computer vision technology to identify the emotion categories of the user. The emotion categories include happy, sad, angry, surprised, fearful, disgusted, and neutral; The voice analysis device analyzes the voice intonation of the user, extracts the emotion features in the voice, and realizes the judgment of the user's emotion category. The voice intonation includes pitch, intensity, duration, and timbre, and the emotion features include fundamental frequency change rate, energy, speech rate, pause frequency, and formant; The analysis unit analyzes the physiological signals, facial expressions, and voice intonation data through an improved deep learning neural network algorithm to realize the judgment of the user's emotion category and emotion intensity. The emotion intensity includes low intensity, medium intensity, and high intensity.

[0021] It should be noted that the heart rate sensor is used to measure the heart rate of the user, including a photoplethysmography sensor: measuring the change in blood flow by an optical method, and then calculating the heart rate; an electrode type heart rate sensor: measuring the electrocardiogram signal through an electrode patch, and calculating the heart rate and heart rate variability.

[0022] The skin conductance sensor is used to measure the skin conductance of the user, including an electrode type skin conductance sensor: measuring the conductivity of the skin through an electrode patch to reflect the sweat gland activity.

[0023] The respiration sensor is used to measure the respiration rate of the user, including a respiration belt: monitoring the respiration rate by measuring the expansion and contraction of the chest and abdomen; a photoplethysmography sensor: measuring the change in blood flow caused by respiration by an optical method.

[0024] The heart rate variability sensor is used to measure the heart rate variability of the user, including an electrode type heart rate sensor: measuring the electrocardiogram signal through an electrode patch and calculating the heart rate variability.

[0025] The blood pressure sensor is used to measure the blood pressure of the user, including a non-invasive blood pressure monitor: measuring the arterial blood pressure through a cuff.

[0026] The body temperature sensor is used to measure the body temperature of the user, including a thermistor: measuring the body temperature through a thermistor.

[0027] Heart rate (HR): The number of heartbeats per minute, with the unit of beats per minute (bpm).

[0028] Galvanic skin response (GSR): The electrical conductivity of the skin, reflecting the activity of sweat glands, with the unit of microsiemens (μS).

[0029] Respiratory rate (BR): The number of breaths per minute, with the unit of breaths / minute.

[0030] Heart rate variability (HRV): The variation in the intervals between heartbeats, reflecting the activity of the autonomic nervous system.

[0031] Blood pressure (BP): Arterial blood pressure, with the unit of millimeters of mercury (mmHg).

[0032] Body temperature (BT): The temperature of the body, with the unit of degrees Celsius (°C).

[0033] Happiness: It is manifested as a positive and pleasant emotional state. The facial expression is the corners of the mouth turning up, the eyes narrowing, and the cheeks rising. It has a relatively high voice energy and speech rate.

[0034] Sadness: It is manifested as a negative and depressed emotional state. The facial expression is the corners of the mouth drooping, the eyes looking down, and the eyebrows furrowed. It has a relatively low voice energy and slow speech rate.

[0035] Anger: It is manifested as an excited and angry emotional state. The facial expression is the eyebrows furrowed, the eyes widened, and the corners of the mouth tightened. It has a relatively high voice energy and fast speech rate.

[0036] Surprise: It is manifested as a surprised and shocked emotional state. The facial expression is the eyes widened, the eyebrows raised, and the mouth opened. It has a relatively high voice energy and short tone length.

[0037] Fear: It is manifested as a scared and nervous emotional state. The facial expression is the eyes widened, the eyebrows raised, and the corners of the mouth drooping. It has a relatively high voice energy and fast speech rate.

[0038] Disgust: It is manifested as a disgusted and dislike emotional state. The facial expression is the eyebrows frowned, the corners of the mouth turned down, and the nose wrinkled. It has a relatively low voice energy and slow speech rate.

[0039] Neutral: It is manifested as a calm and emotionless state. The facial expression is the facial muscles relaxed, without obvious expression changes, and has a relatively stable intonation.

[0040] Pitch refers to the high and low of speech, expressed in hertz (Hz). When the emotion is excited, the pitch will rise; when the emotion is low, the pitch will fall.

[0041] Sound intensity refers to the loudness of speech, expressed in decibels (dB). When you are emotionally excited, the sound intensity will increase; when you are emotionally depressed, the sound intensity will decrease.

[0042] Sound duration refers to the duration of a sound, expressed in seconds. When you are nervous, the sound duration will be shorter; when you are relaxed, the sound duration will be longer.

[0043] Timbre refers to the characteristics of speech, which is determined by the harmonic structure and resonance peaks. Different emotional states will cause changes in timbre. The timbre is sharper when angry, and deeper when sad.

[0044] The fundamental frequency change rate refers to the change of the fundamental frequency over time, which is expressed by the standard deviation (SD). When emotions are excited, the fundamental frequency change rate will increase.

[0045] Energy refers to the intensity of the speech signal, which is expressed as short-term energy. When emotions are excited, the energy increases.

[0046] Speaking speed refers to the number of syllables or words spoken in a unit of time, expressed in syllables / second or words / second. When you are nervous, your speaking speed will increase.

[0047] Pause frequency refers to the number of pauses that occur per unit time, expressed as pauses / second. When people are emotionally tense, the pause frequency will decrease.

[0048] Formant refers to the resonant frequency in a speech signal, expressed in Hertz (Hz). Different emotional states can cause changes in the formant.

[0049] The criteria for judging low intensity include physiological signals: heart rate 60-100 beats / minute, skin conductance less than 10μS, respiratory rate 12-20 times / minute, and high heart rate variability; facial expressions: slight changes in eyebrows, slight changes in corners of the mouth, and changes in the eyes within the normal range; voice intonation: fundamental frequency change rate less than 10Hz, low energy, speaking speed 120-150 words per minute, pause frequency more than 10 times per minute, and small changes in resonance peaks.

[0050] The criteria for judging moderate intensity include physiological signals: heart rate 100-120 beats / minute, skin conductance 10-20μS, respiratory rate 20-25 times / minute, and moderate heart rate variability; facial expressions: moderate changes in eyebrows, moderate changes in corners of the mouth, and moderate changes in eyes; voice intonation: fundamental frequency change rate 10-20Hz, medium energy, speaking speed 150-180 words per minute, pause frequency 5-10 times per minute, and moderate changes in resonance peaks.

[0051] The judgment criteria for high intensity include physiological signals: heart rate exceeding 120 beats per minute, skin conductance exceeding 20 μS, respiratory rate exceeding 25 breaths per minute, and low heart rate variability; facial expressions: significant changes in eyebrows, significant changes in the corners of the mouth, significant changes in eyes; speech intonation: the change rate of fundamental frequency exceeding 20 Hz, high energy, speech rate exceeding 180 words per minute, pause frequency less than 5 times per minute, and significant changes in formants.

[0052] Furthermore, in the emotion monitoring module, by analyzing physiological signals, facial expressions, and speech intonation data through an improved deep learning neural network algorithm, the judgment of the user's emotion category and emotion intensity is realized, where: The improved deep learning neural network algorithm is based on the convolutional neural network CNN, long short-term memory network LSTM, and fully connected neural network FCN. The improvements include multi-modal data fusion, attention mechanism, feature extraction optimization, and model structure optimization; The multi-modal data fusion adopts early fusion, mid-term fusion, and late fusion techniques to comprehensively process the features of physiological signals, facial expressions, and speech intonation; The attention mechanism is to introduce the self-attention mechanism to enhance the weight of the model for features; The feature extraction optimization uses wavelet transform and short-time Fourier transform to extract the time-frequency features of physiological signals, uses a pre-trained CNN model to extract the deep features of facial expressions, and uses mel-frequency cepstrum and fundamental frequency change rate to extract the features of speech intonation; The model structure optimization combines the advantages of CNN and LSTM to construct a hybrid model structure and introduces residual connections to avoid the problem of gradient disappearance; The output of the improved deep learning neural network algorithm is a continuous value between 0 and 1, representing the intensity of emotion. The continuous value is divided into discrete intensity levels. The specific formula is:

[0053]

[0054] Among them, Intensity represents intensity, g(x) is the output function of the regression network, representing the quantization value of emotion intensity, Intensity Level represents the intensity level, Low represents low intensity, Medium represents medium intensity, High represents high intensity, 0 represents the lowest emotion intensity, 1 represents the highest emotion intensity, 0.3 is the threshold between low intensity and medium intensity, and 0.7 is the threshold between medium intensity and high intensity.

[0055] It should be noted that multi-modal data fusion refers to the comprehensive processing of data from different modalities (physiological signals, facial expressions, speech intonations) to make full use of the advantages of each modality data and improve the accuracy and robustness of emotion monitoring.

[0056] Early fusion: In the data preprocessing stage, the data of different modalities are feature concatenated to form a high-dimensional feature vector, and then input into the neural network for training.

[0057] Mid-term fusion: In the middle layer of the neural network, the features of different modalities are fused. Through shared layers or cross-connections, the features of different modalities complement each other.

[0058] Late fusion: At the output layer of the neural network, the prediction results of different modalities are weighted averaged or voted to obtain the final emotion category and intensity judgment.

[0059] The attention mechanism is a neural network technology that simulates human attention. It can make the model pay more attention to the features that have an important impact on emotion judgment. The self-attention mechanism is introduced. By calculating the weights between features, the weights of key features in the model are enhanced, thereby improving the accuracy of emotion monitoring.

[0060] The code for multi-modal data fusion and introducing the attention mechanism includes: import numpy as np import torch import torch.nn as nn import torch.nn.functional as F # Assume that features have been extracted and normalized physiological_features = np.random.rand(1, 10) # Physiological signal features facial_features = np.random.rand(1, 20) # Facial expression features voice_features = np.random.rand(1, 15) # Speech intonation features class MultiModalFusion(nn.Module): def __init__(self, physiological_dim, facial_dim, voice_dim,hidden_dim): super(MultiModalFusion, self).__init__() self.physiological_fc = nn.Linear(physiological_dim, hidden_dim) self.facial_fc = nn.Linear(facial_dim, hidden_dim) self.voice_fc = nn.Linear(voice_dim, hidden_dim) self.fusion_layer = nn.Linear(hidden_dim 3, hidden_dim) self.attention = nn.MultiheadAttention(embed_dim=hidden_dim,num_heads=1) def forward(self, physiological, facial, voice): physiological = self.physiological_fc(physiological) facial = self.facial_fc(facial) voice = self.voice_fc(voice) # Multimodal data fusion fused_features = torch.cat((physiological, facial, voice),dim=1) fused_features = self.fusion_layer(fused_features) # Attention mechanism fused_features, _ = self.attention(fused_features, fused_features, fused_features) return fused_features # Initialize the model fusion_model = MultiModalFusion(physiological_dim=10, facial_dim=20,voice_dim=15, hidden_dim=64) # Convert to tensors physiological_features = torch.tensor(physiological_features, dtype=torch.float32) facial_features = torch.tensor(facial_features, dtype=torch.float32) voice_features = torch.tensor(voice_features, dtype=torch.float32) # Fuse features fused_features = fusion_model(physiological_features, facial_features, voice_features) Physiological signal feature extraction: Wavelet transform and short-time Fourier transform are used to extract the time-frequency features of physiological signals. The extracted features can reflect the changes of physiological signals at different times and frequencies, which helps to judge the user's emotional state.

[0061] Facial expression feature extraction: A pre-trained CNN model is used to extract the deep features of facial expressions. The pre-trained model can capture the subtle changes of facial expressions, so as to accurately identify the emotion categories.

[0062] Speech intonation feature extraction: Mel Frequency Cepstral Coefficients (MFCC) and fundamental frequency change rate are used to extract the features of speech intonation. The extracted features can reflect the emotional information in speech.

[0063] The present invention combines the advantages of CNN and LSTM to construct a hybrid model structure. CNN can extract local features, while LSTM can capture the long-term dependence relationship of time series data. By introducing residual connections, the problem of gradient disappearance is avoided, and the training efficiency and stability of the model are improved.

[0064] The code of the hybrid model structure includes: class EmotionClassifier(nn.Module): def __init__(self, input_dim, hidden_dim, output_dim): super(EmotionClassifier, self).__init__() self.cnn = nn.Conv1d(in_channels=1, out_channels=16, kernel_size=3, padding=1) self.lstm = nn.LSTM(input_size=16, hidden_size=hidden_dim,batch_first=True) self.fc = nn.Linear(hidden_dim, output_dim) self.residual = nn.Linear(input_dim, hidden_dim) def forward(self, x): x = x.unsqueeze(1) # Add channel dimension x = self.cnn(x) x = x.squeeze(1) # Remove channel dimension x, _ = self.lstm(x) x = x[:, -1, :] # Take the last time step of the LSTM residual = self.residual(fused_features) x = x + residual # Residual connection x = self.fc(x) return x # Initialize the model emotion_model = EmotionClassifier(input_dim=64, hidden_dim=32,output_dim=1) # Emotion intensity prediction intensity = emotion_model(fused_features) The code for quantifying emotion intensity includes: def classify_intensity(intensity): if intensity < 0.3: return "Low" elif intensity < 0.7: return "Medium" else: return "High" # Assume the model output is a value between 0 and 1 intensity_value = torch.sigmoid(intensity).item() intensity_level = classify_intensity(intensity_value) print(f"Emotion Intensity: {intensity_value}, Level: {intensity_level}") II. Virtual Reality Devices Virtual reality devices include a head-mounted display, a handle, a spatial positioning sensor, and a scene rendering unit; The head-mounted display, including a display screen, a wide-angle optical system, and eye-tracking technology, can provide a visual experience for users. The display screen is used to present virtual scenes, the wide-angle optical system is used to provide a field of view, and the eye-tracking technology is used to monitor the user's eye movements in real time to achieve visual interaction; The handle, including a motion sensor, a haptic feedback device, buttons, and triggers, enables users to interact with the virtual environment. The motion sensor is used to capture the movements of the hand and fingers, the haptic feedback device is used to provide haptic feedback to enhance the interaction experience, and the buttons and triggers are used to perform operations, including selection operations, navigation operations, interaction operations, and gesture operations; The spatial positioning sensor, including an optical tracking system, an inertial measurement unit, an ultrasonic positioning system, and spatial mapping technology, enables users to move freely and interact in the virtual environment. The optical tracking system captures the movements of the user and the device through a camera to achieve spatial positioning, the inertial measurement unit is used to monitor the movements of the user's head and hands in real time to provide motion data, the ultrasonic positioning system is used to assist the optical tracking system, and the spatial mapping technology maps the spatial information of the real environment into the virtual environment to achieve movement and interaction; A scene presentation unit that stores multiple preset virtual scene templates and can present preset virtual scenes according to the user's emotion category and intensity. The virtual scene templates include natural scenery scenes, urban environment scenes, social interaction scenes, psychological healing scenes, personalized scenes, and emotion regulation scenes.

[0065] It should be noted that the display screen: is used to present virtual scenes, features high resolution and low latency, and can provide clearer and more realistic visual effects. Modern HMDs usually use high-resolution OLED or LCD screens to provide clear and bright images.

[0066] Wide-angle optical system: Through special lens design, it provides a wide field of view (FOV), enabling users to observe the virtual environment more naturally.

[0067] Eye tracking technology: It can monitor the user's eye movements in real time through cameras and sensors to achieve visual interaction. Users can trigger operations by gazing at an object, or the system can dynamically adjust the rendering effect of the scene according to the user's gaze point.

[0068] Motion sensors: Include accelerometers and gyroscopes, which can capture the movements of the hand and fingers, and track the position and orientation of the hand in real time, enabling users to interact naturally in the virtual environment.

[0069] Haptic feedback device: Through vibration motors, it can provide corresponding haptic feedback according to the user's operations, enhancing the user's interaction experience.

[0070] Buttons and triggers: Provide a variety of physical buttons and triggers for performing various operations, which can offer a more intuitive operation method, especially in scenarios that require quick responses.

[0071] Selection operations include single click: quickly press and release the button to select or activate a virtual object; double click: quickly press and release the button twice in succession to quickly select or activate a virtual object; long press: continuously press the button for a period of time to perform specific long-press operations, including opening the menu and activating special functions.

[0072] Navigation operations include arrow keys: used to control the moving direction in the virtual environment, including forward, backward, left turn, and right turn; joystick: used for finer movement control, enabling free movement or perspective adjustment in the virtual environment; trigger: used to perform specific navigation operations, including rapid movement and jumping.

[0073] Interaction operations include grasping: Through buttons or triggers, users can grasp objects in the virtual environment; releasing: Through buttons or triggers, users can release the grasped objects; dragging: Through buttons or triggers, users can drag objects in the virtual environment; rotating: Through buttons or triggers, users can rotate objects in the virtual environment; scaling: Through buttons or triggers, users can scale objects in the virtual environment.

[0074] Gesture operations include gesture recognition: Through the motion sensors of the controller, recognize users' gestures, including waving, grasping, and releasing; gesture triggering: Through buttons or triggers, users can trigger specific gesture operations, including opening the menu and switching scenes.

[0075] Optical tracking system: Captures the movements of users and devices through cameras to achieve high-precision spatial positioning. It can use infrared cameras and marker points (such as the LED lights on VR headsets and controllers) to determine the position and orientation of the devices.

[0076] Inertial Measurement Unit (IMU): Monitors the movements of users' heads and hands in real time and provides motion data. The IMU includes an accelerometer, a gyroscope, and a magnetometer, and can respond quickly to users' actions.

[0077] Ultrasonic positioning system: Assists the optical tracking system to improve the accuracy and stability of positioning. It can measure the distance between devices, thereby providing more accurate position information.

[0078] Spatial mapping technology: Maps the spatial information of the real environment into the virtual environment, enabling users to move freely in the virtual environment while avoiding collisions with real-world obstacles. By scanning the real environment with a depth camera (Kinect), an obstacle model in the virtual environment is generated.

[0079] Natural scenery scenes include the following: Quiet seaside: Provides a peaceful beach environment where users can hear the sound of the waves, feel the sea breeze, and see seagulls flying, helping users relax.

[0080] Forest: Provides a dense forest environment where users can hear the chirping of birds, feel the rustling of leaves, and see the dappled sunlight through the leaves, providing a peaceful natural experience.

[0081] Mountains: Provides a magnificent mountain environment where users can feel the grandeur of the high mountains, see the snow-capped mountains in the distance and the flowers and plants nearby, providing an open view and a peaceful atmosphere.

[0082] Lake: Provides a quiet lake environment where users can hear the sound of the lake waves and see the reflections on the lake surface, providing a peaceful and relaxing experience.

[0083] The urban environment scenarios include the following: Busy city streets: Provide a lively urban street environment where users can hear the hustle and bustle of the city, see the people and vehicles coming and going, and offer a familiar urban atmosphere.

[0084] Parks: Provide a peaceful park environment where users can hear the chirping of birds and the rustling of leaves, see the green lawns and various flowers, and offer a relaxing natural environment.

[0085] Squares: Provide an open square environment where users can hear the music and laughter on the square, see the fountains and sculptures, and offer a lively social atmosphere.

[0086] The social interaction scenarios include the following: Virtual social gatherings: Provide a virtual social gathering environment where users can interact with virtual characters, engage in activities such as conversations and games, and enhance their social skills.

[0087] Classrooms: Provide a virtual classroom environment where users can participate in virtual courses, interact with virtual teachers and classmates, and enhance their learning and social abilities.

[0088] Meeting rooms: Provide a virtual meeting room environment where users can hold virtual meetings, give presentations, have discussions, etc., and enhance their professional social skills.

[0089] The psychological healing scenarios include the following: Warm family environment: Provide a warm family environment where users can feel the warmth of home, see the familiar furniture and decorations, and offer psychological comfort.

[0090] Hopeful future city: Provide a hopeful future city environment where users can see modern urban buildings and technological facilities, and offer a positive psychological experience.

[0091] Virtual meditation space: Provide a peaceful meditation space where users can practice meditation, hear soothing music, see soft lighting effects, and help users relax physically and mentally.

[0092] The personalized scenarios include the following: User's favorite travel destinations: Based on the user's travel experiences, provide virtual scenarios of the user's favorite travel destinations, including the Eiffel Tower in Paris and Times Square in New York, and offer a personalized experience.

[0093] Childhood memory scenarios: Based on the user's childhood memories, provide scenarios from the user's childhood, including the childhood home and school, and offer an emotional connection.

[0094] The emotion regulation scenarios include the following: Happy Scenarios: Include festival celebrations, concerts, providing a lively atmosphere to help users enhance their happy emotions.

[0095] Sad Scenarios: Include a quiet library, a street in the rain, providing a quiet atmosphere to help users express and process their sad emotions.

[0096] Angry Scenarios: Include a virtual boxing gym, a racetrack, providing an environment to release angry emotions and help users manage their anger.

[0097] III. Adaptive Regulation Module The adaptive regulation module includes a scene adjustment unit and a parameter adjustment unit; The scene adjustment unit, including a scene element adjuster and a scene layout adjuster, is used to adjust virtual scene elements and scene layout according to the user's emotion category after the virtual scene is presented. Scene elements include color, lighting effects, and background music. Scene layout includes the openness of the scene and the distribution of objects; The parameter adjustment unit, including an immersion degree adjuster and an element change speed adjuster, can adjust the parameters of the virtual scene according to the user's emotion intensity. The parameters include the immersion degree of the scene and the element change speed.

[0098] It should be noted that for color adjustment: Color has a significant impact on emotions. Blue and green are usually associated with calmness and relaxation, while red and yellow may stimulate vitality or nervousness. According to the user's emotion category, the system can dynamically adjust the color in the scene. If the user feels anxious, the system may adjust the scene color to soft blue or green to help the user relax.

[0099] For lighting effect adjustment: Lighting effects can create different atmospheres. Soft lighting makes people feel comfortable and relaxed, while strong lighting may cause excitement or nervousness. The system can adjust the lighting effects according to the user's emotion category. For users who feel more stressed, the system can dim the light and increase soft shadows to create a peaceful atmosphere.

[0100] For background music adjustment: Music is a powerful tool for emotion regulation. The system can select appropriate background music according to the user's emotion category. For users who feel sad, the system can play soothing music to comfort their emotions; for users who feel tired, the system can play lively music to refresh them.

[0101] For scene openness adjustment: The openness of the scene can affect the user's emotional experience. Open spaces usually make people feel free and relaxed, while narrow spaces may cause a sense of depression. According to the user's emotion category, the system can adjust the openness of the scene. For users who feel lonely, the system can adjust the scene layout to an open social environment and increase interactive elements.

[0102] Object distribution adjustment: The distribution of objects in the scene affects the user's mood. A cluttered environment may cause anxiety, while a tidy environment makes people feel comfortable. The system can adjust the object distribution according to the user's mood category. If the user feels irritable, the system can adjust the object distribution in the scene to a more orderly and tidy state.

[0103] Adjustment of immersion level: For low-intensity emotions, moderately increase the immersion of the scene, enhance visual and auditory effects to gradually relax the user; for medium-intensity emotions, significantly improve the immersion of the scene, add tactile feedback and dynamic light and shadow effects to help the user better engage in the scene; for high-intensity emotions, provide an extreme immersion experience, conduct full-sensory stimulation and real-time interactive feedback to guide the user to relax quickly; Adjustment of the element change speed: For low-intensity emotions, keep the scene elements changing slowly, including slowly floating clouds and gentle music to create a peaceful atmosphere; for medium-intensity emotions, moderately increase the element change speed, including medium-speed water flow and moderate music rhythm to provide moderate stimulation; for high-intensity emotions, quickly change the scene elements, including quickly switching pictures and strong music rhythm to help the user quickly release emotions.

[0104] IV. Feedback Module The feedback module includes a feedback collection unit, a feedback analysis unit, and a data recording unit; The feedback collection unit is used to collect the user's evaluation of the emotion regulation effect through various interaction methods. The various interaction methods include virtual buttons, voice commands, gesture recognition, facial expression recognition, physiological signal feedback, and text input; The feedback analysis unit analyzes the collected user feedback information through emotion analysis algorithms, understands the user's emotional state and satisfaction with the emotion regulation effect, and generates a feedback report based on the analysis results; The data recording unit is used to record the user's emotional change process, regulation effect data, user interaction data, and feedback data, and store the recorded data in a database.

[0105] It should be noted that virtual buttons: Virtual buttons are set in the virtual reality environment. Users can click these buttons to express their emotional state and satisfaction with the regulation effect. The virtual buttons include "Feeling much better", "A little helpful", "No change".

[0106] Voice commands: Through voice recognition technology, users can use voice commands to provide feedback. Users can directly say "I feel much more relaxed" or "This scene makes me more anxious". The system converts the voice into text through voice recognition technology and analyzes it.

[0107] Gesture Recognition: Using gesture recognition technology, users can express their feelings through specific gestures. Thumbs up indicates satisfaction, and waving hands indicates dissatisfaction. The system captures and analyzes these gestures through gesture recognition technology.

[0108] Facial Expression Recognition: Through a facial expression recognition camera, it automatically captures the user's facial expressions and analyzes their emotional states. A smile may indicate satisfaction, and frowning may indicate dissatisfaction. The system recognizes these expressions through computer vision technology and records them.

[0109] Physiological Signal Feedback: Through physiological sensors, it collects the user's physiological signals and analyzes the user's emotional states. A decrease in heart rate may indicate the user's emotional relaxation. The system records the user's feedback based on the changes in these physiological signals.

[0110] Text Input: It provides a text input function. Users can input text through a keyboard or a virtual keyboard to describe in detail their emotional states and feelings about the regulation effect. Users can input "This scene makes me feel very calm" or "I think this scene is a bit noisy".

[0111] Emotion Analysis Algorithm: Using natural language processing (NLP) technology to analyze the user's voice commands and text inputs, extract emotional features, and be able to judge whether the user's emotion is positive, negative, or neutral.

[0112] Feedback Report Generation: Generate a detailed feedback report based on the analysis results, including the user's emotional change trend and satisfaction with the emotion regulation effect. The report can provide a basis for system optimization and personalized adjustment.

[0113] Recording of Emotion Change Process: Record the user's emotional changes during the use of the system, including changes in emotion categories and intensities.

[0114] Recording of Regulation Effect Data: Record the effect data of emotion regulation, including the percentage change in emotion intensity and the improvement rate of emotion stability.

[0115] Recording of User Interaction Data: Record the detailed data of the user's interaction with the system, including virtual button operations, voice commands, and gesture recognition.

[0116] Recording of Feedback Data: Record the user's evaluation and feedback information on the emotion regulation effect.

[0117] V. Emotion Guidance Module The emotion guidance module includes a relaxation exercise suggestion unit, a mindfulness meditation guidance unit, an emotion expression assistance unit, and an emotion release unit; The relaxation exercise suggestion unit can provide relaxation exercise methods according to the user's emotion category. The relaxation exercise methods include deep breathing, progressive muscle relaxation, and imaginal relaxation; The Mindfulness Meditation Guidance Unit, through voice guidance and visual cues, helps users conduct mindfulness meditation practices, which include body scan meditation, breath meditation, and emotion observation meditation; The Emotion Expression Assistance Unit, through the virtual social function in the virtual environment, enables users to communicate with other users or virtual characters, share emotional experiences, and provides a virtual diary function for users to record their emotional states and feelings; The Emotion Release Unit, through interactive activities in the virtual environment, helps users release emotions. The interactive activities include hitting a virtual sandbag, virtual painting, virtual music creation, and virtual nature exploration.

[0118] It should be noted that for deep breathing exercises: Through voice guidance and visual cues, users are guided to conduct deep breathing exercises to adjust the breathing rhythm and relieve tension.

[0119] Progressive muscle relaxation: Through voice guidance and visual cues, users are guided to gradually relax all the muscles in the body, from the head to the feet, to reduce physical tension.

[0120] Imagery relaxation: Through voice guidance, users are allowed to imagine themselves in a peaceful and comfortable place to enhance the relaxation effect.

[0121] Body scan meditation: Through voice guidance, users are allowed to gradually focus on various parts of the body, feel the sensations of the body, and improve body awareness.

[0122] Breath meditation: Through voice guidance and visual cues, users are allowed to focus on the breath, observe the natural flow of the breath, and cultivate concentration and a sense of relaxation.

[0123] Emotion observation meditation: Through voice guidance, users are allowed to observe their emotions, accept the existence of emotions without judgment, and improve emotional awareness.

[0124] Virtual social function: Users can communicate with other users or virtual characters through the virtual environment, share emotional experiences, and the system can set up virtual party scenes where users can communicate their feelings with other users.

[0125] Virtual diary function: Provides a virtual diary function where users can record their emotional states and feelings. The system can provide a virtual diary interface where users can record their emotional changes and feelings through voice input or text input.

[0126] Hitting a virtual sandbag: Users can hit a virtual sandbag through a virtual handle to release tension and anger. The feedback effects of the sandbag (such as vibration, sound) enhance the user experience.

[0127] Virtual Painting: Users can freely paint on a virtual canvas, expressing their emotions through colors and lines. The system provides a variety of painting tools and materials, increasing the creative freedom of users.

[0128] Virtual Music Creation: Users can create music through virtual musical instruments (such as pianos and guitars) to express their emotions. The system provides simple music creation guidance to help users get started quickly.

[0129] Virtual Nature Exploration: Users can freely explore in a virtual natural environment (such as a forest or by the sea), releasing their emotions through interactions with natural elements (such as touching leaves and picking up shells). The sounds and visual effects in the environment enhance the user's sense of immersion.

[0130] Virtual Emotion Release Games: Users can participate in virtual emotion release games, including "Emotion Balloons" (where users breathe deeply to blow emotions into balloons and then release them, symbolizing the release of emotions) and "Emotion Puzzles" (where users complete puzzles to gradually understand and release emotions).

[0131] VI. Effect Evaluation Module The effect evaluation module includes an effect quantification unit and an evaluation unit; The effect quantification unit is used to quantitatively evaluate the effect of emotion regulation, quantifying the effect of emotion regulation into specific indicators, including the percentage change in emotion intensity, the improvement rate of emotion stability, the emotion category conversion rate, and the regulation response time; The evaluation unit is used to evaluate the short-term and long-term effects of emotion regulation based on the quantification results. The short-term effect is to evaluate the emotion changes of users after single use of the system, and the long-term effect is to evaluate the trend of emotion changes of users after multiple uses of the system.

[0132] It should be noted that the percentage change in emotion intensity: measures the change in the user's emotion intensity before and after using the system.

[0133] The improvement rate of emotion stability: measures the degree of improvement in the user's emotion stability, which can be measured by the standard deviation of emotion intensity.

[0134] The emotion category conversion rate: measures the change in the user's emotion category.

[0135] The regulation response time: measures the response time of the system to the user's emotion changes.

[0136] Short-term Effect Evaluation: Evaluate the emotion changes of users after single use of the system. The short-term effect is measured by the percentage change in emotion intensity and the improvement rate of emotion stability. If the percentage change in emotion intensity is 62.5% and the improvement rate of emotion stability is 50% after single use by the user, it can be considered that the system has a good effect on regulating the emotions of users in the short term.

[0137] Long-term effect evaluation: Evaluate the trend of the user's emotional changes after using the system multiple times. The long-term effect is measured by the emotional category conversion rate and the regulation response time. If the emotional category conversion rate gradually decreases and the regulation response time gradually shortens after the user uses the system multiple times, it can be considered that the system has a good effect on regulating the user's emotions in the long term.

[0138] For the emotional regulation system based on the adaptive virtual reality scenario in this embodiment, after the use is completed, each module and device are shut down one by one according to the specified procedure to ensure safe power-off, and the monitoring data and operation records during the use process are sorted out and saved to provide a basis for subsequent clinical research and quality control.

[0139] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An emotion regulation system based on an adaptive virtual reality scenario, characterized in that, Including: An emotion monitoring module, which is used to collect the physiological signals, facial expressions and speech intonations of the user, and analyze and judge the current emotion category and emotion intensity of the user through an improved deep learning neural network algorithm; A virtual reality device, which is used to provide a virtual reality experience for the user and can present a preset virtual scene according to the user's emotion category and emotion intensity; An adaptive regulation module, which is used to adjust the elements and parameters of the virtual scene according to the user's emotion category and emotion intensity; A feedback module, which is used to collect the user's evaluation of the emotion regulation effect through a variety of interaction methods, and analyze the feedback information through an emotion analysis algorithm to generate a feedback report; An emotion guidance module, which is used to provide emotion guidance suggestions according to the user's emotion category, including relaxation practice methods, mindfulness meditation practice, emotion expression assistance and emotion release; An effect evaluation module, which is used to quantify the effect of emotion regulation and evaluate the short-term and long-term effects of emotion regulation according to the quantification results.

2. The emotional regulation system based on an adaptive virtual reality scenario according to claim 1, wherein The emotion monitoring module includes: Physiological sensors, including a heart rate sensor, a skin conductance sensor, a respiration sensor, a heart rate variability sensor, a blood pressure sensor and a body temperature sensor, which are used to collect the physiological signals of the user and realize the real-time monitoring of the user's physiological state. The physiological signals include heart rate, skin conductance, respiration rate, heart rate variability, blood pressure and body temperature; An expression recognition camera, which is used to capture the facial expressions of the user, analyze them using computer vision technology, and identify the user's emotion category. The emotion categories include happy, sad, angry, surprised, fearful, disgusted and neutral; A speech analysis device, which analyzes the speech intonation of the user, extracts the emotion features in the speech, and realizes the judgment of the user's emotion category. The speech intonation includes pitch, intensity, duration and timbre. The emotion features include fundamental frequency change rate, energy, speech rate, pause frequency and formant; An analysis unit, which analyzes the physiological signals, facial expressions and speech intonation data through an improved deep learning neural network algorithm to realize the judgment of the user's emotion category and emotion intensity. The emotion intensity includes low intensity, medium intensity and high intensity.

3. An emotion regulation system based on an adaptive virtual reality scene according to claim 2, characterized in that, The analysis of the physiological signals, facial expressions and speech intonation data through the improved deep learning neural network algorithm realizes the judgment of the user's emotion category and emotion intensity, where: The improved deep learning neural network algorithm is based on a convolutional neural network (CNN), a long short-term memory network (LSTM) and a fully connected neural network (FCN). The improvements include multi-modal data fusion, an attention mechanism, feature extraction optimization and model structure optimization; The multi-modal data fusion adopts early fusion, mid-term fusion and late fusion technologies to comprehensively process the features of physiological signals, facial expressions and speech intonations; The attention mechanism is to introduce a self-attention mechanism to enhance the weight of the model for features; The feature extraction optimization uses wavelet transform and short-time Fourier transform to extract the time-frequency features of physiological signals, uses a pre-trained CNN model to extract the deep features of facial expressions, and uses Mel frequency cepstrum and fundamental frequency change rate to extract the features of speech intonation; The model structure optimizes the advantages of CNN and LSTM, constructs a hybrid model structure, and introduces residual connections to avoid the problem of gradient vanishing; The improved deep learning neural network algorithm outputs a continuous value between 0 and 1, representing the intensity of emotion. The continuous value is divided into discrete intensity levels. The specific formula is: where Intensity represents intensity, g(x) is the output function of the regression network, representing the quantization value of emotion intensity, Intensity Level represents the intensity level, Low represents low intensity, Medium represents medium intensity, High represents high intensity, 0 represents the lowest emotion intensity, 1 represents the highest emotion intensity, 0.3 is the threshold between low intensity and medium intensity, and 0.7 is the threshold between medium intensity and high intensity.

4. An emotion regulation system based on an adaptive virtual reality scenario according to claim 1, characterized in that, The virtual reality device includes: A head-mounted display, including a display screen, a wide-angle optical system, and eye-tracking technology, which can provide a visual experience for users. The display screen is used to present virtual scenes, the wide-angle optical system is used to provide a field of view, and the eye-tracking technology is used to monitor the user's eye movements in real time to achieve visual interaction; A handle, including motion sensors, tactile feedback devices, buttons, and triggers, which can enable users to interact with the virtual environment. The motion sensors are used to capture the movements of the hand and fingers, the tactile feedback devices are used to provide tactile feedback to enhance the interaction experience, and the buttons and triggers are used to perform operations, including selection operations, navigation operations, interaction operations, and gesture operations; A spatial positioning sensor, including an optical tracking system, an inertial measurement unit, an ultrasonic positioning system, and spatial mapping technology, which can enable users to move freely and interact in the virtual environment. The optical tracking system captures the movements of the user and the device through a camera to achieve spatial positioning, the inertial measurement unit is used to monitor the movements of the user's head and hand in real time to provide motion data, the ultrasonic positioning system is used to assist the optical tracking system, and the spatial mapping technology maps the spatial information of the real environment into the virtual environment to achieve movement and interaction; A scene presentation unit, which stores multiple preset virtual scene templates and can present preset virtual scenes according to the user's emotion category and emotion intensity. The virtual scene templates include natural scenery scenes, urban environment scenes, social interaction scenes, psychological healing scenes, personalized scenes, and emotion regulation scenes.

5. An emotion regulation system based on an adaptive virtual reality scene according to claim 1, characterized in that, The adaptive regulation module includes: A scene adjustment unit, including a scene element adjuster and a scene layout adjuster, which is used to adjust virtual scene elements and scene layouts according to the user's emotion category after the virtual scene is presented. The scene elements include color, lighting effects, and background music, and the scene layout includes the openness of the scene and the distribution of objects; A parameter adjustment unit, including an immersion degree adjuster and an element change speed adjuster, which can adjust the parameters of the virtual scene according to the user's emotion intensity. The parameters include the immersion degree of the scene and the element change speed.

6. The emotional regulation system based on an adaptive virtual reality scene according to claim 1, wherein, The feedback module includes: A feedback collection unit, which is used to collect users' evaluations of the emotion regulation effect through various interaction methods, and the various interaction methods include virtual buttons, voice commands, gesture recognition, facial expression recognition, physiological signal feedback, and text input; A feedback analysis unit, which analyzes the collected user feedback information through an emotion analysis algorithm, understands the users' emotional states and satisfaction with the emotion regulation effect, and generates a feedback report according to the analysis results; A data recording unit, which is used to record the users' emotional change processes, regulation effect data, user interaction data, and feedback data, and store the recorded data in a database.

7. An emotion regulation system based on an adaptive virtual reality scenario according to claim 1, characterized in that, The emotion guidance module includes: A relaxation exercise suggestion unit, which can provide relaxation exercise methods according to the users' emotion categories, and the relaxation exercise methods include deep breathing, progressive muscle relaxation, and imaginal relaxation; A mindfulness meditation guidance unit, which helps users conduct mindfulness meditation exercises through voice guidance and visual cues, and the mindfulness meditation exercises include body scan meditation, breath meditation, and emotion observation meditation; An emotion expression assistance unit, which realizes the communication between users and other users or virtual characters through the virtual social function in the virtual environment, shares emotion experiences, and provides a virtual diary function for users to record their emotional states and feelings; An emotion release unit, which helps users release emotions through interactive activities in the virtual environment, and the interactive activities include hitting a virtual sandbag, virtual painting, virtual music creation, and virtual nature exploration.

8. The emotional regulation system based on an adaptive virtual reality scenario according to claim 1, characterized in that, The effect evaluation module includes: An effect quantification unit, which is used to quantitatively evaluate the effect of emotion regulation, and quantifies the effect of emotion regulation into specific indicators, and the indicators include the percentage change in emotion intensity, the improvement rate of emotion stability, the emotion category conversion rate, and the regulation response time; An evaluation unit, which is used to evaluate the short-term effect and long-term effect of emotion regulation according to the quantification results. The short-term effect is to evaluate the emotional changes of users after single use of the system, and the long-term effect is to evaluate the emotional change trend of users after multiple uses of the system.

Images