Multi-modal physiological signal-based personalized normal-feeling and pressure reduction system and multi-modal physiological signal-based personalized normal-feeling and pressure reduction method

Abstract

The invention relates to the field of psychological health, and provides a personalized mind-correcting and pressure-reducing system and method based on multi-modal physiological signals. The system comprises a multi-mode physiological signal acquisition module, an emotion recognition module, a mind content generation module and a virtual / augmented reality presentation module. The acquisition module is used for acquiring and preprocessing various physiological signals by using an advanced sensor; the emotion recognition module establishes a signal and emotion mapping relation according to a deep learning algorithm and classifies the signal and emotion mapping relation; the content generation module generates content according to the emotion and preference of the user by means of a big language model and a retrieval enhancement technology; and the presentation module is used for providing immersive experience by combining Unity 3D with VR / AR (Virtual Reality / Augmented Reality) equipment. The system has the advantages of being capable of accurately monitoring emotions in real time, intelligently matching contents, providing immersion experience and forming closed-loop feedback, having expansibility, achieving the emotion recognition accuracy rate of 95%, and being capable of effectively improving mental health and life quality.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and mental health, and specifically to a personalized mindfulness stress relief system and method based on multimodal physiological signals. The system can monitor the user's emotional state in real time, intelligently match mindfulness content, provide an immersive experience, and improve the level of mental health. Background Art

[0002] Existing mindfulness-based stress reduction technology has obvious limitations. On the one hand, it is impossible to monitor the user's emotional state in real time, which makes the stress reduction process lack specificity. The user's emotions may change at any time, but the existing technology cannot capture these changes in time, and cannot adjust the stress reduction plan according to the real-time emotions. On the other hand, the content is single in form and lacks appeal. It is usually presented in the form of traditional meditation instructions, text materials, etc., which is difficult to arouse the user's continued interest and participation. In addition, the existing technology lacks real-time emotion monitoring and mindfulness content intelligent matching technology, and cannot provide the most suitable mindfulness-based stress reduction content according to the user's specific emotional state. At the same time, there is also a lack of diversified mindfulness-based stress reduction content creation technology based on virtual reality / augmented reality, which cannot provide users with an immersive experience, limiting the effect and appeal of mindfulness-based stress reduction. The present invention aims to solve these problems and provide a personalized mindfulness-based stress reduction system and method based on multimodal physiological signals. Contents of the invention The present invention provides a personalized mindfulness stress reduction system and method based on multimodal physiological signals. The system comprises a multimodal physiological signal acquisition module, an emotional state recognition module, a personalized mindfulness content generation module and a virtual / augmented reality presentation module.

[0003] The multimodal physiological signal acquisition module uses advanced biosensor technology to simultaneously collect multiple physiological signals of the user, such as electrocardiogram (ECG), electrodermal conductivity (EDA), and electroencephalogram (EEG). The module uses high-precision, low-power MEMS sensors combined with flexible electronic technology to achieve a comfortable and portable wearable design. The collected raw signals are pre-processed through low-pass filtering, band-pass filtering, and other steps to remove noise and baseline drift, thereby improving signal quality.

[0004] The emotional state recognition module establishes a mapping relationship between multimodal physiological signals and emotional states based on a deep learning algorithm. The module first extracts time domain features (mean, variance, etc.) and frequency domain features (power spectrum density, etc.) from the preprocessed signal, and then inputs them into the multi-layer perceptron (MLP) neural network for feature fusion and emotion classification. Emotional states are divided into three categories: positive, neutral, and negative, with a classification accuracy of more than 90%. The mathematical model of emotion recognition can be expressed as:

[0005] in, Represents the input physiological signal characteristics, is the corresponding weight, is the bias term, is the activation function (such as ReLU).

[0006] The personalized mindfulness content generation module uses a large language model combined with retrieval-augmented generation (RAG) technology to achieve intelligent mindfulness content creation. The module first designs prompts based on the user's emotional state and personal preferences, such as "create a 5-minute mindfulness meditation guide text for an anxious workplace professional." Then the prompt is input into the large language model to generate the initial text, and then the RAG technology is used to retrieve relevant content from the mindfulness professional knowledge base for optimization, and finally outputs high-quality, personalized mindfulness content. The core steps of the generation algorithm can be expressed as:

[0007] The virtual / augmented reality presentation module is developed using the Unity 3D engine and combined with the latest VR / AR hardware devices (such as Meta Quest 3) to provide users with an immersive mindfulness stress relief experience. The module dynamically constructs 3D scenes (such as forests, beaches, etc.) based on the generated mindfulness content, and enhances the user's sense of immersion through multi-sensory interaction technologies such as spatial audio and tactile feedback. The scene rendering uses real-time ray tracing technology to ensure the realism and smoothness of the picture. Compared with the prior art, the present invention has the following beneficial effects Preferably, the method 1 realizes real-time monitoring and accurate identification of emotional states, providing a scientific basis for personalized mindfulness intervention.

[0008] Preferably, in 2, the generative AI technology is used to realize the intelligent creation and dynamic matching of mindfulness content, which greatly improves the pertinence and diversity of the content.

[0009] Preferably, in embodiment 3, the use of virtual / augmented reality technology provides users with an immersive and interactive mindfulness stress reduction experience, significantly improving user engagement and practice effects.

[0010] Preferably, a closed-loop feedback is formed between the modules of the system, which continuously optimizes the user experience and effectively improves the long-term effect of mindfulness-based stress reduction.

[0011] Preferably, the system has good scalability and adaptability, and can be widely used in multiple fields such as mental health, stress management, and sleep improvement.

[0012] In summary, the present invention provides users with an efficient and personalized mindfulness-based stress reduction solution through the innovative integration of multimodal physiological signal analysis, artificial intelligence algorithms, and virtual reality technology, which is expected to play an important role in improving mental health levels and quality of life.

[0013] It should be understood that the above general description and the following detailed description are only exemplary and explanatory and cannot limit the present disclosure. In order to better understand and implement, the present disclosure is described in detail below in conjunction with the accompanying drawings.

[0014] The end of the hydraulic rod away from the small arm is rotatably connected with a fixed block, and the fixed block is installed in the middle of one side of the big arm.

[0015] Preferably, a joint motor is installed on one side of the rotating frame and the rotating joint, and the joint motor is used to drive the upper arm and the rotating joint to rotate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a structural block diagram of a personalized mindfulness-based stress reduction system based on multimodal physiological signals according to an embodiment of the present disclosure. Figure 2 A schematic diagram of the process of multimodal physiological signal collection and emotion recognition shown in one embodiment of the present disclosure. Figure 3 A schematic diagram of a personalized mindfulness content creation process based on generative AI, showing an embodiment of the present disclosure. Figure 4 The present invention is a schematic diagram of a virtual / augmented reality mindfulness stress reduction content presentation process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0018] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0019] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" / "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0020] The personalized mindfulness-based stress reduction system based on multimodal physiological signals proposed in this invention mainly consists of four core modules: multimodal physiological signal acquisition module, emotional state recognition module, personalized mindfulness content generation module and virtual / augmented reality presentation module. The overall architecture of the system is as follows: Figure 1 shown.

[0021] The multimodal physiological signal acquisition module uses advanced biosensor technology, including electroencephalogram (EEG) acquisition equipment, electrocardiogram (ECG) monitor, skin conductance (GSR) sensor and eye tracking equipment. Among them, the EEG acquisition equipment uses NeuroSky MindWave Mobile 2 EEG headset with a sampling rate of 512Hz, which can accurately capture α, β, γ, θ and δ brain waves; the ECG monitor uses Polar H10 heart rate belt with a sampling rate of 1000Hz, which has extremely high heart rate variability (HRV) measurement accuracy; the GSR sensor uses Empatica E4 wristband with a sampling rate of 4Hz, which can monitor the changes in skin conductance level in real time; the eye tracking device uses Tobii Pro Glasses 3 with a sampling rate of 100Hz, which can accurately capture pupil size and gaze point position. The emotional state recognition module uses a deep learning algorithm to perform real-time emotion recognition based on multimodal physiological signals. The core of this module is a multimodal fusion convolutional neural network (CNN) model that can simultaneously process EEG, ECG, GSR and eye movement data to output the user's current emotional state. The specific structure of the model includes: four parallel 1D-CNN sub-networks, which are used to process physiological signals of different modalities; an attention mechanism layer, which is used to learn the importance weights of signals of different modalities; a fully connected layer, which is used to fuse multimodal features; and a final Softmax classification layer, which is used to output the probability of emotion categories.

[0022] The personalized mindfulness content generation module generates customized mindfulness stress relief content based on the latest large language model, combined with the user's emotional state and personal preferences. This module uses prompt engineering technology to design a set of template libraries specifically for mindfulness content generation, including guided meditation, mindful breathing, body scans and other types. The system will dynamically adjust prompt parameters based on the user's emotional state and historical feedback to generate mindfulness content that best suits the current user's needs.

[0023] The virtual / augmented reality presentation module is developed using the Unity engine and supports a variety of VR / AR devices, such as OculusQuest 2, HTC Vive Pro 2, and Microsoft HoloLens 2. The module contains a rich 3D scene resource library, such as natural environments such as forests, beaches, and mountains, as well as indoor scenes such as meditation rooms and yoga studios. The system dynamically builds an immersive virtual environment based on the generated mindfulness content and user preferences, and provides stereo guidance through spatial audio technology to enhance the user's immersion and relaxation effect.

[0024] Multimodal physiological signals and emotional state mapping algorithm The mapping of multimodal physiological signals to emotional states is one of the core technologies of this system. We propose a multimodal fusion algorithm based on deep learning, which can effectively integrate different types of physiological signals and achieve high-precision emotion recognition. The mathematical model of the algorithm is as follows: set up Represents four different modal physiological signal inputs, where represents the EEG signal, represents the ECG signal, represents the GSR signal, Represents eye movement data. Each modality signal is first processed by its own 1D-CNN sub-network:

[0025] in, Indicates The convolutional neural network corresponding to each modality, is the extracted feature vector.

[0026] Next, we use the attention mechanism to learn the importance weights of features of different modalities:

[0027] in, and is a learnable parameter, Indicates The attention weights of each modality.

[0028] Then, we fuse the weighted feature vectors:

[0029] Finally, the fused feature vector passes through the fully connected layer and the Softmax layer to obtain the probability distribution of the emotion category:

[0030] in, and are the parameters of the fully connected layer, Indicates the emotion category.

[0031] In order to improve the generalization and robustness of the model, we use cross-modal attention mechanism and adversarial training technology. The cross-modal attention mechanism allows the model to interact with information between different modalities and capture the correlation between modalities:

[0032]

[0033] in, , and They are and The query, key, and value matrices of modality features, is the dimension of the key, is the attention weight matrix, is the updated feature representation.

[0034] Adversarial training increases the robustness of the model by adding small perturbations during training:

[0035] in, represents the model parameters, is the training data distribution, is the loss function, is the added disturbance, is the set of constraints on the perturbation.

[0036] Through the above algorithm, our system can achieve an emotion recognition accuracy of up to 95%, greatly improving the personalized matching effect of mindfulness stress relief content. In actual applications, the system will collect the user's multimodal physiological signals in real time, and after preprocessing and feature extraction, input them into the trained model to obtain the user's current emotional state, providing an important basis for the subsequent generation of personalized mindfulness content.

[0037] Personalized mindfulness content generation algorithm based on large language model This system uses a personalized mindfulness content generation algorithm based on a large language model, and provides users with a highly customized mindfulness stress relief experience through carefully designed prompt engineering, advanced text generation technology, and multiple rounds of content optimization. The core steps of the algorithm are as follows: Prompt Design We have developed a library of dynamic prompt templates that cover a variety of mindfulness practice types, including guided meditation, mindful breathing, body scans, etc. Each template contains the following key elements: a) Emotional state description: Select appropriate emotion description words based on the output of the emotion recognition module. b) User preference: Select the user's preferred topics, scenes, and guidance styles based on the user's historical data. c) Exercise type: Select the most suitable mindfulness exercise type based on the user's current emotional state and historical feedback. d) Duration setting: Set the appropriate exercise duration based on the user's available time and historical habits. e) Specific instructions: Contains specific instructions for generating structured, step-by-step mindfulness content.

[0038] The prompt template example is as follows: As a professional mindfulness meditation instructor, please create a [duration] minute [exercise type] guide for a user in [emotional state]. The user prefers [topic preference] topics and [guidance style]. Please create content according to the following structure: 1. Opening (30 seconds): Gently welcome the user and briefly introduce the exercise. 2. Body and Mind Adjustment (1 minute): Guide the user to adjust their posture and relax their body. 3. Main Exercise ([duration - 2] minutes): Describe the steps of [exercise type] in detail, using metaphors and imagery related to [topic preference]. 4. Closing (30 seconds): Gently guide the user to end the exercise and encourage a sense of calmness to be brought into their daily life. Please ensure that the content is progressive, the language is gentle and soothing, and appropriate prompts are included to guide the user to be aware of breathing and physical sensations.

[0039] Text Generation The text generation process uses a large language model combined with retrieval-augmented generation (RAG) technology to improve the quality and relevance of the generated content. The specific steps are as follows: a) Input processing: Input the designed prompt into the model. b) Knowledge retrieval: Use dense passage retrieval (DPR) to retrieve relevant information from a pre-built mindfulness knowledge base. c) Context fusion: Fuse the retrieved knowledge with the original prompt to form an enhanced context. d) Text generation: Use a large language model to generate mindfulness content based on the enhanced context.

[0040] The mathematical representation of the generation process is as follows:

[0041] in, is the input prompt, is the generated text sequence, is the retrieved knowledge, is the length of the generated text.

[0042] To improve the generation quality, we adopt a fine-tuning strategy based on reinforcement learning:

[0043] in, are model parameters, is the training data distribution, is a reward function used to evaluate the quality of generated content.

[0044] Content Optimization The initial content generated goes through multiple rounds of optimization to ensure its quality and personalization: a) Quality Assessment: Use a BERT-based text quality assessment model to score the coherence, relevance, and appropriateness of the generated content.

[0045]

[0046] in, is a scoring function based on BERT, is the generated text, is the original prompt.

[0047] b) Personalized adjustment: Based on user preferences and historical feedback, genetic algorithms are used to fine-tune the content.

[0048]

[0049] in, is a personalized scoring function based on the user model, and is the weight coefficient.

[0050] c) Speech synthesis: Use the latest neural network text-to-speech (TTS) technology to convert optimized text into natural and fluent voice guidance. We use a TTS model based on Tacotron 2 and WaveNet, which can generate speech with emotional color and rhythm:

[0051] in, is the TTS model, is the generated speech signal.

[0052] Application of virtual / augmented reality technology in presenting mindfulness-based stress reduction content This system uses advanced virtual reality (VR) and augmented reality (AR) technologies to provide users with an immersive mindfulness stress relief experience. The specific implementation includes the following aspects: Scene Construction We used the Unity engine to develop a set of highly realistic 3D scene libraries, including natural environments (such as forests, beaches, mountains) and indoor scenes (such as meditation rooms and yoga studios). The scene construction process is as follows: a) Terrain Generation: Use a procedural terrain generation algorithm to create natural terrain based on Fractal Brownian Motion (FBM):

[0053] in, is a function of terrain height, is the Perlin noise function, and are the amplitude and frequency parameters respectively.

[0054] b) Vegetation distribution: Use ecosystem simulation algorithms to simulate plant growth and distribution:

[0055] in, is the plant density, is the growth rate, is the environmental carrying capacity, is the mortality rate, is the diffusion coefficient.

[0056] c) Lighting rendering: Use global illumination techniques, such as path tracing and photon mapping, to achieve realistic lighting and shadow effects:

[0057] in, is the outgoing radiance, It is self-luminous. is the bidirectional reflectance distribution function (BRDF), is the incident radiance.

[0058] Interaction Design To enhance user immersion and engagement, we designed a series of natural and intuitive interaction methods: a) Gesture Recognition: Use deep learning models to achieve accurate gesture recognition, allowing users to control the virtual environment through gestures:

[0059] in, is the gesture category, is the input image sequence.

[0060] b) Breathing synchronization: By analyzing the user’s breathing pattern, elements in the virtual environment (such as light effects, water waves) are synchronized with the user’s breathing rhythm:

[0061] in, It's a breathing signal. is the amplitude, is the frequency, is the phase, is the baseline.

[0062] c) Eye tracking: Using eye tracking technology, the scene details and focus are dynamically adjusted according to the user's gaze point:

[0063] in, is the gaze point coordinate, Controls the width of the Gaussian distribution.

[0064] Multi-sensory integration To create a fully immersive experience, we integrated visual, auditory, and tactile feedback: a) Spatial audio: Use head-related transfer function (HRTF) technology to achieve 3D stereo effect:

[0065] in, It is HRTF. is the frequency, and are the azimuth and elevation of the sound source.

[0066] b) Haptic feedback: Provide subtle tactile stimulation through wearable devices to enhance the body awareness experience: in, It's a tactile signal. and are the vibration amplitude and frequency, and It is the tactile amplitude and frequency.

[0067] Through the comprehensive application of the above technologies, our system can provide users with a highly personalized and immersive mindfulness stress relief experience. Users can follow customized voice guidance in a realistic virtual environment and conduct in-depth mindfulness exercises, thereby effectively relieving stress and improving mental health.

[0068] Application Examples This section will demonstrate specific application examples of the personalized mindfulness-based stress reduction system based on multimodal physiological signals in different scenarios to fully illustrate the practicality and effectiveness of the system.

[0069] Example 1: Office Stress Relief In this example, we will demonstrate how the system can help an office worker who is facing work pressure to quickly relieve stress.

[0070] S101. Multimodal physiological signal acquisition: The user wears NeuroSky MindWave Mobile 2 EEG headset, Polar H10 heart rate belt and Empatica E4 wristband. The system starts to collect the user's electroencephalogram (EEG), electrocardiogram (ECG) and skin conductance (GSR) signals. The acquisition lasts for 30 seconds, with EEG sampling rate of 512Hz, ECG sampling rate of 1000Hz and GSR sampling rate of 4Hz.

[0071] S102. Emotional state recognition: After preprocessing, the collected physiological signals are input into the pre-trained multimodal fusion CNN model. The model analysis results show that the user is currently in a moderately anxious state, which is manifested as follows: •EEG: The energy of beta wave (14-30Hz) is significantly increased, and the energy of alpha wave (8-13Hz) is decreased •ECG: Heart rate variability (HRV) is reduced and RMSSD index is lower than normal •GSR: Increased skin conductance levels and increased frequency of fluctuations The probability distribution of emotional states output by the model is: anxiety (0.72), tension (0.18), calm (0.06), and other (0.04).

[0072] S103. Personalized mindfulness content generation: Based on the identified emotional state and user historical preferences, the system generates the following prompt: As a professional mindfulness meditation instructor, please create a 5-minute mindfulness breathing guide for an office worker who is in a moderately anxious state. The user prefers natural themes and a gentle and calm guidance method. Please create content according to the following structure: 1. Opening remarks (30 seconds): Gently welcome the user and briefly introduce this exercise. 2. Body and mind adjustment (1 minute): Guide the user to adjust their sitting posture and relax their body. 3. Main exercise (3 minutes): Describe the steps of mindfulness breathing in detail, using nature-related metaphors and images. 4. Closing remarks (30 seconds): Gently guide the user to end the exercise and encourage a sense of calmness to work. Please ensure that the content is progressive, the language is gentle and soothing, and appropriate prompts are included to guide the user to be aware of breathing and physical sensations.

[0073] The system inputs this prompt into the large language model and combines it with the retrieval-augmented generation (RAG) technology to retrieve relevant information from the mindfulness knowledge base. After the initial text is quality assessed and personalized, the following mindfulness content is finally generated: "Welcome to this brief moment of mindfulness. In the busy work, let us temporarily put aside all matters and give ourselves a quiet space. Please find a comfortable sitting position, which can be an office chair or a nearby sofa. Gently close your eyes and feel the contact point between your body and the seat.

[0074] Imagine you are sitting in a peaceful forest, surrounded by tall trees, with sunlight filtering through the leaves. Take a deep breath, as if you are breathing in the fresh forest air. As you exhale, let all the tension and anxiety leave your body with your breath.

[0075] Now, focus your attention on your breath. Feel the air entering your nose, flowing down your throat, and filling your lungs. Then, breathe it out slowly, like a breeze in the forest ruffling the leaves. You don't need to change the rhythm of your breath, just be aware of its natural flow.

[0076] If your mind wanders to work or something else, don't beat yourself up. Gently bring your attention back to your breath, just as water in a stream always returns to its channel.

[0077] Continue to be deeply aware of each breath, feeling the rise and fall of your chest and abdomen, just as the land in the forest rises and falls with the changing seasons. Let each breath become an anchor to return to the present moment.

[0078] Slowly, begin to expand your awareness. Feel the touch of your fingertips, hear the sounds of the office, smell the air. Gently move your fingers and toes, and slowly open your eyes.

[0079] Return to your work with that peace and focus. Remember, whenever you feel stressed, you can always return to your breath and find your inner forest of stillness." S104. Virtual / augmented reality presentation: Considering the limitations of the office environment, the system chooses to use augmented reality (AR) technology to present mindfulness content. The user puts on Microsoft HoloLens 2 smart glasses, and the system begins to build a virtual scene.

[0080] Scene construction: The system quickly builds a simple forest scene based on a procedural generation algorithm. The terrain uses the fractal Brownian motion (FBM) algorithm to generate an undulating surface:

[0081] Vegetation distribution uses an ecosystem simulation algorithm to simulate the growth and distribution of trees:

[0082] The lighting effect in the scene uses real-time global illumination technology and a path tracing algorithm to achieve a soft sunlight penetration effect: 2. Interaction design: The system uses the built-in gesture recognition function of HoloLens 2 to allow users to control the scene through simple gestures. For example, users can adjust the transparency of the scene to suit the office environment through a "pinch" gesture.

[0083] The Breath Sync feature identifies breathing patterns by analyzing the rise and fall of the user's chest and abdomen:

[0084] The system dynamically adjusts the swaying of leaves and changes in light and shadow in the scene based on the recognized breathing rhythm to enhance the sense of immersion.

[0085] Multi-sensory fusion: Considering the office environment, the system mainly relies on visual and auditory feedback. Spatial audio technology uses personalized HRTF to create a 3D stereo effect:

[0086] The forest environment sounds generated by the system, such as the rustling of leaves in the wind and the chirping of birds, are presented in this way, enhancing the user's sense of immersion.

[0087] The entire mindfulness stress reduction process lasts for 5 minutes. After the end, the system collects the user's physiological signals again, and the analysis results show: •EEG: Alpha wave power increased significantly, beta wave power decreased •ECG: Heart rate variability (HRV) improved, RMSSD index returned to normal range •GSR: skin conductance levels decrease and fluctuation frequency decreases The probability distribution of emotional states changed to: calm (0.68), focused (0.21), anxious (0.08), and other (0.03), indicating that the user's anxiety state has been significantly improved.

[0088] Example 2: Relaxation before bed In this example, we will demonstrate how the system can help a user who suffers from insomnia to relax before going to bed.

[0089] S201. Collecting multimodal physiological signals: The user lies on the bed, wearing the NeuroSky MindWave Mobile 2 EEG headset and the Empatica E4 wristband. Considering the sleeping posture, the heart rate monitor is not used. The system starts to collect the user's electroencephalogram (EEG), heart rate (through the PPG sensor of the E4 wristband), and skin conductance (GSR) signals. The collection lasts for 60 seconds, with an EEG sampling rate of 512Hz, and a heart rate and GSR sampling rate of 4Hz.

[0090] S202. Signal preprocessing: The original signal undergoes the following preprocessing steps: Bandpass filtering: Use a Butterworth filter to remove 50 Hz power supply interference and high-frequency noise from the EEG signal.

[0091]

[0092] in, is the cutoff frequency, is the filter order.

[0093] Independent component analysis (ICA): removes electrooculographic and electromyographic artifacts from EEG signals.

[0094]

[0095] in, is the observed signal, is the mixing matrix, is the source signal.

[0096] Wavelet denoising: denoise the heart rate and GSR signals.

[0097]

[0098] in, is the wavelet basis function, are the wavelet coefficients.

[0099] S203. Feature extraction: Extract the following features from the preprocessed signal: EEG characteristics: – Frequency band energy: δ (1-4Hz), θ (4-8Hz), α (8-13Hz), β (13-30Hz), γ (30-50Hz) – Power Spectral Density (PSD) – Sample entropy (SampEn) Heart rate features: – Average heart rate – Heart rate variability (SDNN, RMSSD) – Low frequency to high frequency ratio (LF / HF) GSR Features: – Average – Fluctuation frequency – Peak number Feature extraction uses time-frequency analysis methods such as short-time Fourier transform (STFT) and wavelet transform (WT):

[0100]

[0101] S204. Emotion Recognition: The extracted features are input into the pre-trained multimodal fusion CNN model. The model structure includes: Three parallel 1D-CNN sub-networks, processing EEG, heart rate and GSR features respectively; attention mechanism layer, learning importance weights of different modal features; fully connected layer, fusing multi-modal features; Softmax classification layer, outputting emotion category probabilities The forward propagation process of the model is as follows:

[0102]

[0103]

[0104]

[0105] S205. Output emotional state: The model analysis results show that the user is currently in a state of mild anxiety and fatigue, as shown in the following: •EEG: Theta and beta waves are high in energy, alpha waves are low in energy •Heart rate: Low heart rate variability (HRV) and increased LF / HF ratio •GSR: higher skin conductance levels and increased frequency of fluctuations The probability distribution of emotional states output by the model is: anxiety (0.45), fatigue (0.35), tension (0.15), and other (0.05).

[0106] Based on this information, the system generates a personalized mindfulness content for relaxation before bedtime and presents it in audio format. After listening to the 15-minute guidance, the user again collects and analyzes physiological signals, and the results show: •EEG: Significant increase in alpha wave power, decrease in theta and beta wave power •Heart rate: Heart rate decreases, HRV increases, LF / HF ratio decreases •GSR: skin conductance levels decrease and fluctuation frequency decreases The new emotional state probability distribution is: calm (0.62), relaxed (0.28), anxious (0.07), and other (0.03), indicating that the user's anxiety state has been greatly alleviated and it is easier to fall asleep.

[0107] Through these two examples, we can see the application effect of the personalized mindfulness stress reduction system based on multimodal physiological signals in different scenarios. The system can accurately identify the user's emotional state, generate targeted mindfulness content, and present it through appropriate technical means, effectively helping users relieve stress and improve their emotional state.

Claims

1. A personalized mindfulness stress relief system based on multimodal physiological signals, characterized in that: It includes a multimodal physiological signal acquisition module, an emotional state recognition module, a personalized mindfulness content generation module and a virtual / augmented reality presentation module; wherein the multimodal physiological signal acquisition module adopts advanced biosensor technology to simultaneously acquire multiple physiological signals of the user, such as electrocardiogram, skin electricity, and electroencephalogram. The module adopts high-precision, low-power MEMS sensors combined with flexible electronic technology to achieve a comfortable and portable wearable design. The collected raw signals are subjected to pre-processing steps such as low-pass filtering and band-pass filtering to remove noise and baseline drift, thereby improving signal quality. The personalized mindfulness decompression system based on multimodal physiological signals is characterized in that the multimodal physiological signal acquisition module includes an electroencephalogram acquisition device, an electrocardiogram monitor, a skin conductance sensor, and an eye tracking device; wherein the electroencephalogram acquisition device uses NeuroSky MindWave Mobile 2 EEG headset with a sampling rate of 512Hz, which can accurately capture α, β, γ, θ and δ brain waves; the electrocardiogram monitor uses Polar H10 The heart rate belt has a sampling rate of 1000Hz and has extremely high heart rate variability measurement accuracy. The skin conductance sensor uses the Empatica E4 wristband with a sampling rate of 4Hz, which can monitor changes in skin conductance levels in real time. The eye tracking device uses Tobii Pro Glasses3 with a sampling rate of 100Hz, which can accurately capture pupil size and gaze point position.

2. The emotional state recognition module in the personalized mindfulness stress reduction system based on multimodal physiological signals according to claim 2 is characterized in that: It uses a deep learning algorithm to perform real-time emotion recognition based on multimodal physiological signals. The core of this module is a multimodal fusion convolutional neural network model that can simultaneously process EEG, ECG, skin conductance and eye movement data to output the user's current emotional state. The model includes four parallel 1D-CNN sub-networks, an attention mechanism layer, a fully connected layer and a Softmax classification layer.

3. The emotional state recognition module according to claim 3, characterized in that: The mapping of multimodal physiological signals to emotional states uses a multimodal fusion algorithm based on deep learning. Represents the physiological signal input of four different modalities. Each modal signal is first processed by its own 1D-CNN sub-network, and then the attention mechanism is used to learn the importance weights of different modal features. The weighted feature vectors are fused, and finally the probability distribution of emotion categories is obtained through the fully connected layer and the Softmax layer.

4. The personalized mindfulness content generation module in the personalized mindfulness stress reduction system based on multimodal physiological signals according to claim 4, characterized in that: Based on the large language model, combined with the user's emotional state and personal preferences, customized mindfulness stress relief content is generated. This module uses prompt engineering technology to design a dynamic prompt template library, covering guided meditation, mindful breathing, body scans and other mindfulness practice types. The system dynamically adjusts prompt parameters based on the user's emotional state and historical feedback.

5. The personalized mindfulness content generation module according to claim 5, characterized in that: The text generation process uses a large language model combined with retrieval-enhanced generation technology to improve the quality and relevance of the generated content. The specific steps include input processing, knowledge retrieval, context fusion and text generation. The generation process adopts a fine-tuning strategy based on reinforcement learning. The initial generated content undergoes multiple rounds of optimization such as quality assessment, personalization adjustment and speech synthesis.

6. The virtual / augmented reality presentation module in the personalized mindfulness stress reduction system based on multimodal physiological signals according to claim 6, characterized in that: Developed using the Unity engine, it supports a variety of VR / AR devices and contains a rich library of 3D scene resources, such as natural environments and indoor scenes. The system dynamically builds an immersive virtual environment based on the generated mindfulness content and user preferences, and enhances the user's sense of immersion and relaxation through multi-sensory interaction technologies such as spatial audio and tactile feedback.

7. The virtual / augmented reality presentation module according to claim 7, characterized in that: Scene construction uses procedural terrain generation algorithms, ecosystem simulation algorithms and global illumination technology to create realistic 3D scenes. Interaction design uses gesture recognition, breathing synchronization and eye tracking to enhance user immersion and participation. Multi-sensory fusion integrates visual, auditory and tactile feedback to create a full range of immersive experience for users.

8. A personalized mindfulness-based stress reduction method based on multimodal physiological signals according to claim 8, characterized in that: The system includes steps such as multimodal physiological signal acquisition, emotional state recognition, personalized mindfulness content generation and virtual / augmented reality presentation. The system collects the user's multimodal physiological signals in real time, and inputs them into the trained model after preprocessing and feature extraction to obtain the user's current emotional state, providing an important basis for the subsequent generation of personalized mindfulness content. Customized mindfulness content is then generated based on the user's emotional state and personal preferences, and an immersive mindfulness stress relief experience is provided to the user through the virtual / augmented reality presentation module. A closed-loop feedback is formed between the modules to continuously optimize the user experience.

9. The personalized mindfulness stress reduction method based on multimodal physiological signals according to claim 9, characterized in that: In the process of multimodal physiological signal acquisition, the specific device described in claim 2 is used for signal acquisition; in the process of emotional state recognition, the model and algorithm described in claims 3 and 4 are used for emotion recognition; in the process of personalized mindfulness content generation, the modules and technologies described in claims 5 and 6 are used for content generation; in the process of virtual / augmented reality presentation, the modules and technologies described in claims 7 and 8 are used for content presentation, and the entire system can achieve an emotion recognition accuracy rate of up to 95%, and effectively improve the long-term effect of mindfulness stress reduction.

Images