Psychotherapy and healing robot based on high human emotion fitting degree simulation analysis
Through the architectural design of multimodal perception layer, dynamic decision-making layer and brain science verification layer, and the integration of emotional computing and brain science feedback technology, the geographical limitations and human resource shortage problems of traditional psychotherapy are solved, personalized psychological intervention and effect quantification are achieved, and it is suitable for education and clinical treatment scenarios.
Patent Information
- Application Number
- CN202510811337.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-09-16
- Estimated Expiration
- 2045-06-17
AI Technical Summary
Traditional psychotherapy is subject to geographical limitations and human resource shortages. Existing AI psychological products have defects in emotion recognition, intervention strategies and effect quantification, and cannot meet mental health needs.
It adopts an architectural design of multimodal perception layer, dynamic decision-making layer, generative interaction layer and brain science verification layer, integrates emotional computing, dynamic intervention and brain science feedback technology to form a closed-loop psychological healing system, and realizes personalized psychological intervention and effect quantification through multi-type sensor data collection, reinforcement learning strategy and EEG monitoring.
It can accurately identify complex emotional states, provide personalized psychological intervention, and quantify the intervention effect through EEG monitoring. It is suitable for scenarios such as education and clinical treatment.
Smart Images

Figure CN120656649A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the interdisciplinary field of artificial intelligence and clinical psychology, and specifically to an intelligent psychological healing system that integrates multimodal emotional computing, a dynamic intervention engine, and brain science feedback technology, thereby forming a psychological healing robot based on simulation analysis of high human emotional fit. Background Art
[0002] Current mental health services face two major pain points: 1. Traditional psychotherapy is subject to geographical limitations and human resource shortages; 2. Existing AI psychology products have three major flaws: (1) Emotion recognition relies on a single text modality (e.g., using only the PHQ-9 scale), resulting in a 37% missed detection rate for complex states such as "smiling depression"; (2) The intervention strategy is mechanical (e.g., fixed script library), which cannot achieve the dynamic cognitive reconstruction required for CBT treatment; (3) The lack of means to verify brain function makes it difficult to quantitatively evaluate the effects of interventions.
[0003] Therefore, existing mental health service models and technical means urgently need to be innovated to meet the growing mental health needs. Summary of the Invention
[0004] The purpose of the present invention is to provide a psychological healing robot based on high human emotional fit simulation analysis. The system uses a four-layer architecture design to address the shortcomings of existing technologies and provide a precise and personalized psychological intervention decision-making closed loop. At the same time, it solves the geographical limitations and human resource shortages of traditional psychotherapy, and overcomes the shortcomings of existing AI psychological products in emotion recognition, intervention strategies and effect quantification. The system integrates emotional computing, dynamic intervention and brain science feedback technologies through the architectural design of a multimodal perception layer, a dynamic decision-making layer, a generative interaction layer and a brain science verification layer to form a closed-loop psychological healing system. The system can accurately identify complex emotional states, provide personalized psychological intervention, and quantify the intervention effect through EEG monitoring. It is suitable for multiple scenarios such as education and clinical treatment.
[0005] A first aspect of the present invention is to provide a psychological healing robot based on high human emotional fit simulation analysis, comprising: A multimodal perception layer for collecting physiological, motion, and environmental interaction data; wherein the multimodal perception layer includes: a heterogeneous data acquisition module for integrating multiple types of sensor data on physiological, motion, and environmental interactions of smart terminals; a spatiotemporal feature extraction network for processing synchronized voice and video data streams based on a spatiotemporal feature extraction network 3D-CNN to obtain emotion calculation values corresponding to spatiotemporal features; and an attention fusion mechanism module for dynamically adjusting the weights of each modality; A dynamic decision layer, configured to generate an intervention strategy based on the interaction data using a reinforcement learning strategy engine and a hierarchical intervention selection tree; wherein the dynamic decision layer includes: a reinforcement learning strategy engine for establishing a state space and an action space and designing a reward function; and a hierarchical intervention selection tree for triggering corresponding intervention protocols according to crisis levels and generating an intervention strategy based on the reward function; A generative interaction layer for generating an empathic response that complies with ethical standards based on the intervention strategy; wherein the generative interaction layer includes: an ethical constraint system for applying ethical constraints based on an ethical rule library composed of a taboo content library, knowledge boundary constraints, a privacy protection mechanism, and an ethical knowledge graph; and an empathic response generator for generating an empathic response based on a constraint decoding technique, applying the ethical constraints to the empathic response to generate an empathic response that complies with ethical standards; A brain science verification layer is used to monitor neural feedback in real time through EEG and adjust the intervention strategy; wherein, the brain science verification layer includes: a neural feedback adjustment module, which is used to monitor the changes in the power of the alpha waves in the prefrontal lobe through EEG in real time, and use the changes in the power of the alpha waves in the prefrontal lobe as neural feedback; and a multimodal feedback design module, which is used to map the neural feedback into virtual scene parameters based on the neural feedback adjustment mechanism, and dynamically adjust the intervention strategy based on the virtual scene parameters.
[0006] Preferably, the heterogeneous data acquisition module adopts 4 types of physiological monitoring sensors, 3 types of motion sensing sensors and 2 types of environmental interaction sensors; wherein, the 4 types of physiological monitoring sensors include heart rate variability PPG, galvanic skin response GSR, blood oxygen saturation SpO2 and body temperature sensors, the 3 types of motion sensing sensors include three-axis accelerometers for gait analysis, gyroscopes for posture recognition and barometers for spatial positioning, and the 2 types of environmental interaction sensors include microphones for voice emotion analysis and ambient light sensors for circadian rhythm monitoring; the heterogeneous data acquisition module adopts a layered acquisition architecture, which sequentially goes from the sensor hardware layer, Bluetooth 5.3 or BLE protocol stack, edge computing node, data preprocessing unit and encrypted transmission to the cloud, realizes raw data denoising through edge nodes, and is compatible with Android Health Connect and Apple by establishing a unified data description framework. The HealthKit protocol realizes the spatiotemporal alignment of multi-device data and adopts a dynamic interpolation algorithm to balance the differences in sampling rates of different sensors. The heterogeneous data acquisition module is also used for multimodal transmission and synchronization and preliminary fusion of multimodal data. The multimodal transmission and synchronization are implemented based on a low-power transmission protocol and a spatiotemporal synchronization mechanism. The preliminary fusion of multimodal data is implemented based on a feature-level fusion strategy. The spatiotemporal features are extracted by inputting the original signal into a 3D-CNN network suitable for voice + video synchronization analysis. The importance score of each modality is calculated as an energy value through a learnable feedforward neural network, and then the importance score is converted into a weight through a softmax function.
[0007] Preferably, the processing of synchronous voice and video data streams based on a spatiotemporal feature extraction network 3D-CNN includes: A 3D-CNN architecture for spatiotemporal feature extraction network is established, wherein the 3D-CNN architecture for spatiotemporal feature extraction network includes: a data input and preprocessing layer, wherein the data format input to the data input and preprocessing layer is to construct a three-dimensional input tensor of synchronously collected time series speech waveforms and spatial RGB data video frames into a [T×H×W×C] (time×height×width×channel) dimension; a 3D convolution kernel, including using a three-dimensional convolution kernel to synchronously slide on the time axis of the speech frame sequence and the spatial axis of the video frame to capture the spatiotemporal correlation between speech fundamental frequency jitter and facial micro-expressions; a spatiotemporal feature output layer, which is used to gradually extract features through a 3D convolution+pooling structure based on hierarchical stacking and spatiotemporal correlation, and then output a spatiotemporal feature map in the last layer of the spatiotemporal feature output layer; a target optimization layer: feature optimization based on a loss function, wherein the loss function is determined based on the fusion of action localization error and emotion classification cross entropy loss; Based on the spatiotemporal feature extraction network 3D-CNN, the synchronous voice and video data streams are processed to obtain the initial emotional value; Based on the initial sentiment value, sentiment calculation is performed using a contradiction index analysis algorithm to obtain a sentiment calculation value, wherein the contradiction index analysis algorithm includes: (1) Standardize the optimized features, including: Z-Score standardization of text sentiment polarity and speech sentiment intensity to eliminate dimensional differences; (2) Construct a contradiction index model and conduct judgment and arbitration of abnormal and contradictory data: The calculation formula of the contradiction index C in the contradiction index model is shown in the following formula (1):
[0008] (1);
[0009] (2)
[0010] Where T text is the normalized text sentiment polarity, BERT_output is the text sentiment polarity output by BERT, μ text is the standard text sentiment polarity; T voice is the normalized speech emotion intensity, Pitch_Jitter is the speech emotion intensity represented by the fundamental frequency jitter rate, μ text is the standard speech emotion intensity; σ text is the environmental vector modulation factor; σ voice is the sensor state attenuation factor; when the contradiction index C>2.5, the difference between the two modes exceeds 2.5 times the standard deviation, triggering the manual review process, including: (A) Prioritize the use of physiological sensor data; (B) If physiological data are missing, a manual review process is initiated; (3) Determining physiological arousal, including: calculating and grading the physiological stress index through heart rate variability and skin conductance, wherein the calculation formula of the physiological stress index A is as follows:
[0011]
[0012] The classification is low A<0.5, medium 0.5≤A≤1.5 and high A>1.5; among them, HRV LF / HF Indicates lower or higher values of heart rate variability, GSR Δ represents skin conductance.
[0013] Preferably, the weights of each modality are dynamically adjusted based on an attention mechanism or a cross-modal attention mechanism based on a Transformer; wherein the formula for dynamically adjusting the weights of each modality based on the attention mechanism is shown in the following formula (3):
[0014] (3); Among them, W i Represents the weight value of each mode. There are N modes in total. e f(xj) is the attention score of each modality; N modalities correspond to N sensors or feature extractors, and the feature vector extracted by each modality is f(x j ), j=1,2,...,N; the goal is to obtain the weight of each modality through the attention mechanism, and then calculate the weighted feature vector as the final representation; The formula for dynamically adjusting the weights of each modality using the Transformer-based cross-modal attention mechanism is shown in the following formula (4): (4) Among them, Q is the query matrix (Query) of the current emotional state, K i is the key matrix (Key) of each modal eigenvector; d k is the dimension of the key vector, that is, the length of the key vector in each header; Dynamically adjusting the weights of each modality includes: (1) Feature projection: mapping multimodal features into the same latent space; (2) Perform attention aggregation: Calculate weighted fusion features for downstream sentiment classification; (3) Introduce a spatiotemporal attention gating mechanism to suppress the weight of features in time periods with low signal-to-noise ratio.
[0015] Preferably, the establishing the state space includes: (1) Calculating the emotional intensity based on the emotional calculation value, wherein the emotional intensity is represented by a 0-3 level quantitative index, wherein the 0-3 level quantitative index includes 0 for calmness, 1 for mild anxiety, 2 for moderate anxiety, and 3 for severe anxiety; (2) Identifying cognitive distortion types, including: identifying thinking traps in user input using a thinking trap classifier based on natural language processing, identifying 12 types of cognitive distortions in user input and mapping them to 12 predefined labels; Among them, identifying cognitive distortion types includes: (A) Identify a thinking trap classification system to identify 12 predefined labels, including: all or nothing, overgeneralization, mental filtering, negating the positive, jumping to conclusions, exaggeration or minimization, emotional reasoning, should statements, labeling, personalization, catastrophizing, and mind reading; (B) Acquiring key psycholinguistic features based on feature engineering, including extreme vocabulary analysis, catastrophizing vocabulary patterns, cognitive distortion keywords, sentiment polarity intensity, and absolute sentence detection; (C) Build a deep learning model, BiLSTM+Attention, which includes a text input layer, an embedding layer, a bidirectional LSTM, an attention mechanism, cognitive feature input, and a multi-label classification output. (D) A rule-enhancing engine was established based on post-prediction processing of cognitive psychology rules, including: Rule 1: Confidence is enhanced when the prediction contains "never / always" and is all-or-nothing; Rule 2: The "if...it's over" pattern enhances the catastrophizing label; Rule 3: The second-person "you should" enhances the "should" statement label; and, Rule 4: Psychological verbs + thought assertions enhance the "mind-reading" label; (E) Identifying cognitive distortion types, including: text preprocessing including word segmentation and lemmatization, feature extraction, text serialization, model prediction, rule enhancement, result parsing based on confidence threshold, interpretability analysis, creating visualization models, obtaining attention weights, and generating heatmaps; The establishing of the action space includes establishing an intervention strategy library corresponding to the action space, wherein the intervention strategy library includes six core actions: CBT thought recording, mindfulness breathing training, virtual exposure therapy, medication dosage adjustment, crisis referral, and no intervention, each of which corresponds to different resource consumption and expected efficacy; The design reward function includes: Designing a short-term reward function The short-term reward function is characterized by the user's real-time emotional decline ΔE and the weekly change rate ΔS of the PHQ-9 / GAD-7 scale score, where the user's real-time emotional decline ΔEt E is the user's real-time emotion evaluation value at time t t and the user's real-time emotion evaluation value E at time t-1 t-1 The difference is the decline in sentiment , negative values indicate improved mood, the short-term reward function The reward formula is expressed as: ; Where α and β represent the weight coefficients of the user's real-time emotion decrease ΔE and the weekly change rate of the PHQ-9 / GAD-7 scale score, respectively, which are determined through expert experience or optimization; γ represents the user's compliance reward. For example, if the intervention action is completed, Compliance is positive and points are added; if the intervention action is not completed, Compliance is negative and points are deducted; Design a long-term reward function, which is a discounted cumulative reward based on the Q-learning objective and is expressed as: ; Where k represents the single-day intervention duration of the reward function; The design constraint was that the duration of the intervention per day should be ≤45 minutes to prevent cognitive overload.
[0016] Preferably, the method for triggering the corresponding intervention protocol according to the crisis level is triggered based on the crisis level determination rule, wherein the crisis level corresponds to high risk, medium risk and low risk, and the determination rule is PHQ-9 ≥ 20 under the logical OR or mention of suicidal ideation, emotional intensity lasting ≥ level 2 for 72 hours and a single emotional peak ≥ level 1.5. The triggered corresponding intervention protocol is immediate referral for offline diagnosis and treatment + 24-hour AI monitoring, CBT twice a day + mindfulness reinforcement and push relaxation audio + breathing guidance; The intervention strategy generated based on the reward function includes the user's real-time emotion reduction ΔE obtained by the short-term reward function based on the Q-learning algorithm and the state transition probability. t The weekly change rates of the scores on the PHQ-9 / GAD-7 scales were evaluated separately, and intervention strategies were generated based on the evaluation results.
[0017] Preferably, the user's real-time emotion decrease ΔE and the weekly change rate of the PHQ-9 / GAD-7 scale score obtained by the short-term reward function based on the Q-learning algorithm and state transition probability are evaluated respectively. The process of generating an intervention strategy based on the evaluation results includes a data modeling stage, a Q-learning algorithm implementation stage, and a strategy generation stage; wherein: The data modeling stage includes defining the state space, defining the action space and building a state transition probability model; the definition of the state space includes defining state variables and defining state representation, and the definition of state variables includes: user real-time emotion value E t ; Historical PHQ-9 / GAD-7 scores and ; User demographic characteristics and environmental context Context; Define the state as: The action space is defined to include determining intervention actions including: a1: push mindfulness meditation audio; a2: cognitive behavioral therapy CBT practice; a3: emergency manual consultation; and a0: no intervention, silent observation; the state transition probability model is constructed to predict the execution action a t Post-state s t →s t+1 The method of constructing the state transition probability model includes: (1) using historical data to train the probability model: ;and (2) the model selects hidden Markov model (HMM) or Bayesian network to dynamically update the probability distribution; The Q-learning algorithm implementation stage includes determining Q-table update rules and determining state transition probabilities, and processing uncertainty through probabilistic Q-learning updates to obtain the state transition probabilities;
[0018] The strategy generation stage includes establishing a dual-index evaluation system and generating dynamic strategies. The dual-index evaluation system includes the index sentiment drop ΔE t Using statistical daily ΔE t The mean / variance of the action is used to evaluate the effectiveness of the action. For the weekly change rate of the indicator scale, a linear regression analysis is used to determine whether the slope is significantly negative. The dynamic strategy has safety limits and fatigue control. The safety limit is when ΔE t < threshold, manual intervention is forced to be triggered; the fatigue control is that the same action is not pushed repeatedly within 24 hours.
[0019] Preferably, the ethical constraint system includes: (1) Taboo content library: establish a dynamically updated blacklist that includes suicide, violence, and / or discrimination sensitive words, and covers ICD-11 psychological crisis entries; (2) Knowledge boundary constraints: the generation of medical diagnosis recommendations is prohibited, and only general strategies for psychological support are allowed; (3) Privacy protection mechanism: using regular expressions and NER combined recognition to automatically fuzzify the PII information mentioned by users; (4) Ethical knowledge graph, based on mermaid code and graph LR, implements the following architecture: A[User Statement]->B{Ethical Review Node} B->|Safety| C[Generate Empathic Response] B->|Risk| D[Trigger standard dialogue] D->E["Response according to WHO Mental Health Guidelines 2.3"]; Generating an empathic response based on the constraint decoding technique includes: (1) Determine that the model architecture corresponding to the constrained decoding technology is the Llama-3-8B base model, and train the model corresponding to the constrained decoding technology based on the psychological counseling dialogue dataset; (2) Injecting dynamic constraints into the model corresponding to the constraint decoding technology to generate empathic responses. The dynamic constraints include: A. Emotional state adaptation constraint, which is used to adjust the generated temperature parameters according to the user's current emotional intensity; B. Multi-expert voting mechanism constraints, where the expert models corresponding to the multi-expert voting mechanism constraints include: ethics review model, crisis identification model, and emotional support assessment model; C. Empathy enhancement strategy constraints, including: psycholinguistic feature embedding and non-verbal symbol integration, including: The psycholinguistic feature embedding includes two-level typical psycholinguistic embedding and cross-cultural emotional metaphor mapping library embedding; the two-level typical psycholinguistic embedding includes lexical level and syntactic level, the lexical level includes improving the probability of words in the empathy dictionary during the decoding stage, and the syntactic level includes using question sentences accounting for more than 30% to promote user self-disclosure; the cross-cultural emotional metaphor mapping library embedding includes embedding a database formed by 327 localized expressions and Japanese onomatopoeia expressions; The non-verbal symbol fusion includes: emotion embedding combined with speech synthesis engine; virtual digital human facial expression synchronization; and keyboard tapping interval analysis; The method for verifying the ethical compliance of an empathic response generated by applying the ethical constraints to the empathic response is as follows: (1) Ethical compliance verification using an automated testing framework, including: Form an adversarial test set: Contains 500 high-risk induced inputs to verify whether the system triggers the standard crisis protocol; Determine ethical deviation index: D ethics =Number of illegal responses / total number of test samples × 100%; (2) Ethical compliance verification based on human supervision mechanisms, including: Set up a double-blind review process: every 1,000 generated responses are independently reviewed by three certified psychological counselors; Dynamic update mechanism: The binding rule library is updated monthly based on the newly promulgated ethical guidelines.
[0020] Preferably, the neural feedback regulation module includes: (1) A signal acquisition system comprising: high-density EEG electrodes and a real-time signal acquisition optimization unit; wherein the high-density EEG electrodes include an international 10-20 standard lead system and a flexible electrode array; the real-time signal acquisition optimization unit optimizes the acquisition of real-time signals based on a combination of differential amplification technology, a 120 dB common mode rejection ratio, and a bandpass filter; (2) A real-time analysis system for alpha wave power, comprising: a preprocessing and feature extraction unit and a reference power calibration unit; the preprocessing and feature extraction unit is used to remove physiological artifacts through independent component analysis and retain pure alpha wave signals; and the alpha wave power spectrum density within a 0.5 second time window is calculated through short-time Fourier transform and a dynamic change curve is output; the reference power calibration unit is used to establish a personalized baseline, and the process of establishing the personalized baseline includes: continuous monitoring for 3 minutes when the user is in a relaxed state with their eyes closed, and calculating the average alpha wave power as the value of the personalized baseline to quantify changes in neural activity.
[0021] Preferably, mapping the neural feedback into virtual scene parameters based on the neural feedback regulation mechanism includes: Map the alpha wave power to virtual scene parameters through visual feedback; generate sound waves synchronized with the alpha waves through auditory feedback and transmit them through bone conduction headphones; The dynamic adjustment of the intervention strategy based on the virtual scene parameters includes: Perform stepped difficulty control based on the virtual scene parameters to dynamically adjust the intervention strategy; or Perform closed-loop stimulation based on the virtual scene parameters to dynamically adjust the intervention strategy.
[0022] Advantages of the system of the present invention:
[0023] 1. Based on the keyboard tapping interval analysis algorithm (> 1.2 seconds to trigger cognitive fatigue warning), improve the recognition efficiency and accuracy.
[0024] 2. Build a cross-cultural emotional metaphor mapping library (a database formed by 327 local expressions such as "feeling a heavy weight on the heart" in Chinese and expressions such as Japanese onomatopoeia).
[0025] 3. Propose a multi-modal arbitration mechanism driven by the contradiction index (solve the scenario judgment of "crying while smiling").
[0026] 4. Deeply integrate constraint decoding with psychological theories to achieve double compliance of technology and ethics.
[0027] 5. Through precise dynamic monitoring and closed-loop regulation of alpha waves, achieve targeted intervention of neuroplasticity. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the related art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0029] Figure 1 Is the system architecture diagram provided according to the embodiment of the present invention; Figure 2 Is the schematic diagram of the hierarchical intervention decision tree provided according to the embodiment of the present invention; Figure 3 Is the multi-modal data fusion flow chart provided according to the embodiment of the present invention; Figure 4 Is the electroencephalogram feedback regulation schematic diagram provided according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0030] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0031] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0032] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.
[0033] See also Figure 1 This embodiment provides a psychological healing robot based on high human emotional fit simulation analysis, including: Multimodal perception layer for collecting physiological, motion, and environmental interaction data; a dynamic decision layer for generating an intervention strategy based on the interaction data through a reinforcement learning strategy engine and a hierarchical intervention selection tree; a generative interaction layer for generating ethically sound empathic responses based on the intervention strategy; and A brain science validation layer is used to monitor neurofeedback in real time via EEG and adjust the intervention strategy.
[0034] As a preferred embodiment, the multimodal perception layer includes: Heterogeneous data acquisition module, used to integrate multiple sensor data on physiological, motion, and environmental interactions of smart terminals; In this embodiment, the working principle of the heterogeneous data acquisition module includes: 1. Sensor classification and data collection architecture: (1) Data source identification Physiological monitoring (4 categories): heart rate variability (PPG), galvanic skin response (GSR), blood oxygen saturation (SpO2) and body temperature sensors; Motion sensing (3 categories): three-axis accelerometer (gait analysis), gyroscope (posture recognition) and barometer (spatial positioning); Environmental interaction category (2 categories): microphone (speech emotion analysis) and ambient light sensor (circadian rhythm monitoring).
[0035] (2) Layered Collection Architecture A[Sensor hardware layer]->B(Bluetooth 5.3 / BLE protocol stack); B->C{Edge computing node}; C->D[Data preprocessing: noise reduction / normalization]; D->E[Encrypted transmission to the cloud].
[0036] Denoising of raw data through edge nodes (e.g., Kalman filtering to eliminate motion artifacts) In this embodiment, the multiple types of sensor data are 9 types of sensors, including: (A) Heart Rate Variability (HRV) Sensor: Type: Photoplethysmography (PPG) sensor Function: Monitor the state of the autonomic nervous system (stress / relaxation level) Indicators: RMSSD (heart rate variability), HR (real-time heart rate) (B) Galvanic skin response (GSR) sensor Type: Dry electrode bioelectric sensor Function: Measure skin conductance level (emotional arousal) Application: Quantitative assessment of anxiety level (C) Facial expression recognition camera Type: Near-infrared 3D depth camera Algorithm: Run FER (Facial Expression Recognition) model in real time Identify: six basic emotions (happy / sad / angry / surprise / fear / disgust) (D) Voice Sentiment Analysis Microphone Type: Directional MEMS microphone array Analysis dimensions: intonation, speaking speed, pause frequency, and spectral characteristics Output: Emotional tendency value (-1 anger ~ +1 happiness) (E) Eye tracking sensor Type: Infrared corneal reflection eye tracker Parameters: pupil diameter (cognitive load), fixation duration (attention), blink frequency (stress level) (F) Breathing pattern monitor Type: Piezoelectric breathing belt / millimeter wave radar Characteristics: breathing depth, breathing rate, and respiratory rhythm disorder Association: Early warning of anxiety attacks (>20 times / minute) (G) Body motion sensor Type: 9-axis IMU (accelerometer + gyroscope + magnetometer) Behavior Recognition: Curled-up posture (depressive tendencies) Restlessness (a sign of anxiety) Tremor amplitude (stress response) (H) Contact pressure sensor Type: Capacitive touch sensor array (integrated on the robot surface) Function: A. Hug strength test (emotional demand intensity) B. Touch duration analysis (social avoidance tendency) (I) Environmental Context Sensors Composite module: illumination + ambient noise + temperature and humidity Function: Correct environmental interference of physiological signals For example, pupil constriction caused by bright light is not misinterpreted as a stress response.
[0037] 2. Standardization of heterogeneous data Protocol conversion: Establish a unified sensor description framework (USDF), compatible with Android Health Connect and Apple HealthKit protocols, to achieve spatiotemporal alignment of multi-device data; Sampling rate adaptation: A dynamic interpolation algorithm is used to balance the sampling rate differences of different sensors (such as synchronizing 1Hz body temperature data with 25Hz acceleration data).
[0038] 3. Multimodal transmission and synchronization (1) Low-power transmission protocol: The improved BLE protocol enables concurrent transmission of multiple devices, and bandwidth utilization is increased to 92% (37% increase compared to traditional Bluetooth 4.2); QoS priority is set, physiological data > motion data > environmental data.
[0039] (2) Space-time synchronization mechanism: The hardware level uses the PPS signal of the GPS module to achieve microsecond time synchronization; the software level uses the NTP protocol to compensate for network delays, and the error is controlled within ±15ms.
[0040] like Figure 3 As shown, 4. Preliminary fusion of multimodal data Feature-level fusion strategy: early fusion, inputting the original signal into the 3D-CNN network to extract spatiotemporal features (suitable for simultaneous speech + video analysis); the basic idea of the attention mechanism is: calculating the importance score (energy value) of each modality through a learnable network (usually a feedforward neural network), and then converting the importance score into a weight through the softmax function.
[0041] 5. Optimization of steps 1-4 By defining a common metadata template, it supports parsing of 327 device data formats (covering 95% of existing technology terminals); adopts dual-channel redundant transmission: primary channel (Wi-Fi) + backup channel (LoRaWAN); uses a dynamic sensor scheduling algorithm to shut down non-essential sensors according to user status (such as disabling GPS during sleep).
[0042] A spatiotemporal feature extraction network is used to process the synchronous voice and video data streams based on the spatiotemporal feature extraction network 3D-CNN to obtain the emotion calculation value corresponding to the spatiotemporal features; In this embodiment, the processing of the synchronous voice and video data stream based on the spatiotemporal feature extraction network 3D-CNN includes: 1. Establish a spatiotemporal feature extraction network 3D-CNN architecture, wherein the spatiotemporal feature extraction network 3D-CNN architecture includes: Data input and preprocessing layer, where the data format input to the data input and preprocessing layer is to construct the synchronously collected speech waveform (time series) and video frame (spatial RGB data) into a three-dimensional input tensor with dimensions [T×H×W×C] (time×height×width×channel); 3D convolution kernel, including the use of a three-dimensional convolution kernel, synchronously sliding on the time axis (speech frame sequence) and the spatial axis (video frame) to capture the spatiotemporal correlation between speech fundamental frequency jitter and facial micro-expressions (such as AU4 frown unit).
[0043] The spatiotemporal feature output layer is used to gradually extract features through a 3D convolution + pooling structure based on hierarchical stacking and spatiotemporal correlation, and then output a spatiotemporal feature map at the last layer of the spatiotemporal feature output layer (example dimensions: 8×16×16×512); Target optimization layer: performs feature optimization based on a loss function that is determined by combining action localization error (e.g., speech-expression asynchrony) with the cross-entropy loss for emotion classification.
[0044] 2. Based on the spatiotemporal feature extraction network 3D-CNN, the synchronous voice and video data streams are processed to obtain the initial emotional value; 3. Based on the initial sentiment value, sentiment calculation is performed using a contradiction index analysis algorithm to obtain a sentiment calculation value, wherein the contradiction index analysis algorithm includes: (1) Standardize the optimized features, including: Z-Score standardization of text sentiment polarity and speech sentiment intensity to eliminate dimensional differences.
[0045] (2) Construct a contradiction index model and conduct judgment and arbitration of abnormal and contradictory data: Taking audio data as an example, the calculation formula of the contradiction index C in the contradiction index model is shown in the following formula (1):
[0046] (1);
[0047] (2)
[0048] Where T text is the normalized text sentiment polarity, BERT_output is the text sentiment polarity output by BERT, μ text is the standard text sentiment polarity; T voice is the normalized speech emotion intensity, Pitch_Jitter is the speech emotion intensity represented by the fundamental frequency jitter rate, μ text is the standard speech emotion intensity; σ text is the environmental vector modulation factor; σ voice is the sensor state attenuation factor;
[0049] An exemplary encoding for abnormal data arbitration is as follows: def dynamic_weight_calculation(h_modalities, env_vector, sensor_status, prev_context): """ h_modalities: list of eigenvectors of the 9 modes env_vector: environmental sensor data [light, noise, temperature and humidity] sensor_status: Each sensor status dictionary {'SNR', 'last_valid_time'} prev_context: LSTM state at the previous moment """ # 1. Calculate basic attention score e_scores = [] for h_i in h_modalities: score = vT @ np.tanh(Wh @ h_i + Wc @ prev_context + b) e_scores.append(score) # 2. Calculate the environmental modulation factor gamma_factors = [] for h_i in h_modalities: combined = np.concatenate([h_i, env_vector]) gamma = sigmoid(UT @combined) gamma_factors.append(gamma) # 3. Calculate the time decay factor tau_factors = [] current_time = time.time() for status in sensor_status: delta_t = current_time - status['last_valid_time'] tau = np.exp(-0.05 * delta_t * status['SNR']) tau_factors.append(tau) # 4. Calculate the final weight numerator = [] for i in range(9): num = gamma_factors[i] * tau_factors[i] * np.exp(e_scores[i]) numerator.append(num) denominator = sum(numerator) alpha_weights = [num / denominator for num in numerator] return alpha_weights
[0050] When the contradiction index C>2.5, the manual review process is triggered. When the contradiction index C>2.5 (that is, the difference between the two modalities exceeds 2.5 times the standard deviation), the following strategy is triggered as a manual review process: (A) Prioritize physiological sensor data (e.g., galvanic skin response (GSR) > 5μS is considered true anxiety); (B) If physiological data is missing, a manual review process is initiated.
[0051] Attention fusion mechanism module is used to dynamically adjust the weights of each modality.
[0052] In this embodiment, in the late fusion stage, the weights of each modality are dynamically adjusted based on the attention mechanism or the Transformer-based cross-modal attention mechanism.
[0053] 1. The formula for dynamically adjusting the weights of each modality based on the attention mechanism is shown in the following formula (3): (3); Among them, W i Represents the weight value of each mode. There are N modes in total. e f(xj) is the attention score of each modality.
[0054] Experiments show that this method improves the recognition accuracy of contradictory scenarios such as "trembling voice but calm expression" by 23%.
[0055] N modalities (corresponding to N sensors or feature extractors), the feature vector extracted by each modality is f(x j ) (j=1,2,...,N). The goal is to obtain the weight of each modality through the attention mechanism, and then calculate the weighted feature vector as the final representation.
[0056] 2. Based on the Transformer-based cross-modal attention mechanism, the weights are dynamically adjusted through formula (4): (4) Among them, Q is the query matrix (Query) of the current emotional state, K i is the key matrix (Key) of each modal eigenvector; d k is the dimension of the key vector, that is, the length of the key vector in each header.
[0057] As a preferred embodiment, the dynamic adjustment of each modal weight includes: (1) Feature projection: mapping multimodal features (text, speech, video) into the same latent space (dimension: 256); (2) Perform attention aggregation: Calculate weighted fusion features for downstream sentiment classification.
[0058] (3) Introduce a spatiotemporal attention gating mechanism to suppress the weight of features in low signal-to-noise ratio time periods (such as speech pauses).
[0059] The above solution achieves accurate analysis of multimodal emotional conflict scenes through 3D-CNN spatiotemporal feature extraction and dynamic attention fusion.
[0060] As a preferred embodiment, the dynamic decision layer includes: Reinforcement learning policy engine, used to establish state space and action space and design reward function; In this embodiment, establishing the state space includes: (1) Calculating the emotional intensity based on the emotional calculation value, wherein the emotional intensity is represented by a 0-3 level quantitative index, wherein the 0-3 level quantitative index includes 0 for calmness, 1 for mild anxiety, 2 for moderate anxiety, and 3 for severe anxiety; (2) Identifying cognitive distortion types, including: using a natural language processing (NLP)-based thinking trap classifier to identify thinking traps in user input (such as “all or nothing” and “catastrophic thinking”), identifying 12 types of cognitive distortions in user input and mapping them to 12 predefined labels; The Natural Language Processing (NLP)-based mind-trap classifier combines deep learning, rule engines, and psycholinguistic features to form the following system architecture: A[User input text] --> B(Text preprocessing) B --> C{Multi-level feature extraction} C --> D [word-level features] C --> E[Syntactic features] C --> F[semantic feature] D --> G[Feature Fusion] E --> G F --> G G --> H[Multi-label classifier] H --> I [output 12 class probabilities] I --> J[rule post-processing] As a preferred embodiment, the identification of cognitive distortion types includes the following steps: 1. Determine the classification system of thinking traps (12 categories) THINKING_TRAPS = { 0: "all_or_nothing", # all or nothing 1: "overgeneralization", # Overgeneralization 2: "mental_filter", # Mental filter 3: "disqualifying_positive", # Negate the positive 4: "jumping_to_conclusions", # jumping to conclusions 5: "magnification_minimization", # exaggeration / reduction 6: "emotional_reasoning", # Emotional reasoning 7: "should_statements", # should be stated 8: "labeling", # labeling 9: "personalization", # Personalization 10: "catastrophizing", # catastrophizing 11: "mind_reading" # Mind reading } 2. Obtaining key psycholinguistic features based on feature engineering Python def extract_cognitive_features(text): features = {} # Extreme vocabulary analysis extreme_words = ["always", "never", "every", "complete", "total","perfect"] features["extreme_count"] = sum(text.lower().count(word) for wordin extreme_words) # Catastrophizing Word Pattern catastrophe_patterns = [ r"disaster", r"worst ever", r"unbearable", r"ruin", r"destroy" ] features["catastrophe_score"] = sum(len(re.findall(p, text)) forp in catastrophe_patterns) # Cognitive Distortion Keywords distortion_triggers = { "should": r"\b(should|must|ought to)\b", "labeling": r"\b(stupid|failure|loser|idiot)\b", "mind_reading": r"\b(knows|thinks|believes)\b that I'm? " } for key, pattern in distortion_triggers.items(): features[key] = len(re.findall(pattern, text)) # Sentiment polarity strength blob = TextBlob(text) features["polarity_intensity"] = abs(blob.sentiment.polarity) # Absolute statement detection features["absolute_statements"] = len(re.findall(r"\b(all|none|noone|everyone)\b", text)) return features 3. Building a deep learning model (BiLSTM + Attention) Python import tensorflow as tf from tensorflow.keras.layers import Input, Embedding, Bidirectional,LSTM, Dense, Attention def create_trap_detection_model(vocab_size=20000, max_len=100): # Text input layer text_input = Input(shape=(max_len,)) # Embedding layer embedding = Embedding(vocab_size, 128)(text_input) Bidirectional LSTM bilstm = Bidirectional(LSTM(64, return_sequences=True))(embedding) # Attention Mechanism attention = Attention()([bilstm, bilstm]) # Cognitive feature input feature_input = Input(shape=(12,)) # 12 manual features concat = tf.keras.layers.concatenate([attention, feature_input]) # Multi-label classification output output = Dense(64, activation='relu')(concat) output = Dense(12, activation='sigmoid')(output) # 12 categories of thinking traps model = tf.keras.Model(inputs=[text_input, feature_input],outputs=output) model.compile(loss='binary_crossentropy', optimizer='adam',metrics=['accuracy']) return model 4. Build a rule enhancement engine Python def apply_cognitive_rules(text, predictions): """Prediction post-processing based on cognitive psychology rules""" # Rule 1: Increase confidence when including "never / always" and the prediction is all or nothing if ('never' in text or 'always' in text) and predictions[0] >0.3: predictions[0] = min(1.0, predictions[0] + 0.2) # Rule 2: "If only..." patterns reinforce catastrophizing labels if re.search(r"if .* (ruin|end|disaster)", text) and predictions
[10] > 0.4: predictions
[10] = min(1.0, predictions
[10] + 0.3) # Rule 3: Second person "you should" enhances should statement tags if re.search(r"you (should|must|ought to)", text) and predictions[7] > 0.3: predictions[7] = min(1.0, predictions[7] + 0.25) # Rule 4: Psychological verbs + thought assertions strengthen mind reading tags mind_read_verbs = ["know", "think", "believe", "feel"] if any(verb in text for verb in mind_read_verbs) and "that" intext and predictions
[11] > 0.4: predictions
[11] = min(1.0, predictions
[11] + 0.15) return predictions 5. Identify the types of cognitive distortions Python def detect_thinking_traps(user_input): Text preprocessing cleaned_text = preprocess_text(user_input) # Including word segmentation, lemmatization, etc. Feature extraction linguistic_features = extract_linguistic_features(cleaned_text) cognitive_features = extract_cognitive_features(cleaned_text) # Text serialization sequence = tokenizer.texts_to_sequences([cleaned_text]) padded_seq = pad_sequences(sequence, maxlen=100) # Model prediction raw_predictions = model.predict([padded_seq, np.array([cognitive_features])])[0] # Rule enhancement final_predictions = apply_cognitive_rules(cleaned_text, raw_predictions) # Result parsing detected_traps = [] for i, prob in enumerate(final_predictions): if prob > 0.65: # Confidence threshold detected_traps.append((THINKING_TRAPS[i], round(float(prob), 2))) # Interpretability analysis explanation = generate_explanation(cleaned_text, detected_traps) return { "traps": detected_traps, "explanation": explanation, "original_text": user_input } def visualize_attention(text, model): # Create a visualization model attention_model = tf.keras.Model( inputs=model.input, outputs=model.get_layer("attention").output ) # Get attention weight attn_weights = attention_model.predict(prepare_input(text)) # Generate heatmap plt.figure(figsize=(15, 2)) sns.heatmap(attn_weights[0], annot=True, xticklabels=text.split()) plt.title("Cognitive Distortion Attention Weights") plt.show() This embodiment uses multimodal feature fusion and attention visualization interpretation, especially using word embedding features for semantic understanding, syntactic dependency features for relationship analysis, and cognitive linguistic features to expand domain knowledge. It achieves good recognition results through an incremental learning mechanism, including the use of a user feedback loop and weekly incremental training: when a user marks a sample as "inaccurate", it enters a correction queue, and contrastive learning is used to enhance the recognition of difficult samples.
[0061] (3) Determining physiological arousal, including: calculating and grading the physiological stress index through heart rate variability and skin conductance, wherein the calculation formula of the physiological stress index A is as follows:
[0062]
[0063] The classification is low (A<0.5), medium (0.5≤A≤1.5), and high (A>1.5); among them, HRV LF / HF Indicates lower or higher values of heart rate variability, GSR Δ represents skin conductance.
[0064] In this embodiment, establishing the action space includes establishing an intervention strategy library corresponding to the action space. The intervention strategy library includes six core actions: CBT thought recording, mindfulness breathing training, virtual exposure therapy, drug dosage adjustment, crisis referral, and no intervention. Each core action corresponds to different resource consumption and expected therapeutic effects.
[0065] In this embodiment, the design reward function includes:
[0066] Designing a short-term reward function The short-term reward function is characterized by the user's real-time emotional decline ΔE and the weekly change rate ΔS of the PHQ-9 / GAD-7 scale score, where the user's real-time emotional decline ΔEt E is the user's real-time emotion evaluation value at time t t and the user's real-time emotion evaluation value E at time t-1 t-1 The difference is the decline in sentiment , negative values indicate improved mood, the short-term reward function The reward formula is expressed as: ;
[0067] Where α and β represent the weight coefficients of the user's real-time emotion decrease ΔE and the weekly change rate of the PHQ-9 / GAD-7 scale score, respectively, which are determined through expert experience or optimization; γ represents the user's compliance reward. For example, if the intervention action is completed, Compliance is positive and points are added; if the intervention action is not completed, Compliance is negative and points are deducted;
[0068] Design a long-term reward function, which is a discounted cumulative reward based on the Q-learning objective and is expressed as:
[0069] ;
[0070] Where k represents the single-day intervention duration of the reward function;
[0071] The design constraint was that the duration of the intervention per day should be ≤45 minutes to prevent cognitive overload.
[0072] like Figure 2 As shown, a hierarchical intervention selection tree is used to trigger corresponding intervention protocols according to crisis levels and generate intervention strategies based on the reward function.
[0073] In this embodiment, the method for triggering the corresponding intervention protocol according to the crisis level is triggered based on the crisis level determination rules shown in Table 1.
[0074] Table 1
[0075] Risk Level Decision condition (logical OR) Trigger Protocol high risk PHQ-9 ≥20 or mention of suicidal ideation Immediate referral to offline diagnosis and treatment + 24-hour AI monitoring Moderate risk Emotional intensity ≥ level 2 for 72 hours CBT twice a day + mindfulness intensive Low risk Single emotional peak ≥ 1.5 levels Push relaxation audio + breathing guidance
[0076] In this embodiment, the intervention strategy generated based on the reward function includes the user's real-time emotion reduction ΔE obtained by the short-term reward function based on the Q-learning algorithm and the state transition probability. t The weekly change rates of the PHQ-9 / GAD-7 scale scores were evaluated respectively, and intervention strategies were generated based on the evaluation results. The Q-learning algorithm was used to update the state-action value matrix according to the historical intervention effects. Experiments showed that the strategy convergence speed was increased by 41% compared with the random strategy. The state transition probability was determined by constructing a Markov decision model based on more than 100,000 clinical cases.
[0077] The user's real-time emotion reduction ΔE and the weekly change rate of the PHQ-9 / GAD-7 scale score obtained by the short-term reward function based on the Q-learning algorithm and state transition probability are evaluated respectively. The process of generating an intervention strategy based on the evaluation results is divided into three stages: data modeling, Q-learning algorithm implementation, and strategy generation, combining reinforcement learning with probabilistic models:
[0078] 1. Data Modeling Phase
[0079] 1. Define the State Space
[0080] Define state variables including: user real-time emotion value E t (collected in real time via sensors / questionnaires); historical PHQ-9 / GAD-7 scores and (Updated weekly); user demographics (static features such as gender and age); and context (time, location, activity type);
[0081] Define the state representation:
[0082]
[0083] 2. Define the Action Space
[0084] The intervention actions were determined to include: a1: pushing mindfulness meditation audio; a2: cognitive behavioral therapy (CBT) exercises; a3: emergency manual consultation; and a0: no intervention (silent observation);
[0085] 3. Construct a state transition probability model, where the goal of constructing the state transition probability model is to predict the execution action a t Post-state s t →s t+1 The probability of state transition probability model includes:
[0086] (1) Using historical data to train a probability model:
[0087] ;
[0088] (2) Model selection: Hidden Markov Model (HMM) or Bayesian Network, dynamically updating the probability distribution.
[0089] 2. Q-learning algorithm implementation stage
[0090] 1. Determine the Q-table update rule as follows:
[0091] ;
[0092] Among them, η represents the learning rate and γ represents the discount factor;
[0093] 2. Determine the state transition probability and process the uncertainty by probabilistic Q-learning update to obtain the state transition probability as:
[0094] ;
[0095] 3. Algorithm Process
[0096] for each user session:
[0097] Observe the current state s_t
[0098] for each timestep:
[0099] Select action a_t with ε-greedy strategy (exploration vs. exploitation)
[0100] Execute a_t, observe reward r_t = R_short(s_t, a_t) and transition state s_{t+1}
[0101] Update the state transition probability model P(s_{t+1}|s_t, a_t)
[0102] Update Q-table:
[0103] Q(s_t, a_t) += η * [r_t + γ * max_a Q(s_{t+1}, a) - Q(s_t,a_t)]
[0104] s_t = s_{t+1}
[0105] 3. Strategy Generation Phase
[0106] 1. Establish a dual-indicator evaluation system
[0107] index Evaluation Method <![CDATA[Emotional decline ΔE t > <![CDATA[Statistically analyze the daily ΔE t mean / variance to verify the effectiveness of the action]]> Weekly change rate of the scale Is the slope of the linear regression analysis significantly negative (improvement trend)?
[0108] 2. Generate dynamic strategies
[0109] Input: Current state ;
[0110] Output: Optimal action ;
[0111] Constraints:
[0112] oSafety limit: When ΔE t < threshold, manual intervention is forced to be triggered;
[0113] o Fatigue control: The same action will not be pushed repeatedly within 24 hours;
[0114] 4. Verification and Iteration
[0115] 1. Offline verification: Use historical data backtesting to compare the indicator improvement rate of Q-learning strategy vs. rule-based strategy.
[0116] 2. Online A / B testing: Divide users into groups and verify the effect of new strategies on ΔE t and the significance of scale improvement rate (t-test).
[0117] 3. Dynamic parameter adjustment: Adjust the reward weights α, βα, β based on user feedback (e.g., PHQ-9 has a higher weight than GAD-7).
[0118] Through a real-time decision-making mechanism, the edge computing node performs a policy evaluation every 5 minutes (delay <200ms). This solution achieves a precise and personalized psychological intervention decision-making closed loop through quantified state space and dynamic policy optimization. Of course, the basis for this implementation is the realization of "high human emotional fit simulation analysis."
[0119] Technical effects of hierarchical intervention selection trees:
[0120] 1. Probabilistic Q-learning: Introducing state transition probability to resolve the uncertainty of psychological state;
[0121] 2. Dual-target reward: Simultaneously optimize short-term mood reduction (ΔE) and long-term clinical indicators (PHQ-9 / GAD-7);
[0122] 3. Safety mechanism: Set the emotional breakdown threshold to force manual intervention.
[0123] As a preferred embodiment, the generative interaction layer includes:
[0124] An ethical constraint system, which is used to enforce ethical constraints based on an ethical rule base consisting of a taboo content library, knowledge boundary constraints, privacy protection mechanisms, and an ethical knowledge graph;
[0125] In this embodiment:
[0126] (1) Taboo content library: establish a dynamically updated blacklist (including sensitive words such as suicide / violence / discrimination, covering ICD-11 psychological crisis entries);
[0127] (2) Knowledge boundary constraints: It is prohibited to generate medical diagnosis suggestions (such as "you should take XX medicine"), and only general strategies for psychological support are allowed;
[0128] (3) Privacy protection mechanism: Automatically fuzzify PII information such as names and addresses mentioned by users (using regular expressions + NER joint recognition)
[0129] (4) Ethical knowledge graph, based on mermaid code and graph LR, implements the following architecture:
[0130] A[User Statement]->B{Ethical Review Node}
[0131] B->|Safety| C[Generate Empathic Response]
[0132] B->|Risk| D[Trigger standard dialogue]
[0133] D-> E["Response according to WHO Mental Health Guidelines 2.3"]
[0134] The map contains more than 2,000 standard psychological support scripts (certified by the APA Ethics Committee).
[0135] An empathy response generator, configured to generate an empathy response based on a constraint decoding technique, and apply the ethical constraints to the empathy response to generate an empathy response that complies with ethical standards;
[0136] In this embodiment, generating an empathic response based on the constraint decoding technology includes:
[0137] (1)Determine that the model architecture corresponding to the constrained decoding technology is the Llama-3-8B base model (Normalization layer: RMSNorm (Root Mean Square Normalization) is adopted, with a 30% higher computational efficiency than LayerNorm; Position encoding: upgraded to RoPE (Rotary Position Embedding), supporting dynamic context expansion to 128K tokens to solve the problem of long-range dependence attenuation; The attention mechanism uses grouped query attention (GQA) and sliding window attention (SWA); It has MoE (Mixture of Experts) support, enabling dynamic sparse activation and video memory optimization, thereby stabilizing the gating training through Z-loss regularization and avoiding routing oscillations), and train the model corresponding to the constrained decoding technology based on the psychological counseling dialogue dataset (including 100,000 labeled samples);
[0138] (2)Inject dynamic constraints into the model corresponding to the constrained decoding technology to generate an empathic response. The dynamic constraints include:
[0139] A. Emotional state adaptation constraint, which is used to adjust the generation temperature parameter according to the user's current emotional intensity; In this embodiment, when the user's current emotional intensity is in an anxious state, the temperature parameter temperature = 0.3, so that the psychological healing robot makes a conservative response; When the user's current emotional intensity is in a calm state, the temperature parameter temperature = 0.7, so that the psychological healing robot makes a creative response;
[0140] B. Multi-expert voting mechanism constraint, where the expert models corresponding to the multi-expert voting mechanism constraint include: ethical review model, crisis identification model, and emotional support degree evaluation model.
[0141] C. Empathy enhancement strategy constraint, including: psycholinguistic feature embedding and non-verbal symbol fusion, where:
[0142] The psycholinguistic feature embedding includes two-level typical psycholinguistic embedding and cross-cultural emotional metaphor mapping library embedding; The two-level typical psycholinguistic embedding includes a lexical level and a syntactic level. The lexical level includes increasing the probability of words in the "empathy dictionary" (such as "feel for others" and "it's really hard for you") during the decoding stage. The syntactic level includes using a question sentence ratio > 30% (such as "Does this thing make you feel a lot of pressure?") to promote user self-disclosure; The cross-cultural emotional metaphor mapping library embedding includes the embedding of a database formed by 327 native expressions such as "having a heavy heart" in Chinese and Japanese onomatopoeia expressions;
[0143] The fusion of non-verbal symbols includes: emotional embedding combined with a speech synthesis engine (such as reducing speech speed by 20% and increasing fundamental frequency by 5% to express concern); synchronization of facial expressions of virtual digital humans (driving the 3D facial action coding system FACS based on the response content); and analysis of keyboard tapping intervals (>1.2 seconds triggers a cognitive fatigue warning).
[0144] In this embodiment, the method for performing ethical compliance verification on the empathy response generated by applying the ethical constraint to the empathy response and complying with ethical standards is as follows:
[0145] (1) Ethical compliance verification using an automated testing framework, including:
[0146] Create an adversarial test set: This includes 500 high-risk, induced inputs (such as "I want to end everything") to verify whether the system triggers standard crisis protocols;
[0147] Determine ethical deviation index: D ethics =Number of illegal responses / total number of test samples × 100%;
[0148] After verification, the system D_{ethics} of this embodiment is 0.17% (better than the 2.3% error rate of human consultants).
[0149] (2) Ethical compliance verification based on human supervision mechanism, including
[0150] A double-blind review process was set up: every 1,000 generated responses had to be independently reviewed by three certified psychological counselors;
[0151] Dynamic update mechanism: The binding rule library is updated monthly based on the newly promulgated ethical guidelines.
[0152] like Figure 4 As shown, as a preferred embodiment, the brain science verification layer includes:
[0153] A neurofeedback adjustment module is used to monitor the change in the power of the alpha wave of the prefrontal lobe in real time through EEG, and use the change in the power of the alpha wave of the prefrontal lobe as neurofeedback;
[0154] As a preferred embodiment, the neural feedback regulation module includes:
[0155] (1) Signal acquisition system, including: high-density EEG electrodes and real-time signal acquisition optimization unit; wherein, the high-density EEG electrodes adopt the international 10-20 standard lead system, deploy silver / silver chloride dry electrodes in the frontal lobe area (Fp1 / Fp2 / Fz), control the contact impedance below 5kΩ, support 256-512Hz sampling rate per second, and adopt a new flexible electrode array to penetrate the stratum corneum and directly contact the dermis to improve the α wave signal-to-noise ratio (SNR>10dB); the real-time signal acquisition optimization unit adopts differential amplification technology combined with 120dB common mode rejection ratio to eliminate environmental electromagnetic interference (such as 50Hz power frequency noise), and includes a bandpass filter (7-13Hz) to directly extract the α band signal, reducing the calculation delay to less than 5ms.
[0156] (2) A real-time analysis system for alpha wave power, comprising: a preprocessing and feature extraction unit and a reference power calibration unit; the preprocessing and feature extraction unit is used to remove physiological artifacts such as blinking and electromyography through independent component analysis (ICA) to retain pure alpha wave signals; and to calculate the alpha wave power spectral density (PSD) within a 0.5 second time window through short-time Fourier transform (STFT) and output a dynamic change curve; the reference power calibration unit is used to establish a personalized baseline, and the process of establishing the personalized baseline includes: continuous monitoring for 3 minutes when the user is in a relaxed state with their eyes closed, and calculating the average alpha wave power as the value of the personalized baseline to quantify changes in neural activity.
[0157] The multimodal feedback design module is used to map the neural feedback into virtual scene parameters based on the neural feedback regulation mechanism, and dynamically adjust the intervention strategy based on the virtual scene parameters.
[0158] As a preferred embodiment, mapping the neural feedback into virtual scene parameters based on the neural feedback regulation mechanism includes:
[0159] Through visual feedback, the alpha wave power is mapped into virtual scene parameters (for example, in the mood fruit tree game scene under the forest scene, the quietness of the forest increases with the enhancement of alpha waves); through auditory feedback, sound waves synchronized with alpha waves are generated (frequency 8-13Hz) and transmitted through bone conduction headphones.
[0160] As a preferred embodiment, the dynamic adjustment intervention strategy based on the virtual scene parameters includes:
[0161] Performing step-by-step difficulty control based on the virtual scene parameters to dynamically adjust the intervention strategy; for example, in this embodiment, when the alpha wave power exceeds the baseline by 1.5σ for 5 consecutive minutes, the complexity of the cognitive training task is automatically increased (e.g., the memory matrix dimension is increased by 1); or
[0162] Closed-loop stimulation is performed based on the virtual scene parameters to dynamically adjust the intervention strategy; for example, in this embodiment, transcranial magnetic stimulation (TMS) is triggered in synchronization with alpha wave oscillations to enhance prefrontal-limbic system functional connectivity (phase-locked accuracy ±15ms).
[0163] In this embodiment, the method for verifying and optimizing the indicator includes:
[0164] (1) Validity verification:
[0165] Alpha wave power improvement rate: Clinical data show that 4 weeks of continuous training can increase the alpha wave power of the prefrontal lobe by 22%-35%.
[0166] Emotional regulation efficacy: Anxiety index (GAD-7) was significantly negatively correlated with changes in α wave power (r=-0.71, p<0.01).
[0167] (2) Security assurance
[0168] Set a dynamic safety threshold: when the α wave power mutation exceeds 3σ, the stimulation is automatically paused and manual review is initiated.
[0169] Biocompatibility testing: The flexible electrodes have passed ISO 10993 biosafety certification and support 8 hours of continuous use without skin irritation.
[0170] Application Example: Intervention Process for Patients with Social Anxiety
[0171] 1. Data collection: Capture facial AU4 (frowning) action unit using a mobile phone camera and simultaneously analyze the voice fundamental frequency jitter rate.
[0172] 2. Status assessment: Voice tremor frequency of 8.5Hz (> anxiety threshold of 7.5Hz) + text showing cognitive distortion of "fear of being laughed at" detected.
[0173] 3. Strategy generation: The dynamic engine selects virtual exposure therapy, and the VR scene generation module builds a supermarket shopping simulation environment.
[0174] 4. Effect verification: After the intervention, the LF / HF ratio of real-time monitoring of HRV (heart rate variability) increased to 1.8 (baseline value 0.9).
[0175] Technical Effects
[0176] 1. The accuracy of emotion recognition has increased to 89.7% (F1 value).
[0177] 2. The early warning time for high-risk cases is 4-6 months earlier than that of traditional scales.
[0178] 3. The intervention efficacy for patients with depression (PHQ-9 reduction ≥ 50%) reached 72.3%.
[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A psychological healing robot based on high human emotional fit simulation analysis, characterized by: include: A multimodal perception layer for collecting physiological, motion, and environmental interaction data; wherein the multimodal perception layer includes: a heterogeneous data acquisition module for integrating multiple types of sensor data on physiological, motion, and environmental interactions of smart terminals; a spatiotemporal feature extraction network for processing synchronized voice and video data streams based on a spatiotemporal feature extraction network 3D-CNN to obtain emotion calculation values corresponding to spatiotemporal features; and an attention fusion mechanism module for dynamically adjusting the weights of each modality; A dynamic decision layer, configured to generate an intervention strategy based on the interaction data using a reinforcement learning strategy engine and a hierarchical intervention selection tree; wherein the dynamic decision layer includes: a reinforcement learning strategy engine for establishing a state space and an action space and designing a reward function; and a hierarchical intervention selection tree for triggering corresponding intervention protocols according to crisis levels and generating an intervention strategy based on the reward function; A generative interaction layer for generating an empathic response that complies with ethical standards based on the intervention strategy; wherein the generative interaction layer includes: an ethical constraint system for applying ethical constraints based on an ethical rule library composed of a taboo content library, knowledge boundary constraints, a privacy protection mechanism, and an ethical knowledge graph; and an empathic response generator for generating an empathic response based on a constraint decoding technique, applying the ethical constraints to the empathic response to generate an empathic response that complies with ethical standards; A brain science verification layer is used to monitor neural feedback in real time through EEG and adjust the intervention strategy; wherein, the brain science verification layer includes: a neural feedback adjustment module, which is used to monitor the changes in the power of the alpha waves in the prefrontal lobe through EEG in real time, and use the changes in the power of the alpha waves in the prefrontal lobe as neural feedback; and a multimodal feedback design module, which is used to map the neural feedback into virtual scene parameters based on the neural feedback adjustment mechanism, and dynamically adjust the intervention strategy based on the virtual scene parameters.
2. A psychological healing robot based on high human emotional fit simulation analysis according to claim 1, characterized in that: The heterogeneous data acquisition module adopts 4 types of physiological monitoring sensors, 3 types of motion perception sensors and 2 types of environmental interaction sensors; among them, the 4 types of physiological monitoring sensors include heart rate variability PPG, skin galvanic response GSR, blood oxygen saturation SpO2 and body temperature sensors, the 3 types of motion perception sensors include three-axis accelerometers for gait analysis, gyroscopes for posture recognition and barometers for spatial positioning, and the 2 types of environmental interaction sensors include microphones for voice emotion analysis and ambient light sensors for circadian rhythm monitoring; the heterogeneous data acquisition module adopts a layered acquisition architecture, which sequentially goes from the sensor hardware layer, Bluetooth 5.3 or BLE protocol stack, edge computing node, data preprocessing unit and encrypted transmission to the cloud, realizes raw data denoising through edge nodes, and establishes a unified data description framework, which is compatible with Android Health Connect and Apple The HealthKit protocol realizes the spatiotemporal alignment of multi-device data and adopts a dynamic interpolation algorithm to balance the differences in sampling rates of different sensors. The heterogeneous data acquisition module is also used for multimodal transmission and synchronization and preliminary fusion of multimodal data. The multimodal transmission and synchronization are implemented based on a low-power transmission protocol and a spatiotemporal synchronization mechanism. The preliminary fusion of multimodal data is implemented based on a feature-level fusion strategy. The spatiotemporal features are extracted by inputting the original signal into a 3D-CNN network suitable for voice + video synchronization analysis. The importance score of each modality is calculated as an energy value through a learnable feedforward neural network, and then the importance score is converted into a weight through a softmax function.
3. A psychological healing robot based on high human emotional fit simulation analysis according to claim 2, characterized in that: The processing of synchronous voice and video data streams based on the spatiotemporal feature extraction network 3D-CNN includes: A 3D-CNN architecture for spatiotemporal feature extraction network is established, wherein the 3D-CNN architecture for spatiotemporal feature extraction network includes: a data input and preprocessing layer, wherein the data format input to the data input and preprocessing layer is to construct a three-dimensional input tensor of synchronously collected time series speech waveforms and spatial RGB data video frames into a [T×H×W×C] (time×height×width×channel) dimension; a 3D convolution kernel, including using a three-dimensional convolution kernel to synchronously slide on the time axis of the speech frame sequence and the spatial axis of the video frame to capture the spatiotemporal correlation between speech fundamental frequency jitter and facial micro-expressions; a spatiotemporal feature output layer, which is used to gradually extract features through a 3D convolution+pooling structure based on hierarchical stacking and spatiotemporal correlation, and then output a spatiotemporal feature map in the last layer of the spatiotemporal feature output layer; a target optimization layer: feature optimization based on a loss function, wherein the loss function is determined based on the fusion of action localization error and emotion classification cross entropy loss; Based on the spatiotemporal feature extraction network 3D-CNN, the synchronous voice and video data streams are processed to obtain the initial emotional value; Based on the initial sentiment value, sentiment calculation is performed using a contradiction index analysis algorithm to obtain a sentiment calculation value, wherein the contradiction index analysis algorithm includes: (1) Standardize the optimized features, including: Z-Score standardization of text sentiment polarity and speech sentiment intensity to eliminate dimensional differences; (2) Construct a contradiction index model and conduct judgment and arbitration of abnormal and contradictory data: The calculation formula of the contradiction index C in the contradiction index model is shown in the following formula (1): (1); (2); Where T text is the normalized text sentiment polarity, BERT_output is the text sentiment polarity output by BERT, μ text is the standard text sentiment polarity; T voice is the normalized speech emotion intensity, Pitch_Jitter is the speech emotion intensity represented by the fundamental frequency jitter rate, μ text is the standard speech emotion intensity; σ text is the environmental vector modulation factor; σ voice is the sensor state attenuation factor; when the contradiction index C>2.5, the difference between the two modes exceeds 2.5 times the standard deviation, triggering the manual review process, including: (A) Prioritize the use of physiological sensor data; (B) If physiological data are missing, a manual review process is initiated; (3) Determining physiological arousal, including: calculating and grading the physiological stress index through heart rate variability and skin conductance, wherein the calculation formula of the physiological stress index A is as follows: ; The classification is low A<0.5, medium 0.5≤A≤1.5 and high A>1.5; among them, HRV LF / HF Indicates lower or higher values of heart rate variability, GSR Δ represents skin conductance.
4. The psychological healing robot based on high human emotional fit simulation analysis according to claim 3 is characterized in that: Dynamically adjust the weights of each modality based on the attention mechanism or dynamically adjust the weights of each modality based on the Transformer cross-modal attention mechanism; wherein, the formula for dynamically adjusting the weights of each modality based on the attention mechanism is shown in the following formula (3): (3); Among them, W i Represents the weight value of each mode. There are N modes in total. e f(xj) is the attention score of each modality; N modalities correspond to N sensors or feature extractors, and the feature vector extracted by each modality is f(x j ), j=1,2,...,N; the goal is to obtain the weight of each modality through the attention mechanism, and then calculate the weighted feature vector as the final representation; The formula for dynamically adjusting the weights of each modality using the Transformer-based cross-modal attention mechanism is shown in the following formula (4): (4); Among them, Q is the query matrix (Query) of the current emotional state, K i is the key matrix (Key) of each modal eigenvector; d k is the dimension of the key vector, that is, the length of the key vector in each header; Dynamically adjusting the weights of each modality includes: (1) Feature projection: mapping multimodal features into the same latent space; (2) Perform attention aggregation: Calculate weighted fusion features for downstream sentiment classification; (3) Introduce a spatiotemporal attention gating mechanism to suppress the weight of features in time periods with low signal-to-noise ratio.
5. The psychological healing robot based on high human emotional fit simulation analysis according to claim 4 is characterized in that: The establishing state space comprises: (1) Calculating the emotional intensity based on the emotional calculation value, wherein the emotional intensity is represented by a 0-3 level quantitative index, wherein the 0-3 level quantitative index includes 0 for calmness, 1 for mild anxiety, 2 for moderate anxiety, and 3 for severe anxiety; (2) Identifying cognitive distortion types, including: identifying thinking traps in user input using a thinking trap classifier based on natural language processing, identifying 12 types of cognitive distortions in user input and mapping them to 12 predefined labels; Among them, identifying cognitive distortion types includes: (A) Identify a thinking trap classification system to identify 12 predefined labels, including: all or nothing, overgeneralization, mental filtering, negating the positive, jumping to conclusions, exaggeration or minimization, emotional reasoning, should statements, labeling, personalization, catastrophizing, and mind reading; (B) Acquiring key psycholinguistic features based on feature engineering, including extreme vocabulary analysis, catastrophizing vocabulary patterns, cognitive distortion keywords, sentiment polarity intensity, and absolute sentence detection; (C) Build a deep learning model, BiLSTM+Attention, which includes a text input layer, an embedding layer, a bidirectional LSTM, an attention mechanism, cognitive feature input, and a multi-label classification output. (D) A rule-enhancing engine was established based on post-prediction processing of cognitive psychology rules, including: Rule 1: Confidence is enhanced when the prediction contains "never / always" and is all-or-nothing; Rule 2: The "if...it's over" pattern enhances the catastrophizing label; Rule 3: The second-person "you should" enhances the "should" statement label; and, Rule 4: Psychological verbs + thought assertions enhance the "mind-reading" label; (E) Identifying cognitive distortion types, including: text preprocessing including word segmentation and lemmatization, feature extraction, text serialization, model prediction, rule enhancement, result parsing based on confidence threshold, interpretability analysis, creating visualization models, obtaining attention weights, and generating heatmaps; The establishing of the action space includes establishing an intervention strategy library corresponding to the action space, wherein the intervention strategy library includes six core actions: CBT thought recording, mindfulness breathing training, virtual exposure therapy, medication dosage adjustment, crisis referral, and no intervention, each of which corresponds to different resource consumption and expected efficacy; The design reward function includes: Designing a short-term reward function The short-term reward function is characterized by the user's real-time emotional decline ΔE and the weekly change rate ΔS of the PHQ-9 / GAD-7 scale score, where the user's real-time emotional decline ΔEt E is the user's real-time emotion evaluation value at time t t and the user's real-time emotion evaluation value E at time t-1 t-1 The difference is the decline in sentiment , negative values indicate improved mood, the short-term reward function The reward formula is expressed as: ; Where α and β represent the weight coefficients of the user's real-time emotion decrease ΔE and the weekly change rate of the PHQ-9 / GAD-7 scale score, respectively, which are determined through expert experience or optimization; γ represents the user's compliance reward. For example, if the intervention action is completed, Compliance is positive and points are added; if the intervention action is not completed, Compliance is negative and points are deducted; Design a long-term reward function, which is a discounted cumulative reward based on the Q-learning objective and is expressed as: ; Where k represents the single-day intervention duration of the reward function; The design constraint was that the duration of the intervention per day should be ≤45 minutes to prevent cognitive overload.
6. The psychological healing robot based on high human emotional fit simulation analysis according to claim 5, characterized in that: The method for triggering the corresponding intervention protocol according to the crisis level is triggered based on the crisis level judgment rule, wherein the crisis level corresponds to high risk, medium risk and low risk, and the judgment rule is PHQ-9 ≥ 20 under the logical OR or mention of suicidal ideation, emotional intensity lasting ≥ level 2 for 72 hours and a single emotional peak ≥ level 1.
5. The corresponding intervention protocol triggered is immediate referral for offline diagnosis and treatment + 24-hour AI monitoring, CBT twice a day + mindfulness reinforcement and push relaxation audio + breathing guidance; The intervention strategy generated based on the reward function includes the user's real-time emotion reduction ΔE obtained by the short-term reward function based on the Q-learning algorithm and the state transition probability. t The weekly change rates of the scores on the PHQ-9 / GAD-7 scales were evaluated separately, and intervention strategies were generated based on the evaluation results.
7. The psychological healing robot based on high human emotional fit simulation analysis according to claim 6, characterized in that: The user's real-time emotion reduction ΔE and the weekly change rate of the PHQ-9 / GAD-7 scale score obtained by the short-term reward function are evaluated based on the Q-learning algorithm and state transition probability. The process of generating an intervention strategy based on the evaluation results includes a data modeling stage, a Q-learning algorithm implementation stage, and a strategy generation stage; wherein: The data modeling stage includes defining the state space, defining the action space and building a state transition probability model; the definition of the state space includes defining state variables and defining state representation, and the definition of state variables includes: user real-time emotion value E t ; Historical PHQ-9 / GAD-7 scores and ; User demographic characteristics and environmental context Context; Define the state as: The action space is defined to include determining intervention actions including: a1: push mindfulness meditation audio; a2: cognitive behavioral therapy CBT practice; a3: emergency manual consultation; and a0: no intervention, silent observation; the state transition probability model is constructed to predict the execution action a t Post-state s t →s t+1 The method of constructing the state transition probability model includes: (1) using historical data to train the probability model: ; and (2) the model selects the hidden Markov model (HMM) or Bayesian network to dynamically update the probability distribution; The Q-learning algorithm implementation stage includes determining Q-table update rules and determining state transition probabilities, and processing uncertainty through probabilistic Q-learning updates to obtain the state transition probabilities; The strategy generation phase includes establishing a dual-index evaluation system and generating dynamic strategies. The dual-index evaluation system includes the index sentiment drop ΔE t Using statistical daily ΔE t The mean / variance of the action is used to evaluate the effectiveness of the action. For the weekly change rate of the indicator scale, a linear regression analysis is used to determine whether the slope is significantly negative. The dynamic strategy has safety limits and fatigue control. The safety limit is when ΔE t < threshold, manual intervention is forced to be triggered; the fatigue control is that the same action is not pushed repeatedly within 24 hours.
8. The psychological healing robot based on high human emotional fit simulation analysis according to claim 7, characterized in that: The ethical constraint system includes: (1) Taboo content library: establish a dynamically updated blacklist that includes suicide, violence, and / or discrimination sensitive words, and covers ICD-11 psychological crisis entries; (2) Knowledge boundary constraints: the generation of medical diagnosis recommendations is prohibited, and only general strategies for psychological support are allowed; (3) Privacy protection mechanism: using regular expressions and NER combined recognition to automatically fuzzify the PII information mentioned by users; (4) Ethical knowledge graph, based on mermaid code and graph LR, implements the following architecture: A[User Statement]->B{Ethical Review Node} B->|Safety| C[Generate Empathic Response] B->|Risk| D[Trigger standard dialogue] D->E["Response according to WHO Mental Health Guidelines 2.3"]; Generating an empathic response based on the constraint decoding technique includes: (1) Determine that the model architecture corresponding to the constrained decoding technology is the Llama-3-8B base model, and train the model corresponding to the constrained decoding technology based on the psychological counseling dialogue dataset; (2) Injecting dynamic constraints into the model corresponding to the constraint decoding technology to generate empathic responses. The dynamic constraints include: A. Emotional state adaptation constraint, which is used to adjust the generated temperature parameters according to the user's current emotional intensity; B. Multi-expert voting mechanism constraints, where the expert models corresponding to the multi-expert voting mechanism constraints include: ethics review model, crisis identification model, and emotional support assessment model; C. Empathy enhancement strategy constraints, including: psycholinguistic feature embedding and non-verbal symbol integration, including: The psycholinguistic feature embedding includes two-level typical psycholinguistic embedding and cross-cultural emotional metaphor mapping library embedding; the two-level typical psycholinguistic embedding includes lexical level and syntactic level, the lexical level includes improving the probability of words in the empathy dictionary during the decoding stage, and the syntactic level includes using question sentences accounting for more than 30% to promote user self-disclosure; the cross-cultural emotional metaphor mapping library embedding includes embedding a database formed by 327 localized expressions and Japanese onomatopoeia expressions; The non-verbal symbol fusion includes: emotion embedding combined with speech synthesis engine; virtual digital human facial expression synchronization; and keyboard tapping interval analysis; The method for verifying the ethical compliance of an empathic response generated by applying the ethical constraints to the empathic response is as follows: (1) Ethical compliance verification using an automated testing framework, including: Form an adversarial test set: Contains 500 high-risk induced inputs to verify whether the system triggers the standard crisis protocol; Determine ethical deviation index: D ethics =Number of illegal responses / total number of test samples × 100%; (2) Ethical compliance verification based on human supervision mechanisms, including: Set up a double-blind review process: every 1,000 generated responses are independently reviewed by three certified psychological counselors; Dynamic update mechanism: The binding rule library is updated monthly based on the newly promulgated ethical guidelines.
9. The psychological healing robot based on high human emotional fit simulation analysis according to claim 8, characterized in that: The neural feedback regulation module includes: (1) A signal acquisition system comprising: high-density EEG electrodes and a real-time signal acquisition optimization unit; wherein the high-density EEG electrodes include an international 10-20 standard lead system and a flexible electrode array; the real-time signal acquisition optimization unit optimizes the acquisition of real-time signals based on a combination of differential amplification technology, a 120 dB common mode rejection ratio, and a bandpass filter; (2) A real-time analysis system for alpha wave power, comprising: a preprocessing and feature extraction unit and a reference power calibration unit; the preprocessing and feature extraction unit is used to remove physiological artifacts through independent component analysis and retain pure alpha wave signals; and the alpha wave power spectrum density within a 0.5 second time window is calculated through short-time Fourier transform and a dynamic change curve is output; the reference power calibration unit is used to establish a personalized baseline, and the process of establishing the personalized baseline includes: continuous monitoring for 3 minutes when the user is in a relaxed state with their eyes closed, and calculating the average alpha wave power as the value of the personalized baseline to quantify changes in neural activity.
10. The psychological healing robot based on high human emotional fit simulation analysis according to claim 9, characterized in that: Mapping the neural feedback to virtual scene parameters based on the neural feedback regulation mechanism includes: Alpha wave power is mapped to virtual scene parameters through visual feedback; sound waves synchronized with alpha waves are generated through auditory feedback and delivered through bone conduction headphones; The dynamic adjustment intervention strategy based on the virtual scene parameters includes: Performing step-by-step difficulty control based on the virtual scene parameters to dynamically adjust the intervention strategy; or Closed-loop stimulation is performed based on the virtual scene parameters to dynamically adjust the intervention strategy.
Citation Information
Patent Citations
Music healing system based on brain wave emotion recognition and processing method thereof
CN112999490A
Session type artificial intelligence driven personality simulation system based on context awareness and operation method
CN117874185A
Empathic Computing System and Methods for Improved Human Interactions With Digital Content Experiences
US20210390366A1
Cited By
Robot control method and system based on multi-modal fusion
CN121061897A
Man-machine conversation psychological stress identification and intervention method, system, medium and product
CN121306436A
Closed-loop adaptive psychological intervention method and system based on multi-modal brain-computer fusion
CN122025028A
Closed-loop adaptive psychological intervention method and system based on multi-modal brain-machine fusion
CN122025028B