Game emotion regulation and control system and method based on multi-mode physiological signal real-time monitoring

Abstract

The invention discloses a game emotion regulation and control system and method based on multi-mode physiological signal real-time monitoring. The method comprises the steps that a wearable device collects a player physiological signal in real time; synchronously recording behavior data such as an operation error rate and team interaction event frequency through a game log capture interface; inputting an HRV low-frequency / high-frequency power ratio (LF / HF), an EDA signal first-order derivative and an operation error feature into a lightweight LSTM prediction model (less than or equal to 5MB) in an edge calculation unit, and generating an emotion risk level; when LF / HFgt; rendering a breathing guide ripple animation with the frequency of 6 times per minute at the edge of the game interface when the time lasts for 5 seconds; when operation errors are continuously carried out for three times and negative chats exist, an emotion semantic prompt label is dynamically generated on the top of the role head; and meanwhile, sending of a vibration instruction to the wearable device is supported. According to the method, the technical blank of real-time emotion regulation and control in a game scene is filled up, the prediction accuracy is improved through physiological-behavior multi-mode fusion, and non-sensitive and scene-adaptive emotion intervention is realized.

Description

Technical Field

[0001] The present application relates to the technical field of adolescent mental health, and in particular to a game emotion regulation system and method based on real-time monitoring of multimodal physiological signals. Background Art

[0002] Analysis of existing technology defects

[0003] 1. Emotional regulation lag:

[0004] Traditional game addiction prevention systems only intervene through time limits or forced offline intervention, but fail to provide real-time control measures for immediate emotional fluctuations during gaming (such as the "flush" phenomenon caused by failure feedback and social conflict). Adolescents' emotional regulation abilities are immature, and emotions like anger and anxiety triggered by physiological arousal during gaming (such as abnormal HRV and elevated skin charge) can, if not addressed promptly, lead to negative behaviors such as cyberbullying and academic burnout.

[0005] 2. Limitations of single modality data:

[0006] Existing emotion monitoring technologies often rely on single physiological indicators (such as heart rate) or subjective reports, lacking multimodal data fusion across physiological, behavioral, and contextual factors. For example, traditional wearable devices only record heart rate and cannot correlate it with contextual factors such as in-game operational errors and team interactions, resulting in insufficient emotion prediction accuracy.

[0007] 3. Intervention methods are out of context:

[0008] Existing intervention methods (such as psychological counseling and offline relaxation training) are separated from the game process, making it difficult to trigger an effective response at the critical point of emotional outburst. Summary of the Invention

[0009] The present application aims to solve one of the technical problems in the related art at least to a certain extent.

[0010] To this end, one purpose of this application is to provide a game emotion regulation system and method based on real-time monitoring of multimodal physiological signals. Through the collaboration of wearable devices and game plug-ins, accurate prediction and scenario-based regulation of teenagers' game emotions can be achieved, filling the gaps in existing technologies in real-time, accuracy and scenario adaptability.

[0011] To achieve the above objectives, the first embodiment of the present application proposes a game emotion regulation method based on real-time monitoring of multimodal physiological signals, comprising the following steps:

[0012] S1. Collecting physiological signals of the player in real time through a wearable device, wherein the physiological signals include at least heart rate variability (HRV) and electrodermal activity (EDA);

[0013] S2. Synchronously record game behavior data through the game log capture interface, including the error rate and frequency of team interaction events;

[0014] S3. Inputting the physiological signals and behavioral data into the machine learning model of the edge computing unit to generate a predicted value of the emotional risk level;

[0015] S4. When the predicted value exceeds a preset threshold, perform at least one of the following actions through the dynamic intervention engine:

[0016] S4.1 Render breathing guidance animation in the game interface;

[0017] S4.2 generates emotional semantic cue labels and overlays them onto the game screen;

[0018] S4.3 sends a vibration control instruction to the wearable device.

[0019] According to the gaming emotion regulation system and method based on real-time monitoring of multimodal physiological signals according to the embodiments of the present application, accurate prediction and scenario-based regulation of teenagers' gaming emotions are achieved through the collaboration of wearable devices and game plug-ins, filling the gaps in existing technologies in terms of real-time performance, accuracy and scenario adaptability.

[0020] In addition, the game emotion regulation system and method based on real-time monitoring of multimodal physiological signals proposed in the present application may also have the following additional technical features:

[0021] In one embodiment of the present application, the machine learning model in step 3 is a quantized LSTM network with a model size of ≤5MB; the input features include: HRV low-frequency / high-frequency power ratio (LF / HF), the first-order derivative of the EDA signal, and the number of skill release errors per unit time.

[0022] In one embodiment of the present application, step 4 is triggered using a dual-channel mechanism: Channel 1: When LF / HF>3.0 and lasts for 5 seconds, operation S4.1 is triggered immediately; Channel 2: When there are three consecutive operation errors and the team chat contains negative keywords, operation S4.2 is triggered.

[0023] In one embodiment of the present application, the breathing guidance animation is rendered in the form of semi-transparent ripples in the edge area of ​​the game interface, and the ripple expansion frequency matches the deep breathing rhythm of 6 times / minute; the emotional semantic prompt label is dynamically displayed above the head of the game character.

[0024] A game emotion regulation system based on real-time monitoring of multimodal physiological signals, wherein the wearable device integrates an HRV sensor, an EDA sensor, a Bluetooth transmission module and a vibration motor, wherein the sampling rate of the HRV sensor is ≥100Hz; the sampling rate of the EDA sensor is ≥50Hz; the edge computing unit is connected to the wearable device via Bluetooth, and includes a lightweight LSTM prediction model and a feature extraction module; the lightweight LSTM prediction model is used to output the emotion risk level; the feature extraction module calculates the LF / HF ratio and the EDA signal slope; the dynamic intervention engine is connected to the edge computing unit via a data bus, and includes an animation rendering submodule, a semantic generation submodule and a device control submodule; the animation rendering submodule calls the game engine graphical interface to generate breathing ripples; the semantic generation submodule outputs emotion prompt text based on NLG technology; and the device control submodule sends a vibration command to the wearable device.

[0025] In one embodiment of the present application, the edge computing unit is deployed on the user's local terminal and shares memory resources with the game client; the dynamic intervention engine is embedded in the game engine (Unity / Unreal) in the form of a plug-in, and obtains the character coordinate data through the engine event system to locate the display position of the emotional semantic prompt label.

[0026] In one embodiment of the present application, a parent supervision module is added, which is connected to the edge computing unit through an encrypted API to obtain emotional risk reports in real time and push them to the guardian terminal.

[0027] In one embodiment of the present application, the HRV sensor and EDA sensor of the wearable device are integrated on the same PCB board, and the sensor spacing is ≤10 mm to reduce motion artifacts.

[0028] The advantages of this application compared with the existing technology are:

[0029] (1) By collecting HRV and EDA physiological signals through wearable devices, and simultaneously recording behavioral data such as game operation error rate and team negative interaction, a three-dimensional emotion analysis model is constructed to improve the prediction accuracy compared with a single indicator.

[0030] (2) Physiological trigger intervention (channel 1): When the LF / HF ratio of HRV is greater than 3.0 and lasts for 5 seconds (sympathetic nerve excitation), a translucent breathing guidance animation (6 times / minute rhythm) on the edge of the game interface is triggered to quickly reduce the level of physiological arousal;

[0031] Behavior-triggered intervention (Channel 2): ​​When three consecutive operational errors + negative chat are made, a semantic prompt label ("Suggested rhythm adjustment") is generated above the character's head to guide cognitive reappraisal and avoid forced interruption of the gaming experience.

[0032] (3) The bracelet integrates HRV / EDA sensors and vibration motors, and communicates with game plug-ins in real time via Bluetooth to form a physical closed loop of "monitoring-analysis-intervention". It also encrypts and pushes emotional risk reports to the guardian's terminal to assist family collaborative intervention and reduce parent-child conflicts.

[0033] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0035] Figure 1 This is a flow chart of a physiological signal acquisition module of a game emotion regulation system and method based on real-time monitoring of multimodal physiological signals according to one embodiment of the present application;

[0036] Figure 2 This is a flow chart of a data processing and prediction module of a game emotion regulation system and method based on real-time monitoring of multimodal physiological signals according to one embodiment of the present application;

[0037] Figure 3 This is a flow chart of an intervention execution module of a game emotion regulation system and method based on real-time monitoring of multimodal physiological signals according to one embodiment of the present application;

[0038] Figure 4 This is a flow chart of the system integration and supervision module of a game emotion regulation system and method based on real-time monitoring of multimodal physiological signals according to one embodiment of the present application.

[0039] As shown in the figure: 1. Wearable device; 11. HRV sensor; 12. EDA sensor; 13. Bluetooth transmission module; 14. Vibration motor;

[0040] 2. Game log capture interface;

[0041] 3. Edge computing unit; 31. Lightweight LSTM prediction model; 32. Feature extraction module;

[0042] 4. Dynamic intervention engine; 41. Animation rendering submodule; 42. Semantic generation submodule; 43. Device control submodule; 401. Breathing guidance animation; 402. Emotional semantic prompt label;

[0043] 5. Game client;

[0044] 6. Parent supervision module;

[0045] 7. Guardian terminal. DETAILED DESCRIPTION

[0046] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application and are not to be construed as limiting the present application. On the contrary, the embodiments of the present application include all variations, modifications, and equivalents that fall within the spirit and scope of the appended claims.

[0047] The following describes the game emotion regulation system and method based on real-time monitoring of multimodal physiological signals according to an embodiment of the present application with reference to the accompanying drawings.

[0048] like Figure 1-Figure 4 As shown, the game emotion regulation method based on real-time monitoring of multimodal physiological signals in an embodiment of the present application includes the following steps:

[0049] Step S1: Multimodal physiological signal acquisition

[0050] The wearable device 1 integrates an HRV sensor 11 and an EDA sensor 12 to monitor the player's heart rate variability (HRV) and electrodermal activity (EDA) in real time at sampling rates of ≥100 Hz and ≥50 Hz, respectively.

[0051] The collected physiological signals are sent to the edge computing unit 3 in real time via the Bluetooth transmission module 13, ensuring low-latency data transmission (≤200ms).

[0052] Step S2: Game behavior data synchronization

[0053] The game log capture interface 2 extracts the operation error rate (such as the number of skill release errors per unit time) and the frequency of team interaction events (such as the number of chat messages containing negative keywords) from the game client 5, aligns them with the physiological signal timestamp of S1 (accuracy ≤100ms), and forms a multimodal data input stream.

[0054] Step S3: Emotional risk level prediction

[0055] The feature extraction module 32 of the edge computing unit 3 preprocesses the physiological signal:

[0056] Calculate the low-frequency / high-frequency power ratio (LF / HF) of HRV (reflecting the autonomic nervous balance, LF corresponds to the sympathetic nervous system, and HF corresponds to the parasympathetic nervous system);

[0057] Calculate the first-order derivative of the EDA signal (reflecting the rate of change of skin electricity and representing the degree of emotional arousal).

[0058] The aforementioned physiological characteristics and S2's operational error rate are fed into a lightweight LSTM prediction model 31 (model size ≤ 5MB). The model is trained using historical multimodal data (e.g., HRV, EDA, operational logs, and SAM scale annotations from 200 adolescent gaming scenarios) and outputs an emotional risk level (1-5, with 5 being the critical threshold).

[0059] Step S4: Dynamic intervention execution

[0060] When the emotional risk level is greater than or equal to the preset threshold (e.g., level 4), the dynamic intervention engine 4 triggers the following actions:

[0061] 1. S4.1: Breathing Guided Animation 401

[0062] The animation rendering submodule 41 calls the game engine (Unity / Unreal) graphics interface to render a translucent ripple animation in the edge area of ​​the game interface. The ripple expansion frequency matches the deep breathing rhythm of 6 times / minute (controlled by the engine timer, completing a ripple expansion-contraction cycle every 10 seconds), guiding the player's physiological arousal level to decrease.

[0063] 2. S4.2: Emotional semantic cue tag 402

[0064] The semantic generation submodule 42 generates prompt text based on NLG technology (the preset template is "The current emotion score is X level, it is recommended to adjust the operation rhythm"), obtains the character coordinates through the event system of the game engine, and dynamically superimposes the label to the top of the game character's head (3D coordinate positioning of the UI layer to ensure that the operation field of view is not blocked).

[0065] 3. S4.3: Vibration control command

[0066] The device control submodule 43 sends instructions to the vibration motor 14 of the wearable device 1 to trigger short-frequency vibration (such as 3 vibrations at intervals of 1 second) to remind the player to pay attention to his emotional state through tactile feedback.

[0067] In one embodiment of the present application, Figure 1-Figure 4 As shown, for the machine learning model of step S3, it can be understood that:

[0068] HRV low frequency / high frequency power ratio (LF / HF) calculation:

[0069] The HRV sensor 11 of the wearable device 1 collects heart rate data at a sampling rate of ≥100 Hz. The feature extraction module 32 of the edge computing unit 3 performs frequency domain analysis (FFT transform) on the heart rate signal, separates the low-frequency (LF, 0.04-0.15 Hz) and high-frequency (HF, 0.15-0.4 Hz) components, and calculates the ratio LF / HF (reflecting the sympathetic-parasympathetic nerve balance, the higher the value, the stronger the emotional excitement).

[0070] EDA signal first-order derivative calculation:

[0071] The EDA sensor 12 collects skin electrical data at a sampling rate of ≥50 Hz. The feature extraction module 32 performs first-order difference processing on the EDA signal to obtain the skin electrical change rate (unit: μS / s), which characterizes the dynamic fluctuations of emotional arousal (for example, the EDA derivative increases significantly when anxious, which serves as a physiological marker of emotional risk).

[0072] Quantification of operational error rate:

[0073] Game log capture interface 2 records the number of skill release errors per unit time (the number of unused skills and positioning errors per minute), which is directly input into the model as a behavioral feature to reflect the stability of game operations (which is highly correlated with emotional state, such as the operation error rate increases significantly when the "temperature" is "red").

[0074] Network architecture: A 3-layer LSTM network (64 neurons per layer) is used, the input layer dimension is 3 (LF / HF, EDA derivative, operation error rate), the output layer is a 5-level emotional risk level (1-5, level 1 is calm, level 5 is red temperature critical), and the probability distribution is calculated through the softmax activation function.

[0075] Quantization optimization: Through weight quantization (8-bit integer quantization) and model pruning, the model size is compressed to ≤5MB, ensuring real-time inference (≥100 times / second) on the edge computing unit 3, meeting the low latency requirements of gaming scenarios (≤200ms).

[0076] Dataset: Training based on multimodal data (HRV, EDA, operation logs, subjective emotion annotation) of 200 adolescent gamers.

[0077] The edge computing unit 3 runs a lightweight LSTM prediction model locally, and the feature extraction module 32 generates a three-dimensional feature vector (LF / HF, EDA derivative, and operation error rate) every second. The input model performs temporal reasoning (using the LSTM memory unit to capture the temporal correlation of emotional fluctuations, such as the cumulative effect of consecutive high LF / HF values), and outputs the emotional risk level (updated once per second).

[0078] In one embodiment of the present application, Figure 1-Figure 4 As shown, for the dual-channel trigger mechanism of step S4, it can be understood that:

[0079] 1. Channel 1: Physiological indicators trigger intervention (LF / HF>3.0 and lasts for 5 seconds → S4.1)

[0080] Data collection and processing:

[0081] The HRV sensor 11 of the wearable device 1 collects heart rate data in real time, and the feature extraction module 32 of the edge computing unit 3 calculates the LF / HF ratio (LF: low-frequency power, reflecting sympathetic nerve activity; HF: high-frequency power, reflecting parasympathetic nerve activity).

[0082] The feature extraction module 32 updates the LF / HF value once per second and determines whether the ratio continuously exceeds the threshold value of 3.0 through time window monitoring (sliding window size of 5 seconds).

[0083] Condition triggering and intervention execution:

[0084] When LF / HF>3.0 and lasts for 5 seconds, the edge computing unit 3 sends a first-level warning instruction to the dynamic intervention engine 4.

[0085] The animation rendering submodule 41 of the dynamic intervention engine 4 immediately triggers the S4.1 operation: a translucent breathing guidance animation 401 is rendered in the edge area of ​​the game interface, and the ripple expansion frequency matches the deep breathing rhythm of 6 times / minute (one cycle every 10 seconds through the game engine timer), guiding the player to adjust the breathing frequency and reduce the sympathetic nerve excitement.

[0086] 2. Channel 2: Behavioral-social trigger intervention (3 consecutive operational errors + negative keywords → S4.2)

[0087] Data collection and processing:

[0088] The game log capture interface 2 records the number of operation errors in real time (such as skill release errors, positioning errors, etc., the counting period is the continuous operation period), and generates a behavior abnormality signal when three consecutive operation errors are detected.

[0089] At the same time, the game log capture interface 2 performs keyword matching on the team chat text (preset negative keyword library such as "junk", "waste", etc.), and generates a social conflict signal when a message containing negative keywords is detected.

[0090] Condition triggering and intervention execution:

[0091] When three consecutive operation error signals and negative keyword signals appear at the same time, the edge computing unit 3 sends a secondary warning instruction to the dynamic intervention engine 4.

[0092] The semantic generation submodule 42 of the dynamic intervention engine 4 triggers the S4.2 operation: generating an emotional semantic prompt label 402, such as "The current operation error rate is high, it is recommended to pause and adjust", and obtaining the character coordinates through the game engine event system, and dynamically superimposing the label to the top of the game character's head (3D coordinate positioning of the UI layer to ensure visibility but not block the operation interface), guiding the player to perform cognitive re-evaluation (such as reflecting on operation strategies and regulating emotions).

[0093] Channel 1 targets the physiological basis of emotions (autonomic nervous system imbalance) and quickly regulates physiological arousal levels through breathing guidance;

[0094] Channel 2 targets situational triggers of emotions (operational failure + social conflict), intervenes in cognitive processes through semantic cues, and achieves dual intervention of "physiological soothing + cognitive regulation".

[0095] Built-in dual-channel state machine, parallel monitoring of two conditions:

[0096] Channel 1 state machine: Trigger when the cumulative duration of LF / HF>3.0 reaches 5 seconds;

[0097] Channel 2 state machine: detects whether the operation error count is ≥ 3 times and the chat contains negative keywords. If both conditions are met, it will be triggered.

[0098] The two conditions are independent of each other and can trigger different interventions at the same time (rendering animation and generating labels at the same time).

[0099] In one embodiment of the present application, Figure 1-Figure 4 As shown, with respect to the breathing guidance animation 401 and the emotional semantic prompt label 402, it can be understood that:

[0100] 1. Rendering and control of breathing guided animation 401

[0101] The breathing guidance animation 401 is generated by the animation rendering submodule 41 of the dynamic intervention engine 4 and is implemented by calling the graphics rendering interface (GPU Instancing of Unity or Niagara system of Unreal) of the game engine (such as Unity / Unreal).

[0102] Translucent corrugated form:

[0103] Create a translucent material through the engine material system (Alpha value is set to 0.3-0.5), define the ripple as a ring or wavy grid, and the edge area refers to the ring area 50-100 pixels away from the boundary around the game interface (such as the top, bottom, left and right edges of the screen).

[0104] Frequency matching 6 times / minute:

[0105] The animation rendering submodule 41 has a built-in timer component, which controls the ripples to gradually expand and contract from the edge of the interface to the center, with a breathing cycle of 10 seconds (6 times / minute corresponds to a complete deep breath every 10 seconds):

[0106] First 5 seconds: The ripples expand from the edge to the center (simulating inhalation) at a rate of 10-20 pixels per second;

[0107] Last 5 seconds: The ripples shrink from the center to the edge (simulating exhalation), and the shrinkage speed is consistent with the expansion, forming a looping animation.

[0108] Intervention trigger logic:

[0109] When the edge computing unit 3 sends an intervention instruction (channel 1 trigger), the animation rendering submodule 41 immediately activates the ripple animation at the edge of the interface until the emotional risk level drops below the threshold (such as ≤ level 3).

[0110] 2. Dynamic display of emotional semantic prompt label 402

[0111] The emotional semantic prompt tag 402 is generated by the semantic generation submodule 42 of the dynamic intervention engine 4 and displayed through the UI system of the game engine.

[0112] Semantic generation logic:

[0113] The semantic generation submodule 42 is based on a preset emotional label template library (such as "The current emotion is quite excited, it is recommended to pause the game" and "The operation error rate is increasing, please pay attention to the rhythm"), combined with the emotional risk level output by the edge computing unit 3, and dynamically combines and generates prompt text (such as "The emotional risk level is high, adjust the status immediately" when the risk level is 4).

[0114] Character head positioning:

[0115] Get the 3D world coordinates (Position) of the game character in real time through the game engine's event system (such as Unity's EventSystem or Unreal's GameplayAbiltySystem);

[0116] Convert the character's coordinates to screen space coordinates (ScreenPointToRay), generate a UI text object 1-2 meters (in-game units) above the character's head, and set the anchor point to the center of the screen to ensure that the label is always displayed directly above the character model.

[0117] Hierarchy management: The label UI layer is set higher than the game character model layer but lower than the operation button layer (for example, the Canvas sorting layer of the UGUI is set to 10) to avoid blocking interactive elements such as skill buttons;

[0118] Dynamic disappearance mechanism: The label is displayed for 5-8 seconds, or is hidden immediately after the emotional risk level decreases.

[0119] It should be noted that the translucent material and edge rendering strategy of the breathing animation ensure that the core game screen (such as the combat area) is not blocked;

[0120] The label is positioned above the character's head rather than in a fixed position on the screen. It moves with the character, which is in line with the player's visual habits (the line of sight naturally focuses on the character).

[0121] A breathing rhythm of 6 breaths per minute corresponds to the recommended frequency of "Mindfulness Breathing Therapy", which can effectively reduce heart rate and anxiety levels;

[0122] The labels above the characters' heads conform to the game UI design specifications (such as the health bar and status prompt positions in MOBA games), reducing the cognitive load on players.

[0123] In one embodiment of the present application, Figure 1-Figure 4 As shown, the wearable device 1 integrates an HRV sensor 11 , an EDA sensor 12 , a Bluetooth transmission module 13 and a vibration motor 14 .

[0124] It is understandable that 1. Hardware integration and functions of wearable devices 1

[0125] HRV sensor 11: Uses a photoelectric heart rate sensor (PPG principle) with a sampling rate of ≥100 Hz to ensure the capture of high-frequency details of heart rate variability (HRV) (such as millisecond-level fluctuations between adjacent heartbeats) for analyzing autonomic nervous activity (the basis for calculating the LF / HF ratio).

[0126] EDA sensor 12: Based on the constant potential method design, with a sampling rate of ≥50Hz, it monitors the microvolt-level changes in skin electrodermal activity (EDA) in real time to reflect the level of emotional arousal (such as sweat gland secretion leading to increased skin electrodermal activity during anxiety).

[0127] Bluetooth transmission module 13: uses Bluetooth 5.0 protocol (transmission rate ≥ 2Mbps, delay ≤ 50ms) to transmit HRV / EDA raw data to the edge computing unit 3 in real time, supporting stable connection within a range of 10 meters.

[0128] Vibration motor 14: Integrated linear resonator (LRA), supports ≥3 levels of vibration intensity adjustment (e.g., 50mA / 100mA / 150mA current corresponds to low / medium / high vibration), and controls the vibration frequency and duration through the PWM signal of the device control submodule 43 (e.g., continuous vibration for 10 seconds when channel 1 is triggered).

[0129] 2. Processing flow of edge computing unit 3

[0130] As an independent embedded module or integrated into a game terminal (such as a PC host, a game phone), it establishes a connection with the wearable device 1 through a Bluetooth controller (such as a CSR8670 chip) and shares memory with the game client 5 to obtain operation logs.

[0131] Feature extraction module 32: pre-processes the HRV sensor 11 data by removing motion artifacts (using a 50 Hz low-pass filter), calculating the interval between adjacent heart beats (RR interval), decomposing the LF (0.04-0.15 Hz) and HF (0.15-0.4 Hz) frequency band power through fast Fourier transform (FFT), and outputting the LF / HF ratio;

[0132] The data of the EDA sensor 12 is subjected to a sliding average filter (window size 1 second), and the first-order derivative (ΔEDA / Δt) is calculated to characterize the rate of change of the skin electrical current.

[0133] Lightweight LSTM prediction model 31: Model input: LF / HF ratio, EDA derivative, operation error rate (provided by game log capture interface 2);

[0134] Model output: emotional risk levels of 1-5 (e.g., level 5 corresponds to the critical state of "red temperature"), converted into probability distribution through the softmax layer, with inference delay ≤ 200ms (due to model size ≤ 5MB, quantitative design of claim 2).

[0135] 3. Response Mechanism of Dynamic Intervention Engine 4

[0136] Communicate with the edge computing unit 3 in real time through the game engine's message system (Unity's EventBus or Unreal's MessagePassing) to receive emotional risk levels and intervention instructions (such as triggering S4.1 / S4.2 / S4.3).

[0137] Animation rendering submodule 41: calls the game engine graphics interface (OpenGLES3.0 or Vulkan) to render translucent ripples at the edge of the interface. The ripple vertex coordinates are calculated based on the screen coordinate system to ensure that the edge area (within 50 pixels from the screen boundary) is displayed without blocking the operation buttons.

[0138] Semantic generation submodule 42: Generates prompt text (e.g., "Current emotional risk level 4, deep breathing recommended") based on a rule engine or pre-trained language model (GPT-2 lightweight version), and uses NLG technology to make the text dynamic (insert specific values ​​"LF / HF = 3.5");

[0139] Text rendering uses the game engine UI component (Unity's TextMeshPro), and the anchor point is set to the screen position corresponding to the 3D coordinates of the character's head.

[0140] Device control submodule 43: Sends instructions to the vibration motor 14 of the wearable device 1 through the Bluetooth Low Energy (BLE) protocol. The instruction format includes vibration intensity (level 1-3), duration (1-10 seconds) and interval frequency (such as 2 seconds of vibration at an interval of 1 second) to achieve graded tactile reminders (such as 3 medium-intensity vibrations during the second-level warning).

[0141] 4. System workflow:

[0142] 1. Data collection stage:

[0143] The HRV sensor 11 / EDA sensor 12 of the wearable device 1 collects physiological signals at a sampling rate of ≥100 Hz / ≥50 Hz, and transmits the signals through the Bluetooth transmission module 13 to the feature extraction module 32 of the edge computing unit 3 .

[0144] 2. Analysis and decision-making stage:

[0145] The feature extraction module 32 calculates LF / HF and EDA derivatives → the lightweight LSTM prediction model 31 outputs the emotional risk level → the dynamic intervention engine 4 triggers the corresponding sub-module according to the threshold.

[0146] 3. Intervention implementation phase:

[0147] The animation rendering submodule 41 generates a breathing guidance animation 401 , the semantic generation submodule 42 displays a prompt label, and the device control submodule 43 triggers the vibration motor 14 , forming a real-time closed loop of “physiological monitoring-risk prediction-multimodal intervention”.

[0148] In one embodiment of the present application, Figure 1-Figure 4 As shown, regarding the deployment of the edge computing unit 3 and the dynamic intervention engine 4, it can be understood that:

[0149] 1. Local deployment and data interaction of edge computing unit 3

[0150] The edge computing unit 3 is deployed on the user's local terminal (such as a PC, game console or high-performance mobile phone), runs as an embedded module (Raspberry Pi 4B) or a software process, directly accesses the terminal's memory bus, and establishes a shared memory channel with the game client 5 (operating system-level shared memory partition, implemented by CreateFileMapping under Windows and shm_open under Linux).

[0151] During the operation, the game client 5 writes behavioral data such as operation error rate and team interaction event frequency in real time through shared memory. The edge computing unit 3 directly reads the data through memory mapping without the need for network transmission or file IO, and the delay is ≤10ms.

[0152] The feature extraction module 32 and the lightweight LSTM prediction model 31 of the edge computing unit 3 complete real-time calculations (LF / HF ratio, emotional risk level) based on shared memory data, and the results are fed back to the game client 5 or the dynamic intervention engine 4 through shared memory.

[0153] 2. Plug-in embedding and coordinate acquisition of dynamic intervention engine 4

[0154] Dynamic Intervention Engine 4 is embedded in the Unity or Unreal game engine as a native plug-in (Unity's C# plug-in or Unreal's C++ plug-in), loaded as part of the engine runtime, and functional integration is achieved through the plug-in interface provided by the engine (Unity's IPluginManager, Unreal's IPlugin).

[0155] Character coordinate acquisition process:

[0156] Event system monitoring:

[0157] In the game engine, the character's movement, attack and other behaviors trigger engine events (Unity's OnPlayerMoved event, Unreal's ActorComponentUpdated event). The semantic generation submodule 42 of the dynamic intervention engine 4 captures the character's 3D world coordinates (X, Y, Z) in real time by registering event callback functions.

[0158] Coordinate transformation logic:

[0159] Get the current view matrix and projection matrix through the camera component of the engine rendering pipeline;

[0160] Use the engine API (Unity's Camera.WorldToScreenPoint, Unreal's ProjectWorldToScreen) to convert the character's world coordinates to screen coordinates (ScreenX, ScreenY);

[0161] An emotional semantic prompt label 402 is generated in the UI layer based on the screen coordinates, and a fixed offset (Y axis +100 pixels) is added to position it at the top of the character's head to ensure that the label is within the field of view and does not block the character model.

[0162] 3. System collaborative workflow:

[0163] 1. Data processing of edge computing unit 3:

[0164] Obtaining behavioral data of the game client 5 from the shared memory, combining it with physiological data of the wearable device 1, and outputting an emotional risk level through a lightweight LSTM prediction model 31;

[0165] The level data is sent to the dynamic intervention engine 4 via the shared memory or the engine message system.

[0166] 2. Response of Dynamic Intervention Engine 4:

[0167] After receiving the intervention instruction, the semantic generation submodule 42 calls the engine event system to obtain the target character coordinates;

[0168] A label UI object is generated according to the coordinates and displayed above the character's head through the engine's UI rendering system (Unity's UGUI, Unreal's Slate). At the same time, the animation rendering submodule 41 triggers the breathing guide animation 401.

[0169] In one embodiment of the present application, Figure 1-Figure 4 As shown, a parent supervision module 6 is added, which is connected to the edge computing unit 3 through an encrypted API to obtain emotional risk reports in real time and push them to the guardian terminal 7.

[0170] It can be understood that the parental supervision module 6 is an independent software module deployed on a cloud server or a user's local network device (home router), and establishes a communication link with the edge computing unit 3 through an encrypted API interface.

[0171] Encryption protocol: HTTPS1.3 protocol (based on TLS1.3 encryption algorithm) is used to ensure the confidentiality of emotional risk reports during data transmission.

[0172] Data acquisition and processing process:

[0173] After generating the emotional risk level (output by the lightweight LSTM prediction model 31), the edge computing unit 3 transmits the data to the data receiving interface of the parent supervision module 6 through shared memory or local API.

[0174] Transmission frequency: Synchronize real-time risk levels once per second; generate daily emotional risk reports (including statistical data such as the number of red temperature occurrences, average duration, and triggering causes).

[0175] The parent monitoring module 6 de-identifies the original physiological data (HRV, EDA) (retaining only the anonymized user ID and timestamp) and only transmits the aggregated risk level and statistical indicators to avoid leakage of sensitive physiological information.

[0176] Push mechanism of guardian terminal 7

[0177] Terminal type: The guardian terminal 7 is a mobile device such as a smartphone or tablet computer, which is installed with a dedicated supervision APP (such as "Youth Game Emotion Manager") and maintains real-time communication with the parent supervision module 6 through a long connection protocol (WebSocket).

[0178] Push content and format:

[0179] Real-time warning: When the edge computing unit 3 detects an emotional risk level ≥ 4 (red temperature warning), the parent supervision module 6 immediately pushes a notification to the guardian terminal 7, which includes the content "The current game emotional risk is high, it is recommended to pay attention to the child's status" and the timestamp of the risk occurrence.

[0180] Daily Report: A visual sentiment report is automatically generated at 10:00 PM every day (a bar chart shows the risk level distribution in each time period, and a line chart shows the correlation between operational error rate and mood fluctuations) and sent via in-app message or email.

[0181] Guardians can set personalized warning thresholds through the terminal (such as customizing the red temperature level to level 4 or level 5), or apply to view historical intervention records (such as the number of times the breathing guidance animation 401 is triggered on a certain day of a certain month). The data is stored in the encrypted database of the parent supervision module 6 (the storage period is ≤3 months, in line with the principle of data minimization).

[0182] In one embodiment of the present application, Figure 1-Figure 4 As shown, the HRV sensor 11 and the EDA sensor 12 of the wearable device 1 are integrated on the same PCB board, and the sensor spacing is ≤10 mm to reduce motion artifacts.

[0183] It is understandable that the core hardware of the wearable device 1 is a multi-layer printed circuit board (PCB board) made of FR-4 material (thickness 1.6 mm) and integrated with the HRV sensor 11 and the EDA sensor 12 using a surface mount (SMT) process.

[0184] The HRV sensor 11 uses a photoplethysmography (PPG) sensor chip (Maxim Integrated MAX30102), which includes a red LED, an infrared LED, and a photodetector to collect heart rate signals at the fingertips or wrist.

[0185] The EDA sensor 12 uses a constant potential method circuit (TIA DS1256 analog-to-digital converter) to measure the change in skin resistance by contacting the skin with two silver / silver chloride electrodes (Ag / AgCl).

[0186] The two sensors are soldered on the same plane of the PCB, with a sensor spacing of ≤10mm (measured spacing is 8-10mm). Mutual interference is reduced by sharing a ground layer and an electromagnetic shielding cover (metal foil covering).

[0187] Causes of motion artifacts:

[0188] When the wearable device 1 is in motion (such as operating a game controller with the hand), mechanical vibration and skin deformation will occur, causing fluctuations in the optical coupling efficiency of the HRV sensor 11 and changes in the EDA electrode contact impedance, resulting in signal noise (such as HRV signal baseline drift and EDA signal glitches).

[0189] The spacing of ≤10mm enables the two sensors to maintain extremely small relative displacement during exercise (for example, when the wrist swings, the displacement difference between the two is ≤0.5mm), ensuring the temporal synchronization (sampling time deviation ≤1ms) and spatial consistency (reflecting the physiological state of the same local skin area) of HRV and EDA signals.

[0190] Use a flexible printed circuit (FPC) to connect the PCB to the device body to reduce the impact of movement on the sensor position (FPC bending radius ≥ 3mm to avoid frequent bending that may cause solder joint breakage).

[0191] Signal processing flow:

[0192] The original photocurrent signal output by the HRV sensor 11 is converted into a voltage signal (gain 10^5V / A) by a transimpedance amplifier (TIA) and then passed through a 50Hz low-pass filter to remove power frequency interference;

[0193] The skin resistance signal collected by the EDA sensor 12 is amplified by an instrumentation amplifier (INA) (gain 100 times) and passed through a 1 Hz high-pass filter to remove DC drift.

[0194] The two analog signals are synchronously quantized by a 24-bit ADC chip (such as ADS1256, sampling rate ≥ 200 Hz) on the PCB board and transmitted to the Bluetooth transmission module 13 (NordicnRF52832 chip) through the SPI bus to ensure that the timestamp error of HRV and EDA data is ≤ 10 μs.

[0195] It should be noted that the control method of the present application can be automatically controlled by a controller, and the control method of the controller can be implemented by simple programming by technicians in this field, which is common knowledge in this field. In addition, the present application is mainly used to protect mechanical structures, so the control method and circuit connection are no longer explained in detail in this application.

[0196] Specifically, in the actual implementation process, taking a 15-year-old teenager using this system to control the emotions of MOBA games as an example, the workflow is as follows:

[0197] 1. Wearing and initializing the device

[0198] The player wears a wearable wristband (wearable device 1) that integrates an HRV sensor 11, an EDA sensor 12, a Bluetooth transmission module 13 and a vibration motor 14. The sensor spacing is 8mm (≤10mm) and it fits tightly to the wrist skin through a conductive gel layer.

[0199] Start the game client 5, the dynamic intervention engine 4 is automatically embedded in the Unity engine as a plug-in, the edge computing unit 3 runs in the background of the local PC, establishes a connection with the wearable device 1 via Bluetooth, and establishes a shared memory channel with the game client 5.

[0200] The parent supervision module 6 subscribes to the emotional data of the edge computing unit 3 through the HTTPS encrypted API, and the guardian's mobile phone (guardian terminal 7) logs in to the dedicated APP to enable the real-time warning function.

[0201] 2. Multimodal data acquisition and processing

[0202] Game start phase:

[0203] The HRV sensor 11 collects heart rate data at a sampling rate of 100 Hz, and the EDA sensor 12 collects skin electrical data at a sampling rate of 50 Hz. After being synchronously quantized by the 24-bit ADC on the PCB board, it is sent to the edge computing unit 3 through the Bluetooth transmission module 13 (delay ≤ 50 ms).

[0204] The game log capture interface 2 records the number of skill release errors (such as 2 empty skill releases per minute) and negative keywords in team chat (such as "waste") in real time, and transmits them to the edge computing unit 3 through shared memory.

[0205] Edge computing unit 3 processes:

[0206] The feature extraction module 32 performs FFT transformation on the HRV data, calculates LF / HF = 2.8 (calm state), EDA derivative = 0.3 μS / s, operation error rate = 2 times / minute, inputs the lightweight LSTM prediction model 31, and outputs risk level 2 (low risk).

[0207] 3. Emotional risk escalation and intervention triggers

[0208] Teamfight failure scenario:

[0209] The player made three consecutive skill release errors (the error rate rose to 5 times / minute), the negative keyword "junk operation" appeared in the team chat, and the game log capture interface 2 generated abnormal behavior signals and social conflict signals.

[0210] At the same time, the HRV sensor 11 detects that LF / HF = 3.2 (lasting for 6 seconds) and EDA derivative = 1.2 μS / s (significantly increased). The feature extraction module 32 determines that the dual-channel trigger condition is met:

[0211] Channel 1: LF / HF>3.0 and lasts for 5 seconds, triggering the animation rendering submodule 41 to render the breathing guidance animation 401 (semi-transparent ripples, 6 times / minute rhythm) at the edge of the game interface;

[0212] Channel 2: 3 consecutive mistakes + negative keywords, triggering the semantic generation submodule 42 to display the emotional semantic prompt label 402 ("The current error rate is high, it is recommended to operate calmly") above the character's head.

[0213] The device control submodule 43 sends a vibration instruction to the wearable device 1, and the vibration motor 14 triggers intermittent vibration with a current of 100 mA (vibration for 2 seconds / pause for 1 second, and a cycle of 3 times).

[0214] 4. Intervention effect feedback and parent supervision

[0215] Player Response:

[0216] The respiratory rate was adjusted following the breathing animation. After 5 minutes, the LF / HF ratio dropped to 2.5, the EDA derivative dropped back to 0.5 μS / s, the operation error rate dropped to 1 time / minute, and the lightweight LSTM prediction model 31 outputted a risk level of 1, and the intervention measures were automatically stopped.

[0217] Parent Supervision Module 6:

[0218] When the intervention is triggered, the edge computing unit 3 sends a real-time warning to the parent supervision module 6 through the encrypted API, and the guardian terminal 7 receives the notification "The child's gaming emotional risk level is level 4, and intervention has been triggered."

[0219] At 10 p.m. every day, the Parents APP generates a report: There was 1 red temperature event today, and the emotional recovery time after intervention was shortened by an average of 40% compared with the previous week, with a line graph of the correlation between operational error rate and emotional fluctuations.

[0220] In summary, the game emotion regulation system and method based on real-time monitoring of multimodal physiological signals in the embodiments of the present application, through the collaboration of wearable devices and game plug-ins, can achieve accurate prediction and scenario-based regulation of teenagers' game emotions, filling the gaps in existing technologies in terms of real-time performance, accuracy and scenario adaptability.

[0221] In the description of this specification, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include at least one of such features. In the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0222] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0223] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and deform the above embodiments within the scope of the present application.

Claims

1. A game emotion regulation method based on real-time monitoring of multimodal physiological signals, characterized in that: The following steps are involved: S1. collecting physiological signals of the player in real time through the wearable device (1), wherein the physiological signals include at least heart rate variability (HRV) and electrodermal activity (EDA); S2. Synchronously record game behavior data through the game log capture interface (2), the behavior data including operation error rate and team interaction event frequency; S3. Inputting the physiological signals and behavioral data into the machine learning model of the edge computing unit (3) to generate a predicted value of the emotional risk level; S4. When the predicted value exceeds a preset threshold, the dynamic intervention engine (4) performs at least one of the following operations: S4.1 Rendering a breathing guidance animation on the game interface (401); S4.2 generates an emotional semantic prompt tag (402) and superimposes it on the game screen; S4.3 sends a vibration control instruction to the wearable device (1).

2. The game emotion regulation method based on real-time monitoring of multimodal physiological signals according to claim 1 is characterized in that: The machine learning model in step 3 is a quantized LSTM network with a model size of ≤5MB; Input features include: HRV low-frequency / high-frequency power ratio (LF / HF), EDA signal first-order derivative, and the number of skill release errors per unit time.

3. The game emotion regulation method based on real-time monitoring of multimodal physiological signals according to claim 1 is characterized in that: The trigger of step 4 adopts a dual-channel mechanism: Channel 1: When LF / HF>3.0 and lasts for 5 seconds, operation S4.1 is triggered immediately; Channel 2: When there are three consecutive operational errors and the team chat contains negative keywords, operation S4.2 is triggered.

4. The game emotion regulation method based on real-time monitoring of multimodal physiological signals according to claim 1 is characterized in that: The breathing guidance animation (401) is rendered in the form of semi-transparent ripples at the edge of the game interface, and the ripple expansion frequency matches the deep breathing rhythm of 6 times / minute; The emotional semantic prompt label (402) is dynamically displayed above the head of the game character.

5. A game emotion regulation system based on real-time monitoring of multimodal physiological signals, used to implement the method described in any one of claims 1 to 4, characterized in that: The wearable device (1) integrates an HRV sensor (11), an EDA sensor (12), a Bluetooth transmission module (13) and a vibration motor (14), wherein: The HRV sensor (11) has a sampling rate of ≥100 Hz; The sampling rate of the EDA sensor (12) is ≥50 Hz; The edge computing unit (3) is connected to the wearable device (1) via Bluetooth and includes a lightweight LSTM prediction model (31) and a feature extraction module (32); The lightweight LSTM prediction model (31) is used to output the emotional risk level; The feature extraction module (32) calculates the LF / HF ratio and the EDA signal slope; The dynamic intervention engine (4) is connected to the edge computing unit (3) via a data bus, and includes an animation rendering submodule (41), a semantic generation submodule (42), and a device control submodule (43); The animation rendering submodule (41) calls the game engine graphic interface to generate breathing ripples; The semantic generation submodule (42) outputs the emotion prompt text based on NLG technology; The device control submodule (43) sends a vibration instruction to the wearable device (1).

6. The game emotion regulation system based on real-time monitoring of multimodal physiological signals according to claim 5 is characterized in that: The edge computing unit (3) is deployed on the user's local terminal and shares memory resources with the game client (5); The dynamic intervention engine (4) is embedded in the game engine (Unity / Unreal) in the form of a plug-in, and obtains character coordinate data through the engine event system to locate the display position of the emotional semantic prompt label (402).

7. The game emotion regulation system based on real-time monitoring of multimodal physiological signals according to claim 5 is characterized in that: A parent supervision module (6) is added, which is connected to the edge computing unit (3) through an encrypted API to obtain emotional risk reports in real time and push them to the guardian terminal (7).

8. The game emotion regulation system based on real-time monitoring of multimodal physiological signals according to claim 5 is characterized in that: The HRV sensor (11) and the EDA sensor (12) of the wearable device (1) are integrated on the same PCB board, and the sensor spacing is ≤10 mm to reduce motion artifacts.

Images