A human-computer interaction control method and system based on physiological state gating

Abstract

This invention discloses a human-computer interaction control method and system based on physiological and behavioral synergy, relating to the field of cognitive training and assessment technology. This method aims to address the technical problems of existing cognitive intervention methods lacking objective physiological feedback constraints and failing to ensure effective investment of subjects' neural resources. Its core scheme includes: real-time acquisition of subjects' physiological characteristic signals (such as electroencephalogram signals), mapping their characteristic index values ​​to state attribute parameters (such as stability) of virtual interactive objects in the display interface; receiving external behavioral input commands from subjects during cognitive tasks presenting target stimuli; executing a dual-gating judgment logic upon response to the command, i.e., simultaneously judging the accuracy of the behavioral response and the qualification of the physiological state parameters within the instantaneous time window when the behavioral input is triggered; determining the current interaction as a valid success event and providing positive feedback only when both simultaneously meet preset conditions. This invention significantly improves the targeting and accuracy of cognitive training by constructing a closed-loop mechanism with "behavior-physiology" as mutual prerequisites, and provides objective assessment data with high ecological validity for early auxiliary screening of cognitive impairments such as MCI.

Description

Technical Field

[0001] Background Technology: With the accelerating aging of the global population, neurodegenerative diseases, represented by Alzheimer's disease (AD), have become a serious public health challenge. Mild cognitive impairment (MCI) is a transitional stage between normal aging and dementia, and is considered the "golden window" for cognitive intervention. Research shows that cognitive training targeting key brain regions such as the prefrontal cortex can utilize the principle of neuroplasticity to slow cognitive decline.

[0002] Current cognitive assessment and training methods have the following main shortcomings: 1) Lack of objective real-time physiological feedback constraints: Traditional computerized cognitive training (such as simple memory games or reaction speed tests) mainly relies on external behavioral data (such as click accuracy and reaction time) to evaluate training effectiveness. However, for older adults, the same behavioral results may originate from different neural mechanisms (e.g., whether by actually activating damaged neural circuits or simply relying on residual habitual reflexes or guesswork). Existing systems cannot distinguish between these two situations, resulting in training failing to accurately target neural pathways and leading to "ineffective training."

[0003] 2) The assessment methods have low ecological validity and are outdated: Commonly used cognitive scales in clinical practice (such as MMSE and MoCA) are cross-sectional "snapshot" assessments, which are easily affected by the subject's emotions, educational background and the pressure of the testing environment, and are difficult to capture the subtle fluctuations in cognitive abilities in daily life.

[0004] 3) The training process is tedious and compliance is poor: Traditional neurofeedback training often requires users to control abstract waveforms or charts, which is too tedious and obscure for the elderly and difficult to persist in for a long time.

[0005] Therefore, there is an urgent need for a new type of human-computer interaction system that can deeply integrate objective real-time physiological state monitoring with specific cognitive behavioral tasks, thereby improving the neural accuracy of training while also taking into account both fun and objective evaluation.

[0006] Summary of the Invention The present invention aims to solve the technical problems existing in the prior art, such as the lack of objective physiological constraints in cognitive training, difficulty in ensuring the effective investment of subjects' neural resources, and single and lagging assessment data, and proposes a human-computer interaction control method and system based on physiological state gating.

[0007] The core idea of ​​this invention lies in constructing a "physiological gating" mechanism: physiological state indicators are used as necessary prerequisites for evaluating the effectiveness of a human-computer interaction, rather than merely as background data. Specifically, this invention introduces a mandatory rejection mechanism: when a user's external behavior is correct but their internal physiological state is not up to standard, the interaction is deemed invalid and not counted in the score, thus forcing users to complete cognitive tasks while maintaining a specific physiological state (such as high concentration). This invention also constructs a "dual-gating" interaction judgment mechanism: upgrading the traditional task success judgment that relies solely on external behavioral input (such as clicks) to a collaborative judgment that relies on the simultaneous satisfaction of both "accuracy of external behavior" and "qualification of internal physiological state."

[0008] To achieve the above objectives, the present invention provides a human-computer interaction control method based on physiological state gating, comprising the following steps: S1: During the process of the user performing human-computer interaction tasks, at least one physiological characteristic signal of the user (such as EEG, EDA, etc.) is collected in real time, and the physiological characteristic signal is processed within a preset instantaneous time window to extract the corresponding physiological state index value (such as focus index, relaxation index). S2: Present the user with stimulus information (e.g., target stimulus Go and interference stimulus No-Go) corresponding to the current human-computer interaction task through the interactive interface. S3: Receive external behavioral input instructions (e.g., clicks, buttons) from the user in response to the stimulus information; S4: Within the same time frame or a preset millisecond-level time window when the external behavior input instruction is triggered, perform an interaction validity determination. This determination includes at least two parallel or serial sub-determinations: S4-1: Determine whether the external behavior input instruction meets the preset behavior correctness conditions (e.g., whether the target stimulus was clicked correctly). S4-2: Determine whether the physiological state index value corresponding to the instruction trigger falls within the preset qualified threshold range (e.g., whether the focus is greater than 60). S5: Only when the correctness condition of the behavior and the qualification condition of the physiological state are met simultaneously, the external behavior input instruction is determined to be a valid interaction behavior, and a corresponding human-computer interaction feedback (such as a score) is generated accordingly; in particular, if the correctness condition of the behavior is met but the qualification condition of the physiological state is not met, the external behavior input instruction is determined to be an invalid interaction behavior, and it will not be used as a valid interaction result in the scoring, status update or result statistics of the subsequent human-computer interaction process.

[0009] Furthermore, in order to provide a better user experience and training assistance, as a preferred implementation, the method further includes mapping the physiological state index values ​​extracted in step S1 to the state attribute parameters (such as position stability, transparency, etc.) of at least one virtual interactive object in the interactive interface in real time, so as to provide the user with real-time feedback on their current physiological state level in a visual manner.

[0010] The present invention also provides a system for implementing the above method, including a signal acquisition and processing module, an interactive interface display module, a behavior input receiving module, and a central gate control determination module.

[0011] Compared with the prior art, the present invention has the following beneficial effects: 1) A strict physiological constraint mechanism has been established: Through a unique "gating" logic, especially the negative judgment of the situation of "behavior is correct but physiology is wrong", it effectively eliminates the lucky success operation generated by the user in a state of low neural resource input (such as distraction or guessing), and ensures that every recorded "effective interaction" corresponds to the real activation of the target neural circuit.

[0012] 2) Provides objective assessment data with high ecological validity: This system can simultaneously collect behavioral and physiological data during gamified natural interaction and analyze the coupling relationship between the two (such as the "neuro-behavioral decoupling" phenomenon), providing more sensitive and objective biomarkers for the early identification of MCI than simple behavioral scales.

[0013] 3) Adaptive and personalized intervention is achieved: The system can adjust the difficulty of the task in real time according to physiological signals (such as controlling the game speed through relaxation level), preventing the elderly from giving up due to anxiety or frustration, and improving the safety and compliance of training. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the human-computer interaction control method provided in an embodiment of the present invention.

[0015] Detailed Description of Embodiments To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0016] Example 1: General Method and Core Mechanism. This example provides a human-computer interaction control method based on physiological state gating, the core of which is as follows: During system operation, step S1 is continuously executed, where sensors worn by the user (such as an EEG headband) collect physiological signals in real time, and the current "physiological state index value" (e.g., a 0-100 value reflecting the prefrontal cortex's focus level) is extracted through algorithm processing. Simultaneously, step S2 is executed, presenting stimulus information (such as randomly appearing targets and distractions) on the screen according to the cognitive task requirements. In step S3, the system waits for and captures "external behavioral input commands" such as key presses or clicks from the user. Once an input command is detected, the system immediately initiates the core "interaction validity determination" in step S4. This determination has extremely high real-time requirements, requiring a dual snapshot check of behavioral and physiological data within the same microsecond / millisecond time window triggered by the command: * Check 1 (behavioral level): Did the user click correctly? * Check 2 (physiological level): Does the user's current physiological state index meet the requirements? Step S5 involves triage based on the inspection results. The most crucial feature of this embodiment lies in its handling of the following situation: Suppose the user correctly clicks the target stimulus (behavioral correctness condition met), but the system detects that the user's focus index is below a preset threshold at the moment of click (physiological state qualification condition not met). In this case, the system resolutely determines the operation as an "invalid interaction." This means that although the user performed the correct action, due to a lack of necessary neurophysiological resources, the operation will not be included in the total score, nor will it trigger the progression of the game (such as level upgrades). This mechanism forces the user to adjust their internal state before performing any external action.

[0017] As a preferred implementation, to help users better regulate their own state, the physiological state index values ​​extracted in step S1 can be visualized in real time. For example, the focus value can be mapped to the shaking amplitude of a cursor or virtual object on the screen. When focus is high, the object is stable; when focus is low, the object shakes violently. This explicit visual feedback assists users in self-regulation, but regardless of whether such visual feedback is present, the gating judgment logic in S4 and S5 above always exists and is effective.

[0018] Example 2: Specific Application Scenario – “Zen Fishing” (Inhibition Control Task) This example demonstrates the application of the above method in a specific cognitive training game. This application includes the visual feedback feature described in claim 3.

[0019] Signal and task configuration: EEG signals were collected to calculate the "attention index". The cognitive task was a Go / No-Go paradigm, with "goldfish" (Go stimulus) or "crab" (No-Go stimulus) randomly appearing on the water surface.

[0020] Visual feedback (preferred feature): The virtual "fishhook" in the center of the screen has a stability parameter controlled by the focus index. When the focus is high, the fishhook glows steadily; when the focus is low, the fishhook shakes violently and dims.

[0021] Gating decision logic execution: When a goldfish appears, the user taps the screen (correct action), and the hook is in a stable state at the moment of tapping (focus meets the standard): it is considered a valid success, the goldfish is caught, and the system scores +10 points.

[0022] Key scenario: When the goldfish appears, the user taps the screen (correct action), but the fishhook is violently shaking at the moment of tapping (lack of focus): this is considered an invalid interaction. In this case, although the user taps the fish, it will escape. The system does not score this interaction (the score remains unchanged) and may provide a prompt such as "Too hasty, please calm down." This handling method precisely reflects the technical feature of claim 1, "not included in the scoring as a valid interaction result."

[0023] To achieve the above objectives, the present invention provides a human-computer interaction control method based on physiological and behavioral synergy, comprising the following steps: S1. Real-time acquisition of physiological characteristic signals of the subject (e.g., EEG, EDA, etc.), extraction of characteristic index values ​​(e.g., focus, relaxation) representing the current nervous system state or psychological load level, and real-time mapping of the characteristic index values ​​to state attribute parameters (e.g., stability, transparency, speed) of at least one virtual interactive object in the display interface; S2. Presenting a sequence of perceptual events containing target stimuli in the display interface according to preset cognitive task rules; S3. Receiving external behavioral input instructions triggered by the subject in response to the currently presented perceptual events; S4. In response to receiving the external behavioral input instructions, executing real-time validity judgment logic to determine whether the following two conditions are met simultaneously: (a) behavioral accuracy condition: whether the instruction is a correct response to the target stimulus; (b) physiological state qualification condition: whether the state attribute parameters of the virtual interactive object are within a preset qualified range within the instantaneous time window of the instruction triggering; S5. The current interaction is considered successful and positive feedback is generated only if both conditions in step S4 are met simultaneously; otherwise, it is considered a failure or invalid and corresponding negative or suggestive feedback is generated.

[0024] Furthermore, when the behavioral accuracy condition is met but the physiological state qualification condition is not met, the system will generate a specific feedback signal to indicate that although the subject's operation is correct, the current physiological / psychological state is not up to standard.

[0025] The present invention also provides a system for implementing the above method, including a signal acquisition and processing module, an interactive interface display module, a behavior input receiving module, and a central collaborative judgment module.

[0026] Example 1: This example provides a human-computer interaction control system based on physiological and behavioral coordination, including: Signal Acquisition and Processing Module: A portable wearable device (such as a single-channel prefrontal EEG headband, electrode positions Fp1 or Fp2) is used to acquire the subject's raw EEG signals in real time. This module incorporates algorithms to denoise and filter the signals, perform spectral analysis, and calculate characteristic indicators representing the subject's mental state in real time, such as the "Attention Index" based on the Beta / Theta wave ratio, or the "Meditation Index" based on Alpha wave intensity. The numerical range is typically normalized to 0-100.

[0027] Interactive interface display module: Typically a tablet or touch screen, used to present gamified virtual scenes, virtual interactive objects (such as fishhooks, crosshairs, control consoles, etc.), and cognitive task stimulus sequences (such as target objects and distracting objects). This module receives feature index values ​​from the signal processing module and changes the visual or physical properties of the virtual objects in real time accordingly.

[0028] Behavioral input receiving module: used to capture the subject's external operations, such as touch screen clicks, button presses, or gestures, and record the timestamps of the actions.

[0029] Central Coordination Decision Module: This is the core logical unit of the system. It synchronously receives current physiological characteristic index values ​​and behavioral input events, makes decisions based on preset dual gating logic, and controls the display module to output corresponding audiovisual feedback.

[0030] Reference Figure 1 The system operates as follows: After startup, step S1 is executed continuously to monitor the subject's physiological characteristics (such as concentration) in real time and map them to the attributes of virtual objects in the interface (such as the stability of a fishhook). Simultaneously, step S2 is executed, randomly presenting a target stimulus (Go) or a distractor stimulus (No-Go) according to a preset program. In step S3, the system waits for and receives clicks from the subject. Once a click is detected, step S4 is immediately initiated, where the system checks the moment the click occurs: ① Is the target stimulus on the screen? ② Are the physiological characteristics at this moment higher than a preset threshold (e.g., concentration > 60)? Finally, in step S5, feedback is given based on the results of these two checks.

[0031] Example 2: Specific Application Scenario - "Zen Fishing" (for inhibitory control and sustained attention training) This example demonstrates how to use the above system to implement a training task for the inhibitory function of the prefrontal cortex in the elderly.

[0032] Physiological signal mapping (S1): The system collects EEG signals and calculates the "attention index". The virtual object in the center of the interface is a "fishhook".

[0033] When concentration is high, the hook remains stable and still in the water, emitting a faint glow, indicating that the current state is good.

[0034] When concentration is low (distracted), the hook shakes violently in the horizontal direction and becomes semi-transparent, indicating that it cannot effectively bear force at present.

[0035] Cognitive Task (S2): Using the Go / No-Go paradigm. "Goldfish" (goal stimulus Go, accounting for 70%) or "crab / garbage" (distractor stimulus No-Go, accounting for 30%) randomly appear on the water surface of the screen.

[0036] Scenario A (Successful Capture): A goldfish appears on the screen, the subject taps the screen, and their focus index is high at the moment of tapping (the hook is stable). -> Judgment: Success, the fish is caught, and a score is awarded.

[0037] Scenario B (Neural Feedback Failure / Disengagement): A goldfish appears on the screen, and the subject taps the screen, but their focus index is low at the moment of tapping (the fishhook wobbles). -> Judgment: Failure, message "Too hasty, the hook wasn't steady, the fish got away," no points awarded. (Note: This is the invention) Unlike the key mechanics of ordinary games, it emphasizes the necessity of neural states. Scenario C (Suppression Failure): A crab appears on the screen, and the subject accidentally clicks on the screen. -> Judgment: Failure, fishing up trash, points deducted.

[0038] Example 3: Specific Application Scenario - "Magic Potion Shop" (for working memory and emotion regulation training) This example demonstrates the universality of the mechanism and applies it to working memory (N-Back) training.

[0039] Physiological signal mapping (S1): The system calculates the EEG "relaxation index" (based on alpha waves). The virtual objects in the interface are a "magic conveyor belt" that transports raw materials and a "pot" that is being simmered.

[0040] When the relaxation level is high, the conveyor belt speed is slow, and the liquid surface in the medicine pot is calm.

[0041] When the relaxation level is low (tense / anxious), the conveyor belt speed increases, the medicine pot boils violently, increasing the difficulty of operation and visual interference.

[0042] Cognitive Task (S2): Using a 1-Back paradigm. A conveyor belt delivers magical ingredients of different colors in sequence. The participant must determine whether the current ingredient is the same as the previous one.

[0043] When the subject believes they meet the rules and clicks the "Synthesis" button, the system checks: ① Is the memory judgment correct? ② Is the level of relaxation at the moment of clicking up to the required level (i.e., is the conveyor belt speed in the slow range)? Only actions performed with correct judgment and in a relaxed state are counted as valid synthesis. If a subject clicks correctly while under extreme tension (the conveyor belt is moving very fast), the system will prompt "Too tense, the medicine is unstable," guiding the subject to first take deep breaths to relax, thereby combining emotional regulation with cognitive training.

[0044] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A human-computer interaction control method based on physiological state gating, characterized in that, Includes the following steps: S1: During the process of the user performing human-computer interaction tasks, at least one physiological characteristic signal of the user is collected in real time, and the physiological characteristic signal is processed within a preset instantaneous time window to extract the corresponding physiological state index value. S2: Present stimulation information corresponding to the current human-computer interaction task to the user through the interactive interface; S3: Receive external behavioral input instructions from the user in response to the stimulus information; S4: Within the same time frame or a preset millisecond-level time window when the external action input command is triggered, perform an interaction validity determination, the interaction validity determination including at least: S4-1: Determine whether the external behavior input instruction meets the preset behavior correctness conditions; S4-2: Determine whether the physiological state index value corresponding to the external behavioral input command falls within the preset qualified threshold range. S5: Only when the behavior correctness condition and the physiological state qualification condition are met simultaneously, the external behavior input instruction is determined to be a valid interaction behavior, and a corresponding human-computer interaction feedback is generated accordingly. If the correctness condition of the behavior is met but the qualification condition of the physiological state is not met, then the external behavior input instruction is determined to be an invalid interaction behavior and will not be used as a valid interaction result in the scoring, status update or result statistics of the subsequent human-computer interaction process.

2. The method according to claim 1, characterized in that, The physiological characteristic signals include at least one of electroencephalogram (EEG), electrocardiogram (ECG), electrodermal conductance (EDA), or electrooculogram (EOG); the physiological state index values ​​include focus index, relaxation index, emotional arousal index, or cognitive load index calculated based on the above signals.

3. The method according to claim 1, characterized in that, The method further includes: mapping the physiological state index values ​​extracted in step S1 to state attribute parameters of at least one virtual interactive object in the interactive interface in real time, so as to provide users with real-time feedback on their current physiological state level in a visual manner; the state attribute parameters include at least one of the following or a combination thereof: spatial position stability (shaking amplitude) of the virtual object, movement speed, visual transparency, geometric size or color saturation.

4. The method according to claim 1, characterized in that, The human-computer interaction task in step S2 is configured as an inhibition control task paradigm (Go / No-Go), a working memory task paradigm (N-Back), or a sustained attention task paradigm (CPT); the stimulus information includes a target stimulus that requires a specific response, and interfering stimuli that require suppression of the response or a different response.

5. The method according to claim 1, characterized in that, The generation of corresponding human-computer interaction feedback in step S5 includes: when the judgment result is that the behavior correctness condition is met but the physiological state qualification condition is not met, a specific type of prompting feedback signal is generated to clearly inform the user that although the behavior response is correct, the current physiological state does not meet the requirements, thus distinguishing it from the feedback signal generated for behavior errors.

6. A human-computer interaction control system based on physiological state gating, characterized in that, include: The signal acquisition and processing module is used to acquire at least one physiological characteristic signal of the user in real time and extract the corresponding physiological state index value. An interactive interface display module is used to present stimulus information corresponding to the current human-computer interaction task to the user through the interactive interface; a behavior input receiving module is used to receive external behavior input instructions generated by the user in response to the stimulus information; a central gating judgment module is communicatively connected to the above modules, stores computer program instructions, and when the instructions are executed, executes the interaction validity judgment logic as described in claim 1 at the moment of receiving the external behavior input instruction, and controls the system to generate corresponding feedback based on the judgment result, including not scoring, updating the status, or statistically analyzing the results when the interaction behavior is judged to be invalid.

Images