Method for providing neurofeedback with minimal delay

Abstract

The claimed technical solution relates generally to the field of computing, and more particularly to a method for providing, in real time and with minimal delay, feedback that characterizes brain function. The technical result consists in achieving minimal delays in the real-time provision of feedback. The claimed technical result is achieved by a method for providing, in real time and with minimal delay, feedback that characterizes brain function, which comprises the steps of: recording EEG signals from EEG sensors attached to the scalp, or MEG signals from magnetometers placed in direct proximity to the head; and carrying out, with the aid of the recorded signals, a training session in which the user is shown the following, simultaneously but separately: an instantaneous feedback component that reflects a change in recorded brain activity, with a delay of not more than 100 ms, represented in the form of a change in the feedback signal; an integral feedback component that reflects changes in recorded brain activity which are accumulated over a minimum of 1 second, represented in the form of a gradual change in the feedback signal.

Description

[0001] A METHOD FOR IMPLEMENTING A NEUROFEEDBACK METHOD WITH MINIMUM DELAY

[0002] AREA OF TECHNOLOGY

[0003] The claimed technical solution generally relates to the field of computer technology, and in particular to a method for implementing feedback characterizing brain function in real time with minimal delay.

[0004] LEVEL OF TECHNOLOGY

[0005] Known from the prior art is patent for utility model RU207767U1 "LOW-LATENT NEUROFEEDBACK DEVICE", FEDERAL STATE AUTONOMOUS EDUCATIONAL INSTITUTION OF HIGHER EDUCATION "NATIONAL RESEARCH UNIVERSITY "HIGHER SCHOOL OF ECONOMICS", published 11 / 15 / 2021.

[0006] This patent describes a low-latency neurofeedback device comprising an elastic carrier in which electrodes are placed that record electroencephalography signals, characterized in that the elastic carrier is connected to a rim on which a multi-channel analog-to-digital converter unit is placed for digitizing a multi-channel electroencephalogram recorded using electrodes, and a compact high-speed monitor for presenting neurofeedback, equipped with a system for monitoring the total delay in presenting a neurofeedback signal, the multi-channel analog-to-digital converter unit for digitizing a multi-channel electroencephalogram contains an on-board computer implemented on a microprocessor, control buttons for setting neurofeedback parameters, such as electroencephalogram recording, electroencephalogram frequency band, status indicators,a power button and a micro-USB connector for communication with a personal computer for reprogramming the computer, a power battery and is connected to headphones for presenting neurofeedback via an acoustic channel.

[0007] The prior art includes patent RU2818466C1 "DEVICE FOR TRANSCRANIAL ELECTROSTIMULATION OF THE BRAIN", SIDORUK NIKOL, published 02.05.2024.

[0008] This patent describes a device for transcranial electrical stimulation of the brain, comprising a pulsed monopolar current signal generator, the output of which is connected to a first electrode, and a second electrode, characterized in that an electrode block for recording brain biopotentials, an electroencephalogram analyzer, a Bluetooth transmitting unit, a smartphone, and a Bluetooth receiving unit, the output of which is connected to the control input of the introduced digital potentiometer, the output of which is connected to the second electrode, and the input is connected to the output of the pulsed monopolar current signal generator, which is configured to supply the first electrode with a pulsed monopolar current with a frequency of 77 Hz and an amplitude value of current of 3 mA, wherein the electrode block for recording brain biopotentials contains three electrodes for installation in the ear leads Al,A2 and frontal lead Fz according to the "10-20" system and is configured to use both a monopolar and a bipolar circuit for recording the alpha rhythm of the electroencephalogram, the electrodes are connected by a shielded cable to an electroencephalogram analyzer, which is configured to pre-filter and amplify the biopotentials of the brain and convert the analog signal into a digital one for further processing, the smartphone is configured to analyze the alpha rhythm by amplitude and transmit a control feedback signal wirelessly to the Bluetooth receiving unit for controlling the digital potentiometer, which is configured to supply a pulsed stimulating current to the second electrode, the value of which depends on the amplitude value of the alpha rhythm of the electroencephalogram and varies from 3 to 30 mA.

[0009] Also known from the prior art is invention application RU2021133014A "Method for training skills based on biofeedback using brain-computer interface and virtual reality technologies", Limited Liability Company "REHAB-ZERO", published on May 15, 2023.

[0010] This application describes a method for training skills based on biofeedback using brain-computer interface technologies and virtual reality, characterized in that the training is carried out by presenting the patient in a virtual environment with exercises to imagine the movement of a paretic limb using a virtual reality helmet with the help of moving objects in the form of schematic hands of the patient with control of the patient's imagination of movement based on the analysis of the rhythmic characteristics of the electroencephalogram (EEG) specific to the imagination of limb movement, while the recording of EEG signals of the brain is carried out from a system of active electrodes of a neuroheadset placed on the surface of the head, and the transmission of this data to a computer for their synchronous processing and classification using the method of common spatial components and linear discriminant analysis,The results of the recognition of the task being performed are presented to the patient via visual feedback in a virtual reality helmet using animation of moving objects of the required movement task for the paretic or healthy limb.

[0011] In addition, the prior art includes patent RU2806480C1 "A METHOD FOR OPTIMIZING THE FUNCTIONAL STATE OF ATHLETES' BODY USING BRAIN RHYTHM NEUROCONTROL", published on 01.11.2023.

[0012] This patent describes a method of biofeedback implemented through graphical and gaming sessions. In the graphical session, the user monitors changes in the 0-rhythm level, while in the gaming session, they control the scenario, which depends on the success of rhythm modulation. Despite its high effectiveness in improving functional state, the method is limited to working with a single brain rhythm and is characterized by simplified gaming scenarios. This known solution does not provide for the use of complex forms of integrated and instantaneous feedback or adaptation to a wide range of tasks.

[0013] The disadvantages of the above-mentioned known solutions are the lack of the ability to visualize instantaneous changes in patterns of rhythmic brain activity with a delay of less than 100 ms in combination with the integral (accumulated) effect of training.

[0014] The technical solution proposed for consideration is aimed at eliminating the shortcomings of the current level of technology.

[0015] In addition, the claimed solution differs from the prior art solutions in that it uses a combination of the instantaneous and integral components of the change in the target parameter to generate a feedback signal, which significantly increases the efficiency of training.

[0016] Instant feedback is represented by changes in visual and auditory stimuli, such as the size of a visual effect or the speed of a game character. Integral feedback, on the other hand, reflects cumulative results, such as the number of points earned or the level achieved. Furthermore, the method enables the integration of game scenarios of varying complexity, including adaptation to the user's individual goals and modulation of multiple brainwaves simultaneously. This makes the proposed solution versatile and applicable in neurorehabilitation, cognitive training, and recreational gaming, expanding the capabilities of existing methods.

[0017] ESSENCE OF THE INVENTION

[0018] The technical challenge that the proposed solution aims to address is the creation of a new method for providing feedback characterizing brain function in real time with minimal delay.

[0019] An additional technical task of the claimed method is the creation of a method for presenting a feedback signal to implement an operant conditioning paradigm aimed at developing the user's skills in controlling the target parameter.

[0020] The developed method must implement the simultaneous presentation of instantaneous and integral feedback components.

[0021] The instantaneous feedback component is presented in real time with a delay of less than 100 ms and reflects the change in the target parameter on a characteristic time scale of the order of tens of milliseconds.

[0022] Integral feedback reflects accumulated training results on a time scale of one second or more, expressed in metrics such as the number of points, coins, or goals achieved in gameplay.

[0023] This method allows for the stimulation of targeted brain rhythms in real time through interactive brain-computer gaming interfaces and maintains user motivation at a high level.

[0024] Moreover, the claimed method allows for broad integration with various neurophysiological systems for stimulating nervous system activity. This makes it applicable for research purposes, the development of therapeutic applications, and use in entertainment and educational games.

[0025] Feedback can be provided through changes in visual, auditory, or other stimuli, such as the size of a visual effect or the speed of a game character. The game scenario can also be adapted to specific tasks, such as controlling the movement of a virtual object (a character, vehicle, or other element) in a space of varying complexity.

[0026] The primary focus should be on minimizing delays in the generation and presentation of the feedback signal and enhancing the visibility of graphical or other feedback components reflecting instantaneous changes in brain activity. This can be achieved through the use of modern visualization software and hardware and the development of an intuitive game scenario. The goal is to ensure high feedback speed, thereby ensuring the effective acquisition of biofeedback skills by leveraging the proven effect of reducing feedback signal latency.

[0027] The technical result fully corresponds to the set task and consists of achieving minimal delays in providing feedback in real time.

[0028] High speed is achieved through the use of highly effective low-level software mechanisms for controlling the actuator (monitor, video card, etc.). An instant feedback component ensures the engagement of subconscious regulatory mechanisms. The presented integrated component reflects the accumulated results, the overall success of the training, and increases the user's emotional engagement.

[0029] The claimed technical result is achieved through a method for implementing feedback characterizing the work of the brain in real time with minimal delay, comprising the following stages:

[0030] - record EEG signals from EEG sensors attached to the scalp, or MEG signals using magnetometers located in close proximity to the head;

[0031] - using the recorded signals, a training session is carried out, during which the user is presented simultaneously, but separately:

[0032] • an instantaneous feedback component reflecting a change in the recorded brain activity with a delay not exceeding 100 ms, presented as a change in the feedback signal;

[0033] • an integral feedback component reflecting changes in the recorded brain activity accumulated over at least 1 second and presented in the form of a smooth change in the feedback signal.

[0034] DESCRIPTION OF DRAWINGS

[0035] The invention will be further described in accordance with the accompanying drawings, which are provided to illustrate the invention and in no way limit its scope. The following drawings are attached to the application:

[0036] Figure 1 illustrates the positioning of the method for implementing instant feedback (block 5) as part of a real-time system for neurofeedback through a game interface. Figure 2 illustrates an example of implementing a feedback method using integral and instantaneous indicators in the “Pip the penguin!” game interface.

[0037] Figure 3 illustrates the visualization of feedback in the alternative game interface “Fountain of force”.

[0038] DETAILED DESCRIPTION OF THE INVENTION

[0039] The following detailed description of the invention includes numerous implementation details to provide a clear understanding of the present invention. However, it will be apparent to one skilled in the art how the present invention may be used with or without these implementation details. In other instances, well-known methods, procedures, and components have not been described in detail to avoid obscuring the features of the present invention.

[0040] Furthermore, it will be clear from the foregoing description that the invention is not limited to the embodiment described. Numerous possible modifications, changes, variations, and substitutions, while preserving the spirit and form of the present invention, will be apparent to those skilled in the art.

[0041] The claimed technical solution proposes a method for providing feedback characterizing brain function in real time with minimal delay.

[0042] Figure 1 shows a structural diagram of a system that implements a method for providing feedback characterizing brain function in real time with minimal delay, integrated with the game process, which includes the following components:

[0043] - A device for recording EEG signals (1), which ensures their transmission in the form of an LSL stream and contains an additional channel (AUX1) for connecting a photosensor.

[0044] - Data processing module (2), which includes several key stages: ICA (Independent Component Analysis): An algorithm for extracting independent components to exclude artifacts (e.g., myograms or signals from oculomotor activity).

[0045] Filters: Frequency filtering of signals to suppress noise and highlight target ranges.

[0046] Kalman: Application of dynamic filtering (a variant of the Kalman filter) to assess the phase and amplitude characteristics of brain rhythms.Scale: Data normalization for correct display of parameters in the game interface.

[0047] - Shared memory (3), which implements the mechanism for transferring processed data between the data processing module (2) and the game interface, ensuring minimal latency.

[0048] - Signal Latency Evaluation Window (4). This window (4) measures system latency using a photosensor. This allows for a precise determination of the time between EEG signal registration and the corresponding changes being displayed in the game.

[0049] A game interface (5) that implements a method for providing instant feedback. The graphical interface (5) is implemented in Python and displays gameplay, where the character's altitude and speed change based on the user's brain activity. The game serves as a feedback element, stimulating targeted brain rhythms.

[0050] The claimed solution is a biofeedback system for a selected parameter, in which the instantaneous and integral feedback components are presented to the user simultaneously.

[0051] The instantaneous feedback component is presented in real time and reflects the change in the target parameter on a characteristic time scale of tens of milliseconds.

[0052] The integral feedback component reflects the accumulated results of the user's activity on a time scale of one second or more, which is expressed in indicators such as the number of points collected, coins, or the achieved goal in the gameplay.

[0053] The proposed technical solution can be implemented as a game scenario in which the instantaneous component is presented as a short-term change in the feedback signal (changes in the size, color, or speed of a visual stimulus, volume of an auditory stimulus, etc.). The integral component corresponds to the accumulated changes and is reflected as a smooth change in the attributes of the visual or auditory stimulus. An alternative implementation could include other scenarios in which the instantaneous and integral components can be presented separately but simultaneously.

[0054] For example, Figure 2 illustrates the implementation of a feedback method using integral and instantaneous indicators in the "Pip the Penguin!" game interface. The integral indicator includes the height and number of coins—a generalized result of the user's activity, demonstrating the overall achievement for the current game session.

[0055] Instantaneous indicator: the intensity of the flame from the character's jetpack changes in real time depending on the current parameters of brain rhythms, recorded and processed by the system and displayed with a delay of no more than 100 ms.

[0056] Figure 3 illustrates the feedback visualization in the alternative game interface "Fountain of Force." Integral indicators: the ball filling level – displayed graphically (blue area) and as a numerical value. This indicator accumulates the user's long-term achievements, stimulating activity throughout the entire game session.

[0057] Instant indicators: the dynamics of the fountain's glowing drops, as well as the height of their rise.

[0058] The proposed technical solution achieves delays of tens of milliseconds when presenting an instantaneous feedback component. This high speed is achieved through the use of highly efficient low-level software mechanisms for controlling the actuator (monitor, video card, etc.), such as direct access to video memory, direct access to the gain control of the computer's audio card output stage, and the use of a high-speed monitor with a high frame refresh rate. The instantaneous feedback component ensures the engagement of subconscious regulatory mechanisms. The presented integral component reflects the accumulated results, the overall success of the training, and increases the user's emotional engagement.

[0059] The essential technical features of the instantaneous feedback component are:

[0060] Presentation time not exceeding 100 ms.

[0061] The presentation is carried out separately from the integral component, but simultaneously with it.

[0062] The essential technical features of the integral feedback component are:

[0063] Reflection of integral changes in the target parameter over a period of no less than 1 second. Presentation is performed separately from, but simultaneously with, the instantaneous component.

[0064] The differences from known analogues are: PC17RU2025 / 000163

[0065] Simultaneous but separate presentation of instantaneous and integral feedback components.

[0066] The instantaneous component is associated with the variability of the target parameter (e.g., alpha rhythm power, sensorimotor rhythm, ratio of alpha and theta EEG rhythm powers), recorded on a time scale not exceeding 100 ms.

[0067] Thus, a combination of new technical features, absent from prior art solutions, enables the technical challenge of providing high-speed and accurate feedback in real time during operant conditioning of brain rhythms to be addressed. This is achieved through the integration of optimized signal processing algorithms (e.g., dynamic filtering algorithms), an efficient system architecture, and a specially designed game interface, ensuring the stated technical result.

[0068] A key technical feature of the described method for presenting feedback in the form of a game scenario is the ability to simultaneously, yet separately, present the instantaneous and integral components of the feedback signal for neurofeedback purposes. The instantaneous component is presented no later than 100 ms after the corresponding change in the biological parameter (e.g., alpha rhythm power, sensorimotor rhythm, or the ratio of alpha and theta EEG powers) that is used to provide feedback. Changes in the game scenario parameters reflecting the instantaneous component are key to the meaning of the scenario or the game's goal, attracting the user's attention and encouraging concentration on the corresponding changes. The integral component of the feedback is presented in parallel with the instantaneous component, reflecting the accumulated changes in the target parameter over a period of at least 1 second.Changes in the parameters of the game scenario that reflect the integral component are key to the meaning of the scenario or the goal of the game and create a positive feeling of achieving the goals of the game.

[0069] The described method of presenting a feedback signal is implemented based on information about changes in the feedback parameter values, reflecting the true parameter dynamics with a delay of no more than 100 ms. The game interface is developed using a minimalist design to reduce the user's cognitive load. It implements two-level feedback: instantaneous (displaying real-time changes, such as the character's movement speed, the intensity of visual effects) and integral (accumulated points or coins, the level of completion). As an example of the interface, a game with a flying character is used, where the level of engine activity, displayed as the length of the flame, depends on the current value of the biological parameter and corresponds to the instantaneous value of the character's movement acceleration, while integral parameters such as speed and altitude reflect accumulated success.The number of coins or other discrete items collected can be used as an additional integral parameter. Another implementation option is a generalized fountain scenario, in which the height / intensity of the jet reflects the instantaneous component, the level of water accumulated in the fountain bowl reflects the first-level integral component, and, for example, the fullness of a second vessel connected to the fountain bowl can be considered an additional integral parameter.

[0070] The achievement of the claimed technical result is confirmed by measuring changes in the graphic elements of the scenario (for example, changes in the size of the jetpack flame for the “Pip the pinguin!” scenario), reflecting the instantaneous component of the feedback, g(t), and the presence of a strong correlation relationship (rl> 0.9) between these changes and the time sequence of values ​​of the biological parameter b(t), according to which biofeedback is carried out (for example, the power of the alpha rhythm, sensorimotor rhythm, the ratio of the powers of the alpha and theta rhythms of the EEG). The achievement of the technical result is confirmed by measuring changes in the graphic elements of the scenario, reflecting the integral component (for example, the number of coins collected) i(t) and the presence of a strong correlation relationship (r2> 0.9) with the sequence of values ​​of the biological parameter a(t) = sum_{k = l} A{T}(b(tk)), obtained by window integration of the original sequence (e.g., the alpha rhythm power, sensorimotor rhythm, or the ratio of the alpha and theta EEG rhythm powers). Achieving the technical result is confirmed by the argument of the cross-correlation sequence peak c(tau) = E{g(t)b(t-tau)} not exceeding 100 ms, i.e., tau max = argmax(c(tau)) <= 100 ms.

[0071] io

Claims

CLAUSES OF THE INVENTION 1. A method for providing feedback characterizing brain function in real time with minimal delay, comprising the following steps: - record EEG signals from EEG sensors attached to the scalp, or MEG signals using magnetometers located in close proximity to the head; - using the recorded signals, a training session is carried out, during which the user is presented simultaneously, but separately: • an instantaneous feedback component reflecting a change in the recorded brain activity with a delay not exceeding 100 ms, presented as a change in the feedback signal; • an integral feedback component reflecting changes in the recorded brain activity accumulated over at least 1 second and presented in the form of a smooth change in the feedback signal. li

Images