A method for displaying brain wave signals using Chladni figures
By generating Krani graphics and mapping the waveform oscillation frequency of the EEG signal to the graphics, the problem that cannot fully and accurately reflect the brain state in the existing technology is solved, the information transmission rate and immersion are improved, and the effect of the brain-computer interface is enhanced. and subject's concentration ability.
Patent Information
- Application Number
- CN202210833637.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-15
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-07-15
AI Technical Summary
The existing closed-loop brain-computer interface technology cannot fully and accurately reflect the subject's cognitive and emotional state, and the information transmission rate is low, which affects the effect and immersion of the brain-computer interface.
By generating Krani graphics, the corresponding graphics are presented using different waveform oscillation frequencies of the EEG signal, and the neural feedback indicators are mapped to the waveform oscillation frequency of the Krani graphics, real-time synchronous display is achieved, and information transmission rate and immersion are improved .
It realizes a more comprehensive and precise reflection of the brain state, improves the information transmission rate and real-time performance, enhances the subject's concentration and brain self-regulation ability, and avoids the disadvantages of the existing technology.
Smart Images

Figure CN115227264B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the fields of brain-computer interface, signal visualization, neural feedback, etc., and specifically relates to a method for displaying brain wave signals using Chladni figures. Background Art
[0002] Scalp electroencephalography (EEG) is an electrophysiological monitoring method that records brain electrical signals non-invasively. It measures the weight and value of postsynaptic potentials at the electrode placement points when a large number of pyramidal cells are excited by placing electrodes on the surface of the human scalp.
[0003] Currently, EEG-based brain-computer interface systems are divided into two categories: exogenous brain-computer interface and endogenous brain-computer interface.
[0004] In exogenous brain-computer interfaces, specific visual or auditory stimuli need to be presented to the subjects in sequence and continuously to generate corresponding brain responses, such as event-related potentials P300 and steady-state visual evoked potentials (SSVEP), etc., and then control external devices after recognition. This is a discrete mode with a low information transmission rate. This discrete mode is inconvenient for brain-computer interface applications that require fast and real-time.
[0005] Endogenous brain-computer interfaces do not require any external stimulation. The subjects adjust the state of the brain according to their own wishes, such as motor imagery and attention intensity, to control external devices. This spontaneous mode does not require the induction of external visual stimulation, so it is a continuous mode with a high information transmission rate. However, for example, motor imagery and attention intensity control, different subjects have different control effects, and there are large individual differences in endogenous EEG signals. Therefore, a large amount of data is required in the early stage to train the algorithm model to identify the common EEG patterns of different people for model training, and the effectiveness is also inconsistent.
[0006] For closed-loop brain-computer interface systems, that is, graphics generated based on EEG signal features, on the one hand, they can be relatively simple graphics, such as amplitude graphs, spectrum graphs, bar graphs based on brain wave energy, tones and scales (Wood and Kober, 2018), etc. These patterns lack aesthetics, which makes the above conventional methods lack immersion, easily distract the subject's attention, affect the current brain state, and thus affect the effect of the brain-computer interface; on the other hand, they can be relatively complex graphics, such as mandala images. In 2009, Grechko and Gontar proposed using a discrete chaotic dynamics mathematical model to convert EEG signals into simulated time series, and then into mandala images. According to the healing power theory of mandala image psychological phenomena, mandala image visual stimulation can promote the training effect of neurofeedback. However, this study only uses the temporal characteristics of EEG signals to generate pictures to represent the current mental state of the subject, while ignoring that the rich spectral characteristics of EEG signals can better represent the cognitive and emotional state of the subject. Therefore, the current closed-loop brain-computer interface technology does not reflect the state of the brain in a comprehensive or accurate manner.
[0007] In the 18th century, German physicist Ernst Chladni conducted an experiment in which he placed a wide metal sheet on a violin and evenly sprinkled sand on it. He then began to play the violin with a bow, and the sand automatically arranged itself into different beautiful patterns. As the tunes played by the strings changed and the frequency continued to increase, the patterns also changed and became more complex. This is the Chladni figure.
[0008] Therefore, based on existing technology, we can design a closed-loop brain-computer interface system to reflect the brain state more comprehensively and accurately. Summary of the invention
[0009] The purpose of the present invention is to provide a method for displaying brain wave signals using Chladni figures to more comprehensively and accurately reflect the state of the brain.
[0010] In order to achieve the above object, the present invention provides a method for displaying brain wave signals using Chladni figures, comprising:
[0011] S1: Generate corresponding Chladni figures according to different waveform oscillation frequencies;
[0012] S2: Collect and process EEG signals to obtain neural feedback indicators;
[0013] S3: mapping the neurofeedback index to the waveform oscillation frequency of the Chladni pattern to present the corresponding Chladni pattern;
[0014] S4: By executing steps S1 to S3 in real time and continuously, the Chladni pattern is kept synchronized with the EEG signal or with a fixed delay.
[0015] In the step S1, matplotlib visualization is used to generate Chladni figures corresponding to different waveform oscillation frequencies.
[0016] Matplotlib visualization is used to generate Chladni figures corresponding to different waveform oscillation frequencies, including: digitally simulating the Chladni plate and setting the natural frequency of the Chladni plate; simulating sound and adjusting the sound frequency parameters to obtain the Chladni figures corresponding to different waveform oscillation frequencies.
[0017] The step S2 comprises:
[0018] S21: Select the recording electrode channel, place the reference electrode and the ground electrode on the ear, and start collecting EEG signals;
[0019] S22: Perform online preprocessing of the EEG signal to obtain a neural feedback index of the EEG signal.
[0020] The step S22 comprises:
[0021] S221: Filtered using a 0.5Hz high-pass and 45Hz low-pass finite impulse response filter;
[0022] S222: Calculation of power spectral density by Welch;
[0023] S223: Calculate the average power within each brain wave frequency range according to the range of each brain wave frequency, and determine the neural feedback index.
[0024] In step S222, the EEG signal of the last 4 seconds is selected every 0.05 seconds, and the power spectral density is calculated by Welch; in step S223, the range of each brain wave frequency includes delta, theta, low alpha, highalpha, low beta, high beta and gamma.
[0025] In step S2, a single-channel EEG acquisition device is used to collect and process EEG signals to obtain a neural feedback index.
[0026] In step S2, a multi-channel EEG device is used to collect and process EEG signals to obtain a neural feedback index. For the multi-channel EEG acquisition device, the neural feedback index is calculated in each recording electrode channel, and then the average value of the neural feedback indexes of different recording electrode channels is taken as the final neural feedback index.
[0027] The neurofeedback index is an attention index, a relaxation index, a stress index or an impulse control index.
[0028] In step S3, the corresponding Chladni figure is selected and presented using pygame.
[0029] The present invention is based on a closed-loop brain-computer interface system. By extracting EEG features, it is used for real-time graphic display to further influence the brain response of the subject. It uses the ratio of different brain wave energies to measure the attention index, so it does not require a lot of early training. At the same time, through continuous visual stimulation, the subject's current brain state is reflected in real time through its signal characteristics, without the need for external visual stimulation. Therefore, it is a continuous mode with high information transmission rate and good real-time performance. In this way, the advantages of the above two brain-computer interfaces can be combined to intuitively reflect the subject's current brain state, influence and change the state of the brain, achieve the purpose of fast and real-time, and avoid their disadvantages.
[0030] The method of displaying brain wave signals using Chladni patterns of the present invention utilizes Chladni pattern visual stimulation, and can intuitively and concisely display the energy of the subject's current brain waves in different frequency bands and the corresponding neural feedback indicators, which provides a sense of immersion and will not distract the subject's attention or affect the current brain state, thereby affecting the effect of the method of displaying brain wave signals using Chladni patterns of the present invention.
[0031] Due to the plasticity mechanism of the brain, the method of presenting brain wave signals using Chladni patterns can be used to train subjects to improve various neurofeedback indicators (such as attention) and enhance the brain's self-regulation ability. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is a flow chart of the method of displaying brain wave signals using Chladni figures of the present invention. DETAILED DESCRIPTION
[0033] The preferred embodiments of the present invention are given below in conjunction with the accompanying drawings and described in detail.
[0034] like Figure 1 As shown, the method of presenting brain wave signals using Chladni figures of the present invention comprises the following steps:
[0035] Step S1: Generate corresponding Chladni figures according to different waveform oscillation frequencies;
[0036] All objects, including Chladni plates, have a set of natural frequencies at which they vibrate. A system, such as the standing waves in a musical instrument, often vibrates at discrete frequencies, such as natural frequencies or eigenfrequencies. When a structure vibrates at a certain waveform oscillation frequency, it deforms into a corresponding shape: the eigenmode.
[0037] In this embodiment, matplotlib visualization is used to generate Chladni figures corresponding to different waveform oscillation frequencies.
[0038] Among them, matplotlib visualization is used to generate Chladni figures, which specifically includes: digitally simulating the Chladni plate and setting the natural frequency of the Chladni plate; simulating sound and obtaining the Chladni figures corresponding to different waveform oscillation frequencies by adjusting the sound frequency parameters.
[0039] The value of the natural frequency of the Chladni plate can be defined by yourself, and there is no specific requirement. The Chladni phenomenon is that the frequency of sound (i.e., the waveform oscillation frequency) affects the Chladni plate, causing the sand on the Chladni plate to present a certain pattern. Therefore, in the process of simulating sound, just changing the frequency of the sound can generate different styles of Chladni figures. The Chladni figures corresponding to these different waveform oscillation frequencies have different complexities. As the waveform oscillation frequency increases, the Chladni figure will become more complex, that is, more complex.
[0040] According to the literature [Fu Jiangsong, Approximate solution and experimental study of lateral vibration of a rectangular plate with four sides free, the 10th Conference on Dynamics and Control, 2016], the calculation of the natural frequency of the Chladni plate includes the following steps:
[0041] A1: According to the thin plate vibration theory, set the governing differential equation of the rectangular plate;
[0042] A2: The Chladni pattern experimental method is used to solve the approximate solution of the vibration mode function and the corresponding natural frequency of the four-sided free rectangular plate. According to the expression of the approximate solution, the two-dimensional standing wave theoretical diagram of the four-sided free rectangular plate is obtained.
[0043] The experimental method of Chladni pattern can be found in the literature [ROSSING TD. Chladni's law for vibrating plates [J]. American Journal of Physics, 1982, 50(3): 271-274]. By using differential equations, the vibration mode function expression at each order of natural frequency can be obtained.
[0044] A3: Using the Chladni pattern experimental method, we can obtain the two-dimensional standing wave experimental graph of the rectangular plate.
[0045] By using the approximate solution expression, we can obtain the theoretical graph of two-dimensional standing waves of a four-sided free rectangular plate. By comparing the experimental graph and the theoretical graph, we can conclude that the approximate solution obtained in step A2 is basically consistent with the experimental solution.
[0046] Step S2: collecting and processing EEG signals to obtain neural feedback indicators;
[0047] The step S2 comprises:
[0048] Step S21: Select the recording electrode channel, place the reference electrode and the ground electrode on the ear, and start collecting EEG signals;
[0049] In this embodiment, the recording electrode channel is selected at FPz. In addition to FPz, the recording electrode channel can also include channels such as Fp1, Fp2, Fz, F3, F4, F7 and F8 located in the frontal lobe area of the brain.
[0050] When collecting EEG signals, the impedance of the recording electrode of the EEG acquisition device is controlled below 10kΩ (by adjusting the electrode angle and increasing the contact area between the electrode and the scalp, the impedance is controlled below 10kΩ), the sampling frequency is 512Hz, and the collected EEG signals are transmitted via Bluetooth.
[0051] When collecting EEG signals, the subjects were asked to remain as still as possible and avoid excessive blinking and head movement.
[0052] Step S22: Perform online preprocessing of the EEG signal to obtain a neural feedback index of the EEG signal.
[0053] Step S22 includes:
[0054] Step S221: Filter using a finite impulse response (FIR) filter with a high pass of 0.5 Hz and a low pass of 45 Hz to eliminate low-frequency drift and unnecessary frequency bands, and remove low-frequency artifacts of eye movements and high-frequency noise of muscles.
[0055] Step S222: Calculate the power spectrum density by Welch;
[0056] In this embodiment, the last 4 seconds of EEG signals are selected every 0.05 seconds, and the power spectrum density is calculated by Welch, with a frequency resolution of 0.25 Hz.
[0057] Step S223: Calculate the average power within each brain wave frequency range according to the range of each brain wave frequency to determine the neural feedback index.
[0058] The ranges of individual brainwave frequencies include delta (0.5-4Hz), theta (4-8Hz), low alpha (8-10Hz), high alpha (10-12Hz), low beta (12-20Hz), high beta (20-30Hz) and gamma (30-45Hz).
[0059] In this embodiment, the neural feedback indicator is an attention indicator, and the attention indicator is:
[0060] Attention index = average power of high beta / (average power of theta + average power of low alpha),
[0061] Since beta rhythm is strong in the state of concentration, and theta and alpha rhythms are strong in the state of sleepiness and inattention, the attention index is obtained by calculating the average power of high beta / (average power of theta + average power of low alpha).
[0062] It should be noted that in step S2, either a single-channel EEG acquisition device or a multi-channel EEG device can be used to collect and process EEG signals to obtain a neurofeedback index. As mentioned above, in step S21, in addition to Fpz, the recording electrode channel can also be Fp1, Fp2, Fz, F3, F4, F7 and F8, which are located in the frontal lobe area of the brain. In step S22, for a multi-channel EEG acquisition device, a neurofeedback index (such as an attention index) is calculated in each recording electrode channel, and then the average value of the neurofeedback indexes of different recording electrode channels is taken as the final neurofeedback index.
[0063] In addition, in other embodiments, the neurofeedback index may be a relaxation index, a stress index, and an impulse control index in addition to the attention index, which are used to train relaxation, relieve stress, and improve impulse control ability, respectively. These indexes can be represented by the correspondence between brain waves in different frequency bands of EEG signals. Specific references [Marzbani, H., Marateb, HR, & Mansourian, M.
[0064] (2016). Neurofeedback: a comprehensive review on system design, methodology and clinical applications. Basic and clinical neuroscience, 7(2), 143].
[0065] Step S3: mapping the neural feedback index to the waveform oscillation frequency of the Chladni pattern to present the corresponding Chladni pattern;
[0066] In this embodiment, pygame is used to select the corresponding Chladni figure for presentation.
[0067] In this embodiment, the relationship between the waveform oscillation frequency and the neural feedback index is:
[0068] Waveform oscillation frequency = empirical coefficient × neurofeedback index.
[0069] Therefore, the oscillation frequency of the Chladni pattern (the complexity of the picture) is used to represent the brain's attention state. The greater the oscillation frequency of the Chladni pattern, the more complex the picture, and the greater the brain's attention index.
[0070] Step S4: By executing steps S1 to S3 in real time and continuously, real-time animation is realized, and the Chladni pattern is kept synchronized with the EEG signal (or there is a small fixed delay between the Chladni pattern and the EEG signal. Thus, a continuous Chladni pattern is generated. The fixed delay is very small, less than 10 milliseconds and stable, and can be ignored.
[0071] In addition, in other embodiments, the pattern of EEG frequency distribution and neural feedback indicators such as attention index under different Chladni figure visual stimulation can also be identified by machine learning. The specific steps of machine learning are to use the waveform oscillation frequency of Chladni figure, the pattern of EEG frequency distribution and attention index, these three types of parameters as input parameters, and input a deep learning model for training. Such a method can more accurately correspond Chladni figure, EEG and the attention index of the brain. Specifically, the frequency distribution of EEG signal is calculated by statistical method so as to classify the pattern of EEG frequency distribution.
[0072] In addition to generating Chladni pattern visual stimulation, audio stimulation can also be generated based on EEG signals to jointly regulate the brain state, improve the immersion of the brain-computer interface system, and thus improve the distribution of the subject's attention. Audio stimulation is to select appropriate music with different emotional types to stimulate the brain based on the brain's current state.
[0073] Experimental Results
[0074] As mentioned above, the existing methods for generating graphics based on EEG signals or related features can be divided into four types: generating amplitude graphs based on EEG signal features, generating spectrum graphs based on EEG signal features, generating bar graphs based on brain wave energy based on EEG signal features, tones and scales, and generating mandala images based on EEG signal features proposed by Grechko and Gontar. The following table compares these four methods with the method of the present invention.
[0075] Table 1 Comparison of four methods
[0076]
[0077] The amplitude graph cannot reflect brainwave energy, nor can it reflect brainwaves of different frequency bands according to the refresh rate of the graphics, and lacks a sense of immersion; the spectrum graph can reflect brainwave energy, but cannot reflect brainwaves of different frequency bands according to the refresh rate of the graphics, and lacks a sense of immersion; the bar graph, color tone and scale based on brainwave energy can reflect brainwave energy, and can also reflect brainwaves of different frequency bands according to the refresh rate of the graphics, but lack a sense of immersion. Grechko&Gontar cannot reflect brainwave energy, nor can it reflect brainwaves of different frequency bands according to the refresh rate of the graphics, but there is a sense of immersion. The present invention can reflect brainwave energy, and can also reflect brainwaves of different frequency bands according to the refresh rate of the graphics, and at the same time there is a sense of immersion.
[0078] The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. The above embodiment of the present invention can also be modified in various ways. That is, all simple, equivalent changes and modifications made according to the claims and the description of the present invention application fall within the protection scope of the claims of the present invention.
Claims
1. A method for displaying brain wave signals using Chladni figures, It is characterized in that include: Step S1: Generate corresponding Chladni figures according to different waveform oscillation frequencies; Step S2: collecting and processing EEG signals to obtain neural feedback indicators; Neurofeedback indices were determined by calculating the average power within each brainwave frequency range based on the range of each brainwave frequency; Step S3: Mapping the neural feedback index to the waveform oscillation frequency of the Chladni figure to present the corresponding Chladni figure; wherein the relationship between the waveform oscillation frequency and the neural feedback index is: Waveform oscillation frequency = empirical coefficient × neurofeedback index; Step S4: by executing steps S1 to S3 in real time and continuously, the Chladni pattern is kept synchronized with the EEG signal or with a fixed delay.
2. The method of displaying brain wave signals using Chladni figures according to claim 1, It is characterized in that In the step S1, matplotlib visualization is used to generate Chladni figures corresponding to different waveform oscillation frequencies.
3. The method of displaying brain wave signals using Chladni figures according to claim 2, It is characterized in that Matplotlib visualization is used to generate Chladni figures corresponding to different waveform oscillation frequencies, including: digitally simulating the Chladni plate and setting the natural frequency of the Chladni plate; simulating sound and adjusting the sound frequency parameters to obtain the Chladni figures corresponding to different waveform oscillation frequencies.
4. The method of displaying brain wave signals using Chladni figures according to claim 1, It is characterized in that The step S2 comprises: Step S21: Select the recording electrode channel, place the reference electrode and the ground electrode on the ear, and start collecting EEG signals; Step S22: Perform online preprocessing of the EEG signal to obtain a neural feedback index of the EEG signal.
5. The method of displaying brain wave signals using Chladni figures according to claim 4, It is characterized in that The step S22 comprises: Step S221: Filtering using a 0.5 Hz high-pass and 45 Hz low-pass finite impulse response filter; Step S222: Calculate the power spectrum density by Welch; Step S223: Calculate the average power within each brain wave frequency range according to the range of each brain wave frequency to determine the neural feedback index.
6. The method of displaying brain wave signals using Chladni figures according to claim 5, It is characterized in that In step S222, every 0.05 seconds, the last 4 seconds of EEG signals are selected and the power spectral density is calculated by Welch; In step S223, the ranges of the brain wave frequencies include delta, theta, low alpha, high alpha, low beta, high beta and gamma.
7. The method of displaying brain wave signals using Chladni figures according to claim 1, It is characterized in that In step S2, a single-channel EEG acquisition device is used to collect and process EEG signals to obtain a neural feedback index.
8. The method of displaying brain wave signals using Chladni figures according to claim 1, It is characterized in that In step S2, a multi-channel EEG device is used to collect and process EEG signals to obtain a neural feedback index; For multi-channel EEG acquisition equipment, the neural feedback index is calculated in each recording electrode channel, and then the average value of the neural feedback indexes of different recording electrode channels is taken as the final neural feedback index.
9. The method of displaying brain wave signals using Chladni figures according to claim 1, It is characterized in that The neurofeedback index is an attention index, a relaxation index, a stress index or an impulse control index.
10. The method of displaying brain wave signals using Chladni figures according to claim 1, It is characterized in that In step S3, the corresponding Chladni figure is selected and presented using pygame.
