Method, device, equipment and medium for collecting electroencephalogram signals based on visual stimulation

Abstract

The embodiment of the present invention provides a method, device, equipment and medium for collecting electroencephalogram (EEG) signals based on visual stimulation, which belongs to the field of electroencephalogram (EEG) technology. The method includes: determining the initial stimulation target of visual stimulation, and determining the initial stimulation picture associated with the initial stimulation target; setting the target frequency and target phase corresponding to the initial stimulation picture; determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and target phase; wherein the initial stimulation parameters are used to characterize the parameters corresponding to the initial stimulation picture when it is displayed and played in the target display; establishing a stimulation mapping table according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters; obtaining the current stimulation target corresponding to the subject, and determining the target stimulation picture and target stimulation parameters corresponding to the current stimulation target according to the stimulation mapping table; collecting the target electroencephalogram (EEG) signals corresponding to the subject when the target stimulation picture and the target stimulation parameters are visually stimulated on the current display.

Description

Technical Field

[0001] The present invention relates to the field of electroencephalogram (EEG) technology, and in particular to an EEG signal acquisition method, device, equipment and medium based on visual stimulation. Background Art

[0002] For most EEG signals, the scalp is an ideal location for placing electrodes. However, collecting EEG signals in the ear (around or inside the ear) can promote the practical application of brain-computer interfaces. Therefore, existing technologies often use ear electrodes to collect EEG signals. However, when EEG signals are transmitted from the response cortex to the ear area, the signal undergoes significant attenuation. Taking the steady-state visual evoked potential (SSVEP) as an example, its response cortex is located in the visual cortex and is mainly distributed in the occipital region. In this case, when using ear electrodes to collect SSVEP signals, the signal may attenuate by 10-20db. Therefore, while using the ear to collect EEG signals to ensure the information transmission rate, it is also urgent to solve the problem of signal attenuation when using ear electrodes to collect SSVEP. Summary of the invention

[0003] The main purpose of the embodiments of the present invention is to provide a method, device, equipment and medium for collecting EEG signals based on visual stimulation, aiming to solve the problem in the related technology that when using the ear to collect EEG signals, the information transmission rate can be guaranteed, but the collected EEG signals have signal attenuation.

[0004] In a first aspect, an embodiment of the present invention provides a method for collecting electroencephalogram signals based on visual stimulation, comprising:

[0005] Determining an initial stimulus target corresponding to the visual stimulus, and determining an initial stimulus picture associated with the initial stimulus target;

[0006] Setting the target frequency and target phase corresponding to the initial stimulus image;

[0007] Determining initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase; wherein the initial stimulation parameters are used to characterize the parameters corresponding to the initial stimulation picture when it is displayed and played in the target display;

[0008] Establishing a stimulation mapping table according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters;

[0009] Obtaining a current stimulation target corresponding to the subject, and determining a target stimulation picture and target stimulation parameters corresponding to the current stimulation target according to the stimulation mapping table;

[0010] The target electroencephalogram signal corresponding to the subject when the target stimulation picture and the target stimulation parameters are visually stimulated on the current display is collected.

[0011] In a second aspect, an embodiment of the present invention provides an electroencephalogram signal acquisition device based on visual stimulation, comprising:

[0012] A data determination module, used to determine an initial stimulus target corresponding to the visual stimulus, and to determine an initial stimulus picture associated with the initial stimulus target;

[0013] A data setting module, used to set the target frequency and target phase corresponding to the initial stimulation image;

[0014] A parameter determination module, used to determine the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase; wherein the initial stimulation parameters are used to characterize the parameters corresponding to the initial stimulation picture when it is displayed and played in the target display;

[0015] A mapping establishment module, used to establish a stimulation mapping table according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters;

[0016] A parameter acquisition module, used to obtain the current stimulation target corresponding to the subject, and determine the target stimulation picture and target stimulation parameters corresponding to the current stimulation target according to the stimulation mapping table;

[0017] The signal acquisition module is used to acquire the target EEG signal corresponding to the subject when the target stimulation picture and the target stimulation parameters are visually stimulated on the current display.

[0018] In a third aspect, an embodiment of the present invention further provides a terminal device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of any one of the methods for collecting electroencephalogram signals based on visual stimulation provided in the specification of the present invention are realized.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of the visual stimulation-based EEG signal acquisition methods provided in the specification of the present invention.

[0020] The embodiment of the present invention provides a method, device, equipment and medium for collecting electroencephalogram signals based on visual stimulation. The method includes: determining an initial stimulation target corresponding to the visual stimulation and determining an initial stimulation picture associated with the initial stimulation target, and setting a target frequency and a target phase corresponding to the initial stimulation picture, wherein the target frequency and the target phase are the core parameters of the visual stimulation. The target frequency determines the presentation speed of the stimulation (such as how many frames per second), and the target phase affects the presentation mode of the stimulation (such as regular flashing or changes), so that the initial stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target frequency and the target phase, so that the presentation of the visual stimulation meets the requirements, and then a stimulation mapping table is established according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters, and the establishment of the stimulation mapping table can reduce the possibility of human operation errors. Thus, the current stimulation target corresponding to the subject is obtained, and then the target stimulation picture and the target stimulation parameters corresponding to the current stimulation target are quickly located through the stimulation mapping table to ensure the accurate presentation of the stimulation, and then the target electroencephalogram signals of the subject under visual stimulation are collected in real time on the basis of the dynamic adjustment of the target stimulation picture and the parameters. This real-time acquisition method can capture more subtle neural responses, especially under dynamically changing stimulus conditions, which helps to study the brain's real-time processing mechanism for different visual stimuli. In traditional experiments, although the collection of EEG signals by the ear can ensure the information transmission rate, there is often a problem of signal attenuation, which affects the accuracy of the data. Through the above steps, researchers can achieve precise control of visual stimulation on the target display, combined with efficient target EEG signal acquisition technology, effectively reduce the impact of signal attenuation and improve the signal-to-noise ratio of experimental data. Therefore, by optimizing the stimulation parameters and EEG signal acquisition methods, signal attenuation can be reduced and the accuracy and reliability of the collected EEG signals can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of a flow chart of a method for collecting electroencephalogram signals based on visual stimulation provided by an embodiment of the present invention;

[0023] Figure 2 An initial stimulus image similar to the sun provided for implementing this embodiment;

[0024] Figure 3 An initial stimulus picture similar to the moon provided for implementing this embodiment;

[0025] Figure 4 A schematic diagram of the module structure of an electroencephalogram signal acquisition device based on visual stimulation provided by an embodiment of the present invention;

[0026] Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0029] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0030] The embodiment of the present invention provides a method, device, equipment and medium for collecting electroencephalogram signals based on visual stimulation. The method for collecting electroencephalogram signals based on visual stimulation can be applied to a terminal device, which can be an electronic device such as a tablet computer, a laptop computer, a desktop computer, a personal digital assistant and a wearable device. The terminal device can be a server or a server cluster.

[0031] When EEG is used in the design of brain-computer interface systems, in order to collect EEG signals with a higher signal-to-noise ratio, the collection electrodes are generally placed in hairy areas. When collecting EEG signals in hairy areas, a common practice is to fill conductive paste between the electrode and the skin to reduce the contact impedance between the electrode and the skin. Long-term use of conductive gel has the risk of skin allergies, and the current gel will dehydrate and harden after 2-3 hours, seriously affecting the recording of EEG signals, which hinders the scenario, life and practicality of the brain-computer interface system.

[0032] Previous studies have shown that placing EEG electrodes on hairless areas such as the forehead, ears, and face can collect EEG signals equivalent to those on the scalp. In particular, the ears can be used to place electrodes with the help of headphones to achieve concealment and convenience, which is one of the best choices to replace the scalp as a collection area. In addition to the ears, the area around the ears can also be used as a signal collection area, increasing the selectivity of system design.

[0033] In 2011, Kidmose P. and others from Aarhus University in Denmark first proposed a method for collecting EEG in the ear. Electrodes are placed inside the ear to record EEG signals, and this recording method is called Ear-EEG. Kappel et al. coated iridium dioxide on soft headphones customized according to the anatomical shape of the ear to achieve stable contact, and evaluated four EEG paradigms, namely auditory steady-state response, steady-state visual evoked potential, mismatch negativity, and alpha wave. The study believes that dry-contact ear electrodes are a feasible EEG signal recording technology. In 2023, Zhao Huiqing and others further improved the performance of ear electrodes in SSVEP-based experiments. Through experiments, the number of channels, stimulation frequency, and data processing algorithms of the ear electrodes were compared and optimized. The optimized ear electrodes can achieve the best information transmission rate.

[0034] For most EEG signals, the best electrode position should be on the scalp. When the EEG signal is transmitted from the response cortex to the ear area, the signal attenuation is very large. For example, the response cortex of SSVEP, steady-state visual evoked potential, is located in the visual cortex, which is in the occipital region. When we use ear electrodes to collect SSVEP, the signal is usually attenuated by 10-20dB.

[0035] The human brain has certain functional division characteristics. Previous studies have shown that the occipital lobe is responsible for visual processing, the temporal lobe is responsible for sound, emotion, long-term memory, etc., the parietal lobe is responsible for touch, pressure, body awareness and taste, etc., and the frontal lobe is responsible for memory, emotion, smell, language and attention, etc.

[0036] For most EEG signals, the scalp is an ideal location for placing electrodes. However, collecting EEG signals in the ear (around or inside the ear) can promote the practical application of brain-computer interfaces. Therefore, existing technologies often use ear electrodes to collect EEG signals. However, when EEG signals are transmitted from the response cortex to the ear area, the signal undergoes significant attenuation. Taking the steady-state visual evoked potential (SSVEP) as an example, its response cortex is located in the visual cortex and is mainly distributed in the occipital region. In this case, when using ear electrodes to collect SSVEP signals, the signal may attenuate by 10-20db. Therefore, while using the ear to collect EEG signals to ensure the information transmission rate, it is also urgent to solve the problem of signal attenuation when using ear electrodes to collect SSVEP.

[0037] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0038] Please refer to Figure 1 , Figure 1 A flowchart of a method for collecting electroencephalogram (EEG) signals based on visual stimulation is provided in an embodiment of the present invention.

[0039] like Figure 1 As shown, the method for collecting electroencephalogram signals based on visual stimulation includes steps S101 to S106.

[0040] Step S101: determining an initial stimulation target corresponding to the visual stimulation, and determining an initial stimulation picture associated with the initial stimulation target.

[0041] For example, the initial stimulus target corresponding to the visual stimulus can be determined according to the application requirements of the application scenario. For example, the initial stimulus target that can be determined in the application scenario of a paralyzed patient sitting in a wheelchair can be the wheelchair moving forward, the wheelchair moving backward, the wheelchair turning left, the wheelchair turning right, etc. The initial stimulus target is related to the application requirements in the application scenario.

[0042] Exemplarily, two pictures with opposite semantics or with large semantic differences are used to determine the initial stimulus pictures associated with the initial stimulus target. Figure 2 The sun-like pictures shown and Figure 3 The moon-like picture shown is determined as the initial stimulus picture associated with the initial stimulus target. Similar initial stimulus pictures can also be pictures with opposite semantics such as up and down, male and female, left and right, and each group of two pictures with opposite semantics corresponds to an initial stimulus target.

[0043] Step S102: setting the target frequency and target phase corresponding to the initial stimulation image.

[0044] Exemplarily, the target frequency and target phase corresponding to the initial stimulus image are set according to expert experience or historical experience.

[0045] For example, when the initial stimulation target is 8 targets, such as wheelchair moving forward, wheelchair moving backward, wheelchair turning left, wheelchair turning right, wheelchair stopping, wheelchair accelerating, wheelchair decelerating, wheelchair speed adjusting, etc., the target frequencies can be 4.5, 5, 5.5, 6, 4.7, 5.2, 5.7, 6.2, respectively, and the target phases can be 0, 0, 0, 0, 0.35, 0.35, 0.35, 0.35, respectively.

[0046] Step S103, determining initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase; wherein the initial stimulation parameters are used to characterize the parameters corresponding to the initial stimulation picture when it is displayed and played in the target display.

[0047] Exemplarily, the target frequency refers to the rate at which visual stimuli are presented, and the target phase refers to the pattern of visual stimuli presentation, such as the time point at which the stimulus starts or the periodic changes of the stimulus. Then, according to the target frequency and the refresh rate of the target display, the presentation time interval of the initial stimulus picture is calculated. For example, if the target frequency is 10 Hz and the refresh rate of the target display is 60 Hz, the stimulus picture is displayed once every 6 refresh cycles (60 Hz / 10 Hz = 6). According to the target phase, the time point at which the stimulus starts is determined. For example, if the target phase is 0 degrees (synchronous start), the stimulus picture is displayed at the start point of each refresh cycle. Thus, the presentation of the stimulus picture is simulated on the target display to check whether it meets the expected target frequency and target phase. Confirm whether the parameter settings are correct by observing the flickering pattern or presentation order of the stimulus.

[0048] Exemplarily, the target frequency and the target phase are used to determine initial stimulation parameters corresponding to the initial stimulation picture in the target display. The initial stimulation parameters include but are not limited to picture brightness, picture contrast, and the like.

[0049] In some embodiments, determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase includes: determining the target refresh rate corresponding to the initial stimulation picture in the target display; determining the target semantic picture corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase in combination with a target encoding function; determining the target stimulation parameters corresponding to the initial stimulation picture in the target display according to the target refresh rate and the target semantic picture, wherein the target stimulation parameters are a multi-frame brightness sequence corresponding to the initial stimulation picture.

[0050] Exemplarily, the target refresh rate corresponding to the initial stimulus picture in the target display is determined based on expert experience or historical experience. A suitable target encoding function is selected to determine the presentation mode of the initial stimulus picture. For example, a smoothly changing stimulus pattern can be generated using a sine wave function, while an intermittent stimulus pattern is generated using a square wave function. Thus, a visual stimulus picture is designed based on the target frequency, target phase, and target encoding function. For example, if a sine wave function and a frequency of 10 Hz are used to generate a brightness sequence that changes sinusoidally over time, the presentation time point of the stimulus picture can also be adjusted according to the target phase. For example, if the target phase is 90 degrees, the stimulus picture begins to be presented at a quarter of each cycle.

[0051] Exemplarily, the brightness value of each frame of the image is generated according to the target frequency, the target phase and the target encoding function. For example, for a stimulation frequency of 10 Hz and a refresh rate of 60 Hz, a stimulation image is presented once every 6 refresh cycles, and a brightness sequence corresponding to 6 frames is generated. For example, the brightness value of each frame is calculated according to a target encoding function (such as a sine wave), so that the generated multi-frame brightness value sequence is recorded, and the brightness value is applied to the initial stimulation image to obtain the target semantic image, and then the target stimulation parameters corresponding to the initial stimulation image in the target display are determined according to the target refresh rate and the target semantic image.

[0052] In some embodiments, determining the target semantic picture corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase in combination with a target coding function includes: determining a first time window, and determining a first sampling time according to the first time window and the target refresh rate; determining first phase information, and determining the target coding function according to the target frequency, the first sampling time and the first phase information; the target coding function is a sine function or a cosine function; and determining the target semantic picture corresponding to the initial stimulation picture in the target display according to the target coding function.

[0053] Exemplarily, the first time window may be a time window in seconds, and the first sampling time is determined according to the first time window and the target refresh rate.

[0054] For example, , where t1 represents the first sampling time, represents the first time window, Indicates the target refresh rate.

[0055] Exemplarily, the first phase information is determined, and the first phase information is used to characterize the initial phase information corresponding to the target coding function, and then the target coding function is determined according to the target frequency, the first sampling time and the first phase information; the target coding function is a sine function or a cosine function.

[0056] For example, ,in, It means that the brightness value corresponding to each frame of the initial stimulus image in the first time window is obtained through cosine function encoding. Indicates that the target encoding function is a cosine function, represents the target frequency, represents the first phase information, and t1 represents the first sampling time.

[0057] For example, when the initial stimulation target is 8 targets, such as wheelchair forward, wheelchair backward, wheelchair left turn, wheelchair right turn, wheelchair stop, wheelchair acceleration, wheelchair deceleration, wheelchair speed adjustment, etc., the target frequency can be 4.5, 5, 5.5, 6, 4.7, 5.2, 5.7, 6.2, and the target phase can be 0, 0, 0, 0, 0.35, 0.35, 0.35, 0.35. When the target refresh rate is 60Hz, when the target encoding function is a sine function, the target frequency and target phase can be substituted into the sine function to obtain the corresponding periodic waveform.

[0058] Exemplarily, information such as the display time and display brightness corresponding to the initial stimulus picture in the target display is determined according to the target encoding function, so that the display time and display brightness are applied to the initial stimulus picture to obtain the target semantic picture.

[0059] In some embodiments, determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase includes: determining the target refresh rate corresponding to the initial stimulation picture in the target display; determining a second time window, and determining a second sampling time according to the second time window and the target refresh rate; determining second phase information, and determining the target encoding function according to the target frequency, the second sampling time and the second phase information; the target encoding function is a square wave function; determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target encoding function, wherein the target stimulation parameters are a sequential display sequence corresponding to two different initial stimulation pictures corresponding to the initial stimulation target.

[0060] Exemplarily, the target refresh rate corresponding to the initial stimulus image in the target display is determined based on expert experience or historical experience. The second time window may be a time window in seconds, and the second sampling time is determined based on the second time window and the target refresh rate.

[0061] For example, , where t2 represents the second sampling time, represents the second time window, Indicates the target refresh rate.

[0062] Exemplarily, the second phase information is determined, and the second phase information is used to characterize the initial phase information corresponding to the target coding function, and then the target coding function is determined according to the target frequency, the second sampling time and the second phase information; the target coding function is a square wave function.

[0063] For example, ,in, It means that the display image corresponding to each frame of the initial stimulus image in the second time window is obtained through square wave function encoding. Indicates that the target encoding function is a square wave function, represents the target frequency, represents the second phase information, and t2 represents the second sampling time.

[0064] Exemplarily, the initial stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target coding function, wherein the initial stimulation parameters are the sequential display sequences corresponding to two different initial stimulation pictures corresponding to the initial stimulation target. For example, when the initial stimulation parameter is 1, the initial stimulation picture is displayed as follows Figure 2 When the initial stimulus parameter is 0, the initial stimulus picture is shown as Figure 3 Pictures shown.

[0065] For example, when the initial stimulation target is 8 targets, such as wheelchair forward, wheelchair backward, wheelchair left turn, wheelchair right turn, wheelchair stop, wheelchair acceleration, wheelchair deceleration, wheelchair speed adjustment, etc., the target frequency can be 4.5, 5, 5.5, 6, 4.7, 5.2, 5.7, 6.2, and the target phase can be 0, 0, 0, 0, 0.35, 0.35, 0.35, 0.35. When the target refresh rate is 60Hz, when the target encoding function is a square wave function, the target frequency and target phase can be substituted into the square wave function to obtain the corresponding periodic non-sinusoidal waveform.

[0066] Step S104: establishing a stimulation mapping table according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters.

[0067] Exemplarily, a stimulation mapping table is established according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters, so that after the initial stimulation target is determined, the initial stimulation picture and the initial stimulation parameters corresponding to the initial stimulation target can be quickly obtained through the stimulation mapping table.

[0068] Step S105: obtaining a current stimulation target corresponding to the subject, and determining a target stimulation picture and target stimulation parameters corresponding to the current stimulation target according to the stimulation mapping table.

[0069] Exemplarily, the application scenario used by the subject is obtained, and then the current stimulation target corresponding to the subject is determined, and then the stimulation mapping table is queried according to the current stimulation target, and then when the current stimulation target is the same as the initial stimulation target, the initial stimulation picture and initial stimulation parameters corresponding to the initial stimulation target are determined as the target stimulation picture and target stimulation parameters corresponding to the current stimulation target.

[0070] Step S106 , collecting the target EEG signal of the subject corresponding to the visual stimulation of the target stimulation picture and the target stimulation parameters on the current display.

[0071] Exemplarily, the ears are used to collect the target EEG signals of the subject corresponding to the visual stimulation of the target stimulation picture and the target stimulation parameters on the current display.

[0072] In some embodiments, after obtaining the target EEG signal, the method further includes: classifying the target EEG signal to obtain a gaze picture corresponding to the target EEG signal; and determining a current gaze result corresponding to the subject according to the gaze picture.

[0073] Exemplarily, a deep learning model or a machine learning model is used to classify the target EEG signal to obtain a gaze image corresponding to the target EEG signal.

[0074] Exemplarily, an initial stimulation target corresponding to the fixation image is determined according to the fixation image using a stimulation mapping table, thereby determining a current fixation result corresponding to the subject according to the initial stimulation target.

[0075] For example, when the fixation picture is the initial stimulus picture corresponding to the wheelchair turning left in the initial stimulus target, it can be determined that the current fixation result corresponding to the subject is the wheelchair turning left.

[0076] In some embodiments, the classifying the target EEG signal to obtain a gaze picture corresponding to the target EEG signal includes: when the target EEG signal does not need to be calibrated, generating a corresponding preset EEG signal based on prior frequency coding information; performing a correlation calculation between the target EEG signal and the preset EEG signal to obtain a target correlation value between the target EEG signal and the preset EEG signal; and determining the gaze picture corresponding to the target EEG signal according to the target correlation value.

[0077] Exemplarily, if the target EEG signal does not need to be calibrated, the collected target EEG signal data is directly used, and it is ensured that the collection quality of the target EEG signal is good without obvious artifacts or noise interference.

[0078] Exemplarily, a preset EEG signal is generated based on the prior frequency coding information. The time domain length and sampling rate of the preset EEG signal should be consistent with the target EEG signal actually collected. That is, the preset EEG signal corresponding to each initial stimulation target in the stimulation mapping table is obtained.

[0079] Exemplarily, a suitable correlation calculation method is selected, such as Pearson correlation coefficient, cross-correlation analysis, etc., and then the target EEG signal and the preset EEG signal are correlated to obtain a target correlation value. The target correlation value indicates the degree of similarity between the target EEG signal and the preset EEG signal, so that the initial stimulation image corresponding to the preset EEG signal when the target correlation value is the largest is determined as the fixation image corresponding to the target EEG signal.

[0080] For example, based on the prior frequency coding information, using the sinusoidal and cosine signals generated by a computer as a template, the target EEG signal to be analyzed is correlated with the sinusoidal and cosine templates of all targets, that is, the preset EEG signal, so that the target with the largest target correlation value is defaulted as the result of the user's current gaze. The example can use the classic filter bank canonical correlation analysis FBCCA (Filter bank canonical correlation analysis), based on the prior frequency coding information, using the sinusoidal and cosine signals generated by a computer as a template, the target EEG signal to be analyzed is correlated with the sinusoidal and cosine templates of all initial stimulation targets, and the target with the largest correlation is defaulted as the result of the user's current gaze.

[0081] In addition, a correlation threshold can be set according to the experimental requirements. For example, if the target correlation value is greater than 0.8, the target EEG signal is considered to be highly correlated with the preset signal, and if the maximum target correlation value exceeds the set threshold, it is considered that the subject is looking at the same picture corresponding to the preset EEG signal under visual stimulation. If there is no set threshold for the maximum target correlation value, it means that the current target EEG signal cannot determine the corresponding fixation picture, so the target EEG signal can continue to be collected.

[0082] In some embodiments, the classifying the target EEG signal to obtain the associated image corresponding to the target EEG signal also includes: when the target EEG signal needs to be calibrated, performing calibration analysis on the target EEG signal to obtain a target calibration signal; and performing signal classification on the target calibration signal using a target classification model to obtain the associated image corresponding to the target EEG signal.

[0083] For example, if the target EEG signal needs to be calibrated, the specific problem of the target EEG signal is first confirmed, such as the presence of obvious noise, drift or other interference, and the collected target EEG signal is preliminarily preprocessed, for example: removing low-frequency drift and high-frequency noise, removing physiological interference such as electrooculography and electromyography, and using noise reduction algorithms (such as wavelet noise reduction, independent component analysis, etc.) to further improve the signal quality.

[0084] For example, a segment of EEG signals in a quiet state (baseline) is collected before the experiment begins for subsequent signal calibration, so that the target EEG signal is processed according to the EEG signal in a quiet state (baseline) to obtain a calibrated target calibration signal. For example, if the baseline calibration method is used, the baseline signal can be subtracted from each segment of the target EEG signal to eliminate the influence of baseline drift.

[0085] Exemplarily, a suitable classification model is selected according to the experimental requirements and data characteristics. Common classification models include: support vector machine (SVM), random forest (Random Forest), convolutional neural network (CNN), long short-term memory network (LSTM), and any method such as task related component analysis TRCA (task related component analysis), machine learning, deep learning, etc. can also be used to calculate the spatial domain filter, train the classifier, etc. of the calibration data, and then input the target calibration signal into the trained target classification model to obtain the target classification result. The target classification result is usually a probability distribution or label, indicating the image category that the target EEG signal is most likely to correspond to. Therefore, the image category with the highest probability in the target classification result is used as the associated image corresponding to the target EEG signal.

[0086] The embodiment of the present invention provides a method, device, equipment and medium for collecting electroencephalogram signals based on visual stimulation. The method includes: determining an initial stimulation target corresponding to the visual stimulation and determining an initial stimulation picture associated with the initial stimulation target, and setting a target frequency and a target phase corresponding to the initial stimulation picture, wherein the target frequency and the target phase are core parameters of the visual stimulation. The target frequency determines the presentation speed of the stimulation (such as how many frames per second), and the target phase affects the presentation mode of the stimulation (such as regular flashing or changes), so that the initial stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target frequency and the target phase, so that the presentation of the visual stimulation can be ensured to meet the requirements, and then a stimulation mapping table is established according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters, and the establishment of the stimulation mapping table can reduce the possibility of human operation errors. Thus, the current stimulation target corresponding to the subject is obtained, and then the target stimulation picture and the target stimulation parameters corresponding to the current stimulation target are quickly located through the stimulation mapping table to ensure the accurate presentation of the stimulation, and then the target electroencephalogram signals of the subject under visual stimulation are collected in real time on the basis of the dynamic adjustment of the target stimulation picture and the parameters. This real-time acquisition method can capture more subtle neural responses, especially under dynamically changing stimulus conditions, which helps to study the brain's real-time processing mechanism for different visual stimuli. In traditional experiments, although the collection of EEG signals by the ear can ensure the information transmission rate, there is often a problem of signal attenuation, which affects the accuracy of the data. Through the above steps, researchers can achieve precise control of visual stimulation on the target display, combined with efficient target EEG signal acquisition technology, effectively reduce the impact of signal attenuation and improve the signal-to-noise ratio of experimental data. Therefore, by optimizing the stimulation parameters and EEG signal acquisition methods, signal attenuation can be reduced and the accuracy and reliability of the collected EEG signals can be improved.

[0087] See also Figure 4 , Figure 4An electroencephalogram signal acquisition device 200 based on visual stimulation is provided in an embodiment of the present application. The electroencephalogram signal acquisition device 200 based on visual stimulation includes a data determination module 201, a data setting module 202, a parameter determination module 203, a mapping establishment module 204, a parameter acquisition module 205, and a signal acquisition module 206, wherein the data determination module 201 is used to determine an initial stimulation target corresponding to the visual stimulation, and to determine an initial stimulation image associated with the initial stimulation target; the data setting module 202 is used to set a target frequency and a target phase corresponding to the initial stimulation image; the parameter determination module 203 is used to determine the initial stimulation image according to the target frequency and the target phase. The initial stimulation parameters corresponding to the initial stimulation picture in the target display; wherein the initial stimulation parameters are used to characterize the parameters corresponding to the initial stimulation picture when it is displayed and played in the target display; a mapping establishment module 204 is used to establish a stimulation mapping table according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters; a parameter acquisition module 205 is used to obtain the current stimulation target corresponding to the subject, and determine the target stimulation picture and target stimulation parameters corresponding to the current stimulation target according to the stimulation mapping table; a signal acquisition module 206 is used to collect the target EEG signal corresponding to the subject under the visual stimulation of the target stimulation picture and the target stimulation parameters on the current display.

[0088] In some implementations, the parameter determination module 203, during the process of determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase, performs:

[0089] Determine a target refresh rate corresponding to the initial stimulus image in the target display;

[0090] Determine a target semantic picture corresponding to the initial stimulus picture in the target display according to the target frequency and the target phase in combination with a target encoding function;

[0091] The target stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target refresh rate and the target semantic picture, wherein the target stimulation parameters are a multi-frame brightness sequence corresponding to the initial stimulation picture.

[0092] In some implementations, the parameter determination module 203, in the process of determining the target semantic image corresponding to the initial stimulus image in the target display according to the target frequency and the target phase in combination with the target encoding function, performs:

[0093] Determine a first time window, and determine a first sampling time according to the first time window and the target refresh rate;

[0094] Determine first phase information, and determine the target encoding function according to the target frequency, the first sampling time and the first phase information; the target encoding function is a sine function or a cosine function;

[0095] The target semantic picture corresponding to the initial stimulus picture in the target display is determined according to the target encoding function.

[0096] In some implementations, the parameter determination module 203, during the process of determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase, performs:

[0097] Determine a target refresh rate corresponding to the initial stimulus image in the target display;

[0098] Determine a second time window, and determine a second sampling time according to the second time window and the target refresh rate;

[0099] Determine the second phase information, and determine the target encoding function according to the target frequency, the second sampling time and the second phase information; the target encoding function is a square wave function;

[0100] The initial stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target encoding function, wherein the initial stimulation parameters are a sequential display sequence corresponding to two different initial stimulation pictures corresponding to the initial stimulation target.

[0101] In some embodiments, the visual stimulation-based EEG signal acquisition device 200 further performs, after obtaining the target EEG signal:

[0102] Classifying the target EEG signal to obtain a fixation image corresponding to the target EEG signal;

[0103] A current gaze result corresponding to the subject is determined according to the gaze picture.

[0104] In some embodiments, the EEG signal acquisition device 200 based on visual stimulation performs, during the process of classifying the target EEG signal to obtain the fixation image corresponding to the target EEG signal:

[0105] When the target EEG signal does not need to be calibrated, generating a corresponding preset EEG signal based on a priori frequency coding information;

[0106] Performing correlation calculation on the target EEG signal and the preset EEG signal to obtain a target correlation value between the target EEG signal and the preset EEG signal;

[0107] The fixation picture corresponding to the target EEG signal is determined according to the target correlation value.

[0108] In some implementations, the visual stimulation-based EEG signal acquisition device 200, during the process of classifying the target EEG signal and obtaining the associated image corresponding to the target EEG signal, further performs:

[0109] When the target EEG signal needs to be calibrated, performing calibration analysis on the target EEG signal to obtain a target calibration signal;

[0110] The target calibration signal is classified by using a target classification model to obtain the associated image corresponding to the target EEG signal.

[0111] In some implementations, the visual stimulation-based EEG signal acquisition device 200 may be applied to a terminal device.

[0112] It should be noted that, those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the EEG signal acquisition device 200 based on visual stimulation described above can refer to the corresponding process in the aforementioned embodiment of the EEG signal acquisition method based on visual stimulation, and will not be repeated here.

[0113] See also Figure 5 , Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention.

[0114] like Figure 5 As shown, the terminal device 300 includes a processor 301 and a memory 302 , and the processor 301 and the memory 302 are connected via a bus 303 , such as an I2C (Inter-integrated Circuit) bus.

[0115] Specifically, the processor 301 is used to provide computing and control capabilities to support the operation of the entire terminal device. The processor 301 can be a central processing unit (CPU), and the processor 301 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0116] Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0117] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present invention, and does not constitute a limitation on the terminal device to which the embodiment of the present invention is applied. The specific server may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0118] The processor is used to run a computer program stored in the memory, and implement any one of the visual stimulation-based EEG signal acquisition methods provided by the embodiments of the present invention when executing the computer program.

[0119] In one embodiment, the processor is used to run a computer program stored in the memory, and implements the following steps when executing the computer program:

[0120] Determining an initial stimulus target corresponding to the visual stimulus, and determining an initial stimulus picture associated with the initial stimulus target;

[0121] Setting the target frequency and target phase corresponding to the initial stimulus image;

[0122] Determining initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase; wherein the initial stimulation parameters are used to characterize the parameters corresponding to the initial stimulation picture when it is displayed and played in the target display;

[0123] Establishing a stimulation mapping table according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters;

[0124] Obtaining a current stimulation target corresponding to the subject, and determining a target stimulation picture and target stimulation parameters corresponding to the current stimulation target according to the stimulation mapping table;

[0125] The target electroencephalogram signal corresponding to the subject when the target stimulation picture and the target stimulation parameters are visually stimulated on the current display is collected.

[0126] In some implementations, during the process of determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase, the processor 301 executes:

[0127] Determine a target refresh rate corresponding to the initial stimulus image in the target display;

[0128] Determine a target semantic picture corresponding to the initial stimulus picture in the target display according to the target frequency and the target phase in combination with a target encoding function;

[0129] The target stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target refresh rate and the target semantic picture, wherein the target stimulation parameters are a multi-frame brightness sequence corresponding to the initial stimulation picture.

[0130] In some implementations, the processor 301, in the process of determining the target semantic image corresponding to the initial stimulus image in the target display according to the target frequency and the target phase in combination with the target encoding function, executes:

[0131] Determine a first time window, and determine a first sampling time according to the first time window and the target refresh rate;

[0132] Determine first phase information, and determine the target encoding function according to the target frequency, the first sampling time and the first phase information; the target encoding function is a sine function or a cosine function;

[0133] The target semantic picture corresponding to the initial stimulus picture in the target display is determined according to the target encoding function.

[0134] In some implementations, during the process of determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase, the processor 301 executes:

[0135] Determine a target refresh rate corresponding to the initial stimulus image in the target display;

[0136] Determine a second time window, and determine a second sampling time according to the second time window and the target refresh rate;

[0137] Determine the second phase information, and determine the target encoding function according to the target frequency, the second sampling time and the second phase information; the target encoding function is a square wave function;

[0138] The initial stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target encoding function, wherein the initial stimulation parameters are a sequential display sequence corresponding to two different initial stimulation pictures corresponding to the initial stimulation target.

[0139] In some implementations, after obtaining the target EEG signal, the processor 301 further executes:

[0140] Classifying the target EEG signal to obtain a fixation image corresponding to the target EEG signal;

[0141] A current gaze result corresponding to the subject is determined according to the gaze picture.

[0142] In some implementations, the processor 301, during the process of classifying the target EEG signal to obtain a fixation image corresponding to the target EEG signal, executes:

[0143] When the target EEG signal does not need to be calibrated, generating a corresponding preset EEG signal based on a priori frequency coding information;

[0144] Performing correlation calculation on the target EEG signal and the preset EEG signal to obtain a target correlation value between the target EEG signal and the preset EEG signal;

[0145] The fixation picture corresponding to the target EEG signal is determined according to the target correlation value.

[0146] In some implementations, the processor 301, during the process of classifying the target EEG signal and obtaining the associated image corresponding to the target EEG signal, further executes:

[0147] When the target EEG signal needs to be calibrated, performing calibration analysis on the target EEG signal to obtain a target calibration signal;

[0148] The target calibration signal is classified by using a target classification model to obtain the associated image corresponding to the target EEG signal.

[0149] It should be noted that those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the specific working process of the terminal device described above can refer to the corresponding process in the aforementioned embodiment of the EEG signal acquisition method based on visual stimulation, and will not be repeated here.

[0150] An embodiment of the present invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of the methods for collecting electroencephalogram signals based on visual stimulation provided in the description of the embodiment of the present invention.

[0151] The storage medium may be an internal storage unit of the terminal device described in the foregoing embodiment, such as a hard disk or memory of the terminal device. The storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc., equipped on the terminal device.

[0152] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0153] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0154] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for collecting electroencephalogram signals based on visual stimulation, characterized in that: The method comprises: Determining an initial stimulus target corresponding to the visual stimulus, and determining an initial stimulus picture associated with the initial stimulus target; Setting the target frequency and target phase corresponding to the initial stimulus image, wherein the target frequency refers to the rate at which the visual stimulus is presented, and the target phase refers to the mode in which the visual stimulus is presented; Determining initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase; wherein the initial stimulation parameters are used to characterize the parameters corresponding to the initial stimulation picture when it is displayed and played in the target display; Establishing a stimulation mapping table according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters; Obtaining a current stimulation target corresponding to the subject, and determining a target stimulation picture and target stimulation parameters corresponding to the current stimulation target according to the stimulation mapping table; Collecting the target EEG signal of the subject corresponding to the visual stimulation of the target stimulation picture and the target stimulation parameters on the current display; After obtaining the target EEG signal, the method further includes: When the target EEG signal does not need to be calibrated, a corresponding preset EEG signal is generated based on a priori frequency coding information; a correlation calculation is performed between the target EEG signal and the preset EEG signal to obtain a target correlation value between the target EEG signal and the preset EEG signal; and a fixation image corresponding to the target EEG signal is determined according to the target correlation value; When the target EEG signal needs to be calibrated, performing calibration analysis on the target EEG signal to obtain a target calibration signal; performing signal classification on the target calibration signal using a target classification model to obtain the fixation image corresponding to the target EEG signal; A current gaze result corresponding to the subject is determined according to the gaze picture.

2. The method according to claim 1, characterized in that The step of determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase includes: Determine a target refresh rate corresponding to the initial stimulus image in the target display; Determine a target semantic picture corresponding to the initial stimulus picture in the target display according to the target frequency and the target phase in combination with a target encoding function; The target stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target refresh rate and the target semantic picture, wherein the target stimulation parameters are a multi-frame brightness sequence corresponding to the initial stimulation picture.

3. The method according to claim 2, characterized in that The step of determining the target semantic picture corresponding to the initial stimulus picture in the target display according to the target frequency and the target phase in combination with a target encoding function comprises: Determine a first time window, and determine a first sampling time according to the first time window and the target refresh rate; Determine first phase information, and determine the target encoding function according to the target frequency, the first sampling time and the first phase information; the target encoding function is a sine function or a cosine function; The target semantic picture corresponding to the initial stimulus picture in the target display is determined according to the target encoding function.

4. The method according to claim 1, characterized in that: The step of determining the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase includes: Determine a target refresh rate corresponding to the initial stimulus image in the target display; Determine a second time window, and determine a second sampling time according to the second time window and the target refresh rate; Determine the second phase information, and determine the target encoding function according to the target frequency, the second sampling time and the second phase information; the target encoding function is a square wave function; The initial stimulation parameters corresponding to the initial stimulation picture in the target display are determined according to the target encoding function, wherein the initial stimulation parameters are a sequential display sequence corresponding to two different initial stimulation pictures corresponding to the initial stimulation target.

5. An electroencephalogram signal acquisition device based on visual stimulation, characterized in that: include: A data determination module, used to determine an initial stimulus target corresponding to the visual stimulus, and to determine an initial stimulus picture associated with the initial stimulus target; A data setting module, used to set the target frequency and target phase corresponding to the initial stimulus image, wherein the target frequency refers to the rate at which the visual stimulus is presented, and the target phase refers to the mode in which the visual stimulus is presented; A parameter determination module, used to determine the initial stimulation parameters corresponding to the initial stimulation picture in the target display according to the target frequency and the target phase; wherein the initial stimulation parameters are used to characterize the parameters corresponding to the initial stimulation picture when it is displayed and played in the target display; A mapping establishment module, used to establish a stimulation mapping table according to the initial stimulation target, the initial stimulation picture and the initial stimulation parameters; A parameter acquisition module, used to obtain the current stimulation target corresponding to the subject, and determine the target stimulation picture and target stimulation parameters corresponding to the current stimulation target according to the stimulation mapping table; A signal acquisition module is used to acquire the target EEG signal corresponding to the subject when the target stimulation image and the target stimulation parameters are visually stimulated on the current display, wherein after obtaining the target EEG signal, when the target EEG signal does not need to be calibrated, a corresponding preset EEG signal is generated based on prior frequency coding information; the target EEG signal and the preset EEG signal are correlated to obtain a target correlation value between the target EEG signal and the preset EEG signal; a fixation image corresponding to the target EEG signal is determined according to the target correlation value; when the target EEG signal needs to be calibrated, a calibration analysis is performed on the target EEG signal to obtain a target calibration signal; the target calibration signal is classified using a target classification model to obtain the fixation image corresponding to the target EEG signal; and a current fixation result corresponding to the subject is determined according to the fixation image.

6. A terminal device, characterized in that: The terminal device includes a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program and implement the electroencephalogram signal acquisition method based on visual stimulation as described in any one of claims 1 to 4 when executing the computer program.

7. A computer storage medium for computer storage, characterized in that: The computer storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the EEG signal acquisition method based on visual stimulation as described in any one of claims 1 to 4.

Images