EEG and event-related potential testing method, device, electronic device and storage medium based on extended reality

Abstract

The present application discloses an EEG and event-related potential test method, device, electronic device and storage medium based on extended reality, the method comprising: constructing a virtual environment of the real world based on a preset game engine, wherein the virtual environment comprises at least one environmental scene and at least one perceptual scene; determining a target environmental scene, and determining a target perceptual scene based on the target environmental scene and configuration parameters; dynamically adjusting the configuration parameters of the target environmental scene under the target environmental scene and the target perceptual scene, and collecting event-related potential ERP data of the test subject under different configuration parameters; analyzing the ERP data to determine the ERP response of the test subject under different configuration parameters. The method of the present application can improve the ecological validity of ERP research, enrich the types of stimulation, accurately simulate cognitive processes in the real world, better understand how the brain works in complex cognitive tasks, and provide new tools and perspectives for neuroscience research.

Description

Technical Field

[0001] The present application relates to the field of neuroscience technology, and in particular to an EEG and event-related potential testing method based on extended reality, an EEG and event-related potential testing device based on extended reality, a computer-readable storage medium, and an electronic device. Background Art

[0002] In the fields of neuroscience and cognitive science, event-related potential (ERP) is an important tool for studying the brain's electrophysiological response to specific events or stimuli. Traditional ERP induction methods usually rely on simple visual or auditory stimulation in a laboratory environment. Although this method can control experimental conditions, it has low ecological validity and cannot simulate complex cognitive tasks in the real world. In addition, traditional ERP studies often use a single visual or auditory stimulus, which limits the diversity and depth of the research and makes it difficult to simulate complex cognitive tasks. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the first purpose of the present application is to propose an EEG and event-related potential testing method based on extended reality, to build a virtual environment of the real world based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene, to determine the target environmental scene, and to determine the target perception scene based on the target environmental scene and the configuration parameters, to dynamically adjust the configuration parameters of the target environmental scene under the target environmental scene and the target perception scene, and to collect the event-related potential ERP data of the test subject under different configuration parameters, and to analyze the ERP data to determine the ERP response of the test subject under different configuration parameters, thereby improving the ecological validity of ERP research, enriching the stimulus types, and accurately simulating the cognitive process in the real world, and better understanding the working mode of the brain in complex cognitive tasks, and providing new tools and perspectives for the research of cognitive science and neuroscience.

[0004] The second purpose of this application is to propose an EEG and event-related potential testing device based on extended reality.

[0005] The third object of the present application is to provide a computer-readable storage medium.

[0006] The fourth objective of the present application is to provide an electronic device.

[0007] To achieve the above-mentioned objectives, the first aspect of the present application proposes an EEG and event-related potential testing method based on extended reality, the method comprising: constructing a virtual environment of the real world based on a preset game engine, wherein the virtual environment comprises at least one environmental scene and at least one perception scene; determining a target environmental scene, and determining a target perception scene based on the target environmental scene and configuration parameters; under the target environmental scene and the target perception scene, dynamically adjusting the configuration parameters of the target environmental scene, and collecting event-related potential (ERP) data of the test subject under different configuration parameters; analyzing the ERP data to determine the ERP response of the test subject under different configuration parameters.

[0008] According to the EEG and event-related potential testing method based on extended reality in the embodiment of the present application, a virtual environment of the real world is constructed based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene, a target environmental scene is determined, and a target perception scene is determined based on the target environmental scene and configuration parameters, and the configuration parameters of the target environmental scene are dynamically adjusted under the target environmental scene and the target perception scene, and event-related potential ERP data of the test subject under different configuration parameters are collected, and the ERP data are analyzed to determine the ERP response of the test subject under different configuration parameters. As a result, the method can improve the ecological validity of ERP research, enrich the types of stimulation, and accurately simulate cognitive processes in the real world, and can better understand how the brain works in complex cognitive tasks, providing new tools and perspectives for the study of cognitive science and neuroscience.

[0009] In addition, the EEG and event-related potential testing method based on extended reality according to the above embodiment of the present application may also have the following additional technical features:

[0010] According to one embodiment of the present application, constructing a virtual environment of the real world based on a preset game engine includes: defining the boundaries and materials of the virtual environment, and generating a three-dimensional graphic of the virtual environment; using the preset game engine to construct a virtual environment of the real world in the three-dimensional graphic according to a preset ratio, wherein the preset game engine includes one of the Unity game engine and the Unreal Engine game engine.

[0011] According to one embodiment of the present application, analyzing the ERP data includes: processing the ERP data using short-time Fourier transform to obtain ERP waveform changes of the ERP data at different time points and frequencies; and determining the ERP response of the subject based on the ERP waveform changes.

[0012] According to one embodiment of the present application, analyzing the ERP data includes: processing the ERP data using continuous wavelet transform to obtain time-frequency spectra of the ERP data at different time points and frequencies; and determining the ERP response of the subject based on the time-frequency spectra.

[0013] According to one embodiment of the present application, the target perception scene includes at least one of a visual perception scene, a tactile perception scene, and an auditory perception scene.

[0014] According to one embodiment of the present application, the ERP data includes electroencephalogram data, and the ERP data of the subject is collected through a VR helmet and an electroencephalogram cap.

[0015] According to an embodiment of the present application, the method further includes: using one of statistical analysis, result comparison, and spectrum analysis to compare the ERP response with the ERP data obtained in the experimental environment to obtain a verification result.

[0016] To achieve the above-mentioned purpose, the second aspect embodiment of the present application proposes an EEG and event-related potential testing device based on extended reality, wherein the device includes: a construction module, which is used to construct a virtual environment of the real world based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene; a determination module, which is used to determine the target environmental scene, and determine the target perception scene based on the target environmental scene and the configuration parameters; an adjustment module, which is used to dynamically adjust the configuration parameters of the target environmental scene under the target environmental scene and the target perception scene; an acquisition module, which is used to collect event-related potential ERP data of the test subject under different configuration parameters; and an analysis module, which is used to analyze the ERP data to determine the ERP response of the test subject under different configuration parameters.

[0017] According to the EEG and event-related potential testing device based on extended reality in the embodiment of the present application, the construction module is used to construct a virtual environment of the real world based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene, the determination module is used to determine the target environmental scene, and determine the target perception scene based on the target environmental scene and the configuration parameters, the adjustment module is used to dynamically adjust the configuration parameters of the target environmental scene under the target environmental scene and the target perception scene, the acquisition module is used to collect the event-related potential ERP data of the test subject under different configuration parameters; the analysis module is used to analyze the ERP data to determine the ERP response of the test subject under different configuration parameters. As a result, the device can improve the ecological validity of ERP research, enrich the types of stimulation, and accurately simulate the cognitive process in the real world, and can better understand how the brain works in complex cognitive tasks, providing new tools and perspectives for the study of cognitive science and neuroscience.

[0018] To achieve the above-mentioned purpose, the third aspect of the present application proposes a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the above-mentioned EEG and event-related potential testing method based on extended reality is implemented.

[0019] According to the computer-readable storage medium of the embodiment of the present application, by implementing the above-mentioned EEG and event-related potential testing method based on extended reality during execution, it is possible to improve the ecological validity of ERP research, enrich the stimulation types, and accurately simulate the cognitive process in the real world, so as to better understand how the brain works in complex cognitive tasks, and provide new tools and perspectives for the research of cognitive science and neuroscience.

[0020] To achieve the above-mentioned purpose, an electronic device proposed in the fourth aspect of the present application includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned EEG and event-related potential testing method based on extended reality is implemented.

[0021] According to the electronic device of the embodiment of the present application, by executing the above-mentioned EEG and event-related potential testing method based on extended reality, it is possible to improve the ecological validity of ERP research, enrich the stimulation types, and accurately simulate the cognitive process in the real world. It is possible to better understand how the brain works in complex cognitive tasks, and provide new tools and perspectives for the research of cognitive science and neuroscience.

[0022] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1It is a flowchart of an EEG and event-related potential testing method based on extended reality according to an embodiment of the present application;

[0024] Figure 2 A flowchart of an EEG and event-related potential testing method based on extended reality according to a specific example of the present application;

[0025] Figure 3 Schematic diagram of a block diagram of an EEG and event-related potential testing device based on extended reality according to an embodiment of the present application;

[0026] Figure 4 Schematic diagram of a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0028] The following describes, with reference to the accompanying drawings, an EEG and event-related potential testing method based on extended reality, an EEG and event-related potential testing device based on extended reality, a computer-readable storage medium, and an electronic device proposed in the embodiments of the present application.

[0029] Figure 1 It is a flowchart of an EEG and event-related potential testing method based on extended reality according to an embodiment of the present application.

[0030] like Figure 1 As shown, the EEG and event-related potential testing method based on extended reality in the embodiment of the present application may include the following steps:

[0031] S1, constructing a virtual environment of the real world based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene.

[0032] S2, determining a target environment scene, and determining a target perception scene based on the target environment scene and configuration parameters.

[0033] S3, in the target environment scene and the target perception scene, dynamically adjust the configuration parameters of the target environment scene, and collect the event-related potential ERP data of the test subject under different configuration parameters.

[0034] S4, analyzing the ERP data to determine the ERP responses of the test subjects under different configuration parameters.

[0035] Specifically, in order to solve the problems of low ecological validity, single stimulus, and difficulty in simulating complex cognitive tasks in existing ERP research, complex multi-sensory stimulation and cognitive tasks simulating the real world can be designed in a virtual reality environment. That is, first, a virtual environment of the real world can be built according to a preset game engine, such as Unity or Unreal Engine. These engines provide powerful tools and libraries for creating complex virtual environments. In addition, the virtual environment includes at least one environmental scene and at least one perceptual scene, wherein the environmental scene may include a city street scene (such as a traffic light scene), a forest scene, an office scene, etc. These scenes need to simulate the visual effects and physical properties of the real world as much as possible. The perceptual scene involves the user's sensory experience, such as vision, hearing, touch, etc. For example, the sound and touch of rain can be simulated in a virtual environment. It should be noted that in order for the virtual environment to respond to the user's actions and perceptions, various sensors and input devices need to be integrated, such as VR helmets, handles, tactile gloves, etc.

[0036] Select a specific target environment scene from the constructed environment scenes, which will be used for the experiment, and determine the target perception scene based on the target environment scene and configuration parameters. That is, configuration parameters such as lighting, weather conditions, sound effects, etc. can be set for the selected environment scene, and based on the target environment scene and configuration parameters, the target perception scene is determined, such as visual stimulation (such as color, shape, movement), auditory stimulation (such as the type and volume of sound), and tactile stimulation (such as the intensity of tactile feedback). For example, in a supermarket environment, select a specific shelf as the target environment scene, and set configuration parameters such as the arrangement and color of goods on the shelf. The perception scene may include visual and auditory feedback when customers select goods. For another example, in a traffic light scene, the configuration parameters include the color, duration, intensity, and frequency of change of the traffic light, and the perception scene may include visual and auditory feedback when the driver is waiting for a red light.

[0037] In the target environment scene and the target perception scene, the configuration parameters of the target environment scene can be dynamically adjusted, and the event-related potential ERP data of the test subject under different configuration parameters can be collected. That is to say, during the experiment, the configuration parameters of the target environment scene can be dynamically adjusted according to the preset experimental plan, such as changing the light intensity or sound effect, and while the test subject is experiencing the virtual environment, the electroencephalogram device can be used to collect the electrical activity data of the brain in real time. The ERP response of the test subject under different configuration parameters is recorded, and these data will be used for subsequent analysis. For example, in a supermarket environment, the volume of background music is gradually increased, and the ERP response of the test subject at different volumes is recorded to study the impact of sound on attention and emotion.

[0038] Thus, the ERP data can be analyzed to determine the ERP response of the test subject under different configuration parameters. For example, the collected ERP data is preprocessed, including filtering, artifact removal, etc., to improve the data quality. Analyze the ERP data to identify potential changes related to specific events or stimuli, such as ERP components such as P300 and N400. For example, the time-frequency analysis of event-related potentials is a signal processing technology that combines time domain and frequency domain information. It can provide a detailed description of the changes in EEG signals over time and frequency. Through methods such as short-time Fourier transform or continuous wavelet transform, it reveals the energy changes of ERP signals at different time points and frequencies. It is particularly suitable for detecting event-related synchronization / desynchronization phenomena, that is, the increase or decrease of spectral power in a specific frequency band. This analysis can not only provide dynamic changes in ERP components such as P300 and N400 at different time points and frequencies, but also help researchers to have a deeper understanding of the brain's response mechanism to specific events or tasks.

[0039] Among them, P300 and N400 are two important event-related potential (ERP) components. P300 is a positive ERP component, which usually appears about 300 milliseconds after the stimulus occurs. The amplitude of P300 wave is large and the span range is wide. It is induced by rare and task-related stimuli (called target stimuli). P300 is related to many factors such as subjective probability, relevant tasks, importance of stimulus, decision-making, decision confidence, uncertainty of stimulus, attention, memory and emotion; N400 is a negative ERP component, which usually appears about 400 milliseconds after the stimulus. N400 wave often appears in research situations such as ambiguous words at the end of sentence, related words and irrelevant words, words and non-words, new words and old words, and picture naming. By analyzing P300 and N400, researchers can have a deeper understanding of the brain's response and processing mechanisms in specific cognitive tasks. And statistical methods, such as analysis of variance, can be used to compare the ERP responses of the subjects under different configuration parameters. Based on the analysis results, the ERP responses of the subjects under different configuration parameters can be interpreted to understand their perception and cognitive processing of environmental changes.

[0040] Therefore, conducting ERP research in a virtual reality environment and analyzing the subjects' responses to different environmental stimuli can improve the ecological validity of ERP research, enrich the types of stimulation, and accurately simulate cognitive processes in the real world. This can better understand how the brain works in complex cognitive tasks and provide new tools and perspectives for the study of cognitive science and neuroscience.

[0041] According to an embodiment of the present application, constructing a virtual environment of the real world based on a preset game engine includes: defining the boundaries and materials of the virtual environment, generating a three-dimensional graphic of the virtual environment; using the preset game engine to construct the virtual environment of the real world in the three-dimensional graphic according to a preset ratio, wherein the preset game engine includes one of the Unity game engine or the UnrealEngine game engine. The preset ratio can be determined according to actual conditions.

[0042] Specifically, when building a virtual environment of the real world according to a preset game engine, the boundaries and materials of the virtual environment can be defined to generate three-dimensional graphics of the virtual environment. The boundaries of the virtual environment include the size, shape and range of the environment, which can be achieved by creating an empty object or a parent object as a container for the environment. And the materials of the environment can be defined, which involves selecting or creating textures, which will be applied to the surfaces of the environment, such as the ground, walls, ceilings, etc. The materials can also include normal maps and highlight maps, etc., to increase the realism of the environment. Use 3D modeling software (such as 3ds Max, Maya, Blender, etc.) to create three-dimensional models of the virtual environment. These models can be elements in the real world such as buildings, roads, vegetation, etc. These three-dimensional models are imported into the game engine and adjusted and optimized as needed to ensure that they can run efficiently in the engine.

[0043] In the game engine, a virtual environment of the real world can be built according to a preset scale. This means that all models and textures need to be scaled and positioned according to the size and scale of the real world. For example, use the tools provided by the game engine, such as terrain tools, to create terrain and ground. These tools can help simulate natural landforms such as mountains, plains, and rivers. And place the three-dimensional models in the appropriate position in the virtual environment, and adjust their orientation and scale to ensure the consistency and realism of the environment. You can also set physical properties such as mass, friction, and elasticity for objects in the virtual environment so that they can correctly respond to gravity and other forces in physical simulations, and set environmental interactions, such as doors can be opened, objects can be picked up and moved, etc. Among them, the preset game engine can be the Unity game engine or the Unreal Engine game engine, both of which can provide powerful graphics rendering capabilities and physical simulation functions.

[0044] In addition, the virtual environment can be optimized to ensure that it runs smoothly on the target platform, which can include reducing the number of polygons, optimizing texture resolution and adjusting lighting settings, and testing to check whether the visual effects and physical behavior of the environment are as expected and whether there are any bugs or performance issues. In this way, a highly realistic virtual environment can be created for ERP research and cognitive science experiments.

[0045] According to one embodiment of the present application, analyzing ERP data includes: processing the ERP data using short-time Fourier transform to obtain ERP waveform changes of the ERP data at different time points and frequencies; and determining the ERP response of the subject based on the ERP waveform changes.

[0046] Specifically, when analyzing ERP data, short-time Fourier transform can be used to process ERP data to obtain the ERP waveform changes of ERP data at different time points and frequencies. Among them, short-time Fourier transform is a time-frequency analysis method that provides local information of the signal in time and frequency by dividing the signal into short time periods and performing Fourier transform in each time period. When processing ERP data, the ERP signal is first segmented and a window is added to each segment. This process involves multiplying the window function by a part of the original signal and then calculating the Fourier transform of this part of the signal. The choice of window function is very important for the result of short-time Fourier transform. Common window functions include Hanning window, Hamming window, etc., which will affect the trade-off between time and frequency resolution. By performing Fourier transform on the signal segment in each window, the frequency composition of the signal at different time points can be obtained. The result of short-time Fourier transform is a two-dimensional function that shows the frequency composition of the signal at different time points. This process will be repeated as the window function slides along the time axis to obtain the time-frequency information of the entire signal.

[0047] Therefore, the ERP response of the subject can be determined based on the change of ERP waveform. In other words, based on the time-frequency data obtained by short-time Fourier transform, the change of ERP waveform before and after the stimulus presentation can be analyzed. By observing the time-frequency diagram, it is possible to determine which time and frequency bands have obvious energy changes, which helps to identify the ERP components related to specific events or stimuli. By averaging time and frequency, a topographic map within this time-frequency range can be drawn to further analyze the spatial distribution of ERP responses. Professional ERP analysis software can be used to perform time-frequency analysis on the collected data to identify the ERP components related to the change of traffic lights. Through comparative analysis, the ERP waveform changes can be determined, thereby determining the ERP response of the subject. As a result, researchers can capture the transient changes and time-varying characteristics of the ERP signal, so as to more accurately analyze and understand the brain's electrophysiological response to specific events or stimuli.

[0048] According to another embodiment of the present application, analyzing ERP data includes: processing the ERP data using continuous wavelet transform to obtain time-frequency spectra of the ERP data at different time points and frequencies; and determining the ERP response of the subject based on the time-frequency spectra.

[0049] Specifically, when analyzing ERP data, continuous wavelet transform can also be used to process ERP data to obtain time-frequency spectra of ERP data at different time points and frequencies. Among them, continuous wavelet transform is a time-frequency analysis method that analyzes signals by decomposing them into wavelets obtained by dilation or scaling and translation of basic wavelets (mother wavelets) at different scales. Unlike Fourier transform, wavelet transform provides information on both time and frequency, which is suitable for processing non-stationary signals. By applying continuous wavelet transform to ERP data, a time-frequency spectrum can be obtained, which shows the energy distribution of ERP signals at different times and frequencies. This spectrum is obtained by calculating wavelet coefficients at different scales and wavelet displacements.

[0050] Therefore, the ERP response of the subject can be determined based on the time-frequency spectrum. That is, by analyzing the time-frequency spectrum, the frequency components and time points related to specific events or stimuli in the ERP signal can be identified. For example, it can be observed that after a specific stimulus occurs, the energy of certain frequency bands increases significantly, which may be related to specific EEG activities in the cognitive process. Therefore, researchers can use continuous wavelet transform to conduct in-depth analysis of ERP data, revealing the brain's electrophysiological response to specific events, and providing important information for the study of cognitive science and neuroscience.

[0051] According to one embodiment of the present application, the target perception scene includes at least one of a visual perception scene, a tactile perception scene, and an auditory perception scene.

[0052] Specifically, the target perception scene may include a visual perception scene, that is, in virtual reality (VR), the visual perception scene can be provided by a head-mounted display, and this scene allows users to feel an immersive experience through a computer-generated three-dimensional environment. The virtual scene is projected in front of the user through a head-mounted display device, a handheld device or a projection device, so that the user has an immersive feeling. The target perception scene may also include a tactile perception scene. The tactile perception scene is provided by a tactile glove or a tactile handle in VR. These devices can simulate touch and force feedback, allowing users to feel the texture and hardness of virtual objects. Tactile feedback enhances the user's immersion and interactive experience. The target perception scene also includes an auditory perception scene. The auditory perception scene is realized in VR through spatial audio technology. This technology can create 3D audio effects, allowing users to feel that the sound seems to come from the actual spatial position of the virtual object. For example, equipped with spatial sound technology, this technology enhances the perception of spatial layout and improves design efficiency and experience. In VR, auditory feedback is combined with visual and tactile feedback to provide a more complete and realistic immersive experience.

[0053] Therefore, these three perceptual scenes (vision, touch, and hearing) together constitute a multimodal perceptual environment in VR, which cooperate with each other to provide users with a comprehensive and immersive virtual experience. Through this multimodal perceptual interaction, VR technology can simulate a more realistic environment and enhance the user's cognitive and perceptual experience, which is of great significance to the research of neuroscience and cognitive science.

[0054] Therefore, when multi-sensory stimulation occurs simultaneously, the brain will produce specific event-related potential (ERP) waveforms for each sensory stimulus. These waveforms reflect the brain's processing of different sensory inputs. The following are some of the main sensory stimuli and their corresponding ERP waveforms: Visual stimulation: Visual ERP usually includes several different components, such as P1 (appearing approximately 50-100 milliseconds after stimulation), N1 (approximately 100-150 milliseconds), P2 (approximately 180 milliseconds), and the famous P300 (about 300 milliseconds after stimulation, related to attention and memory updating). Auditory stimulation: The main components of auditory ERP include P50 (approximately 50 milliseconds after stimulation), N1 (approximately 50-100 milliseconds), P2 (approximately 100-200 milliseconds), and N2 (approximately 200-300 milliseconds). Auditory P300 is also an important component, similar to visual P300, and is related to cognitive processing. Tactile stimulation: The waveform of tactile ERP may include early somatosensory evoked potentials, such as S1 (approximately 30-50 milliseconds after stimulation) and S2 (approximately 50-100 milliseconds). These waveforms reflect the initial processing of tactile information in the brain. When multisensory stimuli appear simultaneously, the ERP waveforms may influence each other and produce complex interactions. For example, if visual and auditory stimuli appear simultaneously, temporal superposition or mutual modulation of the ERP components of the two may be observed. In addition, multisensory integration can also lead to the enhancement or suppression of certain ERP components, depending on the consistency of sensory information and the brain's attention allocation. In experimental design, researchers can control factors such as the type, timing, and intensity of stimulation to study how different sensory stimuli affect ERP waveforms and how these waveforms reflect the brain's processing of multisensory information.

[0055] According to one embodiment of the present application, the ERP data includes electroencephalogram data, and the ERP data of the subject is collected through a VR helmet and an electroencephalogram cap.

[0056] Specifically, ERP data may include electroencephalogram data, and the ERP data of the subject can be collected jointly through a VR helmet and an electroencephalogram cap. Among them, the electroencephalogram cap is a wearable device with multiple electrodes on its inner surface for capturing the electrical activity of the brain. These electrodes are arranged according to international systems or other high-density layout standards to ensure that signals can be obtained from different brain regions. The VR helmet provides an immersive virtual environment that allows the subject to experience simulated real-world situations, in which specific events or stimuli can be designed to induce the subject's ERP response. In the VR environment, when specific events or stimuli occur, the electrodes on the electroencephalogram cap capture changes in the brain's electrical activity in real time. These changes are synchronized with the visual, auditory or tactile stimuli in the VR helmet, so that specific cognitive or perceptual events can be accurately associated with changes in the ERP waveform.

[0057] In addition, the raw EEG signals collected are usually very weak and need to be amplified by an amplifier connected to the EEG cap. The amplified signal will be converted into digital form for further analysis and processing. The amplified and digitized EEG signals will be recorded and can be further analyzed. It may include preprocessing steps (such as noise removal, filtering, etc.), as well as the application of various signal processing and analysis techniques (such as spectral analysis, time domain analysis, event-related potentials, etc.) to study brain activity patterns and related cognitive processes. By combining the high-density EEG cap and the VR helmet, scalp EEG signals and brain EEG signals can be collected synchronously, which is of great significance for understanding the functional connectivity and activity patterns of the brain in complex cognitive tasks.

[0058] As a result, researchers can induce and accurately record ERP data in a virtual reality environment, and then analyze the brain's electrophysiological responses to specific stimuli, improving the ecological validity of the experiment. When experiencing a virtual city environment, the brain's electrical activity can be synchronously recorded, ensuring the precise synchronization of the EEG equipment with the virtual reality environment, so that the brain's electrical activity can be recorded in real time when the subjects experience the virtual environment, providing new tools and perspectives for the study of cognitive science and neuroscience.

[0059] According to one embodiment of the present application, the EEG and event-related potential testing method based on extended reality also includes: using one of statistical analysis, result comparison, and spectrum analysis to compare the ERP response with the ERP data obtained in the experimental environment to obtain a verification result.

[0060] Specifically, in the process of comparing the ERP response with the ERP data obtained in the experimental environment to obtain the verification results, one of the methods of statistical analysis, result comparison, and spectrum analysis can be used. Among them, in the laboratory environment, researchers will design a series of ERP experiments, such as using classic experimental paradigms (such as Go / Nogo, Stroop tasks, etc.), and collect ERP data in advance by precisely controlling the stimulus presentation time and task requirements.

[0061] Statistical analysis of ERP data is mainly used to verify the reliability and significance of the results. For example, data preprocessing is first performed to ensure signal quality, including filtering, artifact detection and removal, baseline correction, and data segmentation. Specific events, such as stimulus presentation or response, are marked in the data, and EEG signals of the same event type are averaged to improve the signal-to-noise ratio of the signal and extract a clearer ERP waveform. When analyzing the ERP waveform, the latency and amplitude of each component are extracted. These features can reflect different cognitive processes, so the variance analysis method is used to test whether there are significant differences in ERP components under different conditions.

[0062] Results comparison is an effective method to verify ERP responses by comparing ERP data under different experimental conditions. For example, by selecting an appropriate time window for analysis, usually 200-600 milliseconds after stimulation, averaging multiple experimental results to obtain a stable ERP waveform, and then comparing the average ERP waveforms under different conditions to identify differences in waveforms, such as amplitude changes, latency changes, etc.

[0063] Spectral analysis focuses on the distribution of ERP in different frequency ranges and is used to analyze the frequency domain characteristics of ERP signals. It can convert ERP data from the time domain to the frequency domain to obtain a spectrum diagram that shows the intensity of ERP in different frequency ranges. The spectrum diagram is analyzed to determine the ERP peak and delay, and understand their relationship with behavioral and cognitive processes. Therefore, through the above three methods, researchers can compare ERP responses with ERP data obtained in the experimental environment to obtain verification results and further understand the brain's response patterns under specific events or stimuli.

[0064] Combine the following Figure 2 To describe the method of the present application.

[0065] As a specific example, the EEG and event-related potential testing method based on extended reality of the present application may include the following steps:

[0066] S101, defining the boundary and material of the virtual environment, and generating a three-dimensional graphic of the virtual environment.

[0067] S102, using a preset game engine to construct a virtual environment of the real world in a three-dimensional graphic according to a preset ratio, wherein the preset game engine includes one of a Unity game engine and an Unreal Engine game engine.

[0068] S103, determining a target environment scene, and determining a target perception scene based on the target environment scene and the configuration parameters, wherein the target perception scene includes at least one of a visual perception scene, a tactile perception scene, and an auditory perception scene.

[0069] S104, dynamically adjust the configuration parameters of the target environment scene in the target environment scene and the target perception scene, and collect the event-related potential ERP data of the test subject under different configuration parameters, wherein the ERP data includes electroencephalogram data, and collect the ERP data of the test subject through the VR helmet and the electroencephalogram cap. Then proceed to step S105 or step S108.

[0070] S105, processing the ERP data using short-time Fourier transform to obtain ERP waveform changes of the ERP data at different time points and frequencies.

[0071] S106, determining the ERP response of the subject based on the ERP waveform change.

[0072] S107, using one of statistical analysis, result comparison, and spectrum analysis to compare the ERP response with the ERP data obtained in the experimental environment to obtain a verification result.

[0073] S108, processing the ERP data using continuous wavelet transform to obtain time-frequency spectra of the ERP data at different time points and frequencies.

[0074] S109, determining the ERP response of the subject based on the time-frequency spectrum, and proceeding to step S107.

[0075] In summary, according to the EEG and event-related potential testing method based on extended reality of the embodiment of the present application, a virtual environment of the real world is constructed based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene, the target environmental scene is determined, and the target perception scene is determined based on the target environmental scene and the configuration parameters, and the configuration parameters of the target environmental scene are dynamically adjusted under the target environmental scene and the target perception scene, and the event-related potential ERP data of the test subject under different configuration parameters are collected, and the ERP data are analyzed to determine the ERP response of the test subject under different configuration parameters. Therefore, this method can improve the ecological validity of ERP research, enrich the types of stimulation, and accurately simulate the cognitive process in the real world, and can better understand how the brain works in complex cognitive tasks, and provide new tools and perspectives for the study of cognitive science and neuroscience.

[0076] Corresponding to the above-mentioned embodiments, the present application also proposes an EEG and event-related potential testing device based on extended reality.

[0077] like Figure 3 As shown, the EEG and event-related potential testing device 100 based on extended reality of the embodiment of the present application includes: a construction module 110, a determination module 120, an adjustment module 130, a collection module 140 and an analysis module 150.

[0078] Among them, the construction module 110 is used to build a virtual environment of the real world based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene. The determination module 120 is used to determine the target environmental scene, and determine the target perception scene based on the target environmental scene and the configuration parameters. The adjustment module 130 is used to dynamically adjust the configuration parameters of the target environmental scene under the target environmental scene and the target perception scene. The acquisition module 140 is used to collect event-related potential ERP data of the test subject under different configuration parameters. The analysis module 150 is used to analyze the ERP data to determine the ERP response of the test subject under different configuration parameters.

[0079] According to one embodiment of the present application, the construction module 110 constructs a virtual environment of the real world based on a preset game engine, and is specifically used to: define the boundaries and materials of the virtual environment, and generate three-dimensional graphics of the virtual environment; use the preset game engine to construct a virtual environment of the real world in the three-dimensional graphics according to a preset ratio, wherein the preset game engine includes one of the Unity game engine or the Unreal Engine game engine.

[0080] According to one embodiment of the present application, the analysis module 150 analyzes the ERP data, specifically for: processing the ERP data using short-time Fourier transform to obtain ERP waveform changes of the ERP data at different time points and frequencies; and determining the ERP response of the subject based on the ERP waveform changes.

[0081] According to one embodiment of the present application, the analysis module 150 analyzes the ERP data, specifically for: processing the ERP data using continuous wavelet transform to obtain time-frequency spectra of the ERP data at different time points and frequencies; and determining the ERP response of the subject based on the time-frequency spectra.

[0082] According to one embodiment of the present application, the target perception scene includes at least one of a visual perception scene, a tactile perception scene, and an auditory perception scene.

[0083] According to one embodiment of the present application, the ERP data includes electroencephalogram data, and the ERP data of the subject is collected through a VR helmet and an electroencephalogram cap.

[0084] According to an embodiment of the present application, the analysis module 150 is further used to: use one of statistical analysis, result comparison, and spectrum analysis to compare the ERP response with the ERP data obtained in the experimental environment to obtain a verification result.

[0085] It should be noted that for details not disclosed in the EEG and event-related potential testing device based on extended reality in the embodiment of the present application, please refer to the details disclosed in the EEG and event-related potential testing method based on extended reality in the embodiment of the present application, and the details will not be repeated here.

[0086] According to the EEG and event-related potential testing device based on extended reality in the embodiment of the present application, the construction module is used to construct a virtual environment of the real world based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene, the determination module is used to determine the target environmental scene, and determine the target perception scene based on the target environmental scene and the configuration parameters, the adjustment module is used to dynamically adjust the configuration parameters of the target environmental scene under the target environmental scene and the target perception scene, the acquisition module is used to collect the event-related potential ERP data of the test subject under different configuration parameters; the analysis module is used to analyze the ERP data to determine the ERP response of the test subject under different configuration parameters. As a result, the device can improve the ecological validity of ERP research, enrich the types of stimulation, and accurately simulate the cognitive process in the real world, and can better understand how the brain works in complex cognitive tasks, providing new tools and perspectives for the study of cognitive science and neuroscience.

[0087] Corresponding to the above embodiments, the present application also proposes a computer-readable storage medium.

[0088] The computer-readable storage medium of the embodiment of the present application stores a program thereon, and when the program is executed by a processor, the above-mentioned EEG and event-related potential testing method based on extended reality is implemented.

[0089] According to the computer-readable storage medium of the embodiment of the present application, by executing the above-mentioned EEG and event-related potential testing method based on extended reality, it is possible to improve the ecological validity of ERP research, enrich the stimulation types, and accurately simulate the cognitive process in the real world, so as to better understand how the brain works in complex cognitive tasks, and provide new tools and perspectives for the research of cognitive science and neuroscience.

[0090] Corresponding to the above embodiment, the present application also proposes an electronic device.

[0091] like Figure 4 As shown, the electronic device 200 of the embodiment of the present application may include: a memory 210, a processor 220, and a program stored in the memory 210 and executable on the processor 220. When the processor 220 executes the program, the above-mentioned EEG and event-related potential testing method based on extended reality is implemented.

[0092] According to the electronic device of the embodiment of the present application, by executing the above-mentioned EEG and event-related potential testing method based on extended reality, it is possible to improve the ecological validity of ERP research, enrich the stimulation types, and accurately simulate the cognitive process in the real world. It is possible to better understand how the brain works in complex cognitive tasks, and provide new tools and perspectives for the research of cognitive science and neuroscience.

[0093] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0094] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0095] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0096] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0097] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0098] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for testing EEG and event-related potentials based on extended reality, characterized in that: The method comprises: Constructing a virtual environment of the real world based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene; Determine a target environment scene, and determine a target perception scene based on the target environment scene and configuration parameters; In the target environment scene and the target perception scene, dynamically adjusting the configuration parameters of the target environment scene, and collecting event-related potential (ERP) data of the test subject under different configuration parameters; The ERP data are analyzed to determine the ERP response of the test subject under different configuration parameters.

2. The method for testing EEG and event-related potentials based on extended reality according to claim 1, characterized in that: Build real-world virtual environments based on pre-built game engines, including: Defining the boundaries and materials of the virtual environment and generating a three-dimensional graphic of the virtual environment; The preset game engine is used to construct a virtual environment of the real world in the three-dimensional graphics according to a preset proportion, wherein the preset game engine includes one of the Unity game engine and the Unreal Engine game engine.

3. The method for testing EEG and event-related potentials based on extended reality according to claim 1, characterized in that: The ERP data is analyzed, including: Processing the ERP data using short-time Fourier transform to obtain ERP waveform changes of the ERP data at different time points and frequencies; The ERP response of the subject is determined based on the ERP waveform change.

4. The method for testing EEG and event-related potentials based on extended reality according to claim 1, characterized in that: The ERP data is analyzed, including: The ERP data are processed by continuous wavelet transform to obtain time-frequency spectra of the ERP data at different time points and frequencies; The ERP response of the subject is determined based on the time-frequency spectrum.

5. The method for testing EEG and event-related potentials based on extended reality according to claim 1, characterized in that: The target perception scene includes at least one of a visual perception scene, a tactile perception scene and an auditory perception scene.

6. The method for testing EEG and event-related potentials based on extended reality according to claim 1, characterized in that: The ERP data includes electroencephalogram data, and the ERP data of the subject is collected through a VR helmet and an electroencephalogram cap.

7. The method for testing EEG and event-related potentials based on extended reality according to claim 1, characterized in that: The method further comprises: The ERP response is compared with the ERP data obtained in the experimental environment by using one of statistical analysis, result comparison, and spectrum analysis to obtain a verification result.

8. An EEG and event-related potential testing device based on extended reality, characterized in that: The device comprises: A construction module, used to construct a virtual environment of the real world based on a preset game engine, wherein the virtual environment includes at least one environmental scene and at least one perception scene; A determination module, used to determine a target environment scene, and determine a target perception scene based on the target environment scene and configuration parameters; An adjustment module, used for dynamically adjusting the configuration parameters of the target environment scene under the target environment scene and the target perception scene; An acquisition module, used for acquiring event-related potential (ERP) data of the test subject under different configuration parameters; The analysis module is used to analyze the ERP data to determine the ERP response of the test subject under different configuration parameters.

9. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the EEG and event-related potential testing method based on extended reality is implemented according to any one of claims 1-7.

10. An electronic device, characterized in that: include: A memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the EEG and event-related potential testing method based on extended reality is implemented according to any one of claims 1 to 7.

Images