Virtual reality sickness-oriented neurofeedback training method and system

Abstract

The application discloses a kind of virtual reality motion sickness-oriented neurofeedback training method and system, belong to virtual reality man-machine interaction and neural regulation technical field, contain brain electrical signal acquisition and hypergraph representation, built-in causal constraint interpretable graph neural network, individualized closed-loop neurofeedback regulation and effect evaluation.Integrating multi-channel electroencephalogram, extracting multi-domain features to construct hypergraph model and screening marker features;Real-time output motion sickness state and interpretable neural subgraph by embedding constraints;According to real-time brain state, dynamically adjust scene parameters and multi-sensory feedback, implement individualized adaptive strategy, and quantify intervention effect from multiple dimensions, realize virtual reality motion sickness real-time detection, dynamic regulation and closed-loop intervention integration.

Description

Technical Field

[0001] This invention belongs to the field of virtual reality human-computer interaction and neuromodulation technology, specifically relating to an interpretable neurofeedback training method and system for virtual reality motion sickness. Background Technology

[0002] Virtual reality (VR) technology has been rapidly implemented in industries, medicine, education, and other fields. However, about 80% of first-time users experience motion sickness within 20 minutes, which can lead to nerve damage in severe cases, becoming a core bottleneck for the large-scale promotion of VR. The current mainstream intervention method is habituation training, which improves the brain's adaptability through single repetitive stimulation. However, it has obvious drawbacks: (1) The training paradigm is uniform, ignoring individual neural differences, resulting in poor applicability and limited promotion; (2) It relies on subjective scale assessment, which cannot capture the neural mechanism of motion sickness in real time and objectively; (3) The neural feedback target is singular (such as only regulating alpha waves), resulting in low interpretability and a vague causal chain; (4) It lacks hierarchical representation on a time scale, making it impossible to distinguish the gradual process of "sensory conflict-motion sickness symptoms"; (5) The regulation strategy is fixed and cannot be dynamically adjusted according to the individual's real-time brain state, resulting in unstable intervention effects.

[0003] Existing research on VR motion sickness improvement based on EEG and neurofeedback is mostly an end-to-end black box model, without built-in causal constraints and untraceable decision-making basis; and it has not formed a closed loop of "mechanism analysis - target extraction - personalized regulation", making it difficult to achieve safe, efficient and accurate VR motion sickness intervention. Summary of the Invention

[0004] To address the shortcomings of the aforementioned background technology, the present invention aims to provide an interpretable and personalized neurofeedback training method and system for virtual reality motion sickness. Based on electroencephalogram (EEG) signals, interpretable graph neural networks, and personalized closed-loop regulation, this method intervenes in virtual reality motion sickness, aiming to solve the problems of low interpretability of target neural patterns in neurofeedback, weak causal relationships, lack of personalized adaptation in traditional training, and poor intervention effects. It integrates real-time detection, dynamic regulation, and closed-loop intervention for virtual reality motion sickness, and can be used for the detection, early warning, training, and symptom improvement of motion sickness in virtual reality users. It is applicable to industrial, medical, and other scenarios, improving user safety and comfort.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a neurofeedback training method for virtual reality motion sickness, comprising: In a quiet, VR-free state, resting-state EEG signals were collected from the subjects, and the SSQ motion sickness scale scores were completed to establish individual resting-state EEG baselines and SSQ baseline scores. VR-induced motion sickness was induced by applying graded stimuli to graded scenarios to subjects. The individual motion sickness threshold level was determined based on the resting-state EEG baseline, and the SSQ motion sickness score corresponding to the level was collected. Multi-channel EEG signals were collected in real time during VR motion sickness induction. After preprocessing and multi-domain feature extraction, a multi-attribute brain function network hypergraph reflecting the strength of brain region association was constructed. The hypergraph was based on the SSQ baseline score and SSQ motion sickness score to select effective features. A built-in causal constraint interpretable graph neural network model was constructed based on a multi-attribute brain functional network hypergraph to obtain the real-time motion sickness state and key neural subgraphs of the subjects in VR. Based on the real-time motion sickness state and the current real-time neural pattern represented by key neural subgraphs, and compared with the preset target neural pattern, a personalized control strategy is generated to dynamically adjust the current operating parameters of the VR scene and multi-sensory feedback, thereby achieving closed-loop control of real-time neural feedback. During the real-time neural feedback closed-loop control process, multi-channel EEG data of the entire process of motion sickness induction and neural feedback are continuously collected and continuously input into the causal interpretable graph neural network model for closed-loop iterative training. The model parameters and personalized control strategies are optimized simultaneously until the real-time neural pattern approaches the target neural pattern and reaches the preset termination condition, and then the multi-dimensional model evaluation results are output.

[0006] Furthermore, the determination of an individual's motion sickness threshold level based on the resting-state EEG baseline, and the acquisition of the corresponding SSQ motion sickness score, includes: During the gradual increase in intensity, the subjects' EEG response and subjective motion sickness status were monitored in real time, and multi-stage real-time SSQ motion sickness scale scores were collected simultaneously. When a subject first showed motion sickness tendency or mild motion sickness symptoms, the current stimulus level was the subject's individual motion sickness threshold level, and the real-time SSQ motion sickness scale score corresponding to this level was recorded as the SSQ motion sickness score.

[0007] Furthermore, after preprocessing and multi-domain feature extraction, a multi-attribute brain functional network hypermap reflecting the strength of brain region associations is constructed, including: During the VR motion sickness induction process, the original time-series EEG signals collected are retained at frequencies between 0.5Hz and 60Hz, while electromagnetic interference is filtered out to achieve real-time noise reduction. Multi-dimensional features in time domain, frequency domain, spatial domain, phase domain and Riemann space are extracted simultaneously. Pearson correlation coefficient, phase lock value (PLV) and mutual information (MI) of multi-dimensional features between channels are calculated and multi-indicator fusion is performed to determine the connection weight between channels. Based on the connection weight, the adjacency relationship of the EEG channel network is constructed. By integrating the pairwise channel correlation features, the joint correlation degree of multi-channel combinations is solved, and multi-channel joint correlations greater than the set adaptive threshold θ are defined as hyperedges; From the multi-dimensional features, we selected the iconic features that were significantly related to the SSQ baseline score and the SSQ motion sickness score, and configured them as the feature attributes of the hypergraph nodes. Normalized signature features are used to organize and arrange data according to the correspondence between hypergraph nodes, and then integrated to form a standardized hypergraph vertex feature matrix.

[0008] Further, determining the connection weights between channels includes: Using multi-channel EEG acquisition electrodes as independent nodes in the network, Pearson correlation coefficient, phase lock value (PLV), and mutual information (MI) are calculated between each pair of channels based on the preprocessed raw temporal EEG signals of each channel to obtain multi-dimensional correlation quantification values. The multi-dimensional correlation quantification values ​​are normalized respectively, and combined with the multi-indicator fusion quantification method, the comprehensive correlation strength between channels is obtained and determined as the connection weight between each channel.

[0009] Furthermore, an interpretable graph neural network model with built-in causal constraints was constructed based on a multi-attribute brain functional network hypergraph to obtain the real-time motion sickness state and key neural subgraphs of the subjects in VR, including: Construct a causal interpretable graph neural network model structure that includes an edge convolution module, a point feature convolution module, and a built-in causal relationship module; The multi-attribute brain functional network hypergraph is input into the edge convolution module, and the brain region association strength features are learned through learnable edge weights. The hypergraph is divided into information subgraphs and redundant networks within the edge convolution module. The correlation strength feature vector output by the edge convolution module is input into the point feature convolution module to extract high-dimensional features of the node features; By combining the updated edge weights learned by edge convolution with the node high-dimensional features output by point feature convolution, the connections and nodes in the hypergraph are jointly screened, retaining the connections and nodes that are significantly associated with the halo state, thus completing feature extraction and preliminary subgraph screening. Based on the built-in causal relationship module, the information subgraphs and redundant networks already divided by the edge convolution module are called to enhance the causality between the information subgraphs and the motion sickness state, weaken the causality of the redundant networks, and screen out the key neural subgraphs that are strongly correlated with the motion sickness response. The model is trained based on the features of the key neural subgraphs. Using the real-time VR motion sickness state as a label, the prediction error is constrained by the loss function, enabling the model to output the subject's real-time VR motion sickness state and key neural subgraphs.

[0010] Furthermore, the causality between the enhanced information subgraph and the motion sickness state, and the causality between the weakened redundant network and the motion sickness state, are used to screen out key neural subgraphs that are strongly correlated with the motion sickness response, including: Calculate the causal correlation strength between the information sub-image and the motion sickness state label, and increase its weight in the model loss function; Calculate the causal association strength between redundant networks and motion sickness state labels, and reduce their weight in the model loss function; The model is trained based on the adjusted loss function, so that the model prioritizes learning the edges and nodes in the information subgraph that are strongly related to the motion sickness state. Based on the degree of causal correlation after model training, edges and nodes with significant causal correlation with motion sickness response are retained in the information subgraph to form a key neural subgraph.

[0011] Furthermore, the step of outputting the subject's VR motion sickness state in real time based on the features of the key neural subgraph and updating the key neural subgraph features includes: The node features and edge features of the key neural subgraph are input into the classification layer of the model, and the prediction results of the VR motion sickness state of the subject are obtained by mapping through the fully connected layer. Based on the preset motion sickness level classification standard, the prediction results are mapped to the subject's current VR motion sickness level; Real-time acquisition of EEG data during subsequent VR interaction with subjects, and updating of brain functional network hypergraph; Repeat the processing flow of edge convolution, point feature convolution and causal relationship module, re-select key neural subgraphs and update their features; By combining the motion sickness state output by the model with the updated key neural subgraph features, a real-time basis is provided for the dynamic adjustment of subsequent VR scene parameters and multi-sensory feedback.

[0012] Furthermore, the step of inputting the correlation strength feature vector into the point feature convolution module to extract high-dimensional features of the node features includes: The correlation strength feature vector output by the edge convolution is input into the point feature convolution module. Local features are extracted and mapped to a high-dimensional space through K-means clustering and convolution operations. Then, it is weighted and fused with the original multi-dimensional features of the hypergraph nodes to extract the high-dimensional representation of the node features.

[0013] Furthermore, the graph isomorphic network GIN is used to perform convolutional aggregation on neighboring nodes using learnable edge weights, thereby updating and enhancing node features and obtaining high-dimensional node features.

[0014] This invention also discloses a neurofeedback training system for virtual reality motion sickness, used to implement the method described in any of the preceding claims, including a baseline establishment module, a motion sickness induction module, a hypergraph construction module, a model parsing module, a closed-loop control module, and a training evaluation module, wherein: The baseline establishment module is used to collect resting-state EEG signals of subjects in a quiet, VR-free state and complete the SSQ motion sickness scale score to establish individual resting-state EEG baseline and SSQ baseline score. The motion sickness induction module is used to apply graded stimuli to the subject's graded scene to induce motion sickness using VR. Based on the resting-state EEG baseline, the individual's motion sickness threshold level is determined, and the corresponding SSQ motion sickness score is collected. The hypergraph construction module is used to collect multi-channel EEG signals in real time during VR motion sickness induction. After preprocessing and multi-domain feature extraction, a multi-attribute brain function network hypergraph reflecting the strength of brain region association is constructed. The hypergraph selects effective features based on the SSQ baseline score and the SSQ motion sickness score. The model parsing module is used to construct an interpretable graph neural network model with built-in causal constraints to obtain the real-time motion sickness state and key neural subgraphs of the subjects in VR. Based on the real-time motion sickness state and the current real-time neural pattern represented by key neural subgraphs, and compared with the preset target neural pattern, a personalized control strategy is generated to dynamically adjust the current operating parameters of the VR scene and multi-sensory feedback, thereby achieving closed-loop control of real-time neural feedback. During the real-time neural feedback closed-loop control process, multi-channel EEG data of the entire process of motion sickness induction and neural feedback are continuously collected and continuously input into the causal interpretable graph neural network model for closed-loop iterative training. The model parameters and personalized control strategies are optimized simultaneously until the real-time neural pattern approaches the target neural pattern and reaches the preset termination condition, and then the multi-dimensional model evaluation results are output.

[0015] The beneficial effects of this invention are as follows: Built-in causal constraints output interpretable neural subgraphs from the source, making decisions traceable and biologically meaningful; dynamic regulation based on individual real-time brain state adapts to neural differences, significantly improving intervention effects and training efficiency; real-time closed loop, low latency, can be engineered and implemented, applicable to various VR scenarios and groups; effectively reduces the incidence and intensity of motion sickness, ensures VR usage safety, and promotes large-scale industrial promotion. Attached Figure Description

[0016] Figure 1 An overall flowchart of an interpretable neurofeedback training process for virtual reality motion sickness provided in an embodiment of this application; Figure 2 A flowchart of a neurofeedback training method for virtual reality motion sickness provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the built-in causal constraint interpretable graph neural network model in an embodiment of this application; Figure 4 The following is an overall architecture diagram of neurofeedback modulation for virtual reality motion sickness provided in the embodiments of this application; wherein: (a) is an overall flowchart of the neurofeedback experiment; (b) is a schematic diagram of the neurofeedback modulation mechanism; (c) is a schematic diagram of baseline dynamic adjustment; and (d) is a flowchart of neurofeedback baseline dynamic adjustment. Figure 5 This is a box plot comparing the scores of the SSQ motion sickness scale of the subjects before and after the experiment in a street scene according to an embodiment of this application. Figure 6 This is a schematic diagram illustrating the correlation between the rate of change of eigenvalues ​​and the subject's SSQ value during the motion sickness induction process according to an embodiment of this application. Figure 7 This is a schematic diagram of information sub-graphs and redundant networks in the task process and before and after rest states according to an embodiment of this application; Figure 8 for Figure 7 The diagram shows the characteristic analysis of the information subgraph and redundant network. Figure 9 A block diagram of a neurofeedback training system for virtual reality motion sickness provided in an embodiment of this application. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0018] First, combined Figure 1 This embodiment provides a brief overview of the overall process for interpretable neurofeedback training for VR motion sickness. For example... Figure 1 As shown in the figure, the overall process of interpretable neurofeedback training for VR motion sickness provided in this embodiment is as follows: First, the real-time EEG signal of the subject is acquired through the data acquisition module. After electrode selection and data interception, features are extracted from multiple dimensions such as time domain, frequency domain, spatial domain, phase domain and Riemann space. A multi-attribute brain functional network hypergraph is constructed in combination with the brain functional network. After the hypergraph is input into the interpretable model, the model outputs the motion sickness state detection result and identifies key neural landmark modalities. Based on the identified neural landmark modalities, a personalized control strategy is generated. The VR scene parameters and multi-sensory feedback are adjusted in reverse through the neural feedback pathway. At the same time, new EEG data is continuously collected, forming a closed loop of "EEG acquisition - multi-domain feature extraction - hypergraph construction - interpretable model analysis - neural feedback control - model iterative optimization", realizing personalized, interpretable and real-time neurofeedback training for VR motion sickness.

[0019] The following is in conjunction with the appendix Figure 2 This application details the specific implementation process of a neurofeedback training method for virtual reality motion sickness. For example... Figure 2 As shown, the method includes the following steps: S101. Under the condition of quietness and no VR stimulation, collect the resting-state EEG signals of the subjects and complete the SSQ motion sickness scale score to establish the individual resting-state EEG baseline and SSQ baseline score.

[0020] In this embodiment, the resting-state EEG refers to the natural EEG signal collected from the subject in a quiet, relaxed state, without VR stimulation and without motion sickness symptoms. This signal is used to construct an individual brain activity baseline and ASR (Artifact Subspace Reconstruction) real-time noise reduction parameters. ASR, a real-time EEG noise reduction algorithm, is used in this embodiment to construct artifact reconstruction parameters based on resting-state EEG, automatically identifying and removing noise and interference in subsequent task-oriented EEG, such as removing artifacts from eye movements, electromyography (EMG), and power frequency signals, while preserving genuine neural signals.

[0021] In this embodiment, the SSQ (Simulator Sickness Questionnaire) score is an internationally recognized subjective rating scale for the severity of VR / simulator motion sickness, and is an internationally recognized quantitative indicator of motion sickness severity, used to quantify the degree of motion sickness. In this embodiment, it is mainly used for training the model, evaluating the intervention effect, serving as an individual motion sickness baseline, and determining whether there is improvement before and after training.

[0022] Specifically, the resting-state EEG signal acquired in this embodiment refers to the raw scalp EEG waveform signal acquired by the subject in a quiet, unstimulated, and motion-free state using a multi-channel electrode of a head-mounted EEG acquisition device (ROI brain region). This signal serves as the 0.5Hz–60Hz EEG signal for subsequent preprocessing. The SSQ motion sickness scale score is obtained by scoring the subject on the SSQ scale in a quiet state without any virtual reality scene or stimuli, thus acquiring the baseline score of motion sickness symptoms in the subject's natural state. This embodiment establishes an individual motion sickness baseline based on the resting-state EEG and the SSQ baseline score, constructing an individual EEG benchmark. Simultaneously, the resting-state EEG is used to generate reconstruction parameters for real-time ASR noise reduction.

[0023] It should be noted that the ROI (Region of Interest) brain region refers to the target brain region that is highly related to the mechanism of virtual reality motion sickness, and is the key area for collecting EEG signals in this embodiment.

[0024] S102. Apply graded stimuli VR to the subject in a graded scenario to induce motion sickness. Determine the individual's motion sickness threshold level based on the resting-state EEG baseline and collect the SSQ motion sickness score corresponding to that level.

[0025] This embodiment employs three standardized VR tiered scenarios: street view, roller coaster, and rotating seat. Tiered stimuli are applied to subjects based on movement speed and field of view. VR stimulation is applied progressively from low to high intensity, using the resting-state EEG baseline obtained from S101 as the tiering dimension. During this progressive increase, the subject's EEG response and subjective motion sickness status are monitored in real time, and multi-stage real-time SSQ motion sickness scale scores are collected simultaneously. When a subject first exhibits motion sickness tendency or mild motion sickness symptoms, the current stimulation level is considered the subject's individual motion sickness threshold level, and the corresponding real-time SSQ motion sickness scale score is recorded as the SSQ motion sickness score. By determining the threshold level, the sensory conflict stage and the motion sickness symptom stage are separated, providing a stimulus intensity benchmark for subsequent personalized closed-loop neural feedback regulation, ensuring that the evolutionary process has clear and objective performance evaluation standards.

[0026] It should be noted that during the graded stimulation process, the sensory conflict stage and the motion sickness symptom stage were distinguished based on the subjects' real-time EEG response and subjective feelings, so as to allow for early intervention before motion sickness occurs. Specifically, the sensory conflict stage corresponds to the visual-vestibular sensory integration imbalance caused by VR stimulation; the motion sickness symptom stage corresponds to the state in which the subjects experience obvious dizziness, nausea, and other motion sickness symptoms. The above stage division was achieved using conventional methods in this field.

[0027] S103. During the VR motion sickness induction process, multi-channel EEG signals are collected, and after preprocessing and multi-domain feature extraction, a multi-attribute brain function network hypergraph reflecting the brain region correlation strength is constructed; the hypergraph is based on the SSQ baseline score and the SSQ motion sickness score to select effective features.

[0028] In this embodiment, Figure 3 This is a schematic diagram of the structure of a causal-constrained interpretable graph neural network model, which can intuitively present the complete architecture of a causal-constrained interpretable graph neural network. For example... Figure 3 As shown, the internal processing flow of the causal interpretable graph neural network model in this embodiment is as follows: First, the input graph features are filtered and divided into information hypergraphs and noise hypergraphs. The information hypergraphs and noise hypergraphs are processed by encoders respectively, and combined with mutual information constraints, the effective features and noise features are distinguished and enhanced. The processed features are input into a graph isomorphic network, and the node features are represented and enhanced in a high dimension through K-means clustering and graph point feature convolution modules. Then, after processing by fully connected layers, and then through a subgraph extraction module, the key neural subgraphs strongly correlated with motion sickness response and motion sickness state prediction results are finally output. This model achieves interpretable extraction of key neural patterns and accurate prediction of motion sickness state through the separation of information and noise hypergraphs and causal constraints.

[0029] The following is combined with Figure 3 The process of filtering the input graph features is explained in detail.

[0030] Step 1: Signal Preprocessing: During VR motion sickness induction, the collected raw time-series EEG signals are preprocessed to retain components with frequencies within a preset range (e.g., 0.5Hz to 60Hz), while filtering out extremely low-frequency drift below 0.5Hz and high-frequency noise interference above 60Hz that are unrelated to motion sickness. At the same time, power frequency interference noise caused by mains power (50Hz in China) is filtered out, and 50Hz electromagnetic interference generated by power supply equipment, monitors, etc. in the environment is eliminated to avoid contaminating the EEG signals, thus completing real-time noise reduction.

[0031] The second step is multi-domain feature extraction: Multi-dimensional features in the time domain, frequency domain, spatial domain, phase domain, and Riemann space are extracted simultaneously from the preprocessed clean EEG signal. These features cover Delta, Theta, Alpha, and Beta frequency band features and higher-order correlation features in Riemann space, so as to comprehensively characterize the differences in brain neural activity of subjects during VR motion sickness induction and provide a feature basis for subsequent hypergraph construction.

[0032] Delta, Theta, Alpha, and Beta are four classic frequency bands of EEG signals, each corresponding to different brain activity states, and are key frequency domain features reflecting the VR motion sickness process. Temporal domain features are used to characterize the characteristics of EEG changes over time, including mean, variance, and entropy. Frequency domain features (i.e., Delta / Theta / Alpha / Beta) are used to characterize the energy intensity of different frequencies, which are the power spectra of each frequency band. Spatial domain features are used to characterize which brain regions are activated and which electrodes are active, i.e., brain region activation. Phase domain features are used to characterize the phase synchronization activity patterns between different EEG channels and brain regions. Riemannian space features are high-order neural features extracted by mapping and operation in Riemannian manifold space based on the covariance matrix of EEG signals, and are used to refine the nonlinear correlation between multi-channel EEG signals.

[0033] The third step is to construct channel association and adjacency relationships: calculate the Pearson correlation coefficient, phase lock value (PLV), and mutual information (MI) between each channel, and fuse multiple indicators to determine the connection weights between channels. Based on the connection weights, construct the adjacency relationships of the EEG channel network.

[0034] Specifically, using multi-channel EEG acquisition electrodes as independent nodes in the network (corresponding to an independent EEG signal acquisition channel, which is also a node in the hypergraph model), based on the preprocessed raw time-series EEG signals of each EEG channel, Pearson correlation coefficient, phase lock value (PLV), and mutual information (MI) are calculated between each pair of channels to obtain a multi-dimensional spliced ​​brain network. The three types of correlation indicators are normalized to unify their dimensions, and combined with conventional multi-indicator fusion quantification methods in this field, the information flow intensity of each electrode in the brain is obtained, thereby determining the connection weights between channels and completing the construction of the adjacency relation matrix / adjacency topology of the multi-channel EEG channel network. This adjacency relation provides a topological foundation for subsequent setting of adaptive thresholds, partitioning of hyperedges, and construction of the hypergraph structure.

[0035] Among them, the Pearson correlation coefficient reflects the linear amplitude correlation between channels; the phase lock value (PLV) reflects the phase synchronization correlation between channels; and mutual information (MI) is a correlation feature that characterizes the intensity of nonlinear information interaction between EEG channels.

[0036] Step 4: Definition of Hyperedge: Integrate the pairwise channel correlation features, solve the joint correlation degree of multi-channel combinations, and define the multi-channel joint correlation that is greater than the set adaptive threshold θ as a hyperedge.

[0037] Based on the construction of multi-channel adjacency relationships, the characteristics of pairwise channel associations are integrated to solve the joint association degree of multi-channel combinations. An adaptive threshold θ is set, and multi-channel combinations with a joint association strength greater than this threshold are defined as hyperedges, thus completing the construction of the hypergraph topology. The joint association refers to the synergistic coupling relationship of group neural activity formed by multiple EEG channels as a whole, based on the pairwise adjacency relationships of multi-channels.

[0038] Step 5: Feature Selection: From the multi-dimensional features, select the iconic features that are significantly related to the SSQ baseline score and SSQ motion sickness score, and configure them as feature attributes of the hypergraph node.

[0039] Based on the completion of multi-channel joint correlation analysis and hyperedge partitioning, and the determination of the hypergraph topology, we selected significant features that are significantly related to both the SSQ baseline score and the SSQ halo score from the extracted multi-dimensional features, eliminated redundant interference features, and used the selected effective features as the feature attributes of the hypergraph nodes to improve the construction of the hypergraph model.

[0040] Step 6: Constructing the Vertex Feature Matrix: Normalize the iconic features and map all feature values ​​to the [0,1] interval to eliminate differences in scale and magnitude between different features; arrange the data according to the correspondence between hypergraph nodes and integrate them to form a standardized hypergraph vertex feature matrix.

[0041] Through the above-mentioned signal preprocessing, multi-domain feature extraction, channel adjacency relationship construction, hyperedge definition, feature selection and vertex feature matrix construction steps, the construction of a multi-channel EEG hypermap is completed, and a hypermap feature pattern with preliminary multi-attribute representation ability is obtained. On this basis, the modeling stage of causal interpretable graph neural network in step S104 is entered, and information edges and noise edges are distinguished from hypermap attribute edges according to the distribution law and physical meaning.

[0042] S104. Construct an interpretable graph neural network model with built-in causal constraints based on a multi-attribute brain functional network hypergraph to obtain the user's real-time VR motion sickness state and key neural subgraphs.

[0043] In this embodiment, the construction of the interpretable graph neural network model with built-in causal constraints first involves building a three-layer structure including an edge convolution module, a point feature convolution module, and a built-in causal relationship module. The preprocessed multi-attribute brain functional network hypergraph is input into the edge convolution module, which learns the correlation strength features between brain regions through learnable edge weights. The feature vectors output from the edge convolution are input into the point feature convolution module to further extract high-dimensional representations of node features. Subsequently, the built-in causal relationship module enhances the causality between information subgraphs and motion sickness state while weakening the causality between redundant networks and motion sickness state, filtering out key neural subgraphs strongly correlated with motion sickness response from the brain functional network. Finally, the model is trained based on the features of the key neural subgraphs, using the real-time VR motion sickness state as a label, and the prediction error is constrained by a loss function to complete the model construction, enabling the model to output the subject's real-time VR motion sickness state and key neural subgraphs.

[0044] Furthermore, the preprocessed multi-attribute brain functional network hypergraph is input into the edge convolution module. Since the generated hyperedges are all equipped with multi-dimensional channel correlation quantization information, each hyperedge can be converted into an information edge and a noise edge according to its distribution and physical meaning.

[0045] Specifically, with a coherence network at its core, a coherence value standard is set: a coherence value of 1 represents high coherence, 0 represents complete incoherence, and values ​​within the interval (0,1) correspond to different degrees of noise intrusion (i.e., including some invalid connections). In this embodiment, the Hyper Graph Convolution Network Encoder (HGCN Encoder) is used to perform deep learning on the attribute edge features inside the hypergraph, calculate the hyperedge weight matrix, distinguish between informative hyperedges and noisy hyperedges by the edge weight values, and output the corresponding feature representations A and B respectively, filtering out valid connections related to motion sickness and eliminating invalid noise connections.

[0046] This step distinguishes between informational hyperedges and noisy hyperedges by learning the weights of the edges, dividing the hypergraph into informational subgraphs and redundant networks. However, this optimization process only adjusts the topological connections between nodes and does not perform in-depth extraction and enhancement of the original features of the EEG channel nodes.

[0047] To address the shortcomings of previous methods that only optimized topology without enhancing node features, this embodiment introduces a node feature enhancement process. First, K-means clustering is used to group EEG channel nodes, clarifying local neighborhood relationships and providing a structured foundation for feature extraction. Then, Conv (convolutional layer) operations are initiated to extract local node features through convolutional operations. These features are then mapped to a high-dimensional space using a transformation matrix and weighted and fused with the original multi-dimensional features of the hypergraph nodes, achieving pre-enhancement and preprocessing of node features. Based on the fused features, a subgraph extraction module filters and constructs preliminary subgraphs related to motion sickness response. Finally, a multilayer perceptron integrates and outputs enhanced features for subsequent calculations of cross-entropy loss and conditional mutual information loss. Backpropagation supervises the parameter optimization of the entire model, providing real-time output of the current user's motion sickness state classification. This module compensates for the shortcomings of previous methods that only optimized topological connectivity, ultimately outputting deeply enhanced node features, providing high-quality input for subsequent motion sickness state recognition in the model.

[0048] Specifically, in this embodiment, the graph isomorphic network GIN is used to perform convolutional aggregation on neighboring nodes using learnable edge weights, thereby updating and enhancing node features to obtain a high-dimensional representation of the nodes. The process is as follows: Information characteristics The feature information forms learnable edge weights, and the calculation formula is as follows: By introducing learnable edge weights to perform isomorphic convolution aggregation on neighbor nodes, the following operations are performed on the neighbors: (7), Formula (7) is the node feature update formula in the Graph Isomorphism Network (GIN). Wherein, Let A be the edge weight matrix calculated from information feature A, and let its elements be... For nodes and nodes The edge weights between them are used for subsequent isomorphic convolution aggregation of adjacent nodes to distinguish between informational hyperedges and noisy hyperedges; Represents a node Updated features For the first In the layer, nodes The original feature vector; For learnable scaling parameters; For nodes The set of adjacent nodes; No. In the layer, adjacent nodes The corresponding node feature vector; MLP( () is a multilayer perceptron.

[0049] By using the GIN network and corresponding feature update operations to perform in-depth feature mining and representation enhancement of nodes, optimized specific brain network features are obtained. These features are simplified brain network structures that retain EEG nodes and functional connections strongly correlated with motion sickness response after removing noisy connections. This feature is then input into an overall interpretable network model that integrates the aforementioned causal constraints and mutual information constraints to achieve real-time brain pattern recognition. This not only enables real-time identification of virtual reality motion sickness symptoms in users and achieves accurate detection, but also isolates key neural landmark modalities highly correlated with motion sickness and identifies key brain region synergistic features that induce motion sickness response. This provides clear targeted optimization goals and physiological basis for subsequent neural feedback intervention and vestibular-visual conflict regulation.

[0050] Furthermore, the built-in causal relationship module of this model receives the information hypergraph and noise hypergraph obtained from the edge convolution module, and combines the enhanced node features output by the point feature convolution module with the neural subgraph obtained after preliminary screening, as the input for subsequent loss calculation.

[0051] Furthermore, this embodiment constructs a causal loss through conditional mutual information to remove noise features. Maximizing information features amidst uncertainty The association with label Y strengthens the causal relationship between the two, and the cross-entropy loss of the cross-entropy classification constraint measures the error between the model output Y and the true motion sickness label, ensuring the accuracy of motion sickness state classification. The two types of losses (mutual information loss and causal loss) are weighted and combined, and the model parameters are updated through backpropagation, which improves the interpretability and classification performance of the model while suppressing noise interference.

[0052] Furthermore, in this embodiment, the model is trained based on the adjusted loss function, so that the model prioritizes learning edges and nodes in the information subgraph that are strongly correlated with the motion sickness state.

[0053] Specifically, to ensure that the label only relates to... Based on the premise of a causal relationship and minimizing the influence of noise edges on the results, the loss function of this model is defined as follows: (1), (2), (3), (4), in, The total loss function for model training. Let cross-entropy be the loss function. It is the label state of the model. It is the output of the model. This is achieved by introducing conditional mutual information and eliminating noise features. Uncertainty increases information features With tags causal relationship This is the introduction and The mutual information MI. λ and γ are the causal loss terms. The weighting coefficients for mutual information loss are used to balance the influence ratio of the three types of losses. This refers to the number of classification categories of motion sickness (e.g., no motion sickness, mild motion sickness, moderate motion sickness, severe motion sickness, etc.), which is the total number of categories the model ultimately needs to output.

[0054] Will and The combined loss function can be understood as, in graph features and While maintaining relevance, increase information features The causal relationship with the label is used to weaken the noise features. Relevance to the results. Depending on the input, these represent marginal entropy and joint entropy, respectively. However, in practical calculations, especially for high-dimensional spaces, probability density is difficult to calculate and lacks universality. Therefore, this embodiment proposes to use matrix-based Renyi entropy to calculate marginal entropy and joint entropy, respectively. This method is simple to calculate, mathematically differentiable, supports gradient calculation and backpropagation, can be directly integrated into the end-to-end training process of deep learning models, and is very suitable for network learning. The formulas for calculating the marginal entropy and joint entropy of order Renyi are as follows: (5), (6), in, It is the regularized Gram matrix of the original features of the EEG channel nodes in the feature map. The trace of a matrix is ​​represented by the sum of its diagonals. Representation matrix , , The Hadamard product between the matrices. This is an enhancement feature matrix for higher-order nodes after convolution operations, feature fusion, and graph isomorphic convolution aggregation; the matrix... The final output of the model is the prediction feature matrix, which represents the prediction results of each level of motion sickness and is used to calculate cross-entropy and conditional mutual information loss.

[0055] As can be seen from the above, this embodiment embeds causal constraints and classification constraints into the model training process, thereby achieving the unification of accurate detection of motion sickness and causal interpretation of neural modalities.

[0056] S105. Based on the real-time motion sickness state and the current real-time neural pattern represented by the key neural subgraphs obtained by identification, and in comparison with the preset target neural pattern, a personalized control strategy is generated to dynamically adjust the current operating parameters of the VR scene and multi-sensory feedback, so as to realize the closed-loop control of real-time neural feedback for individual subjects.

[0057] This step uses the key neural subgraphs output by S104 (and key brain network subgraphs) Figure 1 Based on the core criteria of motion sickness (Y) and real-time motion sickness status (Output Y, i.e., the motion sickness level predicted by the model), and relying on... Figure 3 The real-time computing power of the entire model enables personalized closed-loop control.

[0058] In one feasible implementation method, the specific implementation process of this step is as follows: The first step is real-time data acquisition and status acquisition during VR interaction. After completing the selection, construction, and model prediction of key neural subgraphs in step S104, the user's EEG signals are continuously collected in real time. The process of feature extraction, subgraph selection, motion sickness state prediction, and loss calculation is repeated, and the current motion sickness state of the user is output in real time (i.e., the model outputs Output Y, corresponding to the motion sickness level classification result). At the same time, the key neural subgraphs related to motion sickness are updated.

[0059] The second step is the generation of personalized control strategies. Based on the real-time output of motion sickness status and combined with the updated key neural subgraph features, targeted personalized control strategies are generated. The core basis of this strategy is the specific neural response represented by the key neural subgraph, that is, the differentiated changes in the activation intensity of EEG nodes and the functional connectivity between brain regions in the key neural subgraph under VR scene stimulation for different users. This determines the VR scene parameters that need to be adjusted, such as movement speed, rotation speed, field of view, or output of visual, auditory, and tactile multimodal feedback.

[0060] The third step involves adjusting VR scene parameters and multi-sensory feedback in stages. Based on the generated personalized control strategy and the real-time motion sickness level, the field of view, scene movement speed, frame rate, and multi-sensory feedback parameters are dynamically adjusted in stages.

[0061] The fourth step involves feedback and iterative optimization of the control strategy. After adjusting the parameters as described above, the model continuously collects the user's adjusted real-time EEG signals, repeats the process from the first step, and re-acquires the updated motion sickness state and key neural subgraph features. It compares the changes in motion sickness level and the differences in neural responses in the key neural subgraphs before and after parameter adjustment to evaluate the control effect. If the motion sickness level decreases and the abnormal neural responses related to motion sickness in the key neural subgraphs weaken, the control strategy is effective, and the current control logic is maintained with fine-tuning of the parameter amplitude. If the motion sickness level does not improve or increases, and the abnormal neural responses in the key neural subgraphs are not alleviated, the control strategy is reversed, adjusting the type and amplitude of parameter adjustments, and the parameter adjustment steps are executed again. Through the above "collection-evaluation-adjustment-recollection" cycle, feedback and iterative optimization of the control strategy are achieved, forming a complete personalized closed-loop control that continuously suppresses motion sickness-related neural responses and alleviates the user's motion sickness discomfort.

[0062] S106. During the continuous process of real-time neural feedback closed-loop regulation, multi-channel EEG data of the entire process of motion sickness induction and neural feedback are collected synchronously. The collected data is continuously input into the causal interpretable graph neural network model for closed-loop iterative training, and the model parameters and personalized regulation strategies are optimized synchronously until the subject's real-time neural pattern stabilizes and approaches the target neural pattern and meets the preset termination conditions. The model evaluation results are then output from multiple dimensions.

[0063] Continuous data collection and real-time closed-loop training is an extension and deepening of S105's personalized closed-loop control. The specific implementation process is closely linked to the feedback and iteration logic of S105, relying on... Figure 3 The complete model and the core features mentioned above are presented in the following steps: Step 1: Continuous Data Acquisition: Based on the S105 closed-loop control, continuously and in real time collect the EEG signals of the user during the entire process of motion sickness induction and neural feedback during VR interaction. Simultaneously record the user's EEG response data after each VR scene parameter adjustment and multi-sensory feedback optimization in the S105 step, as well as the corresponding motion sickness state (Output Y), key neural subgraph features, and current real-time neural pattern change data to ensure that the collected data covers the complete correspondence between "parameter adjustment" and "neural response".

[0064] Step 2: Real-time closed-loop training: The continuously collected EEG data, motion sickness data, and key neural subgraph update data are input into the graph edge convolution module, graph point feature convolution module, and built-in causal relationship module in real time. The feature extraction, subgraph selection, motion sickness prediction, and loss calculation processes are repeatedly executed. At the same time, combined with the differentiation results of "current real-time neural mode and target neural mode" in S105, the training objective is to make "current real-time neural mode approximate target neural mode". The model parameters (including learnable edge weights) and the regulation strategy logic of S105 are iteratively trained in real time.

[0065] Step 3: Training Termination and Result Output: When continuous training reaches the preset conditions (i.e., the user's current real-time neural pattern stably approaches the target neural pattern, the motion sickness state is maintained at the no-motion / mild motion sickness level, and motion sickness-related abnormal neural responses in key neural subgraphs are completely suppressed), real-time closed-loop training is stopped; after training is completed, quantitative evaluation results are output. These evaluation results focus on model performance and are quantitatively assessed from four dimensions, specifically including: model performance dimension (…). Figure 3 The model comprehensively reflects the closed-loop training and overall system model performance in terms of motion sickness prediction accuracy, key neural subgraph selection precision, neural response dimension (the degree of inhibition of motion sickness-related abnormal neural responses in key neural subgraphs, and the approximation between real-time neural patterns and target neural patterns), behavioral indicator dimension (quantitative data on the behavioral representation of users' motion sickness responses in VR interaction), and system real-time dimension (response latency of EEG data acquisition, model calculation, parameter adjustment, and training iteration).

[0066] This embodiment constructs an interpretable neurofeedback training method for VR motion sickness. By establishing an individual's resting state and motion sickness threshold baseline, it achieves personalized adaptation throughout the entire process. Based on multi-domain features and hypergraph modeling, it comprehensively captures motion sickness-related brain functional connectivity patterns. Relying on a causal interpretable graph neural network, it achieves accurate prediction of motion sickness states and identification of key neural subgraphs, providing interpretable targets for intervention. Based on the differences between real-time and target neural patterns, it dynamically adjusts VR parameters and multi-sensory feedback to form a closed-loop intervention. Through continuous closed-loop training and multi-dimensional model evaluation, it ensures the stability and optimizability of the method. The overall solution achieves personalized, interpretable, and real-time intervention for VR motion sickness, effectively alleviating motion sickness discomfort and providing a reliable neurofeedback pathway for VR motion sickness intervention.

[0067] Figure 4 (a) is the overall flowchart of the neurofeedback experiment for virtual reality motion sickness provided in this embodiment, showing the complete experimental sequence from baseline testing (Session 1), resting-state baseline acquisition (Session 2) to 7 days of neurofeedback training (Session 3). Specifically, subjects completed the baseline test without EEG (SSQ) and the resting-state baseline test with EEG in sequence through questionnaires. They were then randomly divided into experimental and control groups. The experimental group underwent 7 days of neurofeedback training with EEG and simultaneously conducted the SSQ test. ASR real-time noise reduction technology was used throughout the process to ensure data quality.

[0068] Figure 4(b) illustrates the neural feedback modulation mechanism provided in this embodiment. As shown in Figure 4(b), the system first acquires the subject's real-time EEG signals through data acquisition to identify the current real-time neural pattern. The system then compares this real-time pattern with preset first and second target neural patterns. When the real-time neural pattern deviates from the first target neural pattern, an early warning mechanism is triggered, initiating the neural modulation process. Through targeted VR scene parameters and multimodal feedback modulation, the system gradually guides the subject's neural response towards target neural pattern 2. This process achieves dynamic correction of motion sickness-related neural responses through a closed-loop logic of "real-time pattern recognition - target pattern comparison - feedback modulation intervention."

[0069] Figure 4 (c) is a schematic diagram of baseline dynamic adjustment provided in this embodiment. Figure 4 As shown in Figure c, the horizontal axis represents time, and the vertical axis represents the intensity of the subject's neural response (or the level of motion sickness-related indicators). The curves in the figure show the fluctuations in the subject's neural response over time during VR interaction: the system dynamically updates the baseline level of the neural response based on real-time collected EEG data and uses error bars to characterize the stability of the individual response. As feedback modulation continues, the subject's neural response gradually converges towards the preset target baseline, the fluctuation amplitude decreases, and a gradual improvement in motion sickness is achieved.

[0070] Figure 4 (d) is a flowchart of the dynamic adjustment process of the neurofeedback baseline for virtual reality motion sickness provided in this embodiment. As shown in Figure 4d, the dynamic adjustment process of the neurofeedback baseline in this embodiment is the core link of the closed-loop intervention: the subject generates real-time EEG signals during VR interaction. After data acquisition, preprocessing and feature analysis, the signals are input into an embedded interpretable model. The model iteratively updates the link in the background, continuously collecting data to optimize parameters and adapt to the individual, distinguishing between the current real-time neural pattern and the preset target neural pattern (the two are marked with dashed boxes, representing the abstract neural state output by the model). Subsequently, the system uses a dynamic baseline adjustment strategy to reversely regulate VR scene parameters (including movement speed, rotation speed, and field of view) and multimodal sensory feedback (including visual feedback, auditory feedback, and tactile feedback). At the same time, the EEG data enters the real-time preprocessing stage through Lab StreamLayer to support the continuous updating of the neural pattern, ultimately forming a closed loop of "EEG acquisition - state recognition - dynamic regulation - feedback intervention", realizing personalized, interpretable, and real-time neurofeedback training for virtual reality motion sickness. In the diagram, the solid line represents the main link between real-time data and control flow, while the dashed line represents the background auxiliary data flow and baseline update link.

[0071] Specifically, during the neurofeedback experiment, when a change in the subject's impending dizziness was detected, the VR scene parameters were dynamically adjusted. This accelerated the induction of motion sickness in the virtual reality experience when the subject was in a good state, and slowed it down when the dizziness worsened, achieving a personalized training process. To prevent abrupt changes in real-time data processing that could lead to uneven screen or sensory feedback, this study employed dynamic baseline adjustment. The baseline and range of change were based on the average range of change over a short period, rather than the result of a single trial. The neurofeedback experiment design used the Lab Stream Layer (LSL) platform to acquire EEG data in real time. Real-time preprocessing and embedded interpretation model subgraphs were used to obtain real-time neural patterns. The difference between these patterns and the target pattern was calculated to generate feedback information, thereby allowing for changes in the parameters of the stimulus material or the intensity of sensory stimulation. A control experiment was designed to generate noise in real time to provide ineffective neurofeedback to individuals. The neurofeedback experiment procedure first requires detecting the participants' SSQ (Self-Sensitive Questionnaire) values. Secondly, a baseline experiment is conducted, collecting resting-state EEG data. This data will be used to construct the reconstruction parameters for the Artifact Subspace Reconstruction (ASR) algorithm. Subsequent EEG data during the task process will undergo real-time noise reduction based on these parameter values. Participants are randomly selected to either participate in the experimental or control group, and undergo 7 days of neurofeedback training. SSQ values ​​are collected daily during and at the end of the experiment, serving as one of the evaluation criteria for the experimental effect. The experimental group will conduct three types of experiments under sensory stimulation and three types of experiments under regulated stimulation materials. Depending on the effect size, each group will include at least 20 participants.

[0072] Finally, based on the state detection results and neural marker modalities, the system provides personalized neuromodulation signals to the user, dynamically adjusts VR scene parameters and multisensory stimulation intensity, forming a closed-loop system of "EEG acquisition - feature analysis - state recognition - targeted feedback - user control", realizing real-time detection, interpretable intervention and personalized adaptation of virtual reality motion sickness.

[0073] The experimental results show that the virtual reality motion sickness induction experiment based on street view, after comprehensive analysis, yields the following results: First, the SSQ scores of the participants were compared, such as... Figure 5As shown in the figure, this is a box plot comparing the SSQ motion sickness scale scores of the participants before and after the experiment in a street scene, used to verify the effect of VR scenes on inducing motion sickness response. The horizontal axis represents the state before and after the experiment, and the vertical axis represents the SSQ motion sickness scale score. The figure shows that before the experiment, the participants' SSQ scores were at a low level; after the experiment, the participants' SSQ scores increased significantly, and the difference was statistically significant (P<0.01), indicating that the VR street scene can effectively induce motion sickness response in the participants, providing a behavioral basis for subsequent motion sickness level judgment and neurofeedback training.

[0074] Next, the brain activity during the induced dizziness process was analyzed. Based on commonly used EEG features in other fields, the preliminary research extracted 24 features from multiple domains, all of which have the potential to reveal cognition and states in other fields (not listed here). Correlation analysis was performed between the EEG signals during the virtual reality-induced dizziness process and the SSQ values ​​reported by the subjects. Seven brain network features were further extracted and analyzed. These brain network features are also commonly used in other fields and have information mining potential. Features showing significant differences were summarized according to different rhythms, such as... Figure 6As shown in the figure, this is the correlation analysis result between brain network features and SSQ motion sickness scores, showing the correlation between Delta, Theta, Alpha, Beta band feature values, all band feature values, and other feature values ​​and motion sickness scores. Among them, (a) is the Delta feature value, (b) is the Theta feature value, (c) is the Alpha feature value, (d) is the Beta feature value, (e) is the feature value of all bands, and (f) is other feature values. The features involved include entropy indices such as Approximate Entropy, Fuzzy Entropy, and Spectral Entropy; nonlinear dynamic parameters such as Fractal Dimension, Hurst exponent, Lyapunov Exponent, Affine-invariant, and H-index; and PSD (power spectral density), Delta / Theta (δ / θ band power ratio), Delta / Alpha (δ / α band power ratio), and other features. Frequency domain features such as band power ratio, Delta / Beta (δ / β band power ratio), and Spectral Skewness; EEG coupling indices such as Coherence, PLI (phase lag index), and Log-Euclidean distance; and time-series modeling indices such as Auto Regression1 (first-order autoregressive model) and Auto Regression2 (second-order autoregressive model). Each curve in the figure corresponds to one feature. r is the Pearson correlation coefficient between EEG features and SSQ motion sickness scale scores, used to characterize the strength and direction of their linear correlation; p is the significance level of the correlation test, p<0.05 or p<0.01 indicates that the correlation is statistically significant and not due to random error. The results show that multiple multi-domain EEG features are significantly correlated with SSQ motion sickness scores, verifying the effectiveness of these features in reflecting VR motion sickness states and providing a basis for subsequent brain functional network hypergraph construction and model feature selection.

[0075] Preliminary results indicate that each rhythm possesses characterizing features, and these features exhibit varying meanings and inconsistent changes as the evoked process progresses. Furthermore, the electrode characteristics alone demonstrate potential for describing neural mechanisms, making in-depth analysis feasible.

[0076] Furthermore, the core idea behind the design of interpretable graph neural network models based on built-in causal constraints is to increase the causality between information subgraphs and results, while reducing the causality between noisy networks and results, thereby aiming to further extract pattern subgraphs from information subgraphs with more classification information. Therefore, the premise for the model to hold is to prove that information subgraphs possess modalities similar to noisy networks, and the added causal relationships are intended to exclude component information similar to noisy networks from information subgraphs.

[0077] Figure 7 This is a schematic diagram of the information subgraph and redundant network in the task process and before and after rest states according to an embodiment of this application. To verify the rationality of the design of the built-in causal constraint interpretable graph neural network model of this invention, this embodiment conducted preliminary research and verification: Brain functional networks based on street scene scenarios were extracted, and brain networks based on mutual information, phase lock value, and H-index were extracted respectively. Those connections with physical properties are called information subgraphs (IN), and those with weaker connections are called redundant networks (PN). Figure 7 As shown, 50% of the connectivity values ​​were extracted during the task process and before and after the rest period for display. It can be seen that there are obvious differences. The following is a quantitative analysis of its characteristics.

[0078] To fully verify whether there is overlap in the neural modal features between the information subgraph and the redundant network, 5%, 25%, and 50% of the overall brain network connectivity strength were extracted before and after the extraction, and these were used as the information subgraph and the redundant network, respectively. Next, small-world properties, node clustering coefficients, betweenness coefficients, and rich club values ​​were extracted as features, such as... Figure 8 As shown, the features of the two types of brain networks overlap. Therefore, by enhancing the causality between the information subgraph and the motion sickness state and weakening the causality of redundant networks, the model selects key neural subgraphs that are unrelated to redundancy from the information subgraph, providing an interpretable basis for motion sickness state prediction and proving the necessity and feasibility of the model design.

[0079] This application also provides a neurofeedback training system for virtual reality motion sickness that performs the above-described training method, wherein... Figure 9 This is a schematic diagram of a neurofeedback training system for virtual reality motion sickness, such as... Figure 9 As shown, the system includes: a baseline establishment module, a motion sickness induction module, a hypergraph construction module, a model parsing module, a closed-loop control module, and a training and evaluation module, wherein: The baseline establishment module is used to collect resting-state EEG signals of subjects in a quiet, VR-free state and complete the SSQ motion sickness scale score to establish individual resting-state EEG baseline and SSQ baseline score. The motion sickness induction module is used to apply graded stimuli to the subject's graded scene to induce motion sickness using VR. Based on the resting-state EEG baseline, the individual's motion sickness threshold level is determined, and the corresponding SSQ motion sickness score is collected. The hypergraph construction module is used to collect multi-channel EEG signals in real time during VR motion sickness induction. After preprocessing and multi-domain feature extraction, a multi-attribute brain function network hypergraph reflecting the strength of brain region association is constructed. The hypergraph selects effective features based on the SSQ baseline score and the SSQ motion sickness score. The model parsing module is used to construct a causal interpretable graph neural network model based on a multi-attribute brain functional network hypergraph, and to obtain the real-time motion sickness state and key neural subgraphs of the subjects in VR. The closed-loop control module is used to generate personalized control strategies based on the real-time motion sickness state and the current real-time neural pattern represented by key neural subgraphs, and to compare with the preset target neural pattern. It dynamically adjusts the current operating parameters of the VR scene and multi-sensory feedback to achieve real-time neural feedback closed-loop control. The training and evaluation module is used to continuously collect multi-channel EEG data of the entire process of motion sickness induction and neural feedback during the real-time neural feedback closed-loop regulation process, and continuously input the causal interpretable graph neural network model for closed-loop iterative training, simultaneously optimizing model parameters and personalized regulation strategies, until the real-time neural pattern approaches the target neural pattern and reaches the preset termination condition, and outputs multi-dimensional model evaluation results.

[0080] The parts not described in detail in this application are all existing conventional technologies and will not be elaborated here.

[0081] It is understood that the above specific description of the present invention is only for illustrating the present invention and is not limited to the technical solutions described in the embodiments of the present invention. Those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention to achieve the same technical effect; as long as the use needs are met, they are all within the protection scope of the present invention.

Claims

1. A neurofeedback training method for virtual reality motion sickness, characterized in that, include: In a quiet, VR-free state, resting-state EEG signals were collected from the subjects, and the SSQ motion sickness scale scores were completed to establish individual resting-state EEG baselines and SSQ baseline scores. VR-induced motion sickness was induced by applying graded stimuli to graded scenarios to subjects. The individual motion sickness threshold level was determined based on the resting-state EEG baseline, and the SSQ motion sickness score corresponding to the level was collected. Multi-channel EEG signals were collected in real time during VR motion sickness induction. After preprocessing and multi-domain feature extraction, a multi-attribute brain function network hypergraph reflecting the strength of brain region association was constructed. The hypergraph was based on the SSQ baseline score and SSQ motion sickness score to select effective features. A built-in causal constraint interpretable graph neural network model was constructed based on a multi-attribute brain functional network hypergraph to obtain the real-time motion sickness state and key neural subgraphs of the subjects in VR. Based on the real-time motion sickness state and the current real-time neural pattern represented by the key neural subgraphs, and compared with the preset target neural pattern, a personalized control strategy is generated to dynamically adjust the current operating parameters of the VR scene and multi-sensory feedback, so as to realize the real-time neural feedback closed-loop control for individual subjects. During the continuous real-time neurofeedback closed-loop regulation, multi-channel EEG data of the entire process of motion sickness induction and neurofeedback are collected synchronously. The collected data is continuously input into the causal interpretable graph neural network model for closed-loop iterative training, and the model parameters and personalized regulation strategies are optimized synchronously until the subject's real-time neural pattern stabilizes and approaches the target neural pattern and meets the preset termination conditions. The model evaluation results are then output from multiple dimensions.

2. The training method according to claim 1, characterized in that, The method involves determining the individual's motion sickness threshold level based on the resting-state EEG baseline, and collecting the corresponding SSQ motion sickness score, including: During the gradual increase in intensity, the subjects' EEG response and subjective motion sickness status were monitored in real time, and multi-stage real-time SSQ motion sickness scale scores were collected simultaneously. When a subject first showed motion sickness tendency or mild motion sickness symptoms, the current stimulus level was the subject's individual motion sickness threshold level, and the real-time SSQ motion sickness scale score corresponding to this level was recorded as the SSQ motion sickness score.

3. The method according to claim 1, characterized in that, After preprocessing and multi-domain feature extraction, a multi-attribute brain functional network hypermap reflecting the strength of brain region associations is constructed, including: During the VR motion sickness induction process, the original time-series EEG signals collected are retained at frequencies between 0.5Hz and 60Hz, while electromagnetic interference is filtered out to achieve real-time noise reduction. Multi-dimensional features in time domain, frequency domain, spatial domain, phase domain and Riemann space are extracted simultaneously. Pearson correlation coefficient, phase lock value (PLV) and mutual information (MI) of multi-dimensional features between channels are calculated and multi-indicator fusion is performed to determine the connection weight between channels. Based on the connection weight, the adjacency relationship of the EEG channel network is constructed. By integrating the pairwise channel correlation features, the joint correlation degree of multi-channel combinations is solved, and multi-channel joint correlations greater than the set adaptive threshold θ are defined as hyperedges; From the multi-dimensional features, we selected the iconic features that were significantly related to the SSQ baseline score and the SSQ motion sickness score, and configured them as the feature attributes of the hypergraph nodes. Normalized signature features are used to organize and arrange data according to the correspondence between hypergraph nodes, and then integrated to form a standardized hypergraph vertex feature matrix.

4. The training method according to claim 3, characterized in that, The determination of connection weights between channels includes: Using multi-channel EEG acquisition electrodes as independent nodes in the network, Pearson correlation coefficient, phase lock value (PLV), and mutual information (MI) are calculated between each pair of channels based on the preprocessed raw temporal EEG signals of each channel to obtain multi-dimensional correlation quantification values. The multi-dimensional correlation quantification values ​​are normalized respectively, and combined with the multi-indicator fusion quantification method to obtain the comprehensive correlation strength between channels, which is determined as the connection weight between each channel.

5. The training method according to claim 1, characterized in that, A graph neural network model with built-in causal constraints and interpretable graph neural network was constructed based on a multi-attribute brain functional network hypergraph to obtain the real-time motion sickness state and key neural subgraphs of the subjects in VR, including: Construct a causal interpretable graph neural network model structure that includes an edge convolution module, a point feature convolution module, and a built-in causal relationship module; The multi-attribute brain functional network hypergraph is input into the edge convolution module, and the brain region association strength features are learned through learnable edge weights. The hypergraph is divided into information subgraphs and redundant networks within the edge convolution module. The correlation strength feature vector output by the edge convolution module is input into the point feature convolution module to extract high-dimensional features of the node features; By combining the updated edge weights learned by edge convolution with the node high-dimensional features output by point feature convolution, the connections and nodes in the hypergraph are jointly screened, retaining the connections and nodes that are significantly associated with the halo state, thus completing feature extraction and preliminary subgraph screening. Based on the built-in causal relationship module, the information subgraphs and redundant networks already divided by the edge convolution module are called to enhance the causality between the information subgraphs and the motion sickness state, weaken the causality of the redundant networks, and screen out the key neural subgraphs that are strongly correlated with the motion sickness response. The model is trained based on the features of the key neural subgraphs. Using the real-time VR motion sickness state as a label, the prediction error is constrained by the loss function, enabling the model to output the subject's real-time VR motion sickness state and key neural subgraphs.

6. The training method according to claim 5, characterized in that, The causality between the enhanced information subgraph and the motion sickness state, and the causality between the weakened redundant network and the motion sickness state, are used to screen out key neural subgraphs that are strongly correlated with the motion sickness response, including: Calculate the causal correlation strength between the information sub-image and the motion sickness state label, and increase its weight in the model loss function; Calculate the causal correlation strength between redundant networks and motion sickness state labels, and reduce their weight in the model loss function; The model is trained based on the adjusted loss function, so that the model prioritizes learning the edges and nodes in the information subgraph that are strongly related to the motion sickness state. Based on the degree of causal correlation after model training, edges and nodes with significant causal correlation with motion sickness response are retained in the information subgraph to form a key neural subgraph.

7. The training method according to claim 5, characterized in that, The process of outputting the subject's VR motion sickness state in real time based on the features of the key neural subgraph and updating the key neural subgraph features includes: The node features and edge features of the key neural subgraph are input into the classification layer of the model, and the prediction results of the VR motion sickness state of the subject are obtained by mapping through the fully connected layer. Based on the preset motion sickness level classification standard, the prediction results are mapped to the subject's current VR motion sickness level; Real-time acquisition of EEG data during subsequent VR interaction with subjects, and updating of brain functional network hypergraph; Repeat the processing flow of edge convolution, point feature convolution and causal relationship module, re-select key neural subgraphs and update their features; By combining the motion sickness state output by the model with the updated key neural subgraph features, a real-time basis is provided for the dynamic adjustment of subsequent VR scene parameters and multi-sensory feedback.

8. The training method according to claim 5, characterized in that, The step of inputting the correlation strength feature vector into the point feature convolution module to extract high-dimensional features of the node features includes: The correlation strength feature vector output by the edge convolution is input into the point feature convolution module. Local features are extracted and mapped to a high-dimensional space through K-means clustering and convolution operations. Then, it is weighted and fused with the original multi-dimensional features of the hypergraph nodes to extract the high-dimensional representation of the node features.

9. The training method according to claim 8, characterized in that, By using the graph isomorphic network GIN to perform convolutional aggregation on neighboring nodes with learnable edge weights, the node features are updated and enhanced, resulting in high-dimensional node features.

10. A neurofeedback training system for virtual reality motion sickness, characterized in that, It includes a baseline establishment module, a motion sickness induction module, a hypergraph construction module, a model parsing module, a closed-loop control module, and a training and evaluation module, among which: The baseline establishment module is used to collect resting-state EEG signals of subjects in a quiet, VR-free state and complete the SSQ motion sickness scale score to establish individual resting-state EEG baseline and SSQ baseline score. The motion sickness induction module is used to apply graded stimuli to the subject's graded scene to induce motion sickness using VR. Based on the resting-state EEG baseline, the individual's motion sickness threshold level is determined, and the corresponding SSQ motion sickness score is collected. The hypergraph construction module is used to collect multi-channel EEG signals in real time during VR motion sickness induction. After preprocessing and multi-domain feature extraction, a multi-attribute brain function network hypergraph reflecting the strength of brain region association is constructed. The hypergraph selects effective features based on the SSQ baseline score and the SSQ motion sickness score. The model parsing module is used to construct an interpretable graph neural network model with built-in causal constraints to obtain the real-time motion sickness state and key neural subgraphs of the subjects in VR. The closed-loop control module is used to generate personalized control strategies based on the real-time motion sickness state and the current real-time neural pattern represented by the key neural subgraphs, and in comparison with the preset target neural pattern. It dynamically adjusts the current operating parameters and multi-sensory feedback of the VR scene to achieve real-time neural feedback closed-loop control for individual subjects. The training and evaluation module is used to synchronously collect multi-channel EEG data of the entire process of motion sickness induction and neural feedback during the continuous real-time neural feedback closed-loop regulation. The collected data is continuously input into the causal interpretable graph neural network model for closed-loop iterative training, and the model parameters and personalized regulation strategies are optimized synchronously until the subject's real-time neural pattern stabilizes and approaches the target neural pattern and meets the preset termination conditions. The module then outputs the model evaluation results from multiple dimensions.

Images