Immersive space virtual-real interaction method and system based on meta universe

Abstract

The invention discloses an immersive space virtual-real interaction method and system based on element universe, and relates to the technical field of brain-computer interfaces, and the method comprises the steps: capturing an electroencephalogram signal of a user, and obtaining brain wave data of the user in real time; transmitting the brain wave data to a neural feedback adaptive network for analysis, and generating a corresponding interaction intention; virtual environment elements in the meta universe are adjusted in real time according to the interaction intention; the physical environment of the user is monitored in real time, physical environment parameters are obtained, the physical environment parameters are compared with the virtual environment elements, and bidirectional synchronous adjustment is carried out; dynamically adjusting sensory elements in the virtual environment elements by utilizing electroencephalogram resonance based on the electroencephalogram signal of the user and the synchronized physical environment parameters; through bidirectional synchronous adjustment of the physical environment parameters and the virtual environment elements, high consistency between the virtual environment and the physical environment of the user is ensured, and the immersion and authenticity experience of the user are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface technology, and in particular to an immersive space virtual-reality interaction method and system based on a metaverse. Background Art

[0002] With the rapid development of metaverse technology, people have put forward higher requirements for the fusion experience of virtual and reality. As an integrated platform of virtual reality, augmented reality, artificial intelligence and other technologies, the metaverse not only provides a multi-dimensional immersive experience, but also changes various aspects of human social interaction, entertainment, education, etc. through the interaction between the virtual world and the real world. In recent years, the interaction technology between virtual and reality has gradually become the core research direction of the metaverse experience. The improvement of immersive experience depends on the accurate perception and feedback of the user's physiological and psychological state. In particular, the application of brain wave technology has brought new breakthroughs to the interaction mode of the metaverse. By capturing the user's brain wave signals, it can directly reflect the user's internal state such as emotions and intentions, and convert these signals into interactive instructions with the virtual world, greatly improving the natural interaction experience between virtual and reality.

[0003] However, existing virtual-reality interaction technologies have several limitations. First, current interaction methods mainly rely on external devices (such as gesture tracking, voice recognition, etc.), which can only capture the user's external behavior and are difficult to reflect the user's internal emotions and cognitive state, resulting in the virtual environment responding unnaturally. Secondly, the synchronization between the virtual environment and the physical environment is poor, and the user's experience in the virtual world is inconsistent with their real environment, affecting the authenticity of the immersion. In addition, when processing complex physiological signals (such as brain waves), feedback lags, and real-time synchronization between the virtual environment and the user's emotional and cognitive state cannot be achieved. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an immersive space virtual-reality interaction method based on the metaverse to solve the problem in the prior art that it is impossible to achieve real-time synchronization between the virtual environment and the user's emotional and cognitive state.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides an immersive space virtual-reality interaction method based on the metaverse, which includes capturing the user's brain wave signals and obtaining the user's brain wave data in real time; The brain wave data is transmitted to the neurofeedback adaptive network for analysis to generate the corresponding interaction intention; Adjust virtual environment elements in the metaverse in real time based on interaction intent; Monitor the user's physical environment in real time, obtain physical environment parameters, compare the physical environment parameters with virtual environment elements, and make two-way synchronous adjustments; Based on the user's EEG signals and the synchronized physical environment parameters, EEG resonance is used to dynamically adjust the sensory elements in the virtual environment elements.

[0007] As a preferred solution of the immersive space virtual-reality interaction method based on the metaverse of the present invention, the specific steps of capturing the user's EEG signal are as follows: Electrodes are placed in the frontal, central, and occipital regions of the user to collect low-level signals; The low-level signal is transmitted as an EEG signal to the front-end processing device.

[0008] As a preferred solution of the immersive space virtual-reality interaction method based on the metaverse of the present invention, the real-time acquisition of the user's brain wave data comprises the following specific steps: The front-end processing device receives the EEG signal, amplifies it through a high-gain amplifier, and outputs an analog signal; The analog signal is converted into digital brain wave data through an analog-to-digital converter; Use adaptive noise removal algorithm to remove power frequency noise and background noise in EEG data.

[0009] As a preferred solution of the immersive space virtual-reality interaction method based on the metaverse of the present invention, wherein: the brain wave data is transmitted to the neural feedback adaptive network for analysis to generate the corresponding interaction intention, and the specific steps are as follows: Integrate the EEG data after amplification, analog-to-digital conversion and noise removal to form multi-channel time series data ; Time series data The nonlinear dynamic neural network input into the neural dynamic field theory uses a continuous time differential equation to update the hidden state, which is expressed as: ; in, Indicates hidden state Over time The rate of change, The differential symbol represents a small change in quantity. Indicates time A small change in For in time The hidden state of represents the attenuation coefficient, represents a nonlinear activation function, The weight matrix representing the hidden state, represents the weight matrix of time series data, For time Time series data, Bias vector representing the hidden state; Adopting a multi-scale temporal dynamic mechanism to update the hidden state at different time scales; The updated hidden state is passed to the output layer, and the current time is output from the hidden state through a linear transformation The predicted value of ; For the predicted value, the loss function is used to measure the gap between the current predicted value and the expected value output, and the gradient descent method is used to optimize it, and the probability distribution vector of the interaction intention is output. The expression is: ; in, Indicates at time The probability distribution vector of interaction intention at the moment; Setting interaction intent thresholds , through time The probability distribution vector of interaction intention at the moment, filtering out the ones that meet The interaction intention of An index variable representing the interaction intent.

[0010] As a preferred solution of the immersive space virtual-reality interaction method based on the metaverse of the present invention, wherein: the virtual environment elements in the metaverse are adjusted in real time according to the interaction intention, the specific steps are as follows: For each interaction intention, define the corresponding virtual environment adjustment amount, the expression is: ; in, Indicates at time Always monitor virtual environment elements The adjustment amount, The weight coefficient representing the response sensitivity of the environmental elements, Indicates at time Moments and virtual environment elements The probability distribution vector of the relevant interaction intention, An index variable representing a virtual environment element; Based on the calculated adjustment , and generate corresponding adjustment instructions.

[0011] As a preferred solution of the immersive space virtual-reality interaction method based on the metaverse of the present invention, the real-time monitoring of the user's physical environment, obtaining physical environment parameters, and comparing the physical environment parameters with the virtual environment elements to perform two-way synchronous adjustment, the specific steps are as follows: Collect user's physical environment parameters in real time to form physical environment vector; Based on the physical environment vector, define the corresponding elements in the virtual environment to generate a virtual environment vector; Compare the physical environment vector with the virtual environment vector, and calculate the difference vector of each environment parameter by vector subtraction; According to the difference vector, the virtual environment state is updated by the sensitivity coefficient; Setting the difference threshold , if the difference vector is greater than the difference threshold , start external devices to adjust the physical environment.

[0012] As a preferred solution of the immersive space virtual-reality interaction method based on the metaverse of the present invention, the sensory elements in the virtual environment elements are dynamically adjusted by using EEG resonance based on the user's EEG signals and the synchronized physical environment parameters. The specific steps are as follows: For time series data Perform wavelet transform, decompose into different frequency components, and integrate to form cognitive emotion feature set; Through linear transformation, the synchronized physical environment parameters are mapped to the corresponding sensory elements in the virtual environment. , and associated with the cognitive-emotional feature set; The EEG resonance model is defined, and the matching degree between the cognitive emotion feature set and the corresponding sensory elements in the virtual environment is quantified through similarity measurement. The expression is: ; in, Indicates at time The moment The resonance of each sensory element, Indicates time EEG signal at the moment The cognitive emotion feature vector of frequency bands, Indicates the virtual environment The state vector of each sensor element, An index variable representing a sensory element, An index variable representing the frequency band; Setting the resonance threshold ,like , then the sensory elements in the virtual environment do not need to be adjusted; like , it is necessary to dynamically calculate the first The state vector of each sensory element is calculated and the element state in the virtual environment is updated in real time. The expression is: ; in, Indicates The regulatory sensitivity coefficient of each sensory element.

[0013] In a second aspect, the present invention provides an immersive space virtual-reality interaction system based on the metaverse, including an EEG signal capture module, a neural feedback analysis module, a virtual environment adjustment module, a physical environment monitoring module, and a resonance sensory adjustment module; The EEG signal capture module is used to capture the user's EEG signal and obtain the user's EEG data in real time; The neurofeedback analysis module is used to transmit brain wave data to the neurofeedback adaptive network for analysis and generate corresponding interaction intentions; The virtual environment adjustment module is used to adjust the virtual environment elements in the metaverse in real time according to the interaction intention; The physical environment monitoring module is used to monitor the user's physical environment in real time, obtain physical environment parameters, and compare the physical environment parameters with virtual environment elements to perform two-way synchronous adjustment; The resonant sensory adjustment module is used to dynamically adjust sensory elements in the virtual environment elements using EEG resonance based on the user's EEG signals and synchronized physical environment parameters.

[0014] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the immersive space virtual-reality interaction method based on the metaverse as described in the first aspect of the present invention is implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the immersive space virtual-reality interaction method based on the metaverse as described in the first aspect of the present invention.

[0016] The beneficial effects of the present invention are as follows: the present invention can effectively solve the shortcomings of the prior art by capturing the user's EEG signals and combining them with a neural feedback adaptive network to perform interaction intention analysis. First, it can capture and analyze the user's brain wave signals in real time, directly reflecting the user's emotional and cognitive state, and achieving a more natural and intelligent interaction method. Secondly, the present invention ensures a high consistency between the virtual environment and the user's physical environment through the two-way synchronous adjustment of physical environment parameters and virtual environment elements, greatly enhancing the user's immersion and authenticity experience. In addition, the dynamic adjustment mechanism based on EEG resonance can accurately adjust the sensory elements in the virtual environment, so that the changes in the virtual scene are highly synchronized with the user's cognitive and emotional state, thereby significantly enhancing the sensitivity and immersion of the interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 This is a flow chart of the immersive space virtual-reality interaction method based on the metaverse in Example 1.

[0019] Figure 2 This is a module diagram of the immersive space virtual-reality interaction system based on the metaverse in Example 1. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0023] Example 1, reference Figure 1 and Figure 2, which is the first embodiment of the present invention, and provides an immersive space virtual-reality interaction method based on a metaverse, comprising the following steps: S1. Capture the user's EEG signals and obtain the user's EEG data in real time.

[0024] Furthermore, electrodes are placed in the frontal, central and occipital regions of the user to collect low-level signals; The low-level signal is transmitted as an EEG signal to the front-end processing device.

[0025] It should be noted that the front-end processing equipment refers to the hardware equipment used to receive, amplify, process and preliminarily analyze the EEG signals collected from the EEG electrodes. It usually includes several hardware components, which are responsible for amplifying and digitizing the weak analog EEG signals and preparing for subsequent further processing and analysis.

[0026] Specifically, the front-end processing equipment includes the following main parts: high-gain amplifier, analog-to-digital converter (ADC) and pre-processing unit (filtering and noise reduction).

[0027] The front-end processing device amplifies the collected EEG signal through a high-gain amplifier and outputs an analog signal; Convert the analog signal into brain wave data through an analog-to-digital converter (ADC); Among them, the high-gain amplifier is used to significantly enhance the collected weak EEG signals to ensure that the signal strength is sufficient for subsequent processing.

[0028] The adaptive noise cancellation algorithm (ANC) is used to remove power frequency noise (such as 50 Hz power supply interference) and background noise in the EEG data to ensure the purity and accuracy of the signal.

[0029] S2. Transmit the brain wave data to the neurofeedback adaptive network for analysis and generate corresponding interaction intentions.

[0030] Furthermore, the EEG data after amplification, analog-to-digital conversion and noise removal are integrated to form multi-channel time series data. ; Preferably, the integration of multi-channel data can capture the correlation between different brain regions, especially when users perform complex operations, different areas of the brain will produce different activity patterns. In common electroencephalogram (EEG) signal processing technology and model architecture, usually only a single channel or simple multi-channel signals are processed, which makes it difficult to fully utilize the complex relationships across channels.

[0031] Time series data The nonlinear dynamic neural network input to the neural dynamic field theory (DFT) uses a continuous time differential equation to update the hidden state, which is expressed as: ; in, Indicates hidden state Over time The rate of change of (i.e., the derivative of the hidden state), The differential symbol represents a small change in quantity. Indicates time A small change in For in time The hidden state of represents the attenuation coefficient, Represents a nonlinear activation function, which usually uses a Gaussian kernel function or other continuous functions to control the change of the hidden state. The weight matrix representing the hidden state, represents the weight matrix of time series data, For time Time series data, Bias vector representing the hidden state; Preferably, by using dynamic field theory, the network can handle long-term dependencies more naturally and can produce a continuous response to small changes in input data. This dynamic response mechanism is more flexible than traditional RNN and can adapt to complex nonlinear time series data, especially when processing EEG data with non-stationary and nonlinear changes, it can capture more delicate dynamic features; in addition, DFT can also avoid the common gradient vanishing problem when processing continuous time data.

[0032] In view of the dynamic changes of multiple frequency bands and multiple levels in EEG data, a multi-scale time dynamic mechanism is adopted to update the hidden state at different time scales to capture short-term and long-term dynamic processes; Furthermore, by using a multi-scale time dynamic mechanism, the hidden state is no longer a dynamic evolution of a single time scale, but a dynamic field of multiple different time scales. Specifically, multiple different time constants can be introduced into the hidden layer of the neural network. The update of the hidden state is performed on different time scales, and the expression is: ; in, Indicates The time derivative of a hidden state, that is, its rate of change over time, Indicates at time The moment Hidden states on time scales, Indicates control The decay rate of the time scale is An index variable representing the time scale; Preferably, by introducing a multi-scale time mechanism, the network can adaptively capture short-term dynamic changes and long-term dependencies. This mechanism is particularly suitable for processing information in multiple frequency bands in EEG data (such as long-term changes in low-frequency bands and short-term fluctuations in high-frequency bands), and can process multi-level EEG signals better than traditional RNNs.

[0033] The updated hidden state is passed to the output layer, and the current time is output from the hidden state through a linear transformation The predicted value of , the expression is: ; in, represents the mapping function used to generate interaction intent, represents the weight matrix from the hidden layer to the output layer, A bias vector representing the interaction intention; Preferably, the adaptive feedback mechanism can dynamically adapt to the user's brain wave signals without relying entirely on offline training, thereby improving the response speed and accuracy of the interaction. This is difficult to achieve in neural network methods that rely on static models or offline training, especially in real-time interaction scenarios, where traditional neural network models are often unable to adapt to changes in user status.

[0034] Based on the interaction intention obtained, the loss function is used to measure the gap between the current output and the expected output, and the gradient descent method is used for optimization to output the probability distribution vector of the interaction intention, which is expressed as: ; in, Indicates at time The probability distribution vector of interaction intention; Setting interaction intent thresholds , for time The probability distribution vector of interaction intention at the moment, collecting the satisfaction The interaction intent index of An index variable representing the interaction intent.

[0035] For example, if , then generate a lighting adjustment command; if , a color adjustment command is generated.

[0036] Specifically, set the interaction intent threshold , the specific steps are as follows: Start with a simple fixed threshold, such as 0.5 or 0.7, and filter out interaction intentions with a probability higher than the threshold. Use this threshold as the initial benchmark for subsequent dynamic adjustments.

[0037] According to the probability distribution characteristics of the model output, adjust , for example, we can use the maximum probability ratio (such as ) or output-based percentile settings , to dynamically adapt to different output modes.

[0038] By comparing the probability value of each intention with the maximum probability , further filter the intents close to the maximum value (such as the probability greater than ) to improve the accuracy of intent selection.

[0039] On the basis of dynamic adjustment, we combine the prior probability of the task with the cost weight, use Bayesian theory to minimize the error cost, and further optimize choice.

[0040] Finally, the aforementioned dynamic thresholds are verified and tuned through cross-validation or experiments to select the optimal threshold that maximizes key performance indicators (such as accuracy and F1 score). .

[0041] S3. Adjust the virtual environment elements in the metaverse in real time according to the interaction intention.

[0042] Furthermore, for each interaction intention, the corresponding virtual environment adjustment amount is defined, and the expression is: ; in, Indicates at time Always monitor virtual environment elements The adjustment amount, The weight coefficient representing the response sensitivity of the environmental elements, Indicates at time Moments and virtual environment elements The probability distribution vector of the relevant interaction intention, An index variable representing a virtual environment element; It should be noted that the calculated adjustment amount , generate corresponding adjustment instructions and apply them to different elements in the virtual environment; For example, updating light intensity, adjusting the volume of background sound effects, and changing the color or shape of virtual objects.

[0043] It should be noted that this adjustment can ensure that the virtual environment responds to the user's interactive intention in a timely manner and adjusts the elements in the virtual world according to the user's EEG activity or other input. All are calculated dynamically, ensuring flexible response.

[0044] S4. Monitor the user's physical environment in real time, obtain physical environment parameters, and compare the physical environment parameters with virtual environment elements to perform two-way synchronous adjustment.

[0045] Furthermore, the user's physical environment parameters are collected in real time to form a physical environment vector; Specifically, real-time data is continuously collected through a variety of sensors (such as temperature sensors, humidity sensors, light sensors, sound sensors, and air quality sensors) deployed in the user's physical environment. At the same time, these sensors monitor various parameters in the environment at all times and transmit the collected data such as temperature, humidity, light intensity, sound decibels, and air quality to the front-end processing device; the processing device integrates this data into a multi-dimensional vector, called the physical environment vector, which represents the physical environment state at the current moment.

[0046] Based on the physical environment vector, the corresponding elements (temperature, humidity, light, sound and air quality) in the virtual environment are defined to generate a virtual environment vector; Specifically, various elements in the virtual environment (such as temperature, humidity, lighting, sound and air quality) are defined based on the actual parameters of the physical environment. According to the various parameters in the physical environment vector collected in real time, the corresponding elements in the virtual environment are set so that the initial state of the virtual environment is consistent with the physical environment or matches it within a certain range; for example, data such as temperature and humidity in the physical environment are directly mapped to the temperature and humidity parameters in the virtual environment to ensure that the sensory experience of the virtual environment is coordinated with that of the real environment; in this way, by mapping each element of the physical environment vector to the corresponding parameter of the virtual environment, a virtual environment vector is generated to represent the current environmental state in the virtual scene.

[0047] Compare the physical environment vector with the virtual environment vector, and calculate the difference vector of each environment parameter by vector subtraction; According to the difference vector, the virtual environment state is updated by the sensitivity coefficient; Among them, the sensitivity coefficient is a parameter used to adjust the response speed of the virtual environment to changes in the physical environment; The sensitivity coefficient is usually determined based on three aspects: user experience requirements, characteristics of virtual environment elements, and the magnitude of environmental changes; Specifically, user experience requirements: The size of the sensitivity coefficient can be set according to the user's preferences for changes in the virtual environment. For example, some users may want the virtual environment to respond quickly to changes in the physical environment to create a stronger sense of immersion; while other users may prefer slow and smooth transitions to avoid discomfort caused by sudden changes; Characteristics of virtual environment elements: Different virtual environment elements (such as temperature, humidity, lighting, etc.) have different sensitivities to changes in the physical environment. Some elements (such as lighting) may require higher sensitivity to quickly match changes in the physical environment, while other elements (such as temperature) may require lower sensitivity to avoid frequent fluctuations; Amplitude of environmental changes: If changes in the physical environment are frequent and large, the sensitivity coefficient can be adjusted dynamically to avoid frequent fluctuations in the virtual environment.

[0048] It should be noted that, after calculating the difference vector between the physical environment and the virtual environment, the sensitivity coefficient determines the response strength of the virtual environment to each parameter change; For example: if it is large, the virtual environment parameters will be adjusted quickly, so that the virtual environment quickly approaches the current value of the physical environment; If it is smaller, the virtual environment adjusts slowly, causing the virtual environment to quickly approach the current value of the physical environment.

[0049] Setting the difference threshold , if the difference vector is greater than the difference threshold , start external equipment (heater, humidifier, lighting equipment, etc.) to adjust the physical environment to make it gradually closer to the virtual environment.

[0050] It should be noted that setting the difference threshold The specific steps are first to determine the minimum range of environmental changes that users can feel based on their perception. Next, analyze the response capabilities of physical devices (such as heaters, humidifiers, lighting devices, etc.) to determine the minimum unit and response speed that the device can effectively adjust to prevent the device from starting frequently due to minor changes. Then, evaluate the natural fluctuations of environmental parameters to avoid frequent adjustments within the normal fluctuation range. Combining these factors, set an initial difference threshold for each environmental parameter , ensuring a balance between user experience, device performance, and environmental changes. Afterwards, we verified the response through experiments, collected user feedback, and further optimized the thresholds , ensuring that the virtual and physical environments are adjusted synchronously to achieve the best results. Finally, the difference threshold can be dynamically adjusted according to the user's state (such as concentration or relaxation) and the dramatic fluctuations of the environment. , to flexibly adapt to different scene requirements.

[0051] S5. Based on the user's EEG signals and synchronized physical environment parameters, the sensory elements in the virtual environment elements are dynamically adjusted using EEG resonance.

[0052] Furthermore, for time series data Perform wavelet transform, decompose into different frequency components, and integrate to form cognitive emotion feature set; The synchronized physical environment parameters are mapped to the corresponding sensory elements in the virtual environment through linear transformation. (such as temperature perception, volume, light intensity, etc.), and associate them with cognitive emotion feature sets; The EEG resonance model is defined, and the matching degree between the cognitive emotion feature set and the corresponding sensory elements in the virtual environment is quantified through similarity measurement. The expression is: ; in, Indicates at time The moment The resonance of each sensory element, Indicates time EEG signal at the moment The cognitive emotion feature vector of frequency bands, Indicates the virtual environment The state vector of each sensor element, An index variable representing a sensory element, The index variable representing the frequency band, Representation vector and The inner product of is used to calculate their similarity. Representation vector and The norm (or length) of is used for normalization to ensure that the similarity calculation is not affected by the length of the vector; Setting the resonance threshold ,like , then the sensory elements in the virtual environment do not need to be adjusted; Specifically, set the resonance threshold The specific steps are as follows: Collect sufficient resonance data from multiple users' EEG signals and virtual environment interaction experiments. These data should cover the matching degree in a variety of different situations, including high matching and low matching situations; Arrange all collected resonance data into distribution and draw its frequency histogram or density map. This helps to understand the overall distribution trend of resonance and determine the concentration interval of most matches; Select the 90% or 95% quantile of the resonance. This quantile means that in most cases (such as 90% or 95% of the time), the resonance is above this threshold.

[0053] Assume that the 90% quantile is selected as the threshold, which means that in all experimental data, about 90% of the resonance degrees are within the threshold Above, only 10% of the data is below the threshold This ensures that the virtual environment is only adjusted in low-match situations.

[0054] Determine the specific threshold value based on the calculated quantile; for example, if the 90% quantile is obtained through statistical analysis , then Set to 0.75; that is, when the resonance When it is greater than 0.75, the sensory elements of the virtual environment do not need to be adjusted; when it is less than 0.75, the adjustment mechanism is triggered.

[0055] like , it is necessary to dynamically calculate the first The state vector of each sensory element is calculated and the element state in the virtual environment is updated in real time. The expression is: ; in, Indicates The regulatory sensitivity coefficient of each sensory element.

[0056] It should be noted that the sensory elements in the virtual environment are dynamically adjusted through the resonance model to resonate with the user's brain wave data. In this process, the updated EEG signals generated by the user during resonance are captured as feedback data and further input into the neurofeedback adaptive network. In this way, the user's cognitive and emotional state can be perceived in real time, and the sensory element settings of the virtual environment can be further optimized based on these feedback signals, so that the virtual environment is synchronized with the user's emotional and cognitive state; This resonance feedback mechanism based on EEG signals not only improves the intelligence level of the virtual environment, but also enables adaptive selection of the most appropriate sensory element settings through real-time neural feedback, thereby achieving a more natural, organic and efficient interactive experience.

[0057] The present embodiment also provides an immersive space virtual-reality interaction system based on the metaverse, including: an EEG signal capture module, used to capture the user's EEG signal and obtain the user's brain wave data in real time; a neural feedback analysis module, used to transmit the brain wave data to the neural feedback adaptive network for analysis, and generate corresponding interaction intentions; a virtual environment adjustment module, used to adjust the virtual environment elements in the metaverse in real time according to the interaction intention; a physical environment monitoring module, used to monitor the user's physical environment in real time, obtain physical environment parameters, and compare the physical environment parameters with the virtual environment elements for two-way synchronous adjustment; a resonant sensory adjustment module, used to dynamically adjust the sensory elements in the virtual environment elements based on the user's EEG signal and the synchronized physical environment parameters using EEG resonance.

[0058] This embodiment also provides a computer device, which is suitable for the case of an immersive space virtual-reality interaction method based on the metaverse, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the immersive space virtual-reality interaction method based on the metaverse as proposed in the above embodiment.

[0059] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0060] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for realizing immersive space virtual-reality interaction based on the metaverse as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0061] In summary, the present invention can effectively solve the shortcomings of the prior art by capturing the user's EEG signals and combining them with a neural feedback adaptive network to analyze interaction intentions. First, it can capture and analyze the user's EEG signals in real time, directly reflecting the user's emotional and cognitive state, and achieving a more natural and intelligent interaction method. Secondly, the present invention ensures a high consistency between the virtual environment and the user's physical environment through the two-way synchronous adjustment of physical environment parameters and virtual environment elements, greatly enhancing the user's immersion and authenticity experience. In addition, the dynamic adjustment mechanism based on EEG resonance can accurately adjust the sensory elements in the virtual environment, so that the changes in the virtual scene are highly synchronized with the user's cognitive and emotional state, thereby significantly enhancing the sensitivity and immersion of the interaction.

[0062] Example 2, referring to Table 1, is the second example of the present invention. In order to further verify the technical solution of the present invention, experimental simulation data of an immersive space virtual-reality interaction method based on the metaverse is provided.

[0063] This embodiment uses a complete set of brain-computer interface devices, including a 64-channel dry electrode EEG acquisition device (sampling rate 1024Hz), a high-performance computing server (equipped with NVIDIA A100 GPU) and supporting virtual reality equipment. 30 volunteers (age range 20-35 years old, including 15 males and 15 females) were selected to conduct an immersive virtual-reality interactive experience test. The test scene was set as a "virtual museum", which includes multimodal interactive elements such as vision, hearing, and touch.

[0064] First, dry electrodes were placed in the frontal lobe (Fp1, Fp2, F3, F4, Fz), central area (C3, C4, Cz) and occipital area (O1, O2, Oz) of the volunteers using conductive gel to collect EEG signals. The collected raw signals were amplified by 160,000 times the gain of the TI ADS1299 chip from Texas Instruments to amplify the microvolt level signal to the volt level range. Subsequently, the analog signal was converted into a digital signal through a 24-bit analog-to-digital converter, and the improved adaptive least mean square (LMS) algorithm was used to filter out the 50Hz power frequency interference and electromyographic artifacts in real time, and the signal-to-noise ratio was improved by about 40dB, as shown in Table 1 below: Table 1 EEG signal acquisition and processing data table

[0065] Secondly, the preprocessed EEG data was input into a pre-trained neurofeedback adaptive network. The network adopted a hierarchical structure with three time scales (0.1s, 0.5s, 2s), 512 neurons in the hidden layer, and ReLU as the activation function. The interaction intention threshold τ was set to 0.75, and the environmental element response sensitivity The value range is [0.1,1.0], specifically: Time series data Input to a nonlinear dynamic neural network from the neural dynamic field theory. The hidden state of the network Over time The rate of change of is described by the following differential equation, expressed as: ; in, The nonlinear activation function, in this embodiment, uses ReLU (RectifiedLinearUnit), that is, the expression is: ; The network adopts a hierarchical structure with three time scales, namely 0.1s, 0.5s, and 2s. This means that the hidden state The updates are asynchronous at each time scale, and the specific time scale determines the rate at which the hidden state changes.

[0066] Since discrete time steps are usually used in actual calculations, the differential equation can be discretized: ; in, is the time step (e.g. 0.1s, 0.5s, 2s), which represents the time interval between each step in discrete time.

[0067] Updated hidden state It is passed to the output layer and the current moment is obtained through linear transformation The predicted value of : ; in, represents the weight matrix of the output layer, represents the weight matrix of the output layer; Next, the softmax function is used to calculate the probability distribution vector of the interaction intention , the expression is: ; Setting interaction intent thresholds , by filtering to meet The interaction intention is expressed as: ; Furthermore, the details are shown in Table 2 below: Table 2 Time series neural network update table

[0068] Next, in terms of physical environment monitoring, a sensor array of temperature, humidity, light, and sound field was deployed with a sampling frequency of 100 Hz. The difference threshold ω was set to 1.5 times the standard deviation of each parameter. Physical environment adjustment uses intelligent lighting equipment, omnidirectional sound field equipment, and temperature control equipment to work together, as shown in Table 3 below: Table 3 Physical environment monitoring and adjustment parameters

[0069] Finally, in the EEG resonance modeling, Morlet wavelet was used to perform time-frequency analysis on the EEG signal to extract the energy characteristics of the five frequency bands of δ (1-4 Hz), θ (4-8 Hz), α (8-13 Hz), β (13-30 Hz) and γ (30-50 Hz). The resonance threshold δ was set to 0.8, and the value range of the sensory element adjustment sensitivity coefficient γ_k was [0.2, 0.8], as shown in Table 4 below: Table 4 Experimental data comparison table

[0070] By comparing the experimental data, it can be clearly seen that the scheme of the present invention is superior to the existing technology in multiple key performance indicators. For example, in terms of immersive experience, after the EEG resonance mechanism is enabled, the immersive score reaches 95.2 points, which is 31.3% higher than the traditional VR. This improvement is due to the fact that the scheme can capture the user's cognitive and emotional state in real time and adjust the sensory elements in the virtual environment accordingly, so that the virtual experience is more in line with the user's subjective feelings. In terms of interactive performance, the present invention reduces the interaction delay to 28ms, which is 81.3% less than the traditional VR, and at the same time, the interaction accuracy is increased to 96.5%, which is 23.1% higher than the traditional VR. This is mainly due to the multi-scale time dynamic mechanism of the neural feedback adaptive network, which can quickly respond to the user's interactive intention. In terms of user experience, the scheme significantly reduces physiological fatigue, from 61.9% to 25.1, indicating that the interaction method is more natural and reduces the burden of use; the cognitive load index is also reduced from 7.8 to 3.2, with an improvement of 59.0%, which shows that the interaction method of the present invention is more in line with the natural cognitive law of human beings. In terms of virtual-reality collaboration, the environmental synchronization error was reduced from 42.3mm to 12.6mm, an improvement of 70.2%; virtual-reality coordination reached 94.7%, an improvement of 25.3% over traditional VR. Finally, in terms of emotional experience, the emotional engagement reached 93.8 points, an improvement of 36.1%. This is achieved by extracting the user's cognitive emotional characteristics through wavelet transform and establishing a dynamic mapping relationship with the sensory elements in the virtual environment, making the virtual experience more in line with the user's emotional state.

[0071] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An immersive space virtual-reality interaction method based on the metaverse, characterized by: include, Capture the user's EEG signals and obtain the user's EEG data in real time; The brain wave data is transmitted to the neurofeedback adaptive network for analysis to generate the corresponding interaction intention; Adjust virtual environment elements in the metaverse in real time based on interaction intent; Monitor the user's physical environment in real time, obtain physical environment parameters, compare the physical environment parameters with virtual environment elements, and make two-way synchronous adjustments; Based on the user's EEG signals and synchronized physical environment parameters, the sensory elements in the virtual environment elements are dynamically adjusted using EEG resonance; The brain wave data is transmitted to the neurofeedback adaptive network for analysis to generate the corresponding interaction intention. The specific steps are as follows: Integrate the EEG data after amplification, analog-to-digital conversion and noise removal to form multi-channel time series data ; Time series data Input to a nonlinear dynamic neural network of neural dynamic field theory, and update the hidden state using continuous-time differential equations; Adopting a multi-scale temporal dynamic mechanism to update the hidden state at different time scales; The updated hidden state is passed to the output layer, and the current time is output from the hidden state through a linear transformation The predicted value of ; For the predicted value, a loss function is used to measure the gap between the current predicted value and the expected value output, and the gradient descent method is used to optimize it and output the probability distribution vector of the interaction intention; Setting interaction intent thresholds , through time The probability distribution vector of interaction intention at the moment, filtering out the ones that meet The interaction intention of An index variable representing the interaction intent.

2. The immersive space virtual-reality interaction method based on the metaverse as claimed in claim 1, characterized in that: The specific steps of capturing the user's EEG signal are as follows: Electrodes are placed in the frontal, central, and occipital regions of the user to collect low-level signals; The low-level signal is transmitted as an EEG signal to the front-end processing device.

3. The immersive space virtual-reality interaction method based on the metaverse as claimed in claim 2, characterized in that: The specific steps of obtaining the user's brain wave data in real time are as follows: The front-end processing device receives the EEG signal, amplifies it through a high-gain amplifier, and outputs an analog signal; The analog signal is converted into digital brain wave data through an analog-to-digital converter; Use adaptive noise removal algorithm to remove power frequency noise and background noise in EEG data.

4. The immersive space virtual-reality interaction method based on the metaverse as claimed in claim 3, characterized in that: The expression for updating the hidden state using the continuous time differential equation is: ； in, Indicates hidden state Over time The rate of change, The differential symbol represents a small change in quantity. Indicates time A small change in For in time The hidden state of represents the attenuation coefficient, represents a nonlinear activation function, The weight matrix representing the hidden state, represents the weight matrix of time series data, For time Time series data, Bias vector representing the hidden state; The probability distribution vector of interaction intention is calculated by the following formula: ； in, Indicates at time The probability distribution vector of interaction intention at the moment.

5. The immersive space virtual-reality interaction method based on the metaverse as claimed in claim 4, characterized in that: The specific steps of adjusting the virtual environment elements in the metaverse in real time according to the interaction intention are as follows: For each interaction intention, define the corresponding virtual environment adjustment amount, the expression is: ； in, Indicates at time Always monitor virtual environment elements The adjustment amount, The weight coefficient representing the response sensitivity of the environmental elements, Indicates at time Moments and virtual environment elements The probability distribution vector of the relevant interaction intention, An index variable representing a virtual environment element; Based on the calculated adjustment , generate corresponding adjustment instructions.

6. The immersive space virtual-reality interaction method based on the metaverse as claimed in claim 5, characterized in that: The real-time monitoring of the user's physical environment, obtaining physical environment parameters, and comparing the physical environment parameters with virtual environment elements to perform two-way synchronous adjustment, the specific steps are as follows: Collect user's physical environment parameters in real time to form physical environment vector; Based on the physical environment vector, define the corresponding elements in the virtual environment to generate a virtual environment vector; Compare the physical environment vector with the virtual environment vector, and calculate the difference vector of each environment parameter by vector subtraction; According to the difference vector, the virtual environment state is updated by the sensitivity coefficient; Setting the difference threshold , if the difference vector is greater than the difference threshold , start external devices to adjust the physical environment.

7. The immersive space virtual-reality interaction method based on the metaverse according to claim 6, characterized in that: The specific steps of dynamically adjusting the sensory elements in the virtual environment elements by using EEG resonance based on the user's EEG signal and the synchronized physical environment parameters are as follows: For time series data Perform wavelet transform, decompose into different frequency components, and integrate to form cognitive emotion feature set; Through linear transformation, the synchronized physical environment parameters are mapped to the corresponding sensory elements in the virtual environment. , and associated with the cognitive-emotional feature set; The EEG resonance model is defined, and the matching degree between the cognitive emotion feature set and the corresponding sensory elements in the virtual environment is quantified through similarity measurement. The expression is: ； in, Indicates at time The moment The resonance of each sensory element, Indicates time EEG signal at the moment The cognitive emotion feature vector of frequency bands, Indicates the virtual environment The state vector of each sensor element, An index variable representing a sensory element, An index variable representing the frequency band; Setting the resonance threshold ,like , then the sensory elements in the virtual environment do not need to be adjusted; like , it is necessary to dynamically calculate the first The state vector of each sensory element is calculated and the element state in the virtual environment is updated in real time. The expression is: ； in, Indicates The regulatory sensitivity coefficient of each sensory element.

8. An immersive space virtual-reality interaction system based on a metaverse, based on the immersive space virtual-reality interaction method based on a metaverse according to any one of claims 1 to 7, characterized in that: Including, EEG signal capture module, neural feedback analysis module, virtual environment adjustment module, physical environment monitoring module and resonance sensory adjustment module; The EEG signal capture module is used to capture the user's EEG signal and obtain the user's EEG data in real time; The neurofeedback analysis module is used to transmit brain wave data to the neurofeedback adaptive network for analysis and generate corresponding interaction intentions; The virtual environment adjustment module is used to adjust the virtual environment elements in the metaverse in real time according to the interaction intention; The physical environment monitoring module is used to monitor the user's physical environment in real time, obtain physical environment parameters, and compare the physical environment parameters with virtual environment elements to perform two-way synchronous adjustment; The resonant sensory adjustment module is used to dynamically adjust sensory elements in the virtual environment elements using EEG resonance based on the user's EEG signals and synchronized physical environment parameters.

Images