Virtual scene immersion prediction method based on electroencephalogram signals

Abstract

The invention discloses a virtual scene immersion prediction method based on electroencephalogram signals. The method comprises the following steps: firstly, based on a binocular imaging principle, constructing a space distortion model of different distortion effects caused by mismatching of a camera shooting field angle and a human eye field angle, and quantifying a distortion degree by using an index based on a field angle difference; secondly, a man-machine interaction experiment involving virtual reality space distortion is designed, and subjective I PQ scoring data and objective EEG signal data under different distortion conditions are collected; preprocessing such as filtering, artifact removal and spectral analysis is carried out on the collected EEG original signal so as to calculate the absolute power value of a specified brain wave frequency band. And finally, combining the calculated electroencephalogram absolute power value with the distortion index, constructing an immersion prediction model based on a polynomial regression algorithm, and realizing objective prediction of subjective immersion. According to the method, the influence mechanism of virtual space distortion on immersion can be effectively revealed, and a solid theoretical support is provided for optimizing virtual reality space design.

Description

Technical Field

[0001] The present invention relates to the field of virtual scene analysis, and more specifically, to a method for predicting virtual scene immersion based on electroencephalogram signals. Background Art

[0002] As consumers' demand for personalized entertainment experiences grows, and as industries such as medicine, construction, and education increase their demand for simulation technologies, people's demand for immersive experiences in virtual reality (VR) human-computer interaction is also increasing. VR technology integrates advanced technologies such as visual simulation, audio simulation, tactile feedback, and motion capture, bridging the physical world and the digital world. Users experience multi-dimensional sensory experiences in a virtual environment (VE) that are very similar to the real world. Especially in the medical field, VR stereo imaging technology is widely used in surgical navigation and medical education. Therefore, providing high-quality VR immersive experience through real and distortion-free stereo images has become a key task for the further development of current VR visual technology.

[0003] VR immersion refers to the degree of engagement and sense of reality that users gain in a virtual environment. This sense of immersion is largely affected by hardware and software performance such as graphics quality, sound quality, and interactive capabilities. In the field of VR visual display applications, spatial distortion, as a ubiquitous phenomenon, has an impact on user experience that cannot be ignored. This is a spatial distortion phenomenon that occurs when presenting three-dimensional images. The long-term visual discomfort and cognitive confusion caused by it can easily cause user fatigue, motion sickness, and other problems, thereby reducing the overall comfort and efficiency of human-computer interaction. In recent years, scholars have conducted in-depth discussions on the causes, impact mechanisms, measurement methods, and correction strategies of distortion. Some scholars believe that this phenomenon may be caused by hardware reasons. For example, in near-eye display (NED) devices, in order to fully present the image in front of the human eye, the optical lens of the device needs to focus the focal plane of the human eye to the back of the device. However, due to the presence of the lens, the stereoscopic image will have a certain image distortion at the edge. This optical distortion can cause visually induced motion sickness (VIMS).

[0004] In addition, the entire process of acquiring, processing, displaying, and perceiving stereoscopic images may lead to unnatural deformation and distortion of spatial shapes or spatial relationships due to the inconsistency of parameters between each other. Some existing studies have shown that the mismatch between the focal length of the virtual camera in the VR environment and the natural focal length of the human eye will lead to distortion of spatial perception, affecting the accuracy of the user's judgment of the distance of the self-center, and the offset of the camera position relative to the observer's eyes will also aggravate the parallax error. Some scholars have focused on the technical details of the stereoscopic image presentation process, pointing out that the non-parallel arrangement of the left and right virtual cameras will cause trapezoidal distortion, and explaining the direct influence of the shooting angle on the distortion formation. Secondly, some studies have pointed out the barrel distortion phenomenon caused by the difference in pixel offset between the left and right views. Whether it is simultaneous distortion of the left and right or distortion of a single view, it will have different degrees of impact on the perception of stereoscopic sense.

[0005] Optimizing the VR immersive experience is an important research goal of virtual reality stereo imaging technology. Although there are many methods to quantify the subjective experience of VR, which can reflect the cognitive level and psychological state of users in the virtual reality experience to a certain extent, there are still some limitations. For example, existing quantitative methods mostly rely on subjective scoring, which is easily disturbed by individual differences and uncontrollable factors in the experimental environment, thus affecting the objectivity and accuracy of the data. In addition, when describing the sense of immersion, there is a lack of an intuitive and objective model that can clearly characterize this subjective perception level. Furthermore, how to study distortion quantification methods that are more sensitive to structural distortion is an important task in the current VR field. Summary of the invention

[0006] The present invention overcomes the defects of the prior art and proposes a method for predicting the immersive feeling of a virtual scene based on EEG signals.

[0007] The first aspect of the present invention provides a method for predicting immersion in a virtual scene based on electroencephalogram signals, comprising:

[0008] S1: Based on the principle of binocular stereo imaging, in the process of acquiring, displaying and perceiving stereo images, parameter mismatch may lead to spatial distortion. By adjusting the camera field of view, different degrees of spatial distortion are simulated, and the corresponding distortion level index is set. According to different degrees of distortion, spatial distortion models at all levels are constructed;

[0009] S2: Design a human-computer interaction experiment based on virtual reality to collect the subjective perception quality evaluation IPQ of different distortion models and the corresponding objective original EEG signal data;

[0010] S3: Perform IIR filter preprocessing on the original EEG data to obtain preprocessed EEG signals, separate spatially independent signals from the preprocessed EEG signals based on the ICA analysis method to remove mixed noise in the EEG signals, reconstruct the preprocessed EEG signals to obtain artifact-free EEG signals, perform spectrum analysis on the artifact-free EEG signals through fast Fourier transform, and convert the signals into frequency domain for analysis, calculate the power spectral density PSD of the specified frequency on each channel, and calculate the corresponding absolute power value through the power spectral density PSD;

[0011] S4: Construct an immersion prediction model based on machine learning and polynomial regression algorithm. The prediction model uses the absolute power value of the specified frequency and the distortion level index calculated in S3 as input features, combines the input features through an exponential relationship, defines the immersion index, and uses the polynomial regression algorithm to fit the correlation between the predicted value and the immersion index to obtain the polynomial expression of the prediction model.

[0012] S5: Correlation analysis was performed between IPQ and the predicted value, and the coefficient of determination R was used to 2 Evaluate the goodness of fit expressed by the polynomial model and further optimize the prediction model.

[0013] In this scheme, in S1, stereoscopic images are imaging analysis based on virtual reality, including three processes of acquisition, display and perception, and each process corresponds to independent parameter collection.

[0014] In this solution, S1 includes:

[0015] The stereoscopic imaging device includes a left camera and a right camera. A three-dimensional coordinate system is established with the midpoint of the left and right cameras as the origin, which is defined as the object space. The coordinates are expressed as (X o ,Y o ,Z o ), a three-dimensional coordinate system is established with the center position of the left and right eyes of the subject as the origin, and it is defined as the image space. The coordinates are expressed as (X i ,Y i ,Z i ), a geometric mapping relationship is established between the object space and the image space, and the relationship is expressed as follows:

[0016]

[0017] Where M is the magnification factor for mapping the two-dimensional image on the camera sensor to the 3D display device; b e and b c Respectively represent the pupil distance of human eyes and the distance between binocular cameras; f represents the focal length of the camera; V d is the distance between the human eye and the 3D display device; s is the camera sensor offset value, W cis the horizontal width of the camera sensor, C d is the distance between the intersection of the left and right camera optical axes and the camera center point, that is, the camera convergence distance. The plane at this camera convergence distance is called the zero parallax plane. o is the horizontal width of the camera’s shooting range on the zero parallax plane, C d , W o , b c The relationship between the three is shown in the following formula:

[0018]

[0019] Among them, λ is the camera field of view angle. By changing the camera field of view angle, different distortion degrees and corresponding distortion level indexes are formed. The following is the calculation expression of the distortion level index:

[0020]

[0021] Among them, Distortion represents the distortion level index.

[0022] In this solution, in S3, before performing IIR filter preprocessing, high-pass and low-pass cutoff frequencies are set for the IIR filter.

[0023] In this solution, S3 is specifically:

[0024] IIR filter is based on the difference equation, and the output depends on the linear combination of the current input sample and the past input and output samples. The expression is as follows:

[0025]

[0026] Among them, y[n] is the output of the filter, x[n] is the input raw EEG data, and a j and b are filter coefficients, M and N are filter orders, k and j are cumulative calculation variables, nk and nj are corresponding cumulative index values, and n is the index of the window;

[0027] Before the spectrum analysis, the artifact-free EEG signal was divided into 129 frequency intervals, each interval corresponding to an FFT coefficient, and a Hamming window function with 256 sampling points was used for windowing. The sampling rate was set to 220 Hz. After each FFT calculation, the window slid forward 22 sample points to form a 90% overlapping window. FFT is fast Fourier transform, and the expression is as follows:

[0028]

[0029] Where ω(n) is the time domain signal processed by the preset window function, n is the index of the window, and L is the length of the window. Given the original discrete signal x(n), before performing FFT calculation on it, the Hamming window function is applied to the signal to generate the windowed time domain signal x ω (n), the expression is as follows,

[0030] x ω (n) = x(n)·ω(n),

[0031] The windowed time domain signal is subjected to fast Fourier transform to obtain the frequency domain signal. The expression is as follows:

[0032]

[0033] Among them, X(k) is the frequency domain signal after Fourier transformation, N is the sample size of the input signal, k is the input sample index, and n is the index of the window. is the complex exponential kernel function in the frequency domain, j1 is the preset coefficient, and the following is the calculation expression of the power spectral density PSD and the calculation expression of the absolute power value P in the specified frequency range.

[0034]

[0035] where |X(k)| 2 is the square of the modulus of X(k), indicating the power of the kth frequency component; PSD(k) indicates the power distributed per unit frequency, i.e., the power spectral density; k1 and k2 correspond to the start and end indexes of the specified frequency band, respectively; f s is the sampling rate, and k is the input sample index.

[0036] In this solution, in S4, the immersion index and the polynomial model are expressed as follows:

[0037] I IMS =P a (1-Distortion) b ,

[0038] Among them, a and b are fitting parameters, which are 0.4 and 0.9 respectively, P is the absolute power value, Distortion is the distortion level index, I IMS is the immersion index. Further, the nonlinear relationship between the immersion index and subjective immersion is fitted based on the polynomial regression algorithm. The polynomial model expression is as follows:

[0039]

[0040] in, Predict values ​​for the immersion model.

[0041] In this scheme, in S5, the determination coefficient R 2 The specific expression to be evaluated is:

[0042]

[0043] Among them, n1 is the amount of evaluation data, y i is the actual value of the i-th evaluation data, is the predicted value for the i-th evaluation data, For all y i The mean of .

[0044] The second aspect of the present invention further provides a virtual scene immersion prediction system based on EEG signals, the system comprising: a memory, a processor, the memory comprising a virtual scene immersion prediction program based on EEG signals, the virtual scene immersion prediction program based on EEG signals implementing the following steps when executed by the processor:

[0045] S1: Based on the principle of binocular stereo imaging, in the process of acquiring, displaying and perceiving stereo images, parameter mismatch may lead to spatial distortion. By adjusting the camera field of view, different degrees of spatial distortion are simulated, and the corresponding distortion level index is set. According to different degrees of distortion, spatial distortion models at all levels are constructed;

[0046] S2: Design a human-computer interaction experiment based on virtual reality to collect the subjective perception quality evaluation IPQ of different distortion models and the corresponding objective original EEG signal data;

[0047] S3: Perform IIR filter preprocessing on the original EEG data to obtain preprocessed EEG signals, separate spatially independent signals from the preprocessed EEG signals based on the ICA analysis method to remove mixed noise in the EEG signals, reconstruct the preprocessed EEG signals to obtain artifact-free EEG signals, perform spectrum analysis on the artifact-free EEG signals through fast Fourier transform, and convert the signals into frequency domain for analysis, calculate the power spectral density PSD of the specified frequency on each channel, and calculate the corresponding absolute power value through the power spectral density PSD;

[0048] S4: Construct an immersion prediction model based on machine learning and polynomial regression algorithm. The prediction model uses the absolute power value of the specified frequency and the distortion level index calculated in S3 as input features, combines the input features through an exponential relationship, defines the immersion index, and uses the polynomial regression algorithm to fit the correlation between the predicted value and the immersion index to obtain the polynomial expression of the prediction model.

[0049] S5: Correlation analysis was performed between IPQ and the predicted value, and the coefficient of determination R was used to 2 Evaluate the goodness of fit expressed by the polynomial model and further optimize the prediction model.

[0050] The third aspect of the present invention also provides a computer-readable storage medium, which includes a virtual scene immersion prediction program based on EEG signals. When the virtual scene immersion prediction program based on EEG signals is executed by a processor, the steps of the virtual scene immersion prediction method based on EEG signals as described in any of the above items are implemented.

[0051] The fourth aspect of the present invention also provides a VR platform device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and is characterized in that when the processor executes the computer program, the steps of the virtual scene immersion prediction method based on EEG signals as described above are implemented.

[0052] The present invention proposes a method for predicting the immersion of virtual scenes with different spatial distortion levels based on EEG signals. First, based on the principle of binocular imaging, a spatial distortion model of different distortion effects caused by the mismatch between the camera's field of view and the human eye's field of view is constructed, and an index based on the field of view is used to quantify the degree of distortion. Then, a human-computer interaction experiment involving virtual reality space distortion is designed and developed, and subjective IPQ score data and objective EEG signal data under different distortion conditions are collected. The collected EEG original signal is preprocessed by filtering, artifact removal, and spectrum analysis to calculate the absolute power value of the specified EEG frequency band. Finally, the calculated EEG absolute power value is combined with the distortion index to construct an immersion prediction model based on a polynomial regression algorithm to achieve objective prediction of subjective immersion. Through the present invention, the influence mechanism of virtual space distortion on immersion can be effectively revealed, and a solid theoretical support is provided for optimizing the design of virtual reality space. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of a method for predicting virtual scene immersion based on EEG signals of the present invention is shown;

[0054] Figure 2 A schematic diagram of the three-dimensional imaging process of virtual reality of the present invention is shown.

[0055] Figure 3 The two-dimensional model principle diagram and effect diagram of the image space under six distortion level indexes of the embodiment of the present invention are shown;

[0056] Figure 4 The EEG signal processing flow chart of the present invention is shown;

[0057] Figure 5 The relationship between the predicted value of the model of the present invention and the actual IPQ score is shown;

[0058] Figure 6A block diagram of a virtual scene immersion prediction system based on EEG signals of the present invention is shown. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0061] Figure 1 A flowchart of a method for predicting virtual scene immersion based on EEG signals of the present invention is shown.

[0062] like Figure 1 As shown, the first aspect of the present invention provides a method for predicting immersion in a virtual scene based on EEG signals, comprising:

[0063] S1: Based on the principle of binocular stereo imaging, in the process of acquiring, displaying and perceiving stereo images, parameter mismatch may lead to spatial distortion. By adjusting the camera field of view, different degrees of spatial distortion are simulated, and the corresponding distortion level index is set. According to different degrees of distortion, spatial distortion models at all levels are constructed;

[0064] Three-dimensional imaging in virtual reality is mainly divided into three steps, namely the acquisition, display and perception of stereoscopic images.

[0065] A three-dimensional coordinate system is established with the midpoint of the left and right cameras as the origin, which is defined as the object space (X o ,Y o ,Z o ). A three-dimensional coordinate system is established with the center position of the left and right eyes as the origin, and it is defined as the image space (X i ,Y i ,Z i ), a geometric mapping relationship is established between the object space and the image space;

[0066] The specific geometric mapping process is shown in Equation 1:

[0067]

[0068] Where M is the magnification factor for mapping the two-dimensional image on the camera sensor to the 3D display device; b e and b cRespectively represent the interpupillary distance (IPD) of the human eye and the binocular camera distance; f represents the focal length of the camera; V d is the distance between the human eye and the 3D display device; s is the camera sensor offset value (Formula 2), which allows the optical axes of the left and right cameras to intersect while maintaining a horizontal arrangement. d W is the distance between the intersection of the left and right camera optical axes and the camera center point, also known as the camera convergence distance. On the plane at this distance, there is no parallax between the left and right camera views, so this plane is called the zero parallax plane. o is the horizontal width of the camera’s shooting range on the zero parallax plane, C d , W o , b c The relationship between the three is shown in the following formula:

[0069]

[0070] Where λ is the camera field of view (FOV).

[0071] When the camera's field of view is consistent with the human eye's field of view, the image space and the object space are completely matched. Once the camera's field of view is changed, the zero parallax plane width W o The focal length f will also change accordingly, thus changing the mapping position from the object space to the image space, which will affect the entire stereoscopic image perception process, causing the image space to distort (parameter mismatch) and form a distortion effect. Based on this principle, different degrees of spatial distortion models can be constructed by changing the mismatch between the camera's field of view and the human eye's field of view.

[0072] In one embodiment, the viewing distance V d Ensure that the human eye field of view is 60° when watching a 3D display device, and then set six different camera field of view angles, corresponding to six distortion levels. Formula 4 is the calculation expression of the distortion level index.

[0073]

[0074] This formula represents the compression between the horizontal width captured at different camera field of view angles and the horizontal width when the viewing angle is 60°. It can be used as a quantitative value to represent the degree of distortion. The six distortion level indices are 0 (60°), 0.2127 (70°), 0.4532 (80°), 0.7319 (90°), 1.0641 (100°) and 1.4733 (110°).

[0075] Figure 2 A schematic diagram of the three-dimensional imaging process of virtual reality of the present invention is shown.

[0076] like Figure 2 As shown in the figure, Object space is the object space, Image space is the image space, Camera sensor is the camera sensor, and 3D display device is the 3D display device.

[0077] Figure 3 The two-dimensional model principle diagram and effect diagram of the image space under six distortion level indexes of the embodiment of the present invention are shown. The order of distortion degree change is (a) to (f).

[0078] S2: Design a human-computer interaction experiment based on virtual reality to collect the subjective perception quality evaluation IPQ of different distortion models and the corresponding objective original EEG signal data;

[0079] It should be noted that VR human-computer interaction experience is carried out in sequence under different degrees of distortion to induce the corresponding immersive state of the subjects. In addition to filling in the IPQ subjective score after each interactive experience, the subjects always wear EEG equipment throughout the experience to collect objective physiological signal data during the experience. The subjective evaluation method adopts the Igroup Presence Questionnaire (IPQ), which has the advantages of clear structure, concise content, and easy operation. It has shown high reliability and validity in a wide range of applications and is a representative tool for measuring immersion. However, since the subjective score depends on the individual's subjective perception and is easily affected by emotions, cognitive preferences and individual differences, in order to make up for this deficiency, the present invention introduces the process of collecting electroencephalogram (EEG) data to obtain more objective physiological signals.

[0080] S3: Perform IIR filter preprocessing on the original EEG data to obtain preprocessed EEG signals, separate spatially independent signals from the preprocessed EEG signals based on the ICA analysis method to remove mixed noise in the EEG signals, reconstruct the preprocessed EEG signals to obtain artifact-free EEG signals, perform spectrum analysis on the artifact-free EEG signals through fast Fourier transform, and convert the signals into frequency domain for analysis, calculate the power spectral density PSD of the specified frequency on each channel, and calculate the corresponding absolute power value through the power spectral density PSD;

[0081] Figure 4 The flowchart of EEG signal processing of the present invention is shown.

[0082] like Figure 4 As shown in the figure, Hamming window is the windowing process of Hamming window. Figure 4 Below is a schematic diagram of the absolute power calculation for each frequency band.

[0083] It should be noted that the windowed time domain signal is the artifact-free EEG signal, and the output of the filter is the preprocessed EEG signal.

[0084] When preprocessing the EEG raw data (i.e., raw EEG data), the present invention uses an infinite-duration impulse response (IIR) filter for bandpass filtering. The design of the IIR filter has the advantages of less computing resource requirements and efficient frequency domain response. The implementation of the filter is based on a difference equation, and the output depends on the linear combination of the current input sample and the past input and output samples, as shown below:

[0085]

[0086] Among them, y[n] is the output of the filter, x[n] is the input original EEG signal (original EEG data), a j and b k are filter coefficients, M and N are filter orders, k and j are cumulative calculation variables, nk and nj are corresponding cumulative index values, and n is the index of the window.

[0087] In the embodiment of the present invention, by setting the high-pass and low-pass cutoff frequencies of the IIR filter, bandpass filtering in the range of 1-44 Hz can be achieved. This process not only effectively removes the low-frequency drift and high-frequency noise in the signal, but also retains the frequency band components that reflect the main information in the EEG activity, ensuring the effectiveness and accuracy of subsequent analysis.

[0088] Based on the EEG signal preprocessed by bandpass filtering, in order to further remove artifacts in independent source signals, such as non-neural activity artifacts such as eye movements, electromyography, and electronic interference, the present invention uses independent component analysis (ICA) for processing. ICA is a method for separating independent components contained in a signal to extract signal features or remove noise. Here, by analyzing the time-frequency characteristics and spatial distribution of independent components, the components related to artifacts are manually or automatically marked and removed, and then the EEG signal without mixed noise is reconstructed. This processing step can improve the signal-to-noise ratio of EEG data and ensure the reliability and accuracy of subsequent signal analysis.

[0089] In order to further extract the frequency components of the signal and analyze its spectral characteristics, the Fast Fourier Transform (FFT) was used to perform spectral analysis on the EEG signal data after removing artifacts. The time domain signal is converted into a frequency domain representation, and the power spectral density (PSD) of the specified frequency on each channel is calculated to effectively analyze the contribution of each frequency component in the signal. Before spectral analysis, the artifact-free EEG signal is first divided into 129 frequency intervals. Each interval corresponds to an FFT coefficient, which represents the power of the frequency component. In order to ensure the accuracy and resolution of frequency analysis, the present invention uses a Hamming window function with 256 sampling points for windowing, and the sampling rate is set to 220Hz. After each FFT calculation, the window slides forward 22 sample points to form a 90% overlapping window (i.e., the window slides once every 10 milliseconds).

[0090] This process ensures continuity and high temporal resolution of the spectral analysis, which is defined below.

[0091]

[0092] Where ω(n) is the time domain signal processed by the preset window function, n is the index of the window, and L is the length of the window. Given the original discrete signal x(n), before performing FFT calculation on it, the Hamming window function is applied to the signal to generate the windowed time domain signal x ω (n), the expression is as follows,

[0093] x ω (n) = x(n)·ω(n), (7)

[0094] The processed time domain signal is subjected to fast Fourier transform to obtain the frequency domain signal, which is expressed as follows:

[0095]

[0096] Among them, X(k) is the frequency domain signal after Fourier transformation, N is the sample size of the input signal, k is the input sample index, and n is the index of the window. is the complex exponential kernel function in the frequency domain, and j is the preset coefficient. The following is the calculation expression of the power spectral density (PSD) and the calculation expression of the absolute power value in the specified frequency range.

[0097]

[0098] where |X(k)| 2is the square of the modulus of X(k), indicating the power of the kth frequency component; PSD(k) indicates the power distributed per unit frequency, i.e., the power spectral density; k1 and k2 correspond to the start and end indexes of the specified frequency band, respectively; f s is the sampling rate, and k is the input sample index.

[0099] S4: Construct an immersion prediction model based on machine learning and polynomial regression algorithm. The prediction model uses the absolute power value of the specified frequency and the distortion level index calculated in S3 as input features, combines the input features through an exponential relationship, defines the immersion index, and uses the polynomial regression algorithm to fit the correlation between the predicted value and the immersion index to obtain the polynomial expression of the prediction model.

[0100] S5: Correlation analysis was performed between IPQ and the predicted value, and the coefficient of determination R was used to 2 Evaluate the goodness of fit expressed by the polynomial model and further optimize the prediction model.

[0101] In this invention, in order to quantify and characterize the user's immersion in the virtual reality experience, an immersion prediction model based on machine learning polynomial regression algorithm is constructed. The model takes the absolute power value P of the specified frequency band in the EEG signal data and the corresponding distortion index Distortion as input features to capture the potential psychological and physiological correlation in the user's subjective experience. Given that these two input features are in a corresponding relationship in the data set, in order to achieve dimensionality reduction and simplify the model expression, the present invention combines the two features through an exponential relationship to define the optimal immersion index (Immersion index, I IMS ), as shown in the following expression.

[0102] I IMS =P a (1-Distortion) b , (11)

[0103] Wherein, a and b are fitting parameters, which are taken as 0.4 and 0.9 respectively in the present invention. The nonlinear relationship between this index and subjective immersion is fitted using a polynomial regression algorithm to ensure that the model can adapt to complex feature interactions. The following is the second-order polynomial model expression of the immersion prediction model in the present invention:

[0104]

[0105] in, is the predicted value of the immersion model. The model can analyze the quantitative impact of input features on immersion, thereby predicting user immersion. Given the high reliability and validity of IPQ subjective scores in immersion evaluation, the model uses it as the best indicator for measuring immersion and compares it with the predicted value.

[0106] Figure 5 The relationship between the predicted value of the model of the present invention and the actual IPQ score is shown.

[0107] like Figure 5 As shown in the figure, Fitted curve is the fitting curve, Sample points are the sample points, Subjective IPQscore is the actual IPQ score (i.e., subjective perceived quality evaluation IPQ), 95%Confidence interval is the selected 95% confidence interval, and Immersion index is the variable corresponding to the immersion index. It can be seen that the fitting curve coincides with the confidence interval, and the determination coefficient at this time is R 2 =0.6167.

[0108] By R 2 To assess the goodness of fit of the model:

[0109]

[0110] Among them, n1 is the amount of evaluation data, y i is the actual value of the i-th evaluation data, is the predicted value for the i-th evaluation data, For all y i The determination coefficient R of the prediction model of the present invention 2 It is 0.6167, indicating that the immersion prediction model can accurately predict the user's immersion level and has strong robustness and wide applicability.

[0111] It should be noted that the designated frequency band is the designated frequency.

[0112] Figure 6 A block diagram of a virtual scene immersion prediction system based on EEG signals of the present invention is shown.

[0113] The second aspect of the present invention further provides a virtual scene immersion prediction system 6 based on EEG signals, the system comprising: a memory 61 and a processor 62, wherein the memory 61 comprises a virtual scene immersion prediction program based on EEG signals, and when the virtual scene immersion prediction program based on EEG signals is executed by the processor, the following steps are implemented:

[0114] S1: Based on the principle of binocular stereo imaging, in the process of acquiring, displaying and perceiving stereo images, parameter mismatch may lead to spatial distortion. By adjusting the camera field of view, different degrees of spatial distortion are simulated, and the corresponding distortion level index is set. According to different degrees of distortion, spatial distortion models at all levels are constructed;

[0115] S2: Design a human-computer interaction experiment based on virtual reality to collect the subjective perception quality evaluation IPQ of different distortion models and the corresponding objective original EEG signal data;

[0116] S3: Perform IIR filter preprocessing on the original EEG data to obtain preprocessed EEG signals, separate spatially independent signals from the preprocessed EEG signals based on the ICA analysis method to remove mixed noise in the EEG signals, reconstruct the preprocessed EEG signals to obtain artifact-free EEG signals, perform spectrum analysis on the artifact-free EEG signals through fast Fourier transform, and convert the signals into frequency domain for analysis, calculate the power spectral density PSD of the specified frequency on each channel, and calculate the corresponding absolute power value through the power spectral density PSD;

[0117] S4: Construct an immersion prediction model based on machine learning and polynomial regression algorithm. The prediction model uses the absolute power value of the specified frequency and the distortion level index calculated in S3 as input features, combines the input features through an exponential relationship, defines the immersion index, and uses the polynomial regression algorithm to fit the correlation between the predicted value and the immersion index to obtain the polynomial expression of the prediction model.

[0118] S5: Correlation analysis was performed between IPQ and the predicted value, and the coefficient of determination R was used to 2 Evaluate the goodness of fit expressed by the polynomial model and further optimize the prediction model.

[0119] The third aspect of the present invention also provides a computer-readable storage medium, which includes a virtual scene immersion prediction program based on EEG signals. When the virtual scene immersion prediction program based on EEG signals is executed by a processor, the steps of the virtual scene immersion prediction method based on EEG signals as described in any of the above items are implemented.

[0120] The fourth aspect of the present invention also provides a VR platform device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and is characterized in that when the processor executes the computer program, the steps of the virtual scene immersion prediction method based on EEG signals as described above are implemented.

[0121] Based on the principle of binocular stereo imaging, the present invention constructs spatial distortion models of different degrees by adjusting the mismatch of parameters in the acquisition, processing, display and perception of stereo images. The present invention aims to combine subjective scoring data with objective physiological data to deeply explore the influence mechanism of different degrees of spatial distortion on visual perception immersion, so as to more accurately predict and reflect the actual experience level of individuals. At the same time, a polynomial regression algorithm is used to analyze and model the collected subjective and objective data, and a prediction model of virtual space distortion is constructed, so as to accurately and intuitively predict the immersive experience under different distortion conditions.

[0122] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0123] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0124] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0125] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0126] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0127] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for predicting immersion in a virtual scene based on EEG signals, characterized in that: include: S1: Based on the principle of binocular stereo imaging, in the process of acquiring, displaying and perceiving stereo images, parameter mismatch may lead to spatial distortion. By adjusting the camera field of view, different degrees of spatial distortion are simulated, and the corresponding distortion level index is set. According to different degrees of distortion, spatial distortion models at all levels are constructed; S2: Design a human-computer interaction experiment based on virtual reality to collect the subjective perception quality evaluation IPQ of different distortion models and the corresponding objective original EEG signal data; S3: Perform IIR filter preprocessing on the original EEG data to obtain preprocessed EEG signals, separate spatially independent signals from the preprocessed EEG signals based on the ICA analysis method to remove mixed noise in the EEG signals, reconstruct the preprocessed EEG signals to obtain artifact-free EEG signals, perform spectrum analysis on the artifact-free EEG signals through fast Fourier transform, and convert the signals into frequency domain for analysis, calculate the power spectral density PSD of the specified frequency on each channel, and calculate the corresponding absolute power value through the power spectral density PSD; S4: Construct an immersion prediction model based on machine learning and polynomial regression algorithm. The prediction model uses the absolute power value of the specified frequency and the distortion level index calculated in S3 as input features, combines the input features through an exponential relationship, defines the immersion index, and uses the polynomial regression algorithm to fit the correlation between the predicted value and the immersion index to obtain the polynomial expression of the prediction model. S5: Correlation analysis was performed between IPQ and the predicted value, and the coefficient of determination R was used to 2 Evaluate the goodness of fit expressed by the polynomial model and further optimize the prediction model.

2. The method for predicting virtual scene immersion based on EEG signals according to claim 1, characterized in that: In S1, stereoscopic imaging is an imaging analysis based on virtual reality, which includes three processes: acquisition, display and perception, and each process corresponds to independent parameter collection.

3. The method for predicting virtual scene immersion based on EEG signals according to claim 2, characterized in that: The S1 includes: The stereoscopic imaging device includes a left camera and a right camera. A three-dimensional coordinate system is established with the midpoint of the left and right cameras as the origin, which is defined as the object space. The coordinates are expressed as (X o ,Y o ,Z o ), a three-dimensional coordinate system is established with the center position of the left and right eyes of the subject as the origin, and it is defined as the image space. The coordinates are expressed as (X i ,Y i ,Z i ), a geometric mapping relationship is established between the object space and the image space, and the relationship is expressed as follows: Where M is the magnification factor for mapping the two-dimensional image on the camera sensor to the 3D display device; b e and b c Respectively represent the pupil distance of human eyes and the distance between binocular cameras; f represents the focal length of the camera; V d is the distance between the human eye and the 3D display device; s is the camera sensor offset value, W c is the horizontal width of the camera sensor, C d is the distance between the intersection of the left and right camera optical axes and the camera center point, that is, the camera convergence distance. The plane at this camera convergence distance is called the zero parallax plane. o is the horizontal width of the camera’s shooting range on the zero parallax plane, C d , W o , b c The relationship between the three is shown in the following formula: Among them, λ is the camera field of view angle. By changing the camera field of view angle, different distortion degrees and corresponding distortion level indexes are formed. The following is the calculation expression of the distortion level index: Among them, Distortion represents the distortion level index.

4. The method for predicting virtual scene immersion based on EEG signals according to claim 3, characterized in that: In S3, before performing IIR filter preprocessing, high-pass and low-pass cutoff frequencies are set for the IIR filter.

5. The method for predicting virtual scene immersion based on EEG signals according to claim 4, characterized in that: In the S3, specifically: IIR filter is based on the difference equation, and the output depends on the linear combination of the current input sample and the past input and output samples. The expression is as follows: Among them, y[n] is the output of the filter, x[n] is the input raw EEG data, and a j and b are filter coefficients, M and N are filter orders, k and j are cumulative calculation variables, nk and nj are corresponding cumulative index values, and n is the index of the window; Before the spectrum analysis, the artifact-free EEG signal was divided into 129 frequency intervals, each interval corresponding to an FFT coefficient, and a Hamming window function with 256 sampling points was used for windowing. The sampling rate was set to 220 Hz. After each FFT calculation, the window slid forward 22 sample points to form a 90% overlapping window. FFT is fast Fourier transform, and the expression is as follows: Where ω(n) is the time domain signal processed by the preset window function, n is the index of the window, and L is the length of the window. Given the original discrete signal x(n), before performing FFT calculation on it, the Hamming window function is applied to the signal to generate the windowed time domain signal x ω (n), the expression is as follows, x ω (n)=x(n)·ω(n), The windowed time domain signal is subjected to fast Fourier transform to obtain the frequency domain signal. The expression is as follows: Among them, X(k) is the frequency domain signal after Fourier transformation, N is the sample size of the input signal, k is the input sample index, and n is the index of the window. is the complex exponential kernel function in the frequency domain, j1 is the preset coefficient, and the following is the calculation expression of the power spectral density PSD and the calculation expression of the absolute power value P in the specified frequency range. where |X(k)| 2 is the square of the modulus of X(k), representing the power of the kth frequency component; PSD(k) represents the power distributed per unit frequency, i.e., power spectral density; k1 and k2 correspond to the start and end indexes of the specified frequency band, respectively; f s is the sampling rate, and k is the input sample index.

6. The method for predicting virtual scene immersion based on EEG signals according to claim 5, characterized in that: In S4, the immersion index is expressed in the polynomial model, and the specific expression is as follows: I IMS =P a ·(1-Distortion) b , Among them, a and b are fitting parameters, which are 0.4 and 0.9 respectively, P is the absolute power value, Distortion is the distortion level index, I IMS is the immersion index. Further, the nonlinear relationship between the immersion index and subjective immersion is fitted based on the polynomial regression algorithm. The polynomial model expression is as follows: in, Predict values ​​for the immersion model.

7. The method for predicting virtual scene immersion based on EEG signals according to claim 6, characterized in that: In S5, the determination coefficient R 2 The specific expression to be evaluated is: Among them, n1 is the amount of evaluation data, y i is the actual value of the i-th evaluation data, is the predicted value for the i-th evaluation data, For all y i The mean of .

8. A virtual scene immersion prediction system based on EEG signals, characterized in that: The system includes: a memory and a processor, wherein the memory includes a virtual scene immersion prediction program based on EEG signals, and when the virtual scene immersion prediction program based on EEG signals is executed by the processor, the following steps are implemented: S1: Based on the principle of binocular stereo imaging, in the process of acquiring, displaying and perceiving stereo images, parameter mismatch may lead to spatial distortion. By adjusting the camera field of view, different degrees of spatial distortion are simulated, and the corresponding distortion level index is set. According to different degrees of distortion, spatial distortion models at all levels are constructed; S2: Design a human-computer interaction experiment based on virtual reality to collect the subjective perception quality evaluation IPQ of different distortion models and the corresponding objective original EEG signal data; S3: Perform IIR filter preprocessing on the original EEG data to obtain preprocessed EEG signals, separate spatially independent signals from the preprocessed EEG signals based on the ICA analysis method to remove mixed noise in the EEG signals, reconstruct the preprocessed EEG signals to obtain artifact-free EEG signals, perform spectrum analysis on the artifact-free EEG signals through fast Fourier transform, and convert the signals into frequency domain for analysis, calculate the power spectral density PSD of the specified frequency on each channel, and calculate the corresponding absolute power value through the power spectral density PSD; S4: Construct an immersion prediction model based on machine learning and polynomial regression algorithm. The prediction model uses the absolute power value of the specified frequency and the distortion level index calculated in S3 as input features, combines the input features through an exponential relationship, defines the immersion index, and uses the polynomial regression algorithm to fit the correlation between the predicted value and the immersion index to obtain the polynomial expression of the prediction model. S5: Correlation analysis was performed between IPQ and the predicted value, and the coefficient of determination R was used to 2 Evaluate the goodness of fit expressed by the polynomial model and further optimize the prediction model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a virtual scene immersion prediction program based on EEG signals. When the virtual scene immersion prediction program based on EEG signals is executed by a processor, the steps of the virtual scene immersion prediction method based on EEG signals as described in any one of claims 1 to 7 are implemented.

10. A VR platform device, characterized in that: The VR platform device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the virtual scene immersion prediction method based on EEG signals as described in any one of claims 1 to 7 when executing the computer program.

Images