Stereoscopic vision comfort feature extraction system and method based on sparrow search algorithm

Abstract

The invention relates to a sparrow search algorithm-based stereoscopic vision comfort feature extraction system and method, and belongs to the field of stereoscopic vision comfort feature extraction. The system comprises a data preprocessing module, a feature extraction module and a feature optimization module. According to the method, EMD and Hilbert transform in an HHT method are used for adaptively extracting time-frequency features of EEG signals, and then a spatial filter in a CSP method is used for obtaining time-frequency features of multiple spatial dimensions. The two combined methods not only can adaptively obtain the time-frequency characteristics, but also can reveal the interaction between different brain regions in the stereoscopic vision transmission process, and more comprehensively understand the stereoscopic vision comfort characteristics based on the EEG signals. The SSA algorithm is utilized to simulate foraging behaviors and anti-capturing behaviors of sparrows and an early warning mechanism, so that the features can be accurately and quickly optimized, the optimal feature combination can be selected, the time-frequency feature redundancy phenomenon is reduced, and the precision and the performance of the stereoscopic vision comfort classification model are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of stereoscopic vision comfort feature extraction, and in particular relates to a stereoscopic vision comfort feature extraction system and method based on a sparrow search algorithm. Background Art

[0002] Stereoscopic visual comfort evaluation based on electroencephalogram (EEG) signals is widely used in fields such as virtual reality (VR) and augmented reality (AR). In order to improve user experience and reduce visual discomfort, we conduct in-depth research on the construction of stereoscopic visual comfort evaluation model. In this process, feature extraction of stereoscopic visual comfort is crucial, which is related to the accuracy and reliability of model prediction.

[0003] At present, there are many methods for extracting stereoscopic visual comfort features, such as common spatial pattern (CSP), independent component analysis (ICA), wavelet transform, and convolutional neural network (CNN) and recurrent neural network (RNN) in deep learning (DL), which provide diverse and effective feature extraction methods. CSP is good at extracting spatial distribution features from multi-channel EEG data, ICA can separate independent signal components, wavelet transform can capture frequency information at different time scales, and CNN and RNN can automatically learn and extract complex features related to stereoscopic visual comfort.

[0004] However, in the process of EEG feature extraction, if the range of extracted features is too narrow or limited to specific aspects, it may lead to the loss of key information and limited model generalization ability, which means that the model may not be able to give reasonable prediction results when processing new and unknown data, thereby affecting the classification accuracy. On the other hand, when the feature extraction party extracts too many features, the training time of the model will increase significantly, which will not only increase the computing resources and time cost, but also increase the risk of overfitting the model on the training data. Therefore, when constructing a stereoscopic visual comfort classification model for EEG, it is necessary to adaptively extract features and select the optimal feature combination to ensure that the model maintains accuracy while reducing computational complexity and avoiding overfitting problems. Summary of the invention

[0005] The present invention aims to solve the technical problems of limited model generalization ability and overfitting in the prior art when constructing a stereoscopic visual comfort evaluation model, and provides a stereoscopic visual comfort feature extraction system and method based on a sparrow search algorithm. The system and method of the present invention are mainly used to adaptively extract and select the optimal stereoscopic visual comfort feature.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] A stereoscopic visual comfort feature extraction system based on a sparrow search algorithm, comprising a data preprocessing module, a feature extraction module and a feature optimization module connected in sequence;

[0008] The data preprocessing module is used to preprocess the EEG signal to remove noise and interference;

[0009] The feature extraction module adaptively extracts time-frequency features in multiple brain regions from the electroencephalogram (EEG) signal preprocessed by the data preprocessing module through the Hilbert-Huang transform (HHT) and common spatial pattern (CSP) methods;

[0010] The feature optimization module optimizes the time-frequency features adaptively extracted by the feature extraction module through a sparrow search algorithm (SSA) and selects the optimal feature combination.

[0011] In the above technical solution, the Hilbert-Huang transform (HHT) includes empirical mode decomposition (EMD) and Hilbert transform.

[0012] In the above technical solution, the empirical mode decomposition (EMD) and Hilbert transform in the Hilbert-Huang transform (HHT) are used to adaptively extract the time-frequency features in the electroencephalogram (EEG) signal, and then the spatial filter in the common spatial pattern (CSP) method is used to obtain the time-frequency features of multiple spatial dimensions.

[0013] A stereoscopic vision comfort adaptive feature extraction method of a stereoscopic vision comfort feature extraction system based on a sparrow search algorithm comprises the following steps:

[0014] Step 1, the data preprocessing module preprocesses the EEG signal to remove noise and interference to obtain a preprocessed EEG signal;

[0015] Step 2, the feature extraction module adaptively extracts the time-frequency features of multiple brain regions in the preprocessed electroencephalogram (EEG) signal obtained in step 1 by using Hilbert-Huang transform (HHT) and common spatial pattern (CSP) methods;

[0016] Step 3: The feature optimization module optimizes the time-frequency features extracted in step 2 through a sparrow search algorithm (SSA) and selects the optimal feature combination.

[0017] In the above technical solution, the Hilbert-Huang transform (HHT) in step 2 includes empirical mode decomposition (EMD) and Hilbert transform;

[0018] First, the preprocessed electroencephalogram (EEG) signal obtained in step 1 is decomposed into a series of intrinsic mode functions (IMFs) through empirical mode decomposition (EMD);

[0019] Then, each intrinsic mode function (IMF) is Hilbert transformed and its instantaneous frequency and amplitude are adaptively calculated to obtain the time-frequency characteristics.

[0020] In the above technical solution, the processed raw EEG signal The specific steps of decomposing into a series of intrinsic mode functions (IMFs) through empirical mode decomposition (EMD) are as follows:

[0021] Step 2.1: Find The local maximum and minimum points of ; then, the cubic spline interpolation method is used to fit these extreme points; finally, we get The average value of the upper and lower envelopes , using raw EEG signals Subtract the mean get :

[0022] ;

[0023] in, is the first approximation component;

[0024] Step 2.2: The number of extreme points is equal to the number of zero points, or the absolute value of the difference between the two is one; and The average value of the upper and lower envelopes of the signal is zero;

[0025] like When the above conditions are met, it is the first intrinsic mode function (IMF), denoted as ; From the raw EEG signal Bundle Separate it out :

[0026] ;

[0027] in, is the first remainder;

[0028] like If the above conditions are not met, repeat step 2.1 until Until the above conditions are met;

[0029] Step 2.3: Judgment Whether the signal can be further decomposed;

[0030] If it can be decomposed, Repeat steps 2.1 and 2.2 as new raw EEG signals until all intrinsic mode functions (IMFs) are successfully extracted;

[0031] like No longer meets the decomposition conditions 0.3, the iteration process stops:

[0032] ;

[0033] in, is the standard deviation, is the i-th approximation component, is the i-1th approximate component, T is the total length of the signal;

[0034] Step 2.4: Repeat steps 2.1 to 2.3 until the Kth intrinsic mode function (IMF) component is obtained. The final empirical mode decomposition (EMD) result is expressed as:

[0035] ;

[0036] in, For the The intrinsic mode function (IMF) components obtained are is the total number of times, For the remainder.

[0037] In the above technical solution, the steps of performing Hilbert transform on each intrinsic mode function (IMF) and adaptively calculating its instantaneous frequency and instantaneous amplitude are as follows:

[0038] Step 2.5: Perform Hilbert transform on each intrinsic mode function (IMF) component decomposed by empirical mode decomposition (EMD):

[0039] ;

[0040] At the same time, the analytical signal is constructed as:

[0041] ;

[0042] in, is the normalization factor, is the integral variable introduced when the IMF component is transformed into an integral. is the time variable, are different phase shifts, is the analytical signal obtained for the i-th time, is the IMF component obtained for the ith time, for The Hilbert transform of is the imaginary unit, is the instantaneous amplitude obtained for the ith time, is the instantaneous phase obtained for the i-th time;

[0043] Therefore, the instantaneous amplitude is obtained by the following formula and instantaneous phase :

[0044] ;

[0045] ;

[0046] Then find the instantaneous frequency :

[0047] ;

[0048] Step 2.6: Convert the original EEG signal into It is expressed as:

[0049] ;

[0050] in, []express The real part of the signal; is the residual signal, which is an indecomposable signal in the empirical mode decomposition (EMD), so this part can be omitted; is an exponential function, is the imaginary unit;

[0051] The time-frequency characteristics of a single subject are obtained through Hilbert-Huang transform (HHT) post-adaptation , according to the comfort and discomfort of stereoscopic vision Divide and Two categories, including represents the time-frequency feature set under the comfortable state, Represents the time-frequency feature set under discomfort state.

[0052] In the above technical solution, the steps of the common space pattern (CSP) method are as follows:

[0053] Step 2.7: Calculation and Covariance matrix of two types of time-frequency feature matrices:

[0054] ;

[0055] in, express The transpose of the feature matrix, It means summing the elements on the diagonal of the characteristic matrix and solving for the mixed space covariance matrix ,use and They represent the expectation of the spatial covariance matrix of the time-frequency feature set samples in a comfortable state and the time-frequency feature set samples in an uncomfortable state respectively;

[0056] Step 2.8: Use the singular value decomposition theorem to calculate the covariance matrix of the mixed space Perform feature decomposition and whiten transformation get ;

[0057] yes The transposed matrix of is the eigenvector matrix;

[0058] Step 2.9: Whiten the eigenvalue matrix Acts on and :

[0059] ;

[0060] ;

[0061] in, yes The transposed matrix of is the diagonal matrix of eigenvalues, is the covariance matrix of the first set of data, is the covariance matrix of the second set of data, yes The transformed covariance matrix, yes The transformed covariance matrix, is the whitened eigenvalue matrix The transpose of

[0062] and have common eigenvectors and there are two diagonal matrices and and the same eigenvector :

[0063] ;

[0064] ;

[0065] ;

[0066] Among them, the two diagonal matrices , The sum is the identity matrix , is the eigenvector The transpose of

[0067] Step 2.10: If The eigenvalues ​​of are arranged in descending order, then The eigenvalues ​​of will be arranged in ascending order; so for the eigenvector ,when When the eigenvalue is the largest, has the smallest eigenvalue, so using the eigenvector Realize the classification of two types of problems, thereby obtaining the projection matrix :

[0068] ;

[0069] Among them, the projection matrix is the corresponding spatial filter, is the eigenvector The transpose of

[0070] Step 2.11: Get the EEG signal Through the projection matrix Projection to obtain the feature matrix , and the final features are obtained by normalization:

[0071] ;

[0072] in, is the variance of the feature, It is The normalized feature matrix of samples is is the feature matrix In the above technical solution, step 3 includes the following steps:

[0073] Step 3.1, Initialization:

[0074] Set the parameters of the sparrow search algorithm (SSA), randomly initialize the sparrow population, each sparrow represents a feature combination, and its position is composed of a set of feature values;

[0075] A population of sparrows is represented by:

[0076] ;

[0077] in, Indicates the variable dimension to be optimized;

[0078] Step 3.2, evaluate fitness:

[0079] For each sparrow, a fitness function is used to evaluate the stereoscopic visual comfort classification performance, and the fitness value of each sparrow is obtained and sorted by fitness value;

[0080] The fitness value of a sparrow is:

[0081] ;

[0082] in, represents the fitness value;

[0083] Step 3.3, role allocation:

[0084] According to the fitness value, sparrows are divided into finders, followers and vigilants;

[0085] Finders are sparrows with higher fitness values, and they will lead the search direction;

[0086] Followers are sparrows with relatively low fitness values, and they will follow the discoverer in the search;

[0087] Sentinels are responsible for monitoring the environment and issuing warnings when potential threats are detected;

[0088] Step 3.4, Update location:

[0089] The discoverer focuses on the local optimization of the current solution space, and also focuses on finding the global optimal feature combination in a larger range; the follower dynamically adjusts according to the discoverer's position, allowing the follower to jump out of the current search area; the vigilant adjusts the search behavior according to the safety status of the environment, and when the environment is unsafe, it adopts strategies to avoid falling into the local optimum;

[0090] Finder location update:

[0091] ;

[0092] in, is the current iteration number, represents the maximum number of iterations, ∈(0,1] is a random number, and Represent the warning value and safety value respectively. , ; is a random number that follows a normal distribution, Indicates a The matrix of is the j-th dimension position of the i-th sparrow in the t-th iteration, is the j-th dimension position of the i-th sparrow in the t+1-th iteration;

[0093] Follower location update:

[0094] ;

[0095] in, is the optimal position currently occupied by the discoverer, Indicates the current global worst position; Indicates a Each element of the d matrix is ​​randomly assigned a value of 1 or -1, and ;when When the fitness value is lower, Followers need to update their positions;

[0096] Step 3.5: Simulate anti-predator behavior:

[0097] During the search process, when the sentinel detects a potential threat, it will send out a warning signal; the sparrow that receives the warning signal will adjust its search strategy by increasing the search step size or changing the search direction to escape the potential predator, which is the local optimal solution;

[0098] ;

[0099] in, is the global optimal position in the current iteration, To adjust the step size factor; Indicates the direction in which the sparrow moves. is a constant, and Respectively represent the optimal and worst fitness values ​​of the current population, Indicates The fitness value of a sparrow;

[0100] Step 3.6, iterative optimization:

[0101] In each iteration, steps 3.2 to 3.5 are repeated to optimize the feature combination by continuously updating the position and role assignment of the sparrows. As the iteration proceeds, the sparrow search algorithm (SSA) will gradually approach the optimal feature combination.

[0102] Step 3.7, select the optimal feature combination:

[0103] After the iteration, the optimal feature combination is selected according to the fitness value.

[0104] The beneficial effects of the present invention are:

[0105] The stereoscopic visual comfort feature extraction system and method based on the sparrow search algorithm of the present invention have the following advantages:

[0106] The present invention uses EMD and Hilbert transform in the HHT method to adaptively extract the time-frequency features of EEG signals, and then uses the spatial filter in the CSP method to obtain the time-frequency features of multiple spatial dimensions. These two combined methods can not only adaptively obtain time-frequency features, but also reveal the interaction between different brain regions during stereoscopic vision transmission, and more comprehensively understand the stereoscopic vision comfort characteristics based on EEG signals.

[0107] The present invention utilizes the SSA algorithm to simulate the foraging behavior, anti-predation behavior and early warning mechanism of sparrows to accurately and quickly optimize features and select the optimal feature combination, reduce the redundancy of time-frequency features, and thus improve the accuracy and performance of the stereoscopic visual comfort classification model.

[0108] The present invention can adaptively divide the frequency band:

[0109] The Hilbert-Huang transform used adaptively divides the frequency band according to the characteristics of the signal itself, rather than relying on a fixed frequency band division method. This adaptive frequency band division can not only capture the key information in the signal more accurately, avoid information loss and precision reduction, but also improve the generalization ability of the model.

[0110] The present invention can reduce redundant features:

[0111] The present invention adopts the sparrow search algorithm to simulate the sparrow's foraging, anti-prediction, early warning and other behaviors in feature selection, and iteratively searches to find the features most relevant to stereoscopic visual comfort. This method reduces redundant features, shortens training time, reduces computational complexity and resource requirements, and effectively avoids overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0113] Figure 1 It is a structural schematic diagram of the stereoscopic visual comfort feature extraction system based on the sparrow search algorithm of the present invention.

[0114] Figure 1 Middle: CSP stands for common spatial mode and IMF stands for intrinsic mode function. DETAILED DESCRIPTION

[0115] The inventive concept of the present invention is: the stereoscopic visual comfort feature extraction method in the prior art is to divide the frequency bands in a fixed manner. In order to study the neural activities in different frequency ranges, the EEG signals are usually fixedly divided into different frequency bands (such as theta, alpha, beta, etc.). However, this fixed division method cannot dynamically reflect the EEG signal characteristics in the brain cognitive process, because the EEG activity may show more subtle spectral changes under different cognitive tasks. This limitation limits the effectiveness of feature extraction, resulting in information loss when the model captures key information, thereby reducing the accuracy of model classification and affecting the generalization ability of the model. There is also a feature redundancy phenomenon: in order to comprehensively extract the relevant features of stereoscopic visual comfort, the existing feature extraction method usually extracts features from multiple dimensions such as time, frequency, and space, or extracts a large number of features under one dimension, which may include irrelevant or redundant features. This feature redundancy phenomenon not only increases the computational complexity and resource requirements of the model, prolongs the training time, but also may carry interference information, affect the classification accuracy of the model, and even cause overfitting problems. The stereoscopic visual comfort feature extraction system and method based on the sparrow search algorithm of the present invention adopts the Hilbert-Huang transform to adaptively divide the frequency band according to the characteristics of the signal itself, rather than relying on a fixed frequency band division method. This adaptive frequency band division can not only capture the key information in the signal more accurately, avoid information loss and reduced accuracy, but also improve the generalization ability of the model. The sparrow search algorithm is used in feature selection to simulate the foraging, anti-prediction, and early warning behaviors of sparrows, and iteratively search to find the features most relevant to stereoscopic visual comfort. This method shortens the training time, reduces the computational complexity and resource requirements, and effectively avoids overfitting by reducing redundant features.

[0116] The present invention is described in detail below with reference to the accompanying drawings.

[0117] Combination Figure 1 Specifically describe the stereoscopic visual comfort feature extraction system based on the sparrow search algorithm of the present invention, including a data preprocessing module, a feature extraction module and a feature optimization module connected in sequence;

[0118] The data preprocessing module is used to preprocess the EEG signal to remove noise and interference;

[0119] EEG signals are easily contaminated by various noises during the acquisition process, such as electromyographic interference, electrooculographic interference, etc. These noises will seriously affect the quality and reliability of the signal. Therefore, before feature extraction, the main purpose of preprocessing the EEG signal is to ensure the quality of the signal, remove noise and interference, and thus improve the accuracy of subsequent feature extraction.

[0120] The feature extraction module adaptively extracts time-frequency features in multiple brain regions from the EEG signal preprocessed by the data preprocessing module through Hilbert-Huang Transform (HHT) and Common Spatial Pattern (CSP) methods;

[0121] HHT transformation includes empirical mode decomposition (EMD) and Hilbert transform. First, the original EEG signal is decomposed into a series of intrinsic mode functions (IMFs) through EMD. These IMFs represent the oscillation modes of different frequency components in the signal and have good anti-noise performance. Then, each IMF is Hilbert transformed and its instantaneous frequency and instantaneous amplitude are adaptively calculated to reveal the dynamic characteristics of the EEG signal. At the same time, HHT is not limited by linearity and stationarity, and has great advantages in processing non-stationary and nonlinear signals.

[0122] The feature optimization module optimizes the time-frequency features adaptively extracted by the feature extraction module through a sparrow search algorithm (SSA) and selects the optimal feature combination.

[0123] The time-frequency features obtained by the HHT and CSP methods have feature redundancy, because EMD can decompose the EEG signal into many IMF components, and each IMF component contains different effective components. The instantaneous time-frequency features of each IMF are obtained through Hilbert transform. This process will generate a large number of time-frequency features; the CSP method will also extract too many features when there are a large number of channels or complex task categories. Therefore, it is necessary to optimize the time-frequency features and select representative feature combinations to improve the accuracy of the stereoscopic visual comfort classification model, reduce training time and calculation, and avoid overfitting.

[0124] The SSA algorithm has a unique optimization mechanism and powerful global search capability. By combining the foraging and anti-predation behaviors of sparrows and the behavioral patterns of different roles, it can effectively search and optimize feature combinations in complex feature spaces, reduce feature redundancy, and improve model classification performance.

[0125] The stereoscopic visual comfort degree adaptive feature extraction method based on the stereoscopic visual comfort degree feature extraction system based on the sparrow search algorithm of the present invention comprises the following steps:

[0126] Step 1, the data preprocessing module preprocesses the EEG signal to remove noise and interference to obtain a preprocessed EEG signal;

[0127] EEG signals are easily contaminated by various noises during the acquisition process, such as electromyographic interference, electrooculographic interference, etc. These noises will seriously affect the quality and reliability of the signal. Therefore, before feature extraction, the main purpose of preprocessing the EEG signal is to ensure the quality of the signal, remove noise and interference, and thus improve the accuracy of subsequent feature extraction;

[0128] Step 2, the feature extraction module adaptively extracts the time-frequency features of multiple brain regions in the preprocessed electroencephalogram (EEG) signal obtained in step 1 by using Hilbert-Huang transform (HHT) and common spatial pattern (CSP) methods;

[0129] HHT transformation includes empirical mode decomposition (EMD) and Hilbert transform. First, the original EEG signal is decomposed into a series of intrinsic mode functions (IMFs) through EMD. These IMFs represent the oscillation modes of different frequency components in the signal and have good anti-noise performance. Then each IMF is Hilbert transformed and its instantaneous frequency and instantaneous amplitude are adaptively calculated to reveal the dynamic characteristics of the EEG signal. At the same time, HHT is not restricted by linearity and stationarity, and has great advantages in processing non-stationary and nonlinear signals. The specific steps of the empirical mode decomposition (EMD), Hilbert transform, and common spatial pattern (CSP) methods involved in this step are explained below.

[0130] Empirical Mode Decomposition:

[0131] The preprocessed raw EEG signal Decomposed into a series of IMFs, the specific steps are as follows:

[0132] Step 2.1: Find The local maximum and minimum points of ; then, the cubic spline interpolation method is used to fit these extreme points; finally, we get The average value of the upper and lower envelopes , using the original EEG signal Subtract the mean get :

[0133] ;

[0134] ;

[0135] Step 2.2: The number of extreme points is equal to the number of zero points, or the absolute value of the difference between the two is one; and The average value of the upper and lower envelopes of the signal is zero;

[0136] like When the above conditions are met, it is the first IMF, and the first IMF is recorded as ; From the raw EEG signal Bundle Separate it out :

[0137] ;

[0138] in, is the first remainder;

[0139] like If the above conditions are not met, repeat step 2.1 until Until the above conditions are met;

[0140] Step 2.3: Judgment Can the signal be further decomposed? If so, Repeat steps 2.1 and 2.2 as the new raw EEG signal until all IMFs are successfully extracted; No longer meets the decomposition conditions 0.3, the iteration process stops, where The expression is as follows:

[0141] ;

[0142] in, is the standard deviation, is the i-th approximation component, is the i-1th approximate component, T is the total length of the signal;

[0143] Step 2.4: Repeat steps 2.1 to 2.3 until the Kth IMF component is obtained. The final result of the empirical mode decomposition can be expressed as:

[0144] ;

[0145] in, For the The intrinsic mode function (IMF) components obtained are is the total number of times, For the remainder.

[0146] Perform Hilbert transform on each IMF component obtained above. The specific steps are as follows:

[0147] Step 2.5: Perform Hilbert transform on each IMF component after EMD decomposition:

[0148] ;

[0149] At the same time, the analytical signal is constructed as:

[0150] ;

[0151] in, is the normalization factor, is the integral variable introduced when the IMF component is transformed into an integral. is the time variable, are different phase shifts, is the analytical signal obtained for the i-th time, is the IMF component obtained for the ith time, for The Hilbert transform of is the imaginary unit, is the instantaneous amplitude obtained for the ith time, is the instantaneous phase obtained for the i-th time; thus, the instantaneous amplitude is obtained by the following formula and instantaneous phase :

[0152] ;

[0153] ;

[0154] Then find the instantaneous frequency :

[0155] ;

[0156] Step 2.6: The original EEG signal can be obtained by obtaining the instantaneous frequency and instantaneous amplitude of the IMF component. It is expressed as:

[0157] ;

[0158] in, []express The real part of the signal; is the residual signal, which is the signal that cannot be decomposed in EMD decomposition, so this part can be omitted; is an exponential function, is the imaginary unit;

[0159] The time-frequency characteristics of a single subject are obtained through adaptive HHT transformation , according to the comfort and discomfort of stereoscopic vision Divide and Two categories, including represents the time-frequency feature set under the comfortable state, Represents the time-frequency feature set under discomfort state.

[0160] HHT and CSP extract EEG signal features from different perspectives. HHT focuses on time-frequency feature extraction and can dynamically capture the nonlinear and non-stationary characteristics of EEG signals; CSP mainly distinguishes different EEG activity patterns by finding the optimal spatial filter. Combining the two, the time-frequency features of EEG signals can be fully and adaptively extracted from multiple dimensions to improve the accuracy and reliability of the visual comfort classification model. The specific steps of the CSP method are as follows:

[0161] Step 2.7: Calculation and Covariance matrix of two types of time-frequency feature matrices:

[0162] ;

[0163] in, express The transpose of the feature matrix, It means summing the elements on the diagonal of the characteristic matrix and solving for the mixed space covariance matrix ,use and They represent the expectations of the spatial covariance matrices of the time-frequency feature set samples in a comfortable state and the time-frequency feature set samples in an uncomfortable state, respectively.

[0164] Step 2.8, due to the mixed space covariance matrix It is a positive definite matrix, which is decomposed by the singular value decomposition theorem and whitened. available: ;

[0165] yes The transposed matrix of is the eigenvector matrix;

[0166] Step 2.9: Whiten the eigenvalue matrix Acts on and :

[0167] ;

[0168] ;

[0169] in, yes The transposed matrix of is the diagonal matrix of eigenvalues, is the covariance matrix of the first set of data, is the covariance matrix of the second set of data, yes The transformed covariance matrix, yes The transformed covariance matrix, is the whitened eigenvalue matrix The transpose of

[0170] and have common eigenvectors and there are two diagonal matrices and and the same eigenvector :

[0171] ;

[0172] ;

[0173] ;

[0174] Among them, the two diagonal matrices , The sum is the identity matrix , is the eigenvector The transpose of

[0175] Step 2.10: If The eigenvalues ​​of are arranged in descending order, then The eigenvalues ​​of will be arranged in ascending order. So for the eigenvector ,when When the eigenvalue is the largest, has the smallest eigenvalue, so we can use the eigenvector Realize the classification of two types of problems, thereby obtaining the projection matrix :

[0176] ;

[0177] Among them, the projection matrix is the corresponding spatial filter, is the eigenvector The transpose of

[0178] Step 2.11: Get the EEG signal The feature matrix is ​​obtained by projecting the projection matrix W. , and the final features are obtained by normalization:

[0179] ;

[0180] in, is the variance of the feature, It is The normalized feature matrix of samples is is the feature matrix The variance of .

[0181] Step 3, the feature optimization module optimizes the state time-frequency features extracted in step 2 through the sparrow search algorithm (SSA) and selects the optimal feature combination;

[0182] The time-frequency features obtained by the HHT and CSP methods have feature redundancy, because EMD can decompose the EEG signal into many IMF components, and each IMF component contains different effective components. The instantaneous time-frequency features of each IMF are obtained through Hilbert transform. This process will generate a large number of time-frequency features; the CSP method will also extract too many features when there are a large number of channels or complex task categories. Therefore, it is necessary to optimize the time-frequency features and select representative feature combinations to improve the accuracy of the stereoscopic visual comfort classification model, reduce training time and calculation, and avoid overfitting.

[0183] The SSA algorithm has a unique optimization mechanism and powerful global search capability. By combining the foraging and anti-predation behaviors of sparrows and the behavioral patterns of different roles, it can effectively search and optimize feature combinations in complex feature spaces, reduce feature redundancy, and improve model classification performance.

[0184] The specific operation steps of the SSA algorithm are:

[0185] Step 3.1, Initialization: Set the parameters of the sparrow search algorithm, such as population size (i.e., number of sparrows), number of iterations, search space range, etc. Randomly initialize the sparrow population, each sparrow represents a feature combination, and its position is composed of a set of feature values. A population of sparrows can be expressed as:

[0186] ;

[0187] in, Indicates the dimension of the variable to be optimized.

[0188] Step 3.2, evaluate fitness: For each sparrow, use the fitness function (classification accuracy, cross-validation score) to evaluate the stereoscopic visual comfort classification performance, obtain the fitness value of each sparrow and sort by fitness value. The fitness values ​​of all sparrows are:

[0189] ;

[0190] in, Represents the fitness value.

[0191] Step 3.3, role allocation: According to the fitness value, the sparrows are divided into discoverers, followers and guards. The discoverer has a higher fitness value (the higher fitness value is about 3.5487217452124e -18 ) will lead the search direction; the followers are sparrows with relatively low fitness values ​​(the lower fitness value is about 5.14977325147e -11 ) will follow the discoverer to search; that is, they are sorted by fitness value, the top 20% are discoverers and the last 80% are followers; the sentinels are responsible for monitoring the environment and issuing warnings when potential threats are found.

[0192] Step 3.4, update position: The discoverer focuses on the local optimization of the current solution space, and also focuses on finding the global optimal feature combination in a larger range; the follower dynamically adjusts according to the discoverer's position, so that the follower jumps out of the current search area. The vigilant adjusts the search behavior according to the safety of the environment. When the environment is unsafe, it adopts strategies to avoid falling into the local optimum.

[0193] Finder location update:

[0194] ;

[0195] in, is the current iteration number, represents the maximum number of iterations, ∈(0,1] is a random number. and They represent warning value and safety value respectively. is a random number that follows a normal distribution, Indicates a Matrix of is the j-th dimension position of the i-th sparrow in the t-th iteration, is the j-th dimension position of the i-th sparrow in the t+1-th iteration;

[0196] Follower location update:

[0197] ;

[0198] in, is the optimal position currently occupied by the discoverer, Indicates the current global worst position. Indicates a Each element of the d matrix is ​​randomly assigned a value of 1 or -1, and .when When the fitness value is lower, followers need to update their positions.

[0199] Step 3.5, simulate anti-predator behavior: In the search process, introduce an anti-predator mechanism. When the sentinel detects a potential threat (such as a sudden drop in fitness value or reaching a preset alarm threshold), it will issue a warning signal. The sparrow that receives the warning signal will adjust its search strategy, such as increasing the search step size or changing the search direction, to escape the potential "predator" (i.e., the local optimal solution).

[0200] ;

[0201] in, is the global optimal position in the current iteration, is the step size control parameter, which is a random number that obeys the standard normal distribution and is called the adjustment step size factor. Indicates the direction in which the sparrow moves. is a small constant to prevent the denominator from being zero. and Respectively represent the optimal and worst fitness values ​​of the current population, Indicates The fitness value of a sparrow.

[0202] Step 3.6, iterative optimization: In each iteration, repeat steps 3.2 to 3.5 to optimize the feature combination by continuously updating the position and role assignment of the sparrows. As the iteration proceeds, the algorithm will gradually approach the optimal feature combination.

[0203] Step 3.7, select the optimal feature combination: After the iteration, select the optimal sparrow (i.e. the optimal feature combination) according to the fitness value. Use an independent validation set or test set to verify the performance of the optimal feature combination.

[0204] In summary, the stereoscopic visual comfort feature extraction system and method based on the sparrow search algorithm of the present invention uses the EMD and Hilbert transform in the HHT method to adaptively extract the time-frequency features of the EEG signal, and then uses the spatial filter in the CSP method to obtain the time-frequency features of multiple spatial dimensions. These two combined methods can not only obtain time-frequency features, but also reveal the interaction between different brain regions during stereoscopic vision transmission, and more comprehensively understand the stereoscopic visual comfort characteristics based on EEG signals. By simulating the foraging behavior, anti-prediction behavior and early warning mechanism of sparrows using the SSA algorithm, features can be accurately and quickly optimized and the optimal feature combination can be selected, reducing the redundancy of time-frequency features, thereby improving the accuracy and performance of the stereoscopic visual comfort classification model.

[0205] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A stereoscopic visual comfort feature extraction system based on sparrow search algorithm, characterized in that: It includes a data preprocessing module, a feature extraction module and a feature optimization module which are connected in sequence; The data preprocessing module is used to preprocess the EEG signal to remove noise and interference; The feature extraction module adaptively extracts time-frequency features in multiple brain regions from the EEG signal preprocessed by the data preprocessing module through Hilbert-Huang transform and common spatial pattern method; The feature optimization module optimizes the time-frequency features adaptively extracted by the feature extraction module through a sparrow search algorithm and selects the optimal feature combination.

2. The stereoscopic visual comfort feature extraction system based on the sparrow search algorithm according to claim 1 is characterized in that: The Hilbert-Huang transform includes empirical mode decomposition and Hilbert transform.

3. The stereoscopic visual comfort feature extraction system based on the sparrow search algorithm according to claim 2 is characterized in that: The empirical mode decomposition and Hilbert transform in the Hilbert-Huang transform are used to adaptively extract the time-frequency features in the electroencephalogram signal, and then the spatial filter in the common spatial mode method is used to obtain the time-frequency features of multiple spatial dimensions.

4. The stereoscopic visual comfort adaptive feature extraction method based on the stereoscopic visual comfort feature extraction system based on the sparrow search algorithm according to claim 1 is characterized in that: The following steps are involved: Step 1, the data preprocessing module preprocesses the EEG signal to remove noise and interference to obtain a preprocessed EEG signal; Step 2, the feature extraction module adaptively extracts the time-frequency features of multiple brain regions in the preprocessed EEG signal obtained in step 1 by using Hilbert-Huang transform and common spatial pattern method; Step 3: The feature optimization module optimizes the time-frequency features extracted in step 2 through a sparrow search algorithm and selects the optimal feature combination.

5. The method for adaptive feature extraction of stereoscopic visual comfort according to claim 4, characterized in that: The Hilbert-Huang transform in step 2 includes empirical mode decomposition and Hilbert transform; First, the preprocessed EEG signal obtained in step 1 is decomposed into a series of intrinsic mode functions through empirical mode decomposition; Then, each intrinsic mode function is Hilbert transformed and its instantaneous frequency and amplitude are adaptively calculated to obtain the time-frequency characteristics.

6. The method for adaptive feature extraction of stereoscopic visual comfort according to claim 5, characterized in that: The preprocessed raw EEG signal Decompose it into a series of intrinsic mode functions through empirical mode decomposition. The steps are as follows: Step 2.1: Find The local maximum and minimum points of ; then, the cubic spline interpolation method is used to fit these extreme points; finally, we get The average value of the upper and lower envelopes , using the original EEG signal Subtract the mean get : ； is the first approximation component; Step 2.2: The number of extreme points is equal to the number of zero points, or the absolute value of the difference between the two is one; and The average value of the upper and lower envelopes of the signal is zero; like When the above conditions are met, it is the first intrinsic mode function, recorded as ; From the original EEG signal Bundle Separate it out : ； in, is the first remainder; like If the above conditions are not met, repeat step 2.1 until Until the above conditions are met; Step 2.3: Judgment Whether the signal can be further decomposed; If it can be decomposed, Repeat steps 2.1 and 2.2 as new raw EEG signals until all intrinsic mode functions are successfully extracted; like No longer meets the decomposition conditions 0.3, the iteration process stops, where The expression is as follows: ； in, is the standard deviation, is the i-th approximation component, is the i-1th approximate component, T is the total length of the signal; Step 2.4: Repeat steps 2.1 to 2.3 until the Kth intrinsic mode function component is obtained. The final result of the empirical mode decomposition is expressed as: ； in, For the The intrinsic mode function components obtained are: is the total number of times, For the remainder.

7. The method for adaptive feature extraction of stereoscopic visual comfort according to claim 6, characterized in that: The steps of performing Hilbert transform on each intrinsic mode function and adaptively calculating its instantaneous frequency and instantaneous amplitude are as follows: Step 2.5: Perform Hilbert transform on each intrinsic mode function component decomposed by empirical mode decomposition: ； At the same time, the analytical signal is constructed as: ； in, is the normalization factor, is the integral variable introduced when the intrinsic mode function components are integrally transformed. is the time variable, are different phase shifts, is the analytical signal obtained for the i-th time, is the intrinsic mode function component obtained for the i-th time, for The Hilbert transform of is the imaginary unit, is the instantaneous amplitude obtained for the ith time, is the instantaneous phase obtained for the i-th time; Therefore, the instantaneous amplitude is obtained by the following formula and instantaneous phase : ； ； Then find the instantaneous frequency : ； Step 2.6: Obtain the instantaneous frequency and instantaneous amplitude of the intrinsic mode function components to convert the original EEG signal It is expressed as: ； in, []express The real part of the signal; is the remaining signal; is an exponential function, is the imaginary unit; The time-frequency characteristics of a single subject are obtained through Hilbert-Huang transform (HHT) post-adaptation , according to the comfort and discomfort of stereoscopic vision Divide and Two categories, including represents the time-frequency feature set under the comfortable state, Represents the time-frequency feature set under discomfort state.

8. The method for adaptive feature extraction of stereoscopic visual comfort according to claim 7, characterized in that: The steps of the public space model method are as follows: Step 2.7, Calculation and Covariance matrix of two types of time-frequency feature matrices: ； in, express The transpose of the feature matrix, Express Sum the elements on the diagonal of the characteristic matrix to obtain the covariance matrix of the mixed space ,use and They represent the expectation of the spatial covariance matrix of the time-frequency feature set samples in a comfortable state and the time-frequency feature set samples in an uncomfortable state respectively; Step 2.8: Use the singular value decomposition theorem to calculate the covariance matrix of the mixed space Perform feature decomposition and whiten transformation get ; yes The transposed matrix of is the eigenvector matrix; Step 2.9: Whiten the eigenvalue matrix Acts on and : ； ； in, yes The transposed matrix of is the diagonal matrix of eigenvalues, is the covariance matrix of the first set of data, is the covariance matrix of the second set of data, yes The transformed covariance matrix, yes The transformed covariance matrix, is the whitened eigenvalue matrix The transpose of and have common eigenvectors and there are two diagonal matrices and and the same eigenvector : ； ； ； Among them, the two diagonal matrices , The sum is the identity matrix , is the eigenvector The transpose of Step 2.10: If The eigenvalues ​​of are arranged in descending order, then The eigenvalues ​​of will be arranged in ascending order; so for the eigenvector ,when When the eigenvalue is the largest, has the smallest eigenvalue, so using the eigenvector Realize the classification of two types of problems, thereby obtaining the projection matrix : ； Among them, the projection matrix is the corresponding spatial filter, is the eigenvector The transpose of Step 2.11: Obtained EEG signal Through the projection matrix Projection to obtain the feature matrix , and the final features are obtained by normalization: ； in, is the variance of the feature, It is The normalized feature matrix of samples is is the feature matrix The variance of .

9. The method for adaptive feature extraction of stereoscopic visual comfort according to claim 4, characterized in that: Step 3 includes the following steps: Step 3.1, Initialization: Set the parameters of the sparrow search algorithm, randomly initialize the sparrow population, each sparrow represents a feature combination, and its position is composed of a set of feature values; A population of sparrows is represented by: ； in, Indicates the variable dimension to be optimized; Step 3.2, evaluate fitness: For each sparrow, a fitness function is used to evaluate the stereoscopic visual comfort classification performance, and the fitness value of each sparrow is obtained and sorted by fitness value; The fitness value of a sparrow is: ； in, represents the fitness value; Step 3.3, role allocation: According to the fitness value, sparrows are divided into finders, followers and vigilants; Finders are sparrows with higher fitness values, and they will lead the search direction; Followers are sparrows with relatively low fitness values, and they will follow the discoverer in the search; Sentinels are responsible for monitoring the environment and issuing warnings when potential threats are detected; Step 3.4, Update location: The discoverer focuses on the local optimization of the current solution space, and also focuses on finding the global optimal feature combination in a larger range; the follower dynamically adjusts according to the discoverer's position, allowing the follower to jump out of the current search area; the vigilant adjusts the search behavior according to the safety status of the environment, and when the environment is unsafe, it adopts strategies to avoid falling into the local optimum; Finder location update: ； in, is the current iteration number, represents the maximum number of iterations, ∈(0,1] is a random number, and Represent the warning value and safety value respectively. , ; is a random number that follows a normal distribution, Indicates a Matrix of is the j-th dimension position of the i-th sparrow in the t-th iteration, is the j-th dimension position of the i-th sparrow in the t+1-th iteration; Follower location update: ； in, is the optimal position currently occupied by the discoverer, Indicates the current global worst position; Indicates a d matrix, each element is randomly assigned a value of 1 or -1, and ;when When the fitness value is lower, Followers need to update their positions; Step 3.5: Simulate anti-predator behavior: During the search process, when the sentinel detects a potential threat, it will send out a warning signal; the sparrow that receives the warning signal will adjust its search strategy by increasing the search step size or changing the search direction to escape the potential predator, which is the local optimal solution; ； in, is the global optimal position in the current iteration, To adjust the step size factor; Indicates the direction in which the sparrow moves. is a constant, and Respectively represent the optimal and worst fitness values ​​of the current population, Indicates The fitness value of a sparrow; Step 3.6, iterative optimization: In each iteration, repeat steps 3.2 to 3.5 to optimize the feature combination by continuously updating the position and role assignment of the sparrows. As the iteration proceeds, the sparrow search algorithm will gradually approach the optimal feature combination. Step 3.7, select the optimal feature combination: After the iteration, the optimal feature combination is selected according to the fitness value.

Images