Student state analysis method for virtual reality emergency maintenance

Abstract

The invention belongs to the technical field of virtual teaching methods, and particularly relates to a student state analysis method for virtual reality emergency maintenance. The method comprises the following steps: determining the acquisition time of an electroencephalogram signal, an electrocardiogram signal and a skin electric signal according to the stability of pre-acquired data, performing signal preprocessing through filtering, a time sliding window and the like, and constructing an electroencephalogram signal feature matrix; and establishing a state classification network model based on a convolutional neural network, extracting time and space features of the student state data, and performing classification and identification by using the neural network. Aiming at the characteristics of large virtual reality emergency maintenance virtual environment and operation pressure, the physiological and psychological states of the student are analyzed and identified, so that different influences of different stages and different conditions of the emergency maintenance process on the emergency repair teaching of the student can be further confirmed; therefore, the college emergency repair teaching direction is optimized in a targeted manner, and the teaching quality is improved.

Description

Technical Field

[0001] This invention belongs to the field of virtual teaching methods technology, and in particular relates to a student status analysis method for virtual reality emergency maintenance. Background Technology

[0002] Virtual reality (VR) teaching utilizes VR technology to improve students' cognitive efficiency and enhance teaching quality. VR emergency repair teaching, in particular, can simulate various types of emergency situations through VR scenarios, effectively improving students' emergency response capabilities while ensuring safety. Analysis of the teaching process reveals that different students exhibit varying response characteristics to different emergency scenarios and repair content. Even within the same emergency repair process, different students demonstrate differences in attention allocation, focus, and psychological state. Analyzing the different states of students in emergency repair courses can determine whether their attention allocation is reasonable and their resilience under pressure in emergency situations. This is crucial for further improving students' ability to respond to complex scenarios and for developing more scientific and efficient emergency response teaching plans. Summary of the Invention

[0003] The purpose of this invention is to provide a method for analyzing the status of trainees in virtual reality emergency maintenance, based on the above-mentioned needs, and to realize continuous status analysis of trainees in virtual emergency scenarios.

[0004] To achieve the above objectives, the present invention adopts the following technical solution.

[0005] A method for analyzing student status in virtual reality emergency maintenance includes the following steps:

[0006] Step 1: Acquisition and processing of EEG, ECG, and skin conductance signals.

[0007] The method of advance data collection is adopted, which involves pre-collecting data before formal data collection and determining the time of formal data collection based on the stability of the pre-collected data. At the same time as formal data collection begins, the corresponding start time of data collection is stored in the database. After sufficient data is collected, an appropriate end time should be determined and stored in the database. Finally, the data obtained between the start time and the end time of data collection is extracted, and other data is deleted.

[0008] Step 2: Preprocessing of EEG, ECG, and Electrodermal Signals

[0009] For EEG signals, full-channel acquisition is used, and reference electrode signals are selected as the benchmark. High-pass filters, low-pass filters, and notch filters are used to filter out noise interference. Then, the required EEG signals are separated from the full-channel signals.

[0010] For electrocardiogram (ECG) signals, a photoplethysmography (PPG)-based signal separation scheme is adopted. To achieve effective heart rate signal extraction, several fixed time periods are used based on the time-frequency characteristics of the heart rate signal. The sliding time window has a fixed step size. The acquired photoplethysmography (PPG) signals are truncated. To eliminate interference caused by human movement, the PPG signals are synchronously truncated based on the aforementioned time period, and each truncated PPG signal is used as the original signal. The heart rate frequency range of the target population is then considered. The original signal is filtered using a bandpass filter with the same frequency range. To eliminate unnecessary motion signals caused by human movement in the original signal after filtering, wavelet coefficient vectors containing heart rate information are obtained through wavelet decomposition. A suitable threshold is selected to suppress noise-related wavelet signals. Then, inverse wavelet transform is used to reconstruct the denoised signal. In this application, a global threshold is used. ;in Indicates the first The standard deviation of the layer wavelet coefficients This refers to the total number of wavelets; finally, median filtering is used to eliminate baseline drift.

[0011] For the electrodermal signal, high-pass and low-pass filters are used to remove noise interference, and effective signals are extracted according to the signal interval. Finally, baseline calibration is performed. The calibrated signal is analyzed to locate abnormal data, missing data locations and data ranges caused by large movements and system data acquisition failures. Data is deleted or interpolated to repair the data according to the characteristics of the EEG signal acquisition requirements. The above steps are repeated until the positions of spectral peaks within all time windows are determined.

[0012] Step 3: Classification of Signal Feature Data

[0013] Electroencephalogram (EEG) signal feature extraction uses power spectral density features, energy features, time-frequency features, and non-multiplication coefficients in specific frequency bands to construct an EEG signal feature matrix;

[0014] Among them power spectral density characteristics ;

[0015] in It refers to the first The minimum frequency of each power band, among which It refers to the first The maximum frequency of each power band This refers to the signal power within a frequency band;

[0016] The energy characteristic is represented by differential entropy. This indicates that the probability density function of the EEG signal curve is calculated. Obtain differential entropy ;in This refers to the frequency range in which energy characteristic statistics are performed;

[0017] Time-frequency features can be used to obtain the variation characteristics of different signals in an electroencephalogram (EEG) signal. This is achieved by using wavelet coefficients of the EEG signal. The wavelet decomposition can be expressed as ;

[0018] in Indicates the first The first layer of wavelet decomposition Approximate component, Indicates the first The first layer of wavelet decomposition Each detail is significant. ; These are the basis functions of the wavelet transform. It is a threshold function;

[0019] The non-multiplicative coefficient is used to characterize the asymmetric changes in EEG signals during state transitions; it is obtained through rhythmic energy collected by electrodes symmetrically positioned on the left and right sides of the brain. Perform the calculation, expressed as ;

[0020] Heart rate spectrum characteristics, including heart rate value, i.e., the number of beats per minute, are defined as follows: , This refers to the sampling rate. Indicates the spectral length of the sampling time window; when adjacent sampling time windows overlap. At that time, the first The corrected heart rate within a time window can be expressed as: ,in Represents the stack weight, where ,in This refers to the duration of the sampling time window;

[0021] Electrodermal signal characteristics include the skin response signal (SCR) and the skin conductance level signal (SCL).

[0022] Based on local minima of skin electrical signals and its adjacent local maximum values The average amplitude of the skin conductance signal was obtained. ;

[0023] The characteristics of skin conductivity level signals include: average skin conductivity level. Standard deviation and its skewness value The average skin conductivity water Standard deviation The skew value is obtained by direct statistical calculation based on the SCL signal value. ,in This refers to the calculation of the mean. This refers to skin electrical signal samples;

[0024] Step 4: Establish a state classification network model based on a convolutional neural network

[0025] The state classification network model consists of a time feature module, a spatial feature module, and a weight adjustment module;

[0026] The temporal feature module includes sequential convolutional units and a first normalization unit; the sequential convolutional unit includes several convolutional kernels with a size of The convolutional blocks of and, where This refers to the sample sampling rate, where The first convolutional unit is used to extract the temporal dimension features of the electrocardiogram and electroencephalogram (ECG) signals; the first normalization unit is used to normalize the temporal dimension features.

[0027] The spatial feature module includes a cross-channel convolutional unit and a second normalized unit; the cross-channel convolutional unit includes convolutional kernels with sizes of [missing information]. , , Cross-channel convolution; the outputs of the cross-channel convolution units are concatenated and then normalized using a second normalization unit;

[0028] The weight adjustment module is used to suppress weakly correlated features while enhancing strongly correlated features. Its basic steps include: processing the high-dimensional feature sequence after the time feature module and the spatial feature module... First, a dimensionality transformation is performed to facilitate computation. Then, the weight values ​​and weight vectors of each feature in the transformed low-dimensional feature matrix are calculated. The product of the weight vector and the actual feature vector is calculated. After performing multiple consecutive operations, the weights of effective features are enhanced and the weights of ineffective features are reduced. Subsequently, the feature vectors are expanded and passed through a fully connected layer to map the features onto the final recognition result.

[0029] In a further improved or preferred embodiment of the aforementioned student status analysis method for virtual reality emergency repair, step one further includes using the time when the first EEG signal appears after the start of the acquisition time and the time when the last EEG signal ends before the end of the acquisition time as the start and end times of all signals.

[0030] In a further improved or preferred embodiment of the aforementioned student status analysis method for virtual reality emergency repair, step two further includes a step of correcting the original signal based on the distribution characteristics of the original signal spectrum peaks in a virtual reality emergency repair scenario. Specifically, this involves: collecting baseline data of the tested object under static conditions, performing a fast Fourier transform, selecting the spectrum peak with the highest amplitude as the initial reference for the heart rate spectrum peak, and then optimizing the spectrum peak characteristics by truncating the original signal using a time window.

[0031] A further improvement or preferred embodiment of the aforementioned student status analysis method for virtual reality emergency repair includes the following optimization steps:

[0032] a1. Mark the largest spectral peak obtained within the time window as... The position of the largest spectral peak is denoted as All amplitudes not less than Spectral peaks as candidate spectral peaks The position of the spectral peak is recorded as ,

[0033] a2. Based on the previous time window The obtained heart rate spectrum peak position Determine the current time window Search range for spectral peak positions ,in This is a range adjustment parameter;

[0034] a3. Search the current time window sequentially according to the peak size. Corresponding search range Spectral peak positions within ; Arrange the spectral peak positions in order of peak size Spectral peak positions determined in the previous time window To compare, the first one that satisfies The peak positions are marked as the heart rate peak positions. Otherwise, according to In the previous time window The location of the heart rate spectrum peak is determined by locating the nearest spectral peak within the current time window.

[0035] a4. Repeat the above steps until the positions of spectral peaks within all time windows are determined.

[0036] In a further improvement or preferred embodiment of the aforementioned student status analysis method for virtual reality emergency repair, step three further includes a feature signal standardization step, which standardizes each feature signal to obtain the original feature signal. The standardized signal can be represented as ; Refers to the original feature signal The minimum value, Refers to the original feature signal The maximum value. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of electroencephalogram (EEG) signals;

[0038] Figure 2 This is a schematic diagram of a wavelet decomposition and filtering method for photoplethysmography signals. Detailed Implementation

[0039] The present invention will be described in detail below with reference to specific embodiments.

[0040] This invention relates to a student status analysis method for virtual reality emergency repair. The virtual environment and work pressure in virtual reality emergency repair are high, and the physiological and psychological state of students directly affects the efficiency of emergency repair. By analyzing and identifying the physiological and psychological state of students, it is possible to further confirm the impact of different stages and situations on students' emergency repair work, so as to optimize the emergency repair teaching direction of the college and improve the teaching quality.

[0041] Typically, in actual emergency maintenance training, this application mainly analyzes the impact of trainees' psychological state on their pre-emergency state in emergency environments. In the field of virtual reality, there are already some state analysis technologies based on facial micro-expressions and other technical means. Considering the high-pressure rapid response requirements involved in the actual teaching process of emergency maintenance training, this application considers using data such as electroencephalograms, electrocardiograms, and electrodermal signals, which can be collected in real time and efficiently, as the basis for trainee state analysis.

[0042] Step 1: Signal Acquisition

[0043] To ensure the consistency of the acquisition time of EEG signals related to the trainees' states, it is necessary to ensure the time alignment between data. At the same time, it is also necessary to ensure that the data acquisition is carried out when the test subjects are in a relatively stable state. For this reason, a pre-acquisition method is adopted, in which data is pre-acquired before the formal acquisition. The time of formal acquisition is determined based on the stability of the pre-acquisition data. At the same time as the formal acquisition begins, the corresponding start acquisition time is stored in the database. After sufficient data is acquired, an appropriate end time is determined and stored in the database. Finally, the data acquired between the start and end acquisition times is extracted, and the remaining data is deleted. Considering that the sensitivity and sampling efficiency of EEG signals are superior to other signals, the start and end times of all signals are further defined as the time when the first EEG signal appears after the start acquisition time and the time when the last EEG signal ends before the end acquisition time.

[0044] In particular, to ensure that there are no other factors interfering during the data acquisition process, apart from the corresponding auxiliary means used to simulate the online emergency repair process in the emergency repair teaching process, the existence of other interfering factors should be avoided. At the same time, according to the requirements of testing and data acquisition accuracy, corresponding EEG, ECG and skin conductance signal acquisition equipment that meets the standards, has sufficient sampling channels, and can provide sampling frequencies that meet the data analysis requirements should be adopted.

[0045] This application is mainly used to optimize the teaching and training process. Therefore, a non-intrusive data acquisition method should be adopted. At the same time, considering that trainees may be involved in more physical movements and interactive operations during emergency repairs compared to other virtual scenarios, wireless data communication and photoelectric signal acquisition devices should be used as much as possible to reduce the interference of sensors and supporting equipment on the teaching process in order to avoid interference and foreign object interference.

[0046] Step 2: Signal Preprocessing

[0047] like Figure 1 As shown, EEG signals are mainly acquired through a professional EEG data acquisition system. The EEG signals acquired through the full-channel acquisition system need to be based on a suitable reference electrode signal. High-pass filters, low-pass filters, and notch filters are used to filter out noise interference as much as possible. Then, the required EEG signals are separated from the full-channel signals.

[0048] Electrocardiogram (ECG) signals are used to obtain heart rate information. To adapt to the teaching process, it is necessary to ensure that the testing equipment can be quickly and easily put on or taken off. Traditional acquisition methods based on fixed electrodes are not conducive to the teaching process, and have a large interference with the subject being acquired. It is difficult to guarantee the real heart rate signal during the virtual operation process. Therefore, this application adopts a photoplethysmography (PPG) signal separation scheme. The acquisition of PPG signals will not directly interfere with the subject being acquired, which is conducive to achieving non-intrusive acquisition and ensuring the authenticity and validity of the data.

[0049] To achieve effective heart rate signal extraction, several fixed time periods are used based on the time-frequency characteristics of the heart rate signal. The sliding time window has a fixed step size. The acquired photoplethysmography (PPG) signals are truncated. To eliminate interference caused by human movement, the PPG signals are synchronously truncated based on the aforementioned time period, and each truncated PPG signal is used as the original signal. The heart rate frequency range of the target population is then considered. The original signal is filtered using a bandpass filter with the same frequency range; for example... Figure 2 As shown, to eliminate unnecessary motion signals caused by human movement in the original signal after filtering, wavelet coefficient vectors containing heart rate signal information are obtained through wavelet decomposition. A suitable threshold is selected to suppress noise-related wavelet signals. Then, inverse wavelet transform is used to reconstruct the denoised signal. In this application, a global threshold is used. ;in Indicates the first The standard deviation of the layer wavelet coefficients This refers to the total number of wavelets; finally, baseline drift is eliminated through median filtering.

[0050] For the electrodermal signal, appropriate high-pass and low-pass filters are used to remove noise interference from the electrodermal signal, and the effective signal is extracted according to the signal range for baseline calibration. The calibrated signal is analyzed to locate abnormal data, data loss locations and data ranges caused by the subject's large movements and system data acquisition failures, and deletion or interpolation repair is performed according to the characteristics of the EEG signal acquisition requirements.

[0051] In virtual reality emergency repair scenarios, the operator needs to perform large-scale body movements and frequent, continuous operations, which can lead to significant interference even in pre-processed photoplethysmography (PPG) signals. To extract effective heart rate signals from PPG signals, the original signal needs to be corrected based on the signal spectrum peak distribution characteristics. Specifically, this includes:

[0052] Initial estimation involves collecting baseline data of the subject under static conditions, performing a fast Fourier transform, and selecting the spectral peak with the highest amplitude as the initial reference for the heart rate spectral peak. The spectral peak characteristics are then optimized by truncating the original signal using a time window.

[0053] The optimization steps include:

[0054] a1. Mark the largest spectral peak obtained within the time window as... The position of the largest spectral peak is denoted as All amplitudes not less than Spectral peaks as candidate spectral peaks The position of the spectral peak is recorded as ,

[0055] a2. Based on the previous time window The obtained heart rate spectrum peak position Determine the current time window Search range for spectral peak positions ,in This is a range adjustment parameter;

[0056] a3. Search the current time window sequentially according to the peak size. Corresponding search range Spectral peak positions within ; Arrange the spectral peak positions in order of peak size Spectral peak positions determined in the previous time window To compare, the first one that satisfies The peak positions are marked as the heart rate peak positions. Otherwise, according to In the previous time window The location of the heart rate spectrum peak is determined by locating the nearest spectral peak within the current time window.

[0057] a4. Repeat the above steps until the positions of spectral peaks within all time windows are determined;

[0058] Step 3: Classification of Signal Feature Data

[0059] Electroencephalogram (EEG) signal feature extraction: EEG signals are formed by the fusion of multiple signals and contain features of physiological state and other irrelevant information. Based on the identifiability of actual data and the existing research results on the correlation between EEG and physiological state signals, this application uses power spectral density features, energy features, time-frequency features, and non-multiplication coefficients of specific frequency bands to construct an EEG signal feature matrix.

[0060] Among them power spectral density characteristics ;

[0061] in It refers to the first The minimum frequency of each power band, among which It refers to the first The maximum frequency of each power band This refers to the signal power within a frequency band;

[0062] The energy characteristic is represented by differential entropy. This indicates that the probability density function of the EEG signal curve is calculated. Obtain differential entropy ;in This refers to the frequency range in which energy characteristic statistics are performed;

[0063] Time-frequency features can be used to obtain the variation characteristics of different signals in an electroencephalogram (EEG) signal. Generally, wavelet coefficients of the EEG signal are used. The wavelet decomposition can be expressed as ;

[0064] in Indicates the first The first layer of wavelet decomposition Approximate component, Indicates the first The first layer of wavelet decomposition Each detail is significant. ; These are the basis functions of the wavelet transform. It is a threshold function;

[0065] The non-multiplicative coefficient is used to characterize the asymmetric changes in EEG signals during state transitions; it is obtained through rhythmic energy collected by electrodes symmetrically positioned on the left and right sides of the brain. The calculation can be expressed as ;

[0066] Heart rate spectrum characteristics, including heart rate value, i.e., the number of beats per minute, are defined as follows: , This refers to the sampling rate. Indicates the spectral length of the sampling time window; when adjacent sampling time windows overlap. At that time, the first The corrected heart rate within a time window can be expressed as: ,in Represents the stack weight, where ,in This refers to the duration of the sampling time window;

[0067] Electrodermal signal characteristics

[0068] Including skin conductance response (SCR) and skin conductance level (SCL) signals.

[0069] Based on local minima of skin electrical signals and its adjacent local maximum values The average amplitude of the skin conductance signal was obtained. ;

[0070] The characteristics of skin conductivity level signals include: average skin conductivity level. Standard deviation and its skewness value The average skin conductivity water Standard deviation The skew value is obtained by direct statistical calculation based on the SCL signal value. ,in This refers to the calculation of the mean. This refers to skin electrical signal samples;

[0071] 3b. Characteristic signal standardization

[0072] To avoid errors caused by dimensional differences after signal feature fusion, each feature signal is standardized. The standardized signal can be represented as ; Refers to the original feature signal The minimum value, Refers to the original feature signal The maximum value;

[0073] Step 4: Establish a state classification network model based on a convolutional neural network

[0074] The state classification network model is composed of

[0075] Time Feature Module

[0076] It includes sequential convolutional units and a first normalized unit; the sequential convolutional unit includes several convolutional kernels with a size of The convolutional blocks of and, where This refers to the sample sampling rate, where The first convolutional unit is used to extract the temporal dimension features of the electrocardiogram and electroencephalogram (ECG) signals; the first normalization unit is used to normalize the temporal dimension features.

[0077] Spatial feature module

[0078] It includes a cross-channel convolutional unit and a second normalized unit; the cross-channel convolutional unit includes convolutional kernels with sizes of [sizes to be filled in]. , , Cross-channel convolution; the outputs of the cross-channel convolution units are concatenated and then normalized using a second normalization unit;

[0079] The weight adjustment module is used to suppress weakly correlated features while enhancing strongly correlated features. Its basic steps include:

[0080] For high-dimensional feature sequences after processing by time feature module and spatial feature module First, a dimensionality transformation is performed to facilitate computation. Then, the weight values ​​and weight vectors of each feature in the transformed low-dimensional feature matrix are calculated. The product of the weight vector and the actual feature vector is calculated. After performing multiple consecutive operations, the weights of effective features are enhanced and the weights of ineffective features are reduced. Subsequently, the feature vectors are expanded and passed through a fully connected layer to map the features onto the final recognition result.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A method for analyzing student status in virtual reality emergency maintenance, characterized in that, Includes the following steps: Step 1: Acquisition and processing of EEG, ECG, and skin conductance signals. The method of advance data collection is adopted, which involves pre-collecting data before formal data collection and determining the time of formal data collection based on the stability of the pre-collected data. At the same time as formal data collection begins, the corresponding start time of data collection is stored in the database. After sufficient data is collected, an appropriate end time should be determined and stored in the database. Finally, the data obtained between the start time and the end time of data collection is extracted, and other data is deleted. Step 2: Preprocessing of EEG, ECG, and Electrodermal Signals For EEG signals, full-channel acquisition is used, and reference electrode signals are selected as the benchmark. High-pass filters, low-pass filters, and notch filters are used to filter out noise interference. Then, the required EEG signals are separated from the full-channel signals. For electrocardiogram (ECG) signals, a photoplethysmography (PPG)-based signal separation scheme is adopted. To achieve effective heart rate signal extraction, several fixed time periods are used based on the time-frequency characteristics of the heart rate signal. The sliding time window has a fixed step size. The acquired photoplethysmography (PPG) signals are truncated. To eliminate interference caused by human movement, the PPG signals are synchronously truncated based on the aforementioned time period, and each truncated PPG signal is used as the original signal. The heart rate frequency range of the target population is then considered. The original signal is filtered using a bandpass filter with the same frequency range. To eliminate unnecessary motion signals caused by human movement in the original signal after filtering, wavelet coefficient vectors containing heart rate information are obtained through wavelet decomposition. A suitable threshold is selected to suppress noise-related wavelet signals. Then, inverse wavelet transform is used to reconstruct the denoised signal. In this application, a global threshold is used. ;in Indicates the first The standard deviation of the layer wavelet coefficients This refers to the total number of wavelets; finally, median filtering is used to eliminate baseline drift. For the electrodermal signal, high-pass filtering and low-pass filtering are used to remove noise interference from the electrodermal signal, and the effective signal is extracted according to the signal range. Finally, baseline calibration is performed. The calibrated signals are analyzed to locate abnormal data, missing data locations, and data ranges caused by large movements and system data acquisition failures. Data is deleted or interpolated to repair the data according to the characteristics of the EEG signals. The above steps are repeated until the positions of spectral peaks within all time windows are determined. Step 3: Classification of Signal Feature Data Electroencephalogram (EEG) signal feature extraction uses power spectral density features, energy features, time-frequency features, and non-multiplication coefficients in specific frequency bands to construct an EEG signal feature matrix; Among them power spectral density characteristics ; in It refers to the first The minimum frequency of each power band, among which It refers to the first The maximum frequency of each power band This refers to the signal power within a frequency band; in Energy characteristics are derived using differential entropy. This indicates that the probability density function of the EEG signal curve is calculated. Obtain differential entropy ;in This refers to the frequency range in which energy characteristic statistics are performed; Time-frequency features can be used to obtain the variation characteristics of different signals in an electroencephalogram (EEG) signal. This is achieved by using wavelet coefficients of the EEG signal. The wavelet decomposition can be expressed as ; in Indicates the first The first layer of wavelet decomposition Approximate component, Indicates the first The first layer of wavelet decomposition Each detail is significant. ; These are the basis functions of the wavelet transform. It is a threshold function; The non-multiplicative coefficient is used to characterize the asymmetric changes in EEG signals during state transitions; it is obtained through rhythmic energy collected by electrodes symmetrically positioned on the left and right sides of the brain. Perform the calculation, expressed as ; Heart rate spectrum characteristics, including heart rate value, i.e., the number of beats per minute, are defined as follows: , This refers to the sampling rate. Indicates the spectral length of the sampling time window; when adjacent sampling time windows overlap. At that time, the first The corrected heart rate within a time window can be expressed as: ,in Represents the stack weight, where ,in This refers to the duration of the sampling time window; Electrodermal signal characteristics include the skin response signal (SCR) and the skin conductance level signal (SCL). Based on local minima of skin electrical signals and its adjacent local maximum values The average amplitude of the skin conductance signal was obtained. ; The characteristics of skin conductivity level signals include: average skin conductivity level. Standard deviation and its skewness value The average skin conductivity water Standard deviation The skew value is obtained by direct statistical calculation based on the SCL signal value. ,in This refers to the calculation of the mean. This refers to skin electrical signal samples; Step 4: Establish a state classification network model based on a convolutional neural network The state classification network model consists of a time feature module, a spatial feature module, and a weight adjustment module; The temporal feature module includes sequential convolutional units and a first normalization unit; the sequential convolutional unit includes several convolutional kernels with a size of The convolutional blocks of and, where This refers to the sample sampling rate, where The first convolutional unit is used to extract the temporal dimension features of the electrocardiogram and electroencephalogram (ECG) signals; the first normalization unit is used to normalize the temporal dimension features. The spatial feature module includes a cross-channel convolutional unit and a second normalized unit; the cross-channel convolutional unit includes convolutional kernels with sizes of [missing information]. , , Cross-channel convolution; the outputs of the cross-channel convolution units are concatenated and then normalized using a second normalization unit; The weight adjustment module is used to suppress weakly correlated features while enhancing strongly correlated features. Its basic steps include: processing the high-dimensional feature sequence after the time feature module and the spatial feature module... First, a dimensionality transformation is performed to facilitate computation. Then, the weight values ​​and weight vectors of each feature in the transformed low-dimensional feature matrix are calculated. The product of the weight vector and the actual feature vector is calculated. After performing multiple consecutive operations, the weights of effective features are enhanced and the weights of ineffective features are reduced. Subsequently, the feature vectors are expanded and passed through a fully connected layer to map the features onto the final recognition result.

2. The student status analysis method for virtual reality emergency maintenance according to claim 1, characterized in that, Step one further includes using the time when the first EEG signal appears after the start of the acquisition time and the time when the last EEG signal ends before the end of the acquisition time as the start and end times of all signals.

3. The student status analysis method for virtual reality emergency maintenance according to claim 1, characterized in that, Step two also includes a step of correcting the original signal based on the distribution characteristics of the original signal spectrum peaks in a virtual reality emergency repair scenario. Specifically, this involves: collecting baseline data of the tested object under static conditions, performing a fast Fourier transform, selecting the spectrum peak with the highest amplitude as the initial reference for the heart rate spectrum peak, and then optimizing the spectrum peak characteristics by truncating the original signal using a time window.

4. The student status analysis method for virtual reality emergency maintenance according to claim 3, characterized in that, The optimization steps include: a1. Mark the largest spectral peak obtained within the time window as... The position of the largest spectral peak is denoted as All amplitudes not less than Spectral peaks as candidate spectral peaks The position of the spectral peak is recorded as , a2. Based on the previous time window The obtained heart rate spectrum peak position Determine the current time window Search range for spectral peak positions ,in This is a range adjustment parameter; a3. Search the current time window sequentially according to the peak size. Corresponding search range Spectral peak positions within ; Arrange the spectral peak positions in order of peak size Spectral peak positions determined in the previous time window To compare, the first one that satisfies The peak positions are marked as the heart rate peak positions. Otherwise, according to In the previous time window The location of the heart rate spectrum peak is determined by locating the nearest spectral peak within the current time window. a4. Repeat the above steps until the positions of spectral peaks within all time windows are determined.

5. The student status analysis method for virtual reality emergency maintenance according to claim 1, characterized in that, Step three also includes a feature signal standardization step, which involves standardizing each feature signal to obtain the original feature signal. The standardized signal can be represented as ; Refers to the original feature signal The minimum value, Refers to the original feature signal The maximum value.

Images