A method of virtual training of a marine power plant

Abstract

The application belongs to the technical field of virtual training planning, and particularly relates to a virtual training method for a ship power device. The application establishes a mechanism for analyzing the change of attention degree of students in different time periods in a virtual training process and the difference information of the attention degree for different targets in a virtual scene, provides an optimization basis for optimizing the allocation of training content in a virtual teaching process, and provides an improvement reference for virtual scene design. The method is beneficial to improving the teaching time and space classification of virtual teaching content, reducing the interference in the teaching process, and improving the teaching efficiency.

Description

Technical Field

[0001] This invention belongs to the field of virtual training planning technology, and in particular relates to a virtual training method for ship propulsion systems. Background Technology

[0002] As the core of a ship's power control system, the marine propulsion system is both a key focus and a challenging aspect of ship operation training. It involves various teaching topics, including the structural principles, operation, maintenance, and troubleshooting of different components. Compared to other equipment or structures, the marine propulsion system is vast and complex, encompassing a wide range of aspects. In actual teaching, students often only gain a partial understanding, limiting the quality of instruction. To improve teaching effectiveness, various institutions have developed virtual training methods based on simulation equipment and virtual reality technology. However, due to the inherent complexity of the propulsion system and the intricate operational content, the design of virtual reality training content and procedures often relies on individual students' teaching experience. This lack of a scientific and rational analysis and management mechanism for actual teaching quality and rationality results in current virtual training content primarily focusing on simple elements such as structural demonstrations and animation simulations. Students passively learn, with limited interactive content and shallow teaching depth. Summary of the Invention

[0003] The purpose of this invention is to provide a virtual training method for ship propulsion systems that analyzes and processes students' eye movements and electroencephalogram (EEG) signals to establish an information analysis mechanism for changes in students' attention at different times during virtual training and differences in attention to different targets in virtual scenarios. This provides an optimization basis for optimizing the allocation of training content during virtual teaching and provides an improvement reference for virtual scenario design.

[0004] To achieve the above objectives, the present invention adopts the following technical solution.

[0005] A virtual training method for ship propulsion systems includes the following steps:

[0006] S1. Determine the basic types of virtual training tasks based on the virtual training plan for the power unit. These basic types should include at least: power unit configuration power-on operation, control panel standby operation, auxiliary system inspection and standby operation, unit grid connection operation, power unit power adjustment operation, and power unit fault diagnosis and troubleshooting operation.

[0007] S2. Collect data on students' attention levels during the aforementioned virtual training tasks; assign students to conduct training simulation teaching in a standard experimental environment; after students are able to successfully complete the corresponding operations of the virtual training of the power device, conduct virtual training of the power device in batches; collect students' electroencephalogram (EEG) signal data under different training contents; the sampling frequency is 512Hz~1024Hz.

[0008] S3. Extract student EEG signal data for each training content and generate raw data by sampling through the data channel; preprocess the EEG signals based on the main characteristics of the analyzed EEG signals by using filtering methods such as high-pass filtering or band-pass filtering, and then downsample to compress the data volume.

[0009] S4. High-frequency interference in the EEG signal is removed to obtain smoothed data. The processed parameter values ​​can be expressed as follows: Where x n ￠ represents the nth raw EEG signal acquired at time t, and μ t-1 ￠ represents the average value of the raw EEG signal at time t-1, s t-1 ￠ refers to the standard deviation of the original EEG signal at time t-1, n is the total number of samples collected, and α is the attenuation coefficient;

[0010] S5. Since repetitive signal features are inevitably collected during the EEG signal data sampling process, an increase in the proportion of repetitive data will lead to a deviation in the time-domain characteristics reflected by the original data. Therefore, for the EEG signal X at time t collected in step S4... n ={x1,x2...x n ...x N We need to remove duplicate signal features and extract the independent signals s from each EEG signal source that reflect the characteristics of the original EEG signal. n This yields an independent signal S that reflects the actual feature vector of the original data. n ={s1,s2...s n ...s N Then, using the independent signal S n The reconstructed original signal is used as valid data for further processing. Generally, it can be considered that the EEG signal x... n It is composed of independent signals s n X is obtained by linear combination of the independent signals. n =A T S n :

[0011] {x1,x2...x n ...x N} = A T {s1,s2...s n ...s N Let W = A - 1 Then there is S n =WX n Where A is a linear mixing matrix, representing the sum of the values ​​of each independent signal s. n For each EEG signal x n The impact on independent signals S; n ={sn}, in its component signal s n In the case of absolute independence, its data entropy H(S) n H(s) is the data entropy of its component signal. n The sum of ) and the acquisition of EEG signals x n The best independent signal s in n Extracting the result, i.e., solving for the data entropy H(S) n ) and ∑H(s n The classification that minimizes the difference between ) is defined by the objective function Goal1 = min|∑H(s) n )-H(S N )|;

[0012] For independent signal S n , has logP(S n ) = logP(X n )-log|detW|, EEG signals are acquired from the same source signal through independent signal acquisition ports, therefore each component signal s n The variance of the original signal X should be kept consistent. n It can be regarded as each component signal s n After amplification and linear combination, its covariance is obtained as the independent signal S. n The covariance f, i.e. To ensure that each component signal s n If they are completely uncorrelated, then H(S) needs to be guaranteed. N The objective function is independent of W, meaning that the objective function needs to make log|detW|=0, or detW=1, which means that W needs to be a rotation matrix with all eigenvalues ​​equal to 1.

[0013] Meanwhile, based on the principle of independent component analysis, in order to obtain the optimal uncorrelated component signal s n It is necessary to determine the non-Gaussianity J(W) in the W direction. T X n The maximum value is Goal2 = maxJ(W). T X n The parameters of the W matrix are determined by solving the problem using the FSStICS iterative algorithm, and then substituted into the S matrix. n =WX n The independent signal S can then be obtained. n Specific parameters;

[0014] S6. Extract the independent signal S obtained in step S5. n ={s1,s2...s n ...s N}, identify obvious errors or invalid data, and extract the independent signal S. n ={s1,s2...sn ...s N The individual signal components in the EEG signal are reconstructed to obtain the final EEG signal X. n ={x1,x2...x n ...x N};

[0015] S7. Establish an attention analysis model consisting of a temporal feature extraction module, a spatial feature extraction module, and a classification module; the temporal feature extraction module is composed of several temporal feature extraction components of different scales connected in parallel, and the temporal feature extraction components consist of convolutional layers, activation layers, and pooling layers. The one-dimensional convolutional layer performs dimension-wise convolution operations on the original data of EEG signal data in temporal order, and uses the activation function of the activation layer and the pooling operation of the pooling layer to extract temporal features of different dimensions of EEG signal data;

[0016] The real-valued spatial feature extraction module consists of two spatial feature extraction components connected in parallel. The spatial feature extraction components consist of a spatial convolutional layer, an activation layer, and a pooling layer. The sizes of the two spatial convolutional layers are (D,1) and (0.5D,1), respectively, where D is the total number of EEG signal data channels.

[0017] The classification module is used to perform attention identification and analysis on the temporal feature information extracted by the temporal feature extraction module and the spatial feature information extracted by the real number spatial feature extraction module, and to prevent overfitting based on random deactivation;

[0018] S8. Input the raw EEG signal data into the analysis model established in step S7 to determine the students' attention at different times during the training process.

[0019] S9. Interactive data collection and processing refers to collecting information on the intersections of the handle ray, binocular line of sight, and virtual objects in the virtual training scenario of the ship's power unit during student training to determine the information of interest; including:

[0020] Blank data filling: Due to the unavoidable loss of visual data during training, such as blinking, information such as implementation data may be missing. It is necessary to replace the blank data with the last valid data before the time when the blank data appeared.

[0021] Coordinate transformation processing: Since different types or systems of controllers, head-mounted displays, and virtual reality screens may use different coordinate system establishment schemes due to configuration requirements, in order to ensure data consistency, coordinates should be unified according to the coordinate establishment methods of controllers, head-mounted displays, and virtual reality screens, and the original data should be uniformly modified according to the coordinate adjustment method, including: unifying eye-tracking data based on different left and right hand coordinate systems to the controller coordinate system, and uniformly transforming the head-mounted display coordinate system and the controller coordinate system to the Earth coordinate system.

[0022] Interactive Intent Pre-annotation Processing: Through historical data analysis, in the virtual training process of ship propulsion systems, when the intent is clear, there is generally a period of time between the generation of the intent and the actual execution of the interactive action. yc =200ms to 100ms time difference, further considering system data delay Δt xt To predict the intent of eye-tracking interaction, a multi-level classifier needs to be built based on the aforementioned time difference to predict the intent of the user's interaction behavior. The classification result of the classifier is matched with the existence of the interaction intent and the final result of the interaction intent. Generally, it can be divided into a first-level interaction intent classifier and a second-level interaction classifier. The first-level classifier is a binary classifier, and its classification result is used to determine whether the interaction intent exists. The second-level interaction classifier is a multi-classifier, and its classification result is the specific meaning of the interaction intent. It can generally be divided into selection intent, transmission intent, and no intent (including unconscious subconscious behavior or habitual behavior, etc.).

[0023] Based on the actual system characteristics and the students' speed of analyzing the training content, interactive behavior data with clear interactive intentions were extracted, and the data was processed from 200+ Dt before the interactive behavior data was generated. xt ~T yz +Dt xt The interaction data is input into the first-level classifier, and the T data before the interaction behavior data is generated is then processed. yz +Dt xt ~1000+Dt xt The interactive data is input into a two-level classifier, and the interactive data is classified and labeled according to the actual content of the interactive behavior data.

[0024] S10. Extract behavioral features from the classification and labeling results: The behavioral features include at least the sweeping speed at different data point locations, the fixation time for different data points, and the pupil distance at the corresponding time of the data point; Based on the feature values ​​of the corresponding data points in the classification and labeling structure, determine a preset threshold for judging the type of data point; In this embodiment, the preset threshold includes at least a preset continuous fixation time threshold for judging whether it is a fixation point, a preset discrete distance for judging whether adjacent fixation points correspond to the same interactive object, and a preset maximum defocus distance for judging whether the student's visual focus is located in the direction directly in front of the line of sight;

[0025] Extract the sweep speed of the line of sight or the interaction line of the handle through different data points during the training process. If the sweep speed is lower than the preset sweep speed threshold, it is initially identified as a fixation point. If the sweep speed exceeds the preset sweep speed threshold, it is initially identified as a saccade point.

[0026] For data points initially identified as fixation points, the following judgments are made:

[0027] S1. Determine whether the duration of sustained fixation is lower than the preset duration of sustained fixation threshold.

[0028] S2. Determine whether the distance between it and its adjacent gaze point exceeds the preset discrete distance;

[0029] S3. Determine whether the student's pupil distance is lower than the preset maximum defocus distance at the corresponding moment;

[0030] If the above three conditions are met, it is considered a valid fixation point. If S1 is not met, the data point is reclassified as a saccade point. If both S2 and S3 are not met, it is merged with the previous fixation point. In other cases, no processing is performed.

[0031] S11. Using the behavioral features obtained in step S10 as the classification basis, establish a multi-feature data classification model; obtain historical training interaction data, divide it into training data and validation data, and establish a multi-scale feature fusion prediction model for machine learning; its input is the feature value of each data point, and its output is the data point classification result; select an appropriate data point type multi-scale feature fusion prediction model according to the system computing power and data scale, use historical training interaction data to train and optimize the model, adjust the thresholds or parameters in steps S9 and S10, use validation data to verify and optimize the model accuracy, and the prediction result of the model meets the accuracy requirements;

[0032] S12. Collect real-time data during the virtual training process of ship power units, input it into the prediction model established in step S11, and judge and predict the type of each data point; analyze the prediction and judgment results to obtain the distribution of student attention at each moment and the distribution of student attention areas at each moment.

[0033] Determine whether the sustained attention received by the current training content meets the training needs. If it does, continue with the current training content; otherwise, stop or change the training content.

[0034] The order and arrangement of training content are adjusted according to the distribution of student attention at different times and the distribution characteristics of student attention areas at different times in order to obtain the best attention.

[0035] In a further improved or preferred embodiment of the aforementioned virtual training method for marine power plants, step S1 includes the following steps: Power plant configuration power-on operation includes power configuration operation for each sub-component of the power plant, operation of the power plant power distribution panel, and identification of indication information; display and control console standby operation includes standby operation for the seawater system, fuel system, and oxygen supply system; auxiliary system inspection and standby operation includes power plant oil level inspection operation, purging system initialization configuration operation, working fluid system initialization configuration operation, supply and return circuit initialization configuration operation, and gas distribution system initialization configuration operation; unit grid connection operation includes grid connection control operation with the same unit; power plant power adjustment operation includes power switch panel adjustment operation; and power plant fault diagnosis and troubleshooting operation includes common fault diagnosis and troubleshooting operations for marine power systems.

[0036] In a further improved or preferred embodiment of the aforementioned virtual training method for ship propulsion systems, in step S7, the size of the one-dimensional convolutional layer is (1, L), where L = 0.5. M *fs, where fs is the sampling frequency, M is a positive integer representing the number of convolutional layers in the temporal feature extraction module; the activation function of the activation layer is... Where a is the activation function hyperparameter;

[0037] For the original input sample X input at time t t ={x 1,t ,x 2,t ...x n,t ...x N,t}, x n,t Let x represent the nth sample input at time t. n,t Input the temporal feature extraction module, where the m-th layer temporal feature extraction unit outputs out. m =Avgp(acv(Conv(1,x)) n,t Avgp(x) represents the average pooling operation in the pooling layer, acv(x) represents the activation operation in the activation layer, and conv(x) represents the one-dimensional convolution in the convolutional layer; the output of the temporal feature extraction unit at time t is out. t =f z ([out1,out2...out m ...out M ]), where f z (x) represents the standardized operation. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of a virtual training device for ship propulsion systems;

[0039] Figure 2 This is a flowchart illustrating the virtual training method for ship propulsion systems. Detailed Implementation

[0040] The present invention will be described in detail below with reference to specific embodiments.

[0041] Background technology description:

[0042] Attention mechanisms are affected by students' cognitive characteristics, limited information processing capabilities, and unavoidable interference factors during virtual training of ship propulsion systems. It is difficult to guarantee that students can maintain their attention to specific goals based on expectations. In actual training, in most cases, it is possible to determine whether students pay attention to different times or information during various training sessions and the degree of their attention by analyzing where students' eyes are focused during virtual training of ship propulsion systems.

[0043] Eye tracking is a direct representation of a student's focus during virtual training on ship propulsion systems. It can be used to analyze the degree of attention students pay to various types of information during training, and it is the basis of eye control in most virtual training devices. Currently, mainstream virtual reality systems and various virtual reality equipment are equipped with eye tracking detection to achieve functions such as virtual interaction and optimized virtual rendering control.

[0044] 2. Technical Solution Description

[0045] This application discloses a virtual training method for ship propulsion systems. This method primarily controls the effectiveness of teaching content by analyzing and predicting students' attention areas and levels during virtual training sessions, thereby increasing the attention given to important content and improving training outcomes. The method mainly includes the following steps:

[0046] S1. Determine the basic types of virtual training tasks based on the virtual training plan for the power unit. These basic types should include at least: power unit configuration power-on operation, control panel standby operation, auxiliary system inspection and standby operation, unit grid connection operation, power unit power adjustment operation, and power unit fault diagnosis and troubleshooting operation.

[0047] The power unit configuration and power-on operation includes power configuration operation of each sub-component of the power unit, operation of the power unit power distribution panel and identification of indication information; the display and control console standby operation includes standby operation of the seawater system, standby operation of the fuel system, and standby operation of the oxygen supply system; the auxiliary system inspection and standby operation includes power unit oil level inspection operation, purging system initialization configuration operation, working fluid system initialization configuration operation, supply and return circuit initialization configuration operation, and gas distribution system initialization configuration operation; the unit grid connection operation includes grid connection control operation with the same unit; the power unit power adjustment operation includes power switch panel adjustment operation; the power unit fault diagnosis and troubleshooting operation includes fault diagnosis and troubleshooting operation for common faults in marine power systems;

[0048] S2. Collect data on students' attention levels during the aforementioned virtual training tasks; assign students to conduct training simulation teaching in a standard experimental environment; after students are able to successfully complete the corresponding operations of the virtual training of the power device, conduct virtual training of the power device in batches; collect students' electroencephalogram (EEG) signal data under different training contents; the sampling frequency is 512Hz~1024Hz.

[0049] S3. Extract student EEG signal data for each training content and generate raw data by sampling through the data channel; preprocess the EEG signals based on the main characteristics of the analyzed EEG signals by using filtering methods such as high-pass filtering or band-pass filtering, and then downsample to compress the data volume.

[0050] S4. High-frequency interference in the EEG signal is removed to obtain smoothed data. The processed parameter values ​​can be expressed as follows: Where x n ￠ represents the nth raw EEG signal acquired at time t, and μ t-1 ￠ represents the average value of the raw EEG signal at time t-1, s t-1 ￠ refers to the standard deviation of the original EEG signal at time t-1, n is the total number of samples collected, and α is the attenuation coefficient;

[0051] S5. Since repetitive signal features are inevitably collected during the EEG signal data sampling process, an increase in the proportion of repetitive data will lead to a deviation in the time-domain characteristics reflected by the original data. Therefore, for the EEG signal X at time t collected in step S4... n ={x1,x2...x n ...x N We need to remove duplicate signal features and extract the independent signals s from each EEG signal source that reflect the characteristics of the original EEG signal. n This yields an independent signal S that reflects the actual feature vector of the original data. n ={s1,s2...s n ...s N Then, using the independent signal S n The reconstructed original signal is used as valid data for further processing. Generally, it can be considered that the EEG signal x... n It is composed of independent signals s n X is obtained by linear combination of the independent signals. n =A T S n :

[0052] {x1,x2...x n ...x N} = A T {s1,s2...sn ...s N Let W = A - 1 Then there is S n =WX n Where A is a linear mixing matrix, representing the sum of the values ​​of each independent signal s. n For each EEG signal x n The impact on independent signals S; n ={s n}, in its component signal s n In the case of absolute independence, its data entropy H(S) n H(s) is the data entropy of its component signal. n The sum of ) and the acquisition of EEG signals x n The best independent signal s in n Extracting the result, i.e., solving for the data entropy H(S) n ) and ∑H(s) n The classification that minimizes the difference between ) is defined by the objective function Goal1 = min|∑H(s) n )-H(S N )|;

[0053] For independent signal S n , has logP(S n ) = logP(X n )-log|detW|, EEG signals are acquired from the same source signal through independent signal acquisition ports, therefore each component signal s n The variance of the original signal X should be kept consistent. n It can be regarded as each component signal s n After amplification and linear combination, its covariance is obtained as the independent signal S. n The covariance f, i.e. To ensure that each component signal s n If they are completely uncorrelated, then H(S) needs to be guaranteed. N The objective function is independent of W, meaning that the objective function needs to make log|detW|=0, or detW=1, which means that W needs to be a rotation matrix with all eigenvalues ​​equal to 1.

[0054] Meanwhile, based on the principle of independent component analysis, in order to obtain the optimal uncorrelated component signal s n It is necessary to determine the non-Gaussianity J(W) in the W direction. T X n The maximum value is Goal2 = maxJ(W). T X n The parameters of the W matrix are determined by solving the problem using the FSStICS iterative algorithm, and then substituted into the S matrix. n =WX n The independent signal S can then be obtained.n Specific parameters;

[0055] S6. Extract the independent signal S obtained in step S5. n ={s1,s2...s n ...s N}, identify obvious errors or invalid data, and extract the independent signal S. n ={s1,s2...s n ...s N The individual signal components in the EEG signal are reconstructed to obtain the final EEG signal X. n ={x1,x2...x n ...x N};

[0056] S7. Based on the above, establish an attention analysis model consisting of a temporal feature extraction module, a spatial feature extraction module, and a classification module; the temporal feature extraction module is composed of several temporal feature extraction components of different scales connected in parallel, and the temporal feature extraction components consist of convolutional layers, activation layers, and pooling layers. The one-dimensional convolutional layer performs dimension-wise convolution operations on the original data of EEG signal data in temporal order, and uses the activation function of the activation layer and the pooling operation of the pooling layer to extract temporal features of different dimensions of EEG signal data;

[0057] In a preferred embodiment, the one-dimensional convolutional layer has a size of (1, L), where L = 0.5. M *fs, where fs is the sampling frequency, M is a positive integer representing the number of convolutional layers in the temporal feature extraction module; the activation function of the activation layer is... Where a is the activation function hyperparameter;

[0058] For the original input sample X input at time t t ={x 1,t ,x 2,t ...x n,t ...x N,t}, x n,t Let x represent the nth sample input at time t. n,t Input the temporal feature extraction module, where the m-th layer temporal feature extraction unit outputs out. m =Avgp(acv(Conv(1,x)) n,t Avgp(x) represents the average pooling operation in the pooling layer, acv(x) represents the activation operation in the activation layer, and conv(x) represents the one-dimensional convolution in the convolutional layer; the output of the temporal feature extraction unit at time t is out. t =f z ([out1,out2...out m ...out M]), where f z (x) represents the standardized operation;

[0059] The real-valued spatial feature extraction module consists of two spatial feature extraction components connected in parallel. The spatial feature extraction components consist of a spatial convolutional layer, an activation layer, and a pooling layer. The sizes of the two spatial convolutional layers are (D,1) and (0.5D,1), respectively, where D is the total number of EEG signal data channels.

[0060] The classification module is used to perform attention identification and analysis on the temporal feature information extracted by the temporal feature extraction module and the spatial feature information extracted by the real number spatial feature extraction module, and to prevent overfitting based on random deactivation;

[0061] S8. Input the raw EEG signal data into the analysis model established in step S7 to determine the students' attention at different times during the training process.

[0062] Based on the aforementioned steps, EEG signal detection equipment on the virtual training device is used to acquire students' EEG signal data at different times during the training process. The characteristics of this EEG signal data reflect the students' level of attention to the training content at different times. Since the virtual training of marine power plants involves a large amount of training content, and each training content involves different areas and objects, in order to determine the students' level of attention to specific objects and areas during the virtual training process, it is also necessary to determine the content that the students are paying attention to at the corresponding training time. During the virtual training of marine power plants, the content that students are paying attention to can be divided into two parts: tactile attention content and visual attention content. Tactile attention content refers to the information of specific interactive objects such as physical operating handles and virtual operating buttons that students interact with during the virtual training of marine power plants. The visual attention area refers to the specific area or target information that students collect visually during the training process. Among them, physical interactive objects such as operating handles can be determined by sensors and other devices on physical equipment to determine whether interaction has occurred, while interactive objects or information existing in the virtual screen content generally need to be extracted through the students' eye movement information during the virtual training of marine power plants. The specific steps include the following:

[0063] S9. Interactive data collection and processing refers to collecting information on the intersections of the handle ray, binocular line of sight, and virtual objects in the virtual training scenario of the ship's power unit during student training to determine the information of interest; including:

[0064] Blank data filling: Due to the unavoidable loss of visual data during training, such as blinking, information such as implementation data may be missing. It is necessary to replace the blank data with the last valid data before the time when the blank data appeared.

[0065] Coordinate transformation processing: Since different types or systems of controllers, head-mounted displays, and virtual reality screens may use different coordinate system establishment schemes due to configuration requirements, in order to ensure data consistency, coordinates should be unified according to the coordinate establishment methods of controllers, head-mounted displays, and virtual reality screens, and the original data should be uniformly modified according to the coordinate adjustment method, including: unifying eye-tracking data based on different left and right hand coordinate systems to the controller coordinate system, and uniformly transforming the head-mounted display coordinate system and the controller coordinate system to the Earth coordinate system.

[0066] Interactive Intent Pre-annotation Processing: Through historical data analysis, in the virtual training process of ship propulsion systems, when the intent is clear, there is generally a period of time between the generation of the intent and the actual execution of the interactive action. yc =200ms to 100ms time difference, further considering system data delay Δt xt To predict the intent of eye-tracking interaction, a multi-level classifier needs to be built based on the aforementioned time difference to predict the intent of the user's interaction behavior. The classification result of the classifier is matched with the existence of the interaction intent and the final result of the interaction intent. Generally, it can be divided into a first-level interaction intent classifier and a second-level interaction classifier. The first-level classifier is a binary classifier, and its classification result is used to determine whether the interaction intent exists. The second-level interaction classifier is a multi-classifier, and its classification result is the specific meaning of the interaction intent. It can generally be divided into selection intent, transmission intent, and no intent (including unconscious subconscious behavior or habitual behavior, etc.).

[0067] Based on the actual system characteristics and the students' speed of analyzing the training content, interactive behavior data with clear interactive intentions were extracted, and the data was processed from 200+ Dt before the interactive behavior data was generated. xt ~T yz +Dt xt The interaction data is input into the first-level classifier, and the T data before the interaction behavior data is generated is then processed. yz +Dt xt ~1000+Dt xt The interactive data is input into a two-level classifier, and the interactive data is classified and labeled according to the actual content of the interactive behavior data.

[0068] S10. Extract behavioral features from the classification and labeling results: The behavioral features include at least the sweeping speed at different data point locations, the fixation time for different data points, and the pupil distance at the corresponding time of the data point; Based on the feature values ​​of the corresponding data points in the classification and labeling structure, determine a preset threshold for judging the type of data point; In this embodiment, the preset threshold includes at least a preset continuous fixation time threshold for judging whether it is a fixation point, a preset discrete distance for judging whether adjacent fixation points correspond to the same interactive object, and a preset maximum defocus distance for judging whether the student's visual focus is located in the direction directly in front of the line of sight;

[0069] Extract the sweep speed of the line of sight or the interaction line of the handle through different data points during the training process. If the sweep speed is lower than the preset sweep speed threshold, it is initially identified as a fixation point. If the sweep speed exceeds the preset sweep speed threshold, it is initially identified as a saccade point.

[0070] For data points initially identified as fixation points, the following judgments are made:

[0071] S1. Determine whether the duration of sustained fixation is lower than the preset duration of sustained fixation threshold.

[0072] S2. Determine whether the distance between it and its adjacent gaze point exceeds the preset discrete distance;

[0073] S3. Determine whether the student's pupil distance is lower than the preset maximum defocus distance at the corresponding moment;

[0074] If the above three conditions are met, it is considered a valid fixation point. If S1 is not met, the data point is reclassified as a saccade point. If both S2 and S3 are not met, it is merged with the previous fixation point. In other cases, no processing is performed.

[0075] S11. Using the behavioral features obtained in step S10 as the classification basis, establish a multi-feature data classification model; obtain historical training interaction data, divide it into training data and validation data, and establish a multi-scale feature fusion prediction model for machine learning; its input is the feature value of each data point, and its output is the data point classification result; select an appropriate data point type multi-scale feature fusion prediction model according to the system computing power and data scale, use historical training interaction data to train and optimize the model, adjust the thresholds or parameters in steps S9 and S10, use validation data to verify and optimize the model accuracy, and the prediction result of the model meets the accuracy requirements;

[0076] S12. Collect real-time data during the virtual training process of ship power units, input it into the prediction model established in step S11, and judge and predict the type of each data point; analyze the prediction and judgment results to obtain the distribution of student attention at each moment and the distribution of student attention areas at each moment.

[0077] Determine whether the sustained attention received by the current training content meets the training needs. If it does, continue with the current training content; otherwise, stop or change the training content.

[0078] The order and arrangement of training content are adjusted according to the distribution of student attention at different times and the distribution characteristics of student attention areas at different times in order to obtain the best attention.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A virtual training method for ship propulsion systems, characterized in that, Includes the following steps: S1. Determine the basic types of virtual training tasks based on the virtual training plan for the power unit. These basic types should include at least: power unit configuration power-on operation, control panel standby operation, auxiliary system inspection and standby operation, unit grid connection operation, power unit power adjustment operation, and power unit fault diagnosis and troubleshooting operation. S2. Collect data on students' attention levels during the aforementioned virtual training tasks; assign students to conduct training simulation teaching in a standard experimental environment; after students are able to successfully complete the corresponding operations of the virtual training of the power device, conduct virtual training of the power device in batches; collect students' electroencephalogram (EEG) signal data under different training contents; the sampling frequency is 512Hz~1024Hz. S3. Extract student EEG signal data for each training content and generate raw data through data channel sampling; preprocess the EEG signals based on the main characteristics of the analyzed signals using high-pass or band-pass filtering, and then downsample to compress the data size. S4. High-frequency interference in the EEG signal is removed to obtain smoothed data. The processed parameter values ​​can be expressed as follows: Where x n ' represents the nth raw EEG signal acquired at time t, μ t-1 ' σ represents the average value of the raw EEG signal at time t-1. t-1 ' refers to the standard deviation of the original EEG signal at time t-1, n is the total number of samples collected, and α is the attenuation coefficient; S5. Since repetitive signal features are inevitably collected during the EEG signal data sampling process, an increase in the proportion of repetitive data will lead to a deviation in the time-domain characteristics reflected by the original data. Therefore, for the EEG signal X at time t collected in step S4... n ={x1,x2...x n ...x N We need to remove duplicate signal features and extract the independent signals s from each EEG signal source that reflect the characteristics of the original EEG signal. n This yields an independent signal S that reflects the actual feature vector of the original data. n ={s1,s2...s n ...s N Then, using the independent signal S n The reconstructed original signal is used as valid data for further processing, including the EEG signal x. n It is composed of independent signals s n X is obtained by linear combination of the independent signals. n =A T S n : {x1,x2...x n ...x N } = A T {s1,s2...s n ...s N Let W = A - 1 Then there is S n =WX n Where A is a linear mixing matrix, representing the sum of the values ​​of each independent signal s. n For each EEG signal x n The impact on independent signals S; n ={s n }, in its component signal s n In the case of absolute independence, its data entropy H(S) n H(s) is the data entropy of its component signal. n The sum of ) and the acquisition of EEG signals x n The best independent signal s in n Extracting the result, i.e., solving for the data entropy H(S) n ) and ∑H(s) n The classification that minimizes the difference between ) is defined by the objective function Goal1 = min|∑H(s) n )-H(S N )|; For independent signal S n , has logP(S n ) = logP(X n )-log|detW|, EEG signals are acquired from the same source signal through independent signal acquisition ports, therefore each component signal s n The variance of the original signal X should be kept consistent. n It can be regarded as each component signal s n After amplification and linear combination, its covariance is obtained as the independent signal S. n The covariance f, i.e. To ensure that each component signal s n If they are completely uncorrelated, then H(S) needs to be guaranteed. N The objective function is independent of W, meaning that the objective function needs to make log|detW|=0, or detW=1, which means that W needs to be a rotation matrix with all eigenvalues ​​equal to 1. Meanwhile, based on the principle of independent component analysis, in order to obtain the optimal uncorrelated component signal s n It is necessary to determine the non-Gaussianity J(W) in the W direction. T X n The maximum value is Goal2 = maxJ(W). T X n The parameters of the W matrix are determined by solving the problem using the FSStICS iterative algorithm, and then substituted into the S matrix. n =WX n The independent signal S can then be obtained. n Specific parameters; S6. Extract the independent signals S obtained in step S5. n ={s1,s2...s n ...s N }, identify obvious errors or invalid data, and extract the independent signal S. n ={s1,s2...s n ...s N The individual signal components in the EEG signal are reconstructed to obtain the final EEG signal X. n ={x1,x2...x n ...x N }; S7. Establish an attention analysis model consisting of a temporal feature extraction module, a spatial feature extraction module, and a classification module; the temporal feature extraction module is composed of several temporal feature extraction components of different scales connected in parallel, and the temporal feature extraction components consist of convolutional layers, activation layers, and pooling layers. The convolutional layers perform dimension-wise convolution operations on the original data of EEG signal data in temporal order, and use the activation function of the activation layer and the pooling operation of the pooling layer to extract temporal features of different dimensions of EEG signal data; The real-valued spatial feature extraction module consists of two spatial feature extraction components connected in parallel. The spatial feature extraction components consist of a spatial convolutional layer, an activation layer, and a pooling layer. The sizes of the two spatial convolutional layers are (D,1) and (0.5D,1), respectively, where D is the total number of EEG signal data channels. The classification module is used to perform attention identification and analysis on the temporal feature information extracted by the temporal feature extraction module and the spatial feature information extracted by the real number spatial feature extraction module, and to prevent overfitting based on random deactivation; S8. Input the raw EEG signal data into the analysis model established in step S7 to determine the students' attention at different times during the training process. S9. Interactive data collection and processing refers to collecting information on the intersections of the handle ray, binocular line of sight, and virtual objects in the virtual training scenario of the ship's power unit during the training process to determine the focus information; including: Blank data filling: Due to the unavoidable loss of gaze data during training caused by blinking, gaze data information is missing. It is necessary to replace the blank data with the last valid data before the time when the blank data appeared. Coordinate transformation processing: Since different types or systems of controllers, headsets, and virtual reality screens may use different coordinate system establishment schemes due to configuration requirements, in order to ensure data consistency, coordinates should be unified according to the coordinate establishment methods of controllers, headsets, and virtual reality screens, and the original data should be uniformly modified according to the coordinate adjustment method, including: unifying eye-tracking data based on different left and right hand coordinate systems to the controller coordinate system, and uniformly transforming the headset implementation coordinate system and the controller coordinate system to the Earth coordinate system. Interactive Intent Pre-annotation Processing: Through historical data analysis, during virtual training on ship propulsion systems, when the intent is clear, there exists a time t between the generation of the intent and the actual execution of the interactive action. yc =The time difference between 200ms and 100ms is further considered, taking into account the system data delay Δt. xt To predict the intent of eye-tracking interaction, a multi-level classifier needs to be built based on the aforementioned time difference to predict the intent of the user's interaction behavior. The classification result of the classifier is matched with the existence of the interaction intent and the final result of the interaction intent. It can be divided into a first-level interaction intent classifier and a second-level interaction classifier. The first-level classifier is a binary classifier, and its classification result is used to determine whether the interaction intent exists. The second-level interaction classifier is a multi-classifier, and its classification result is the specific meaning of the interaction intent, which can be divided into selection intent, transmission intent, and no intent. No intent includes unconscious subconscious behavior or habitual behavior. Based on the actual system characteristics and the students' speed of analyzing the training content, interactive behavior data with clear interactive intentions were extracted, and the data was processed from 200+ Dt before the interactive behavior data was generated. xt ~T yz +Dt xt The interaction data is input into the first-level classifier, and the T data before the interaction behavior data is generated is then processed. yz +Dt xt ~1000+Dt xt The interactive data is input into a two-level classifier, and the interactive data is classified and labeled according to the actual content of the interactive behavior data. S10. Extract behavioral features from the classification and labeling results: The behavioral features include at least the sweeping speed at different data point locations, the fixation time for different data points, and the pupil distance at the corresponding time of the data point; Based on the feature values ​​of the corresponding data points in the classification and labeling structure, determine a preset threshold for judging the type of data point; In this embodiment, the preset threshold includes at least a preset continuous fixation time threshold for judging whether it is a fixation point, a preset discrete distance for judging whether adjacent fixation points correspond to the same interactive object, and a preset maximum defocus distance for judging whether the student's visual focus is located in the direction directly in front of the line of sight; Extract the sweep speed of the gaze or handle interaction line as it passes through different data point positions during the training process. If the sweep speed is lower than the preset sweep speed threshold, it is initially identified as a fixation point. If the sweep speed exceeds the preset sweep speed threshold, it is initially identified as a saccade point. For data points initially identified as fixation points, the following judgments are made: S1. Determine whether the duration of sustained fixation is lower than the preset duration of sustained fixation threshold. S2. Determine whether the distance between it and its adjacent gaze point exceeds the preset discrete distance; S3. Determine whether the student's pupil distance is lower than the preset maximum defocus distance at the corresponding moment; If the above three conditions are met, it is considered a valid fixation point. If S1 is not met, the data point is reclassified as a saccade point. If both S2 and S3 are not met, it is merged with the previous fixation point. In other cases, no processing is performed. S11. Using the behavioral features obtained in step S10 as the classification basis, establish a multi-feature data classification model; obtain historical training interaction data, divide it into training data and validation data, and establish a multi-scale feature fusion prediction model for machine learning; its input is the feature value of each data point, and its output is the data point classification result; select an appropriate data point type multi-scale feature fusion prediction model according to the system computing power and data scale, use historical training interaction data to train and optimize the model, adjust the thresholds or parameters in steps S9 and S10, use validation data to verify and optimize the model accuracy, and the prediction result of the model meets the accuracy requirements; S12. Collect real-time data during the virtual training process of ship power units, input it into the prediction model established in step S11, and judge and predict the type of each data point; analyze the prediction and judgment results to obtain the distribution of student attention at each moment and the distribution of student attention areas at each moment. Determine whether the sustained attention received by the current training content meets the training needs. If it does, continue with the current training content; otherwise, stop or change the training content. The order and arrangement of training content are adjusted according to the distribution of student attention at different times and the distribution characteristics of student attention areas at different times in order to obtain the best attention.

2. The virtual training method for ship propulsion systems according to claim 1, characterized in that, In step S1, the power unit configuration power-on operation includes power configuration operation of each sub-component of the power unit, operation of the power unit power distribution panel and identification of indication information; the display and control console standby operation includes standby operation of the seawater system, standby operation of the fuel system, and standby operation of the oxygen supply system; the auxiliary system inspection and standby operation includes power unit oil level inspection operation, purging system initialization configuration operation, working fluid system initialization configuration operation, supply and return circuit initialization configuration operation, and gas distribution system initialization configuration operation; the unit grid connection operation includes grid connection control operation with the same unit; the power unit power adjustment operation includes power switch panel adjustment operation; and the power unit fault diagnosis and troubleshooting operation includes fault diagnosis and troubleshooting operation for common faults in marine power systems.

3. The virtual training method for ship propulsion systems according to claim 1, characterized in that, In step S7, the size of the one-dimensional convolutional layer is (1, L), where L = 0.

5. M *fs, where fs is the sampling frequency, M is a positive integer representing the number of convolutional layers in the temporal feature extraction module; the activation function of the activation layer is... Where a is the activation function hyperparameter; For the original input sample X input at time t t ={x 1,t ,x 2,t ...x n,t ...x N,t }, x n,t Let x represent the nth sample input at time t. n,t Input the temporal feature extraction module, where the m-th layer temporal feature extraction unit outputs out. m =Avgp(acv(Conv(1,x)) n,t Avgp(x) represents the average pooling operation in the pooling layer, acv(x) represents the activation operation in the activation layer, and conv(x) represents the one-dimensional convolution in the convolutional layer; the output of the temporal feature extraction unit at time t is out. t =f z ([out1,out2...out m ...out M ]), where f z (x) represents the standardized operation.

Images