Virtual training method for ship power device

Abstract

The invention belongs to the technical field of virtual training planning, and particularly relates to a virtual training method for a ship power device. According to the method, an optimization basis is provided for optimization of distribution of training contents in a virtual teaching process and an improvement reference is provided for virtual scene design by establishing a difference information analysis mechanism of attention changes of students in different time periods in a virtual training process and attention of different targets in a virtual scene; according to the method, teaching time and space classification of the virtual teaching content are improved in a targeted manner, interference in the teaching process is reduced, and teaching efficiency is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of virtual training planning, and in particular relates to a virtual training method for a ship power plant. Background Art

[0002] As the core of the ship power control system, the ship power plant is the key and difficult point in the ship operation teaching process, which involves different types of teaching content such as the structural principles, operation and maintenance, and troubleshooting of various components. Compared with other equipment or structures, the ship power system is large and complex, involving a wide range of aspects. In the actual teaching process, students can often only get a glimpse of it, and the teaching quality is limited. In order to improve the teaching effect, various colleges and universities have developed virtual training methods based on simulation equipment combined with virtual reality technology. However, due to the structural complexity of the power plant itself and the complicated operation content, the design of virtual reality training content and training steps can often only be planned and designed based on the teaching experience of each student. The actual teaching quality and rationality lack a scientific and reasonable analysis and management mechanism, resulting in the current virtual training teaching content mainly focusing on simple content such as structural display and animation simulation, students passively learning, less actual interactive content, and shallow teaching depth. Summary of the invention

[0003] The purpose of the present invention is to provide a virtual training method for ship power plants based on the analysis and processing of students' eye movements and electroencephalogram signals, to establish a mechanism for analyzing the difference information of students' attention changes in different time periods during virtual training and the attention to different targets in virtual scenes, to provide an optimization basis for optimizing the distribution of training content in the virtual teaching process, and to provide an improved reference for virtual scene design.

[0004] To achieve the above purpose, the present invention adopts the following technical solution.

[0005] A virtual training method for a ship power plant comprises the following steps:

[0006] S1. Determine the basic types of virtual training tasks according to the power plant virtual training plan, and the basic types include at least: power plant configuration and power-on operation, display console vehicle preparation operation, auxiliary system vehicle inspection and preparation operation, unit grid connection operation, power plant power regulation operation, power plant fault diagnosis and troubleshooting operation,

[0007] S2. Collect data on students' required attention in the aforementioned virtual training tasks; assign students to conduct training simulation teaching in a standard experimental environment, and conduct virtual training of power devices in batches after students can successfully complete the corresponding operations of virtual training of power devices; collect students' EEG signal data under different training contents; the sampling frequency is 512Hz~1024Hz;

[0008] S3, extracting the students' EEG signal data under each training content and sampling through the data channel to generate raw data; pre-processing using high-pass filtering or band-pass filtering based on the main features of the analyzed EEG signal, and then down-sampling to compress the data capacity;

[0009] S4. Remove high-frequency interference in the EEG signal to obtain smooth data. The parameter value after processing can be expressed as where x n ￠ is the nth original EEG signal collected at time t, μ t-1 ￠ represents the average value of the original EEG signal at time t-1, s t-1 ￠ refers to the standard deviation of the original EEG signal at time t-1, n is the total number of samples collected, and α refers to the attenuation coefficient;

[0010] S5. Since repeated signal features are inevitably collected during the EEG signal data sampling process, if the proportion of repeated data increases, the time domain features reflected by the original data will be deviated. Therefore, for the EEG signal X at time t collected in step S4, n ={x1,x2...x n ...x N}, we need to remove the repeated signal features and extract the independent signal s from each EEG signal source that reflects the original EEG signal features. n , and obtain an independent signal S that reflects the actual eigenvector of the original data n ={s1,s2...s n ...s N}, and then use the independent signal S n Reconstruct the original signal as valid data for the next step of processing. Generally speaking, it can be considered that the EEG signal x n is composed of independent signals s n The linear combination of the independent signals in , that is, X n =A T S n :

[0011] {x1,x2...x n ...x N =A T {s1,s2...s n ...s N}, let W = A- 1 Then there is S n =WX n ; Where A is a linear mixing matrix, which means that in each independent signal s n For each EEG signal x n The impact of independent signal S n ={sn}, in which its component signal s n In the case of absolute independence, the data entropy H(S n ) is the data entropy of its component signals H(s n ) and obtain the EEG signal x n The best independent signal in n Extract the result, that is, solve the data entropy H(S n ) and ∑H(s n ), that is, the objective function is Goal1=min|∑H(s n )-H(S N )|;

[0012] For an independent signal S n , logP(S n )=logP(X n )-log|detW|, the EEG signal is collected from the same source signal based on independent signal acquisition ports, so each component signal s n The variance of the original signal X n can be regarded as each component signal s n After amplification, the linear combination is obtained, and its covariance is the independent signal S n The covariance f of To ensure that each component signal s n Completely unrelated, it is necessary to ensure that H(S N ) has nothing to do with W, that is, the objective function needs to make log|detW|=0, that is, detW=1, that is, W needs to be a rotation matrix with all eigenvalues ​​1;

[0013] At the same time, based on the principle of independent component analysis, in order to obtain the best uncorrelated component signal s n , we need to determine the non-Gaussianity J(W T X n ) is the largest, that is, Goal2=maxJ(W T X n );Use FSStICS iterative algorithm to solve and determine the W matrix parameters, and then bring them into S n =WX n The independent signal S can be obtained n Specific parameters of

[0014] S6, extracting the independent signal S separated in step S5 n ={s1,s2...s n ...s N}, point out obvious errors or invalid data, and extract independent signal S n ={s1,s2...sn ...s N} to obtain the final EEG signal X n ={x1,x2...x n ...x N};

[0015] S7, establishing an attention analysis model consisting of a time domain feature extraction module, a spatial domain feature extraction module and a classification module; the time domain feature extraction module is composed of a number of time domain feature extraction components of different scales connected in parallel, and the time domain feature extraction component is composed of a convolution layer, an activation layer, and a pooling layer. The one-dimensional convolution layer performs a dimension-by-dimensional convolution operation on the original data of the EEG signal data in a temporal order, and uses the activation function of the activation layer and the pooling operation of the pooling layer to extract the time domain features of the EEG signal data in different dimensions;

[0016] The real number spatial domain feature extraction module is composed of two spatial feature extraction components in parallel. The spatial feature extraction component consists of a spatial convolution layer, an activation layer, and a pooling layer. The sizes of the two spatial convolution layers are (D, 1) and (0.5D, 1), respectively, where D is the total number of EEG signal data channels.

[0017] The classification module is used to perform attention recognition analysis on the time domain feature information extracted by the time domain feature extraction module and the spatial feature information extracted by the real number spatial domain feature extraction module, and to prevent overfitting based on random inactivation;

[0018] S8, inputting the original EEG signal data into the analysis model established in step S7 to determine the students' attention at different times during the training process;

[0019] S9. Interactive data collection and processing refers to collecting the intersection of handle rays, binocular sight lines and virtual objects in the virtual training scene of ship power plant during student training to determine the focus information; including:

[0020] Blank data filling: Due to the inevitable loss of sight data due to blinking and other reasons during the training process, the realization data and other information are missing. It is necessary to replace the blank data with the last valid data before the blank data appears;

[0021] Coordinate conversion processing: Since different types or systems of controllers, head-mounted displays, and virtual reality screens may use different coordinate system establishment schemes due to configuration requirements, in order to ensure data consistency, the coordinates should be unified according to the coordinate establishment methods of the controller, head-mounted display, and virtual reality screen, and the original data should be uniformly modified according to the coordinate adjustment method, including: unifying the eye movement data based on different left and right hand coordinate systems into the controller coordinate system, and converting the head-mounted display coordinate system and the controller coordinate system into the earth coordinate system;

[0022] Interaction intention pre-labeling processing: Through historical data analysis, in the process of virtual training of ship power plants, when the intention is clear, there is generally a period of time from the generation of the intention to the actual execution of the interactive action. yc =200mS~100mS time difference, further considering the system data delay Δt xt , for the prediction of line of sight interaction intention, it is necessary to establish a multi-level classifier based on the aforementioned time difference to predict the intention of the user's interaction behavior. The classification result of the classifier matches whether the interaction intention exists and the final result of the interaction intention. It can generally be divided into a first-level interaction intention classifier and a second-level interaction classifier. The first-level classifier is a binary classifier, and its classification result is used to determine whether the interaction intention exists. The second-level interaction classifier is a multi-classifier, and its classification result is the specific meaning of the interaction intention, which can generally be divided into selection intention, transmission intention and no intention (including unconscious subconscious behavior or habitual behavior, etc.);

[0023] According to the actual system characteristics and the students' analysis speed of the training content, the interactive behavior data with clear interactive intentions are extracted, and the 200+Dt before the interactive behavior data is generated is converted into xt ~T yz +Dt xt The interaction data is input into the first-level classifier, and the T yz +Dt xt ~1000+Dt xt The interaction data is input into the secondary classifier, and the interaction data is classified and labeled according to the actual content of the interaction behavior data;

[0024] S10, extracting behavioral features from the classification and annotation results: the behavioral features at least include the sweeping speed passing through different data point positions, the fixation time for different data points, and the pupil distance at the corresponding moment of the data point; based on the feature value of the corresponding data point in the classification and annotation structure, determining a preset threshold for judging the type of the data point; in this embodiment, the preset threshold at least includes a preset continuous fixation time threshold for judging whether it is a fixation point, a preset discrete distance for judging whether adjacent fixation points correspond to the same interactive object, and a preset maximum defocus distance for judging whether the student's visual focus is located in the direction of the line of sight;

[0025] Extract the sweeping speed of the sight line or the handle interaction line passing through different data points during the training process. If the sweeping speed is lower than the preset sweeping speed threshold, it is preliminarily identified as a fixation point. If the sweeping speed exceeds the preset sweeping speed threshold, it is preliminarily identified as a saccade point.

[0026] For the data points initially identified as fixation points, the following judgments are made:

[0027] S1, determining whether the continuous gaze time is lower than a preset continuous gaze time threshold;

[0028] S2, determining whether the distance between the adjacent gaze point and the adjacent gaze point exceeds a preset discrete distance;

[0029] S3, determining whether the pupil distance of the student at the corresponding moment is lower than the preset maximum defocus distance;

[0030] If the above three conditions are met, it will be regarded as a valid fixation point. If S1 is not met, the data point will be reclassified as a saccade point. If both S2 and S3 are not met, it will be merged with the previous fixation point. In other cases, it will not be processed.

[0031] S11, using the behavioral features obtained in step S10 as the classification basis, establishing a multi-feature data classification model; obtaining historical training interaction data, dividing it into training data and verification data, and establishing a multi-scale feature fusion prediction model for machine learning; its input is the feature value of each data point, and its output is the data point classification result; selecting a suitable data point type multi-scale feature fusion prediction model according to the system computing power and data scale, using historical training interaction data to train and optimize the model, adjusting the thresholds or parameters in steps S9 and S10, using verification data to verify and optimize the model accuracy, and the prediction result of the model meets the accuracy requirements;

[0032] S12, collecting real-time data during the virtual training of ship power plants, inputting it into the prediction model established in step S11, judging and predicting the type of each data point; according to the prediction and judgment results, obtaining the distribution of students' attention at each moment and the distribution of students' attention areas at each moment;

[0033] Determine whether the sustained attention of the current training content meets the training needs. If so, continue with the current training content. If not, stop or change the training content.

[0034] Adjust the order and arrangement of training content according to the distribution of students' attention at each moment and the distribution characteristics of students' attention areas at each moment to obtain the best attention.

[0035] A further improvement or preferred implementation of the aforementioned ship power plant virtual training method, in step S1, the power plant configuration and power-on operation includes the power plant sub-component assembly power supply configuration operation, the power plant distribution panel operation and the indication information identification operation; the display console preparation operation includes the seawater system preparation operation, the fuel system preparation operation, and the oxygen supply system preparation operation; the auxiliary system inspection and preparation operation includes the power plant oil level inspection operation, the purge system initialization configuration operation, the working fluid system initialization configuration operation, the supply and return circuit initialization configuration operation, and the gas distribution system initialization configuration operation; the unit grid connection operation includes the same unit grid connection control operation; the power plant power adjustment operation includes the power switch panel adjustment operation; the power plant fault diagnosis and troubleshooting operation includes the common fault diagnosis and troubleshooting operation for the ship power system;

[0036] A further improvement or preferred implementation of the aforementioned ship power plant virtual training method is that in step S7, the size of the one-dimensional convolutional layer is (1, L), where L = 0.5 M *fs, where fs is the sampling frequency, M is a positive integer representing the number of convolutional layers in the time domain feature extraction module; the activation function of the activation layer is Where a is the activation function hyperparameter;

[0037] For the original input sample X at time t t ={x 1,t ,x 2,t ...x n,t ...x N,t}, x n,t represents the nth sample input at time t, and x n,t Input time domain feature extraction module, where the mth layer time domain feature extraction unit outputs out m =Avgp(acv(Conv(1,x n,t )); Avgp(x) represents the average pooling operation of the pooling layer, acv(x) represents the activation operation of the activation layer, conv(x) represents the one-dimensional convolution of the convolution layer; the output of the time domain feature extraction unit at time t is out t =f z ([out1,out2...out m ...out M ]), where f z (x) indicates a normalization operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a schematic diagram of a virtual training device for a ship power plant;

[0039] Figure 2 It is a flow chart of the virtual training method for ship power plant. DETAILED DESCRIPTION

[0040] The present invention is described in detail below in conjunction with specific embodiments.

[0041] Background technology description:

[0042] Attention mechanism: During the virtual training of ship power plant, it is difficult to ensure that students can continue to pay attention to specific goals as expected due to the students' cognitive characteristics, limited information processing ability and inevitable interference factors. In the actual training process, in most cases, it is possible to confirm whether students pay attention to different times or information in various training processes and the degree of their attention by analyzing the focus of students' eyes during the virtual training of ship power plant.

[0043] Eye movement signal tracking: Eye movement signals are a direct expression of students’ target focus during the virtual training of ship power plants. They can be used to analyze the degree of attention paid to various types of information during the training process. This is also the basis for eye movement control in most virtual training devices. Currently, mainstream virtual reality systems and various virtual reality equipment are equipped with eye movement signal tracking detection to achieve virtual interaction, optimize virtual rendering control and other functions.

[0044] 2. Technical solution description

[0045] The present application discloses a method for virtual training of a ship power plant, which is mainly used to control the effect of teaching content by analyzing and predicting students' attention areas and attention during the virtual training of a ship power plant, thereby increasing the attention paid to important content during the learning process and improving the training effect. The method mainly includes the following steps:

[0046] S1. Determine the basic types of virtual training tasks according to the power plant virtual training plan, and the basic types include at least: power plant configuration and power-on operation, display console vehicle preparation operation, auxiliary system vehicle inspection and preparation operation, unit grid connection operation, power plant power regulation operation, power plant fault diagnosis and troubleshooting operation,

[0047] The power unit configuration and power-on operation includes the power unit sub-components assembly power supply configuration operation, the power unit power distribution panel operation and the indication information identification operation; the display console preparation operation includes the seawater system preparation operation, the fuel system preparation operation, and the oxygen supply system preparation operation; the auxiliary system inspection and preparation operation includes the power unit oil level inspection operation, the purge system initialization configuration operation, the working fluid system initialization configuration operation, the supply and return circuit initialization configuration operation, and the gas distribution system initialization configuration operation; the unit grid connection operation includes the same unit grid connection control operation; the power unit power adjustment operation includes the power switch panel adjustment operation; the power unit fault diagnosis and troubleshooting operation includes the common fault diagnosis and troubleshooting operation for the ship power system;

[0048] S2. Collect data on students' required attention in the aforementioned virtual training tasks; assign students to conduct training simulation teaching in a standard experimental environment, and conduct virtual training of power devices in batches after students can successfully complete the corresponding operations of virtual training of power devices; collect students' EEG signal data under different training contents; the sampling frequency is 512Hz~1024Hz;

[0049] S3, extracting the students' EEG signal data under each training content and sampling through the data channel to generate raw data; pre-processing using high-pass filtering or band-pass filtering based on the main features of the analyzed EEG signal, and then down-sampling to compress the data capacity;

[0050] S4. Remove high-frequency interference in the EEG signal to obtain smooth data. The parameter value after processing can be expressed as where x n ￠ is the nth original EEG signal collected at time t, μ t-1 ￠ represents the average value of the original EEG signal at time t-1, s t-1 ￠ refers to the standard deviation of the original EEG signal at time t-1, n is the total number of samples collected, and α refers to the attenuation coefficient;

[0051] S5. Since repeated signal features are inevitably collected during the EEG signal data sampling process, if the proportion of repeated data increases, the time domain features reflected by the original data will be deviated. Therefore, for the EEG signal X at time t collected in step S4, n ={x1,x2...x n ...x N}, we need to remove the repeated signal features and extract the independent signal s from each EEG signal source that reflects the original EEG signal features. n , and obtain an independent signal S that reflects the actual eigenvector of the original data n ={s1,s2...s n ...s N}, and then use the independent signal S n Reconstruct the original signal as valid data for the next step of processing. Generally speaking, it can be considered that the EEG signal x n is composed of independent signals s n The linear combination of the independent signals in , that is, X n =A T S n :

[0052] {x1,x2...x n ...x N =A T {s1,s2...sn ...s N}, let W = A- 1 Then there is S n =WX n ; Where A is a linear mixing matrix, which means that in each independent signal s n For each EEG signal x n The impact of independent signal S n ={s n}, in which its component signal s n In the case of absolute independence, the data entropy H(S n ) is the data entropy of its component signals H(s n ) and obtain the EEG signal x n The best independent signal in n Extract the result, that is, solve the data entropy H(S n ) and ∑H(s n ), that is, the objective function is Goal1=min|∑H(s n )-H(S N )|;

[0053] For an independent signal S n , logP(S n )=logP(X n )-log|detW|, the EEG signal is collected from the same source signal based on independent signal acquisition ports, so each component signal s n The variance of the original signal X n can be regarded as each component signal s n After amplification, the linear combination is obtained, and its covariance is the independent signal S n The covariance f of To ensure that each component signal s n Completely unrelated, it is necessary to ensure that H(S N ) has nothing to do with W, that is, the objective function needs to make log|detW|=0, that is, detW=1, that is, W needs to be a rotation matrix with all eigenvalues ​​1;

[0054] At the same time, based on the principle of independent component analysis, in order to obtain the best uncorrelated component signal s n , we need to determine the non-Gaussianity J(W T X n ) is the largest, that is, Goal2=maxJ(W T X n );Use FSStICS iterative algorithm to solve and determine the W matrix parameters, and then bring them into S n =WX n The independent signal S can be obtainedn Specific parameters of

[0055] S6, extracting the independent signal S separated in step S5 n ={s1,s2...s n ...s N}, point out obvious errors or invalid data, and extract independent signal S n ={s1,s2...s n ...s N} to obtain the final EEG signal X n ={x1,x2...x n ...x N};

[0056] S7. On the basis of the above, establish an attention analysis model consisting of a time domain feature extraction module, a spatial domain feature extraction module and a classification module; the time domain feature extraction module is composed of a number of time domain feature extraction components of different scales connected in parallel, and the time domain feature extraction component is composed of a convolution layer, an activation layer, and a pooling layer. The one-dimensional convolution layer performs a dimension-by-dimensional convolution operation on the original data of the EEG signal data in a time sequence, and uses the activation function of the activation layer and the pooling operation of the pooling layer to extract the time domain features of the EEG signal data in different dimensions;

[0057] As a preferred implementation, the one-dimensional convolutional layer has a volume size of (1, L), where L = 0.5 M *fs, where fs is the sampling frequency, M is a positive integer representing the number of convolutional layers in the time domain feature extraction module; the activation function of the activation layer is Where a is the activation function hyperparameter;

[0058] For the original input sample X at time t t ={x 1,t ,x 2,t ...x n,t ...x N,t}, x n,t represents the nth sample input at time t, and x n,t Input time domain feature extraction module, where the mth layer time domain feature extraction unit outputs out m =Avgp(acv(Conv(1,x n,t )); Avgp(x) represents the average pooling operation of the pooling layer, acv(x) represents the activation operation of the activation layer, conv(x) represents the one-dimensional convolution of the convolution layer; the output of the time domain feature extraction unit at time t is out t =f z ([out1,out2...out m ...out M]), where f z (x) indicates a standardized operation;

[0059] The real number spatial domain feature extraction module is composed of two spatial feature extraction components in parallel. The spatial feature extraction component consists of a spatial convolution layer, an activation layer, and a pooling layer. The sizes of the two spatial convolution layers are (D, 1) and (0.5D, 1), respectively, where D is the total number of EEG signal data channels.

[0060] The classification module is used to perform attention recognition analysis on the time domain feature information extracted by the time domain feature extraction module and the spatial feature information extracted by the real number spatial domain feature extraction module, and to prevent overfitting based on random inactivation;

[0061] S8, inputting the original EEG signal data into the analysis model established in step S7 to determine the students' attention at different times during the training process;

[0062] Based on the above steps, the EEG signal detection device on the virtual training equipment is used to obtain the EEG signal data of the students at different times during the training process. The characteristics of the EEG signal data show the degree of students' attention to the training content at different times. Since the virtual training of ship power plants involves a large amount of training content, each training content also involves different regional objects. In order to determine the degree of students' attention to specific objects and regions during the virtual training process, it is also necessary to determine the content that students are paying attention to at the corresponding training time. During the virtual training of ship power plants, the content that students are paying attention to can range from two parts, including tactile attention content and visual attention content. Tactile attention content refers to students' specific interactive object information such as physical operating handles and virtual operating buttons during the virtual training of ship power plants. The visual attention area refers to the specific area or target information collected by vision during the training of students; among which the physical interactive objects such as operating handles can be determined by sensors on physical equipment to determine whether interaction occurs, while the interactive objects or information existing in the virtual screen content generally need to be extracted through the eye movement information of students during the virtual training of ship power plants. Specifically, the following steps are included:

[0063] S9. Interactive data collection and processing refers to collecting the intersection of handle rays, binocular sight lines and virtual objects in the virtual training scene of ship power plant during student training to determine the focus information; including:

[0064] Blank data filling: Due to the inevitable loss of sight data due to blinking and other reasons during the training process, the realization data and other information are missing. It is necessary to replace the blank data with the last valid data before the blank data appears;

[0065] Coordinate conversion processing: Since different types or systems of controllers, head-mounted displays, and virtual reality screens may use different coordinate system establishment schemes due to configuration requirements, in order to ensure data consistency, the coordinates should be unified according to the coordinate establishment methods of the controller, head-mounted display, and virtual reality screen, and the original data should be uniformly modified according to the coordinate adjustment method, including: unifying the eye movement data based on different left and right hand coordinate systems into the controller coordinate system, and converting the head-mounted display coordinate system and the controller coordinate system into the earth coordinate system;

[0066] Interaction intention pre-labeling processing: Through historical data analysis, in the process of virtual training of ship power plants, when the intention is clear, there is generally a period of time from the generation of the intention to the actual execution of the interactive action. yc =200mS~100mS time difference, further considering the system data delay Δt xt , for the prediction of line of sight interaction intention, it is necessary to establish a multi-level classifier based on the aforementioned time difference to predict the intention of the user's interaction behavior. The classification result of the classifier matches whether the interaction intention exists and the final result of the interaction intention. It can generally be divided into a first-level interaction intention classifier and a second-level interaction classifier. The first-level classifier is a binary classifier, and its classification result is used to determine whether the interaction intention exists. The second-level interaction classifier is a multi-classifier, and its classification result is the specific meaning of the interaction intention, which can generally be divided into selection intention, transmission intention and no intention (including unconscious subconscious behavior or habitual behavior, etc.);

[0067] According to the actual system characteristics and the students' analysis speed of the training content, the interactive behavior data with clear interactive intentions are extracted, and the 200+Dt before the interactive behavior data is generated is converted into xt ~T yz +Dt xt The interaction data is input into the first-level classifier, and the T yz +Dt xt ~1000+Dt xt The interaction data is input into the secondary classifier, and the interaction data is classified and labeled according to the actual content of the interaction behavior data;

[0068] S10, extracting behavioral features from the classification and annotation results: the behavioral features at least include the sweeping speed passing through different data point positions, the fixation time for different data points, and the pupil distance at the corresponding moment of the data point; based on the feature value of the corresponding data point in the classification and annotation structure, determining a preset threshold for judging the type of the data point; in this embodiment, the preset threshold at least includes a preset continuous fixation time threshold for judging whether it is a fixation point, a preset discrete distance for judging whether adjacent fixation points correspond to the same interactive object, and a preset maximum defocus distance for judging whether the student's visual focus is located in the direction of the line of sight;

[0069] Extract the sweeping speed of the sight line or the handle interaction line passing through different data points during the training process. If the sweeping speed is lower than the preset sweeping speed threshold, it is preliminarily identified as a fixation point. If the sweeping speed exceeds the preset sweeping speed threshold, it is preliminarily identified as a saccade point.

[0070] For the data points initially identified as fixation points, the following judgments are made:

[0071] S1, determining whether the continuous gaze time is lower than a preset continuous gaze time threshold;

[0072] S2, determining whether the distance between the adjacent gaze point and the adjacent gaze point exceeds a preset discrete distance;

[0073] S3, determining whether the pupil distance of the student at the corresponding moment is lower than the preset maximum defocus distance;

[0074] If the above three conditions are met, it will be regarded as a valid fixation point. If S1 is not met, the data point will be reclassified as a saccade point. If both S2 and S3 are not met, it will be merged with the previous fixation point. In other cases, it will not be processed.

[0075] S11, using the behavioral features obtained in step S10 as the classification basis, establishing a multi-feature data classification model; obtaining historical training interaction data, dividing it into training data and verification data, and establishing a multi-scale feature fusion prediction model for machine learning; its input is the feature value of each data point, and its output is the data point classification result; selecting a suitable data point type multi-scale feature fusion prediction model according to the system computing power and data scale, using historical training interaction data to train and optimize the model, adjusting the thresholds or parameters in steps S9 and S10, using verification data to verify and optimize the model accuracy, and the prediction result of the model meets the accuracy requirements;

[0076] S12, collecting real-time data during the virtual training of ship power plants, inputting it into the prediction model established in step S11, judging and predicting the type of each data point; according to the prediction and judgment results, obtaining the distribution of students' attention at each moment and the distribution of students' attention areas at each moment;

[0077] Determine whether the sustained attention of the current training content meets the training needs. If so, continue with the current training content. If not, stop or change the training content.

[0078] Adjust the order and arrangement of training content according to the distribution of students' attention at each moment and the distribution characteristics of students' attention areas at each moment to obtain the best attention;

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A virtual training method for a ship power plant, characterized in that: The steps include: S1. Determine the basic types of virtual training tasks according to the power plant virtual training plan, and the basic types include at least: power plant configuration and power-on operation, display console vehicle preparation operation, auxiliary system vehicle inspection and preparation operation, unit grid connection operation, power plant power regulation operation, power plant fault diagnosis and troubleshooting operation, S2. Collect data on students' required attention in the aforementioned virtual training tasks; assign students to conduct training simulation teaching in a standard experimental environment, and conduct virtual training of power devices in batches after students can successfully complete the corresponding operations of virtual training of power devices; collect students' EEG signal data under different training contents; the sampling frequency is 512Hz~1024Hz; S3, extracting the students' EEG signal data under each training content and sampling through the data channel to generate raw data; pre-processing using high-pass filtering or band-pass filtering based on the main features of the analyzed EEG signal, and then down-sampling to compress the data capacity; S4. Remove high-frequency interference in the EEG signal to obtain smooth data. The parameter value after processing can be expressed as where x n ￠ is the nth original EEG signal collected at time t, μ t-1 ￠ represents the average value of the original EEG signal at time t-1, s t-1 ￠ refers to the standard deviation of the original EEG signal at time t-1, n is the total number of samples collected, and α refers to the attenuation coefficient; S5. Since repeated signal features are inevitably collected during the EEG signal data sampling process, if the proportion of repeated data increases, the time domain features reflected by the original data will be deviated. Therefore, for the EEG signal X at time t collected in step S4, n ={x1,x2...x n ...x N }, we need to remove the repeated signal features and extract the independent signal s from each EEG signal source that reflects the original EEG signal features. n , and obtain an independent signal S that reflects the actual eigenvector of the original data n ={s1,s2...s n ...s N }, and then use the independent signal S n Reconstruct the original signal as valid data for the next step of processing. Generally speaking, it can be considered that the EEG signal x n is composed of independent signals s n The linear combination of the independent signals in, that is, X n =A T S n : {x1,x2...x n ...x N =A T {s1,s2...s n ...s N }, let W = A- 1 Then there is S n =WX n ; Where A is a linear mixing matrix, which means that in each independent signal s n For each EEG signal x n The impact of independent signal S n ={s n }, in which its component signal s n In the case of absolute independence, the data entropy H(S n ) is the data entropy of its component signals H(s n ) and obtain the EEG signal x n The best independent signal in n Extract the result, that is, solve the data entropy H(S n ) and ∑H(s n ), that is, the objective function is Goal1=min|∑H(s n )-H(S N )|; For an independent signal S n , logP(S n )=logP(X n )-log|detW|, the EEG signal is collected from the same source signal based on independent signal acquisition ports, so each component signal s n The variance of the original signal X n can be regarded as each component signal s n After amplification, the linear combination is obtained, and its covariance is the independent signal S n The covariance f of To ensure that each component signal s n Completely unrelated, it is necessary to ensure that H(S N ) has nothing to do with W, that is, the objective function needs to make log|detW|=0, that is, detW=1, that is, W needs to be a rotation matrix with all eigenvalues ​​1; At the same time, based on the principle of independent component analysis, in order to obtain the best uncorrelated component signal s n , we need to determine the non-Gaussianity J(W T X n ) is the largest, that is, Goal2=maxJ(W T X n );Use FSStICS iterative algorithm to solve and determine the W matrix parameters, and then bring them into S n =WX n The independent signal S can be obtained n Specific parameters of S6, extracting the independent signal S separated in step S5 n ={s1,s2...s n ...s N }, point out obvious errors or invalid data, and extract independent signal S n ={s1,s2...s n ...s N } to obtain the final EEG signal X n ={x1,x2...x n ...x N }; S7, establishing an attention analysis model consisting of a time domain feature extraction module, a spatial domain feature extraction module and a classification module; the time domain feature extraction module is composed of a number of time domain feature extraction components of different scales connected in parallel, and the time domain feature extraction component is composed of a convolution layer, an activation layer, and a pooling layer. The one-dimensional convolution layer performs a dimension-by-dimensional convolution operation on the original data of the EEG signal data in a temporal order, and uses the activation function of the activation layer and the pooling operation of the pooling layer to extract the time domain features of the EEG signal data in different dimensions; The real number spatial domain feature extraction module is composed of two spatial feature extraction components in parallel. The spatial feature extraction component consists of a spatial convolution layer, an activation layer, and a pooling layer. The sizes of the two spatial convolution layers are (D, 1) and (0.5D, 1), respectively, where D is the total number of EEG signal data channels. The classification module is used to perform attention recognition analysis on the time domain feature information extracted by the time domain feature extraction module and the spatial feature information extracted by the real number spatial domain feature extraction module, and to prevent overfitting based on random inactivation; S8, inputting the original EEG signal data into the analysis model established in step S7 to determine the students' attention at different times during the training process; S9. Interactive data collection and processing refers to collecting the intersection of handle rays, binocular sight lines and virtual objects in the virtual training scene of ship power plant during the academy training process to determine the focus information; including: Blank data filling: Due to the inevitable loss of sight data due to blinking and other reasons during the training process, the realization data and other information are missing. It is necessary to replace the blank data with the last valid data before the blank data appears; Coordinate conversion processing: Since different types or systems of controllers, head-mounted displays, and virtual reality screens may use different coordinate system establishment schemes due to configuration requirements, in order to ensure data consistency, the coordinates should be unified according to the coordinate establishment methods of the controller, head-mounted display, and virtual reality screen, and the original data should be uniformly modified according to the coordinate adjustment method, including: unifying the eye movement data based on different left and right hand coordinate systems into the controller coordinate system, and converting the head-mounted display coordinate system and the controller coordinate system into the earth coordinate system; Interaction intention pre-labeling processing: Through historical data analysis, in the process of virtual training of ship power plants, when the intention is clear, there is generally a period of time from the generation of the intention to the actual execution of the interactive action. yc =200mS~100mS time difference, further considering the system data delay Δt xt , for the prediction of line of sight interaction intention, it is necessary to establish a multi-level classifier based on the aforementioned time difference to predict the intention of the user's interaction behavior. The classification result of the classifier matches whether the interaction intention exists and the final result of the interaction intention. It can generally be divided into a first-level interaction intention classifier and a second-level interaction classifier. The first-level classifier is a binary classifier, and its classification result is used to determine whether the interaction intention exists. The second-level interaction classifier is a multi-classifier, and its classification result is the specific meaning of the interaction intention, which can generally be divided into selection intention, transmission intention and no intention (including unconscious subconscious behavior or habitual behavior, etc.); According to the actual system characteristics and the students' analysis speed of the training content, the interactive behavior data with clear interactive intentions are extracted, and the 200+Dt before the interactive behavior data is generated is converted into xt ~T yz +Dt xt The interaction data is input into the first-level classifier, and the T yz +Dt xt ~1000+Dt xt The interaction data is input into the secondary classifier, and the interaction data is classified and labeled according to the actual content of the interaction behavior data; S10, extracting behavioral features from the classification and annotation results: the behavioral features at least include the sweeping speed passing through different data point positions, the fixation time for different data points, and the pupil distance at the corresponding moment of the data point; based on the feature value of the corresponding data point in the classification and annotation structure, determining a preset threshold for judging the type of the data point; in this embodiment, the preset threshold at least includes a preset continuous fixation time threshold for judging whether it is a fixation point, a preset discrete distance for judging whether adjacent fixation points correspond to the same interactive object, and a preset maximum defocus distance for judging whether the student's visual focus is located in the direction of the line of sight; Extract the sweeping speed of the sight line or the handle interaction line passing through different data points during the training process. If the sweeping speed is lower than the preset sweeping speed threshold, it is preliminarily identified as a fixation point. If the sweeping speed exceeds the preset sweeping speed threshold, it is preliminarily identified as a saccade point. For the data points initially identified as fixation points, the following judgments are made: S1, determining whether the continuous gaze time is lower than a preset continuous gaze time threshold; S2, determining whether the distance between the adjacent gaze point and the adjacent gaze point exceeds a preset discrete distance; S3, determining whether the pupil distance of the student at the corresponding moment is lower than the preset maximum defocus distance; If the above three conditions are met, it will be regarded as a valid fixation point. If S1 is not met, the data point will be reclassified as a saccade point. If both S2 and S3 are not met, it will be merged with the previous fixation point. In other cases, it will not be processed. S11, using the behavioral features obtained in step S10 as the classification basis, establishing a multi-feature data classification model; obtaining historical training interaction data, dividing it into training data and verification data, and establishing a multi-scale feature fusion prediction model for machine learning; its input is the feature value of each data point, and its output is the data point classification result; selecting a suitable data point type multi-scale feature fusion prediction model according to the system computing power and data scale, using historical training interaction data to train and optimize the model, adjusting the thresholds or parameters in steps S9 and S10, using verification data to verify and optimize the model accuracy, and the prediction result of the model meets the accuracy requirements; S12, collecting real-time data during the virtual training of ship power plants, inputting it into the prediction model established in step S11, judging and predicting the type of each data point; according to the prediction and judgment results, obtaining the distribution of students' attention at each moment and the distribution of students' attention areas at each moment; Determine whether the sustained attention of the current training content meets the training needs. If so, continue with the current training content. If not, stop or change the training content. Adjust the order and arrangement of training content according to the distribution of students' attention at each moment and the distribution characteristics of students' attention areas at each moment to obtain the best attention.

2. The ship power plant virtual training method according to claim 1, characterized in that: In the step S1, the power unit configuration and power-on operation includes the power unit sub-component assembly power supply configuration operation, the power unit distribution panel operation and the indication information identification operation; the display console preparation operation includes the seawater system preparation operation, the fuel system preparation operation, and the oxygen supply system preparation operation; the auxiliary system inspection and preparation operation includes the power unit oil level inspection operation, the purge system initialization configuration operation, the working fluid system initialization configuration operation, the supply and return circuit initialization configuration operation, and the gas distribution system initialization configuration operation; the unit grid-connected operation includes the same unit grid-connected control operation; the power unit power adjustment operation includes the power switch panel adjustment operation; the power unit fault diagnosis and troubleshooting operation includes the common fault diagnosis and troubleshooting operation for the ship power system.

3. The ship power plant virtual training method according to claim 1, characterized in that: In step S7, the size of the one-dimensional convolutional layer is (1, L), where L = 0.5 M *fs, where fs is the sampling frequency, M is a positive integer representing the number of convolutional layers in the time domain feature extraction module; the activation function of the activation layer is Where a is the activation function hyperparameter; For the original input sample X at time t t ={x 1,t ,x 2,t ...x n,t ...x N,t }, x n,t represents the nth sample input at time t, and x n,t Input time domain feature extraction module, where the mth layer time domain feature extraction unit outputs out m =Avgp(acv(Conv(1,x n,t )); Avgp(x) represents the average pooling operation of the pooling layer, acv(x) represents the activation operation of the activation layer, conv(x) represents the one-dimensional convolution of the convolution layer; the output of the time domain feature extraction unit at time t is out t =f z ([out1,out2...out m ...out M ]), where f z (x) indicates a normalization operation.

Images