Data augmentation method and apparatus

Abstract

This invention relates to the field of data augmentation technology, specifically to a data augmentation method and apparatus. The method involves acquiring an initial training dataset corresponding to the training pilots. This initial training dataset includes initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight stages, and corresponding cognitive load levels. Feature extraction is performed on the initial training dataset, and initial fusion features are generated based on the extracted features. These initial fusion features are then input into a preset data augmentation model to augment the initial training dataset, generating a target training dataset. This addresses the problem in existing technologies where the collection of actual pilot flight data is limited by high costs and scenario safety constraints, resulting in a limited training dataset size and a low proportion of samples from high-load, complex task scenarios. This ensures the accuracy of the target cognitive load identification model trained on the target training dataset.

Description

Technical Field

[0001] This invention relates to the field of data expansion technology, and more specifically to data expansion methods and apparatus. Background Technology

[0002] In a human-machine interface closed-loop system, pilots constantly engage in cognitive activities such as information perception, processing, and decision-making while performing flight missions. With the rapid development of civil aviation and artificial intelligence technologies, modern aircraft cockpit systems are highly information-intensive, making the human-machine system increasingly complex and placing higher demands on pilots' cognitive abilities. The assessment of cognitive load began in the early 1970s, and current methods for assessing cognitive load can be divided into subjective and objective assessments. Therefore, accurately identifying the real-time cognitive load level of pilots has become a crucial link in the aviation safety assurance system.

[0003] However, the collection of actual flight data by pilots is limited by high costs and scenario safety constraints, resulting in a limited training dataset size and a low proportion of samples from high-load and complex task scenarios, which can easily lead to insufficient model recognition of extreme scenarios.

[0004] Therefore, how to expand the limited training dataset has become the core issue in improving the performance of the cognitive load identification model, and it is also the key breakthrough point for promoting this technology from the laboratory to actual aviation applications. Summary of the Invention

[0005] In view of this, the present invention provides a data augmentation method and apparatus to solve the problem of how to augment a limited training dataset.

[0006] In a first aspect, the present invention provides a data augmentation method, the method comprising: Obtain the initial training dataset corresponding to the pilots. The initial training dataset includes the initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight stages, and training cognitive load levels corresponding to the pilots. Extract features from the initial training dataset and generate initial fusion features based on the extracted features. Input the initial fusion features into a preset data augmentation model to augment the initial training dataset and generate the target training dataset.

[0007] In one optional implementation, feature extraction is performed on the initial training dataset, and initial fusion features are generated based on the extracted features, including: feature extraction of initial electrocardiogram (ECG) data in the initial training dataset to obtain total initial ECG features corresponding to the initial ECG data; the total initial ECG features include initial ECG time-domain features, initial ECG frequency-domain features, and initial ECG nonlinear features; feature extraction of initial respiratory data in the initial training dataset to obtain total initial respiratory features corresponding to the initial respiratory data; the total initial respiratory features include initial respiratory time-domain features and initial respiratory frequency-domain features; feature extraction of initial eye-tracking data in the initial training dataset to obtain initial eye-tracking features; fusion of the total initial ECG features, total initial respiratory features, and initial eye-tracking features to generate initial physiological features; generation of task complexity features based on the operational elements of each stage of the five-sided flight; fusion of the training flight stage, training cognitive load level, and task complexity features to generate initial conditional features; and fusion of the initial physiological features and initial conditional features to generate initial fusion features.

[0008] In one optional implementation, the initial fusion features are input into a preset data augmentation model to augment the initial training dataset and generate a target training dataset. This includes: inputting the initial fusion features into a preset generation network based on the preset data augmentation model to generate noise data corresponding to the initial training dataset; inputting the noise data into a preset discrimination network in the preset data augmentation model to discriminate the noise data; and augmenting the initial training dataset based on the discrimination results to generate the target training dataset.

[0009] In one optional implementation, the initial fusion features are input into a preset data augmentation model to augment the initial training dataset and generate a target training dataset. This includes: inputting the initial fusion features into a first hidden layer in a preset generator network to expand the initial fusion features and generate a first hidden feature; inputting the first hidden feature into a second hidden layer in the preset generator network to scale the first hidden feature and generate a second hidden feature; and activating the second hidden feature to generate noisy data.

[0010] In one optional implementation, initial fused features are input into a first hidden layer of a preset generator network, and the initial fused features are expanded to generate first hidden features, including: determining the training flight stage and training cognitive load level from the initial fused features; determining the target expanded dimension corresponding to the initial fused features from a preset mapping table according to the training flight stage and training cognitive load level; generating a first initial weight matrix according to the initial dimension corresponding to the initial fused features and the target expanded dimension; correcting the first initial weight matrix according to the training flight stage and training cognitive load level to obtain a first target weight matrix; expanding the initial fused features based on the first target weight matrix to generate a target expanded dimension feature vector; activating the target expanded dimension feature vector based on a preset activation function to output the first hidden features.

[0011] In one optional implementation, the first hidden feature is input into the second hidden layer of a preset generator network, and the first hidden feature is scaled to generate the second hidden feature, including: calculating the first mutual information value between each initial sub-fusion feature in the initial fusion feature; constructing a modal correlation matrix based on each first mutual information value; labeling each first hidden sub-feature according to the mapping relationship between each first hidden sub-feature in the first hidden feature and each initial sub-fusion feature in the initial fusion feature; calculating the mean vector corresponding to each first hidden sub-feature dominated by ECG as the first cluster center based on the labeling result; calculating... The mean vectors of each first hidden sub-feature dominated by respiration are used as the second cluster centers; the mean vectors of each first hidden sub-feature dominated by eye movement are calculated as the third cluster centers; the mean vectors of each first hidden sub-feature dominated by condition are calculated as the fourth cluster centers; based on the modal correlation matrix, the second mutual information value between each first hidden sub-feature is calculated; based on the second mutual information value, each first hidden sub-feature is clustered to obtain the ECG cluster group, the respiration cluster group, the eye movement cluster group, and the conditional cluster group; based on the ECG cluster group, the respiration cluster group, the eye movement cluster group, and the conditional cluster group, the second hidden feature is output.

[0012] In one optional implementation, a second hidden feature is output based on the ECG cluster group, respiratory cluster group, eye-tracking cluster group, and conditional cluster group, including: calculating the average correlation degree among the ECG cluster group, respiratory cluster group, eye-tracking cluster group, and conditional cluster group based on the modal correlation matrix; determining the connection weights among the ECG cluster group, respiratory cluster group, eye-tracking cluster group, and conditional cluster group based on each average correlation degree; generating a second initial weight matrix based on the initial dimension corresponding to the initial fusion feature and the target extended dimension; and arranging the second initial weight matrix according to the in-group... The system divides the region into regional and inter-group regions. Based on the connection weights between the ECG cluster group, the respiratory cluster group, the eye-tracking cluster group, and the respiratory cluster group, the system determines the first target weight value corresponding to the inter-group region in the second initial weight matrix. Based on the criticality of each cluster sub-feature in the ECG cluster group, the respiratory cluster group, the eye-tracking cluster group, and the conditional cluster group, the system determines the second target weight value corresponding to the intra-group region. Based on the first target weight and the second target weight value, the system generates the second target weight matrix. The system multiplies the first hidden feature by the second target weight matrix to obtain the second hidden feature.

[0013] In one optional implementation, noise data is input into a preset discriminant network in a preset data augmentation model to discriminate the noise data, including: calculating the load change rate based on initial ECG data and initial eye movement data; fusing the noise data, training flight phase, training cognitive load level, and load change rate to generate target fusion features; compressing the target fusion features based on a first branch to obtain first branch features; compressing the target fusion features based on a second branch to obtain second branch features; and dividing and labeling the target fusion features into four specific groups according to physiological modalities and functional attributes; the four specific groups are the ECG group, ... The system includes a respiratory group, an eye-tracking group, and a load change rate label group. The first and second branch features are validated respectively, yielding ECG, respiratory, and eye-tracking scores for each feature. The first and second branch features, along with their respective ECG, respiratory, and eye-tracking scores, are fused to obtain a discriminative fusion feature. This discriminative fusion feature is then input into the authenticity discrimination branch, outputting the probability that noisy data is real data. The discriminative fusion feature is also input into the load level discrimination branch, outputting the load level probability corresponding to the noisy data. Finally, the noisy data is discriminated based on a preset loss function.

[0014] In one optional implementation, the noise data is judged based on a preset loss function, including: calculating authenticity loss based on the probability that the noise data is real data; calculating load level loss based on the probability of the load level corresponding to the noise data; calculating compliance loss based on ECG dimension score, respiratory dimension score, and eye movement dimension score; generating a preset loss function based on authenticity loss, load level loss, and compliance loss; and judging the noise data based on the preset loss function.

[0015] In a second aspect, the present invention provides a data expansion device, the device comprising: The acquisition module is used to acquire the initial training dataset corresponding to the trainees. The initial training dataset includes the initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight phase, and training cognitive load level corresponding to the trainees. The generation module is used to extract features from the initial training dataset and generate initial fusion features based on the extracted features; The expansion module is used to input the initial fused features into the preset data expansion model to expand the initial training dataset and generate the target training dataset.

[0016] The cognitive load identification model training method and apparatus provided in this application acquire an initial training dataset corresponding to the pilots being trained. Then, initial fusion features are generated based on the initial training dataset. This allows different modalities of data to complement each other, helping to capture more complex physiological and task-related relationships and improving the model's generalization ability.

[0017] Then, the initial fusion features are input into a pre-defined generative network in a pre-defined data augmentation model to generate noisy data corresponding to the initial training dataset. The generated noisy data can be used to simulate noise or uncertainty in real-world data, enhancing the model's robustness. The noisy data is then input into a pre-defined discriminant network in the pre-defined data augmentation model to discriminate the noisy data. The load level probability corresponding to the noisy data is output. This fusion method integrates multiple aspects of information, enabling the target cognitive load identification model to evaluate data from both realism and load level perspectives, providing a more comprehensive judgment for cognitive load assessment and helping to improve the accuracy and reliability of the assessment results. Based on the discrimination results, the initial training dataset is expanded to generate a target training dataset. This ensures the accuracy and comprehensiveness of the generated target training dataset. It solves the problem in existing technologies where the collection of actual pilot flight data is limited by high costs and scenario safety constraints, resulting in a limited training dataset size and a low proportion of samples from high-load, complex task scenarios. This, in turn, ensures the accuracy of the target cognitive load identification model trained on the target training dataset. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating a data augmentation method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating another data augmentation method according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating another data augmentation method according to an embodiment of the present invention; Figure 4 This is a structural block diagram of a data expansion device according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that the data expansion method provided in this application embodiment can be executed by a data expansion device. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The electronic device can be a server or a terminal. In this application embodiment, the server can be a single server or a server cluster composed of multiple servers. The terminal in this application embodiment can be a smartphone, personal computer, tablet computer, wearable device, or other intelligent hardware device such as an intelligent robot. The following method embodiments will use an electronic device as the execution subject for illustration.

[0022] According to an embodiment of the present invention, a data augmentation method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] This embodiment provides a data augmentation method that can be used in electronic devices. Figure 1This is a flowchart of a data augmentation method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the initial training dataset corresponding to the pilots being trained.

[0024] The initial training dataset includes initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight phases, and training cognitive load levels corresponding to the training pilots.

[0025] Specifically, electronic devices can collect various types of data from pilots during flight training, including: physiological data: initial electrocardiogram data (such as heart rate and heart rate variability), initial respiratory data (such as respiratory rate and depth), and initial eye movement data (such as pupil diameter and fixation point); scenario and label data: training flight phases (such as takeoff, cruise, and landing), and training cognitive load levels (such as load assessment results from level 0 to 5). These data collectively constitute the basic dataset for model training, reflecting the pilot's cognitive load status in different scenarios.

[0026] Step S102: Extract features from the initial training dataset and generate initial fusion features based on the extracted features.

[0027] Specifically, the electronic device can extract features from the initial electrocardiogram data, initial respiratory data, initial eye movement data, and training flight phase data in the initial training dataset to generate initial fused features.

[0028] This step will be explained in detail below.

[0029] Step S103: Input the initial fusion features into the preset data augmentation model to augment the initial training dataset and generate the target training dataset.

[0030] Specifically, electronic devices use a preset data augmentation model to process the initial fusion features, thereby expanding the data scale and enriching data diversity by generating noise data that conforms to physiological laws, supplementing scarce scenario samples such as high load / low load, and strengthening cross-modal feature correlation.

[0031] The resulting target training dataset can more comprehensively cover the cognitive load scenarios that pilots may encounter, reducing model bias caused by insufficient or imbalanced data.

[0032] This step will be explained in detail below.

[0033] The data augmentation method provided in this embodiment obtains an initial training dataset corresponding to the pilots. Then, initial fusion features are generated based on the initial training dataset. This allows different modalities of data to complement each other, helping to capture more complex physiological and task-related relationships and improving the model's generalization ability. The initial fusion features are input into a preset data augmentation model to expand the initial training dataset and generate a target training dataset. This ensures the accuracy and comprehensiveness of the generated target training dataset. It solves the problem in existing technologies where the collection of actual pilot flight data is limited by high costs and scenario safety constraints, resulting in a limited training dataset size and a low proportion of samples from high-load, complex task scenarios. Therefore, it ensures the accuracy of the target cognitive load identification model trained based on the target training dataset.

[0034] This embodiment provides a data augmentation method that can be used in electronic devices. Figure 2 This is a flowchart of a data augmentation method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain the initial training dataset corresponding to the pilots being trained.

[0035] The initial training dataset includes initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight phases, and training cognitive load levels corresponding to the training pilots.

[0036] Step S202: Extract features from the initial training dataset and generate initial fusion features based on the extracted features.

[0037] Specifically, step S202 may include the following steps: Step S2021: Extract features from the initial ECG data in the initial training dataset to obtain the total initial ECG features corresponding to the initial ECG data.

[0038] The total initial ECG characteristics include initial ECG time-domain characteristics, initial ECG frequency-domain characteristics, and initial ECG nonlinear characteristics.

[0039] Specifically, electronic devices can extract initial ECG time-domain features, initial ECG frequency-domain features, and initial ECG nonlinear features from initial ECG data.

[0040] For example, as shown in Table 1, the extracted initial ECG time-domain features, initial ECG frequency-domain features, and initial ECG nonlinear features are shown.

[0041] Table 1 Initial ECG time-domain characteristics, initial ECG frequency-domain characteristics, and initial ECG nonlinear characteristics

[0042] For the initial ECG time-domain characteristics (corresponding to items 1-4 in Table 1): the electronic device can extract the RR interval sequence from the initial ECG data and calculate: average heart rate (HR), count the number of R wave peaks per unit time, and convert it to bpm; RR interval standard deviation (SDNN): calculate the standard deviation of continuous RR interval values, in ms; RR interval standard deviation mean (SDANN): calculate the standard deviation of the RR sequence segment and take the mean; root mean square of adjacent RR interval differences (RMSSD): take the square root of the mean of the squares of the differences between adjacent RR intervals.

[0043] Regarding the initial ECG frequency domain characteristics (corresponding to items 5-8 in Table 1): Electronic devices can perform Fourier transform on the RR interval sequence to decompose the following: High-frequency power (HF): power value in the 0.15-0.4Hz frequency band, in ms²; High-frequency power percentage (HFPowerPercent): the proportion of HF to the total power; Low-frequency power (LF): power value in the 0.04-0.15Hz frequency band, in ms²; Low-frequency power percentage (LFPowerPercent): the proportion of LF to the total power.

[0044] For the initial ECG nonlinear characteristics (corresponding to items 9-12 in Table 1): the electronic device can draw a Poincaré scatter plot and extract: the standard deviation of the major axis of the ellipse (SD1) and the standard deviation of the minor axis (SD2); the number of points in the first quadrant (A++) and the number of points in the third quadrant (B--) of the difference scatter plot, expressed as a percentage.

[0045] Step S2022: Extract features from the initial respiratory data in the initial training dataset to obtain the total initial respiratory features corresponding to the initial respiratory data.

[0046] The total initial respiratory characteristics include the initial respiratory time domain characteristics and the initial respiratory frequency domain characteristics.

[0047] Specifically, the electronic device can extract initial respiratory time-domain features and initial respiratory frequency-domain features from initial respiratory data. For example, Table 2 shows the initial respiratory time-domain features and initial respiratory frequency-domain features.

[0048] Table 2. Initial respiratory temporal and frequency domain characteristics.

[0049] Specifically, for the initial respiratory time-domain characteristics (corresponding to items 1-5 in Table 2), the electronic device can perform peak and trough detection on the initial respiratory data and calculate: the average value of the respiratory peak and trough interval (AVRESP): in rpm; the standard deviation of the respiratory peak interval (Std), maximum value (Max), minimum value (Min), and range (Range), all in rpm.

[0050] Based on the initial respiratory frequency domain characteristics (corresponding to items 6-7 in Table 2), electronic devices can perform spectral analysis on the initial respiratory data to extract: respiratory signal energy (Power): unit %²; respiratory peak (Peak): the frequency value with the highest energy in the spectrum, unit Hz.

[0051] Step S2023: Extract features from the initial eye-tracking data in the initial training dataset to obtain initial eye-tracking features.

[0052] Specifically, electronic devices can extract initial eye movement features from initial eye movement data.

[0053] For example, as shown in Table 3, are the initial eye movement features.

[0054] Table 3 Initial eye movement characteristics

[0055] Specifically, electronic devices can extract features from initial eye-tracking data based on the I-VT algorithm to obtain initial eye-tracking features (corresponding to items 1-5 in Table 3), including: average pupil diameter (PupilMean): the average pupil diameter during fixation, in mm; fixation count per second (FixationCount): in N / s; total fixation duration (FixationTotalduration) and average fixation duration (FixationMeanduration): in s; and average saccades (MeanVariances): in mm².

[0056] Step S2024: The total initial electrocardiogram features, total initial respiratory features, and initial eye movement features are fused to generate initial physiological features.

[0057] Specifically, the electronic device can normalize the total initial electrocardiogram features (12 items), total initial respiratory features (7 items), and initial eye movement features (5 items), and then splice the normalized total initial respiratory features and initial eye movement features to generate initial physiological features.

[0058] Step S2025: Based on the operational elements of each stage of the five-sided flight, generate mission complexity features.

[0059] Specifically, electronic equipment can generate mission complexity characteristics by setting key operational parameters for each stage of the five-way flight (e.g., controlling the climb rate to 500 ft / min in the first leg and the touchdown speed to 40 ft / min in the fifth leg). For example, a difficulty coefficient (1-5 points) can be set for four key operational parameters in each stage. These four key operational parameters can be heading maintenance, altitude control, airspeed adjustment, and position calibration, which are extended to 20 dimensions through one-hot encoding. Example: When the "heading maintenance" parameter in the third leg (downwind) has a difficulty of 3 points, the corresponding encoding is [0,0,1,0,0]; when the altitude control parameter has a difficulty of 4 points, the corresponding encoding is [0,0,0,1,0]; when the airspeed adjustment parameter has a difficulty of 2 points, the corresponding encoding is [0,1,0,0,0]; and when the position calibration parameter has a difficulty of 5 points, the corresponding encoding is [0,0,0,0,1], which are then concatenated into 20 dimensions.

[0060] Step S2026: The training flight phase, training cognitive load level, and task complexity features are fused to generate initial condition features.

[0061] Specifically, the electronic device can encode the training flight phase and the training cognitive load level, and then splice the encoded training flight phase, training cognitive load level and task complexity features to generate initial condition features.

[0062] For example, 3D one-hot encoding is used for the training flight phases (first / third / fifth phases), and 2D one-hot encoding is used for the training cognitive load levels (low / high). The five-phase flight in this field is a classic task in pilot training, which includes key phases such as takeoff (first phase), leeward (third phase), and landing (fifth phase). The operational complexity and cognitive load of different phases are different (for example, the fifth phase is the landing phase, which has high accuracy requirements and usually higher cognitive load).

[0063] One-hot encoding is a method for converting categorical variables (here, the "flight phase") into binary vectors: when a sample belongs to the "first side", it is represented by the vector [1,0,0] (the first bit is 1, and the rest are 0); when a sample belongs to the "third side", it is represented by the vector [0,1,0] (the second bit is 1, and the rest are 0); when a sample belongs to the "fifth side", it is represented by the vector [0,0,1] (the third bit is 1, and the rest are 0).

[0064] Step S2027: The initial physiological features and initial conditional features are fused to generate initial fused features.

[0065] Specifically, the electronic device fuses initial physiological features and initial conditional features to generate initial fused features. For example, the electronic device splices 24-dimensional initial physiological features with 25-dimensional initial conditional features to ultimately form 49-dimensional initial fused features.

[0066] Step S203: Input the initial fusion features into the preset data augmentation model to augment the initial training dataset and generate the target training dataset.

[0067] The data augmentation method provided in this application extracts time-domain, frequency-domain, and nonlinear features from initial electrocardiogram (ECG) data to comprehensively capture the changing patterns of ECG signals across different dimensions. These multi-dimensional features cover the static level of heart rate, the dynamic balance of autonomic nervous system regulation, and signal complexity, providing rich cardiovascular physiological evidence for cognitive load assessment. Extracting respiratory time-domain and frequency-domain features accurately characterizes the dynamic changes in breathing patterns. This distinguishes between the speed of breathing (time domain) and rhythm stability (frequency domain), effectively capturing the typical characteristic of "rapid breathing and rhythmic disorder" under high load, supplementing the deficiencies of ECG features. Feature extraction is performed on the initial eye-tracking data in the initial training dataset to obtain initial eye-tracking features. Eye-tracking features respond more quickly to changes in cognitive load, compensating for the lag in ECG and respiratory signals and improving the real-time performance of the features. The total initial ECG features, total initial respiratory features, and initial eye-tracking features are fused to form a multimodal physiological feature vector. This integrates the synergistic changes in the cardiovascular, respiratory, and visual attention systems, overcoming the limitations of a single modality and enhancing the overall representational ability of features for cognitive load. By integrating features from training flight phases, cognitive load levels, and mission complexity, the impact of scenarios on load is quantified, linking physiological characteristics with specific flight scenarios. This allows the model to learn the mapping relationship between "scenario-load-physiology" (e.g., even with small fluctuations in physiological signals during landing, the load level may still be high), improving scenario adaptability. Initial physiological and conditional features are integrated to form initial fusion features encompassing "physiological state + scenario constraints." This preserves the details of the pilot's physiological response while incorporating the objective constraints of the mission scenario, ensuring that subsequent data expansion generates logically consistent samples within a reasonable scenario framework (e.g., avoiding contradictory samples such as "cruise phase + extreme physiological signals"). A pre-defined generation network expands the initial fusion features, generating noise data that conforms to physiological and scenario patterns. This noise data is not random perturbation but rather "quasi-realistic samples" generated based on real feature distributions, effectively supplementing scarce scenario samples while simulating individual differences. A pre-defined discrimination network verifies the authenticity and physiological rationality of the noise data. Invalid samples that do not conform to physiological logic or scenario constraints are filtered out to ensure the quality of the expanded data and avoid misleading model training. By combining the discrimination results with the selection of effective noisy data, the dataset is merged with the initial dataset to form the target training dataset. This addresses the issues of insufficient sample size and uneven distribution in the initial data, enabling the dataset to cover a more comprehensive range of load levels and flight scenarios while maintaining the physiological rationality of the data. This provides high-quality and diverse materials for subsequent model training, improving the model's generalization ability and recognition accuracy.

[0068] This embodiment provides a data augmentation method that can be used in electronic devices. Figure 3 This is a flowchart of a data augmentation method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps: Step S301: Obtain the initial training dataset corresponding to the pilots being trained.

[0069] The initial training dataset includes initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight phases, and training cognitive load levels corresponding to the training pilots.

[0070] Step S302: Extract features from the initial training dataset and generate initial fusion features based on the extracted features.

[0071] Step S303: Input the initial fusion features into the preset data augmentation model to augment the initial training dataset and generate the target training dataset.

[0072] Specifically, step S303 may include the following steps: Step S3031: Input the initial fusion features into the preset generation network based on the preset data augmentation model to generate noisy data corresponding to the initial training dataset.

[0073] Specifically, step S3031 may include the following steps: Step a1: Input the initial fused features into the first hidden layer of the preset generator network, and perform expansion processing on the initial fused features to generate the first hidden features.

[0074] Specifically, step a1 may include the following steps: Step a11: Determine the training flight phase and training cognitive load level from the initial fusion features.

[0075] Specifically, electronic devices can determine the training flight phase and the training cognitive load level from the initial fusion characteristics.

[0076] Step a12: Based on the training flight phase and the training cognitive load level, determine the target expansion dimension corresponding to the initial fusion feature from the preset mapping table.

[0077] Specifically, the electronic device can determine the target expansion dimension corresponding to the initial fusion feature from a preset mapping table based on the combination of training cognitive load level and training flight phase.

[0078] For example, low load + first / third side: target expansion dimension 88 (expansion ratio 1.8), because the operational complexity of this scenario is low and the physiological signal fluctuations are gentle, so there is no need for too many feature dimensions; high load + fifth side (landing phase): target expansion dimension 108 (expansion ratio 2.2), because the cognitive load is the highest during the landing phase and the physiological signals (such as pupil fluctuations and heart rate changes) are complex, so more dimensions are needed to capture details. Other scenarios (such as low load + fifth side, high load + first side): default target expansion dimension 97 (expansion ratio 1.98).

[0079] Step a13: Generate the first initial weight matrix based on the initial dimension corresponding to the initial fusion feature and the target extended dimension.

[0080] Specifically, the electronic device can initialize the weight matrix using a Xavier normal distribution to generate the first initial weight matrix, ensuring that the variances of the input and output features are consistent (avoiding gradient vanishing or exploding), as shown in the formula: Where in_dim is the dimension of the initial fused feature, for example, 49, and out_dim is the target expanded dimension (e.g., 97).

[0081] Step a14: Based on the training flight phase and the training cognitive load level, the first initial weight matrix is ​​modified to obtain the first target weight matrix.

[0082] Specifically, the electronic device can initially determine the current scene features by fusing the training flight phase and training cognitive load level from the initial fusion features. Then, the initial weight matrix is ​​modified based on the current scene features to obtain the first target weight matrix. For example, if the training flight phase and training cognitive load level are "high load + fifth side", then the current scene features are determined to be the first scene. The first scene is a highly stressful scene, so the weight values ​​related to eye movement features (the last 5 dimensions) are enhanced (multiplied by a factor of 1.2), because eye movement features such as pupil sway during the landing phase are more sensitive to cognitive load.

[0083] If the training flight phase and the training cognitive load level are "low load + first side", then the current scenario characteristics are determined to be the second scenario. The second scenario is a relatively easy scenario, so the weight value related to the high-frequency characteristics of ECG (such as HF, LF) is reduced (multiplied by a coefficient of 0.8), because the ECG frequency domain characteristics fluctuate less under low load.

[0084] Optionally, the electronic device can also differentiate and strengthen the weights of corresponding regions in the initial weight matrix based on the importance differences of various physiological feature modalities in the initial fusion features. For example, for the region corresponding to the total initial ECG features (the first 12 dimensions of the input vector), the standard deviation of the weights is reduced by 20% (compared to other regions) to suppress the excessive amplification of high-frequency noise in the ECG signal (meeting the low activation requirement of the subsequent LeakyReLU slope of 0.1). For the region corresponding to the initial eye-tracking features (the last 5 dimensions of the input vector), the standard deviation of the weights is increased by 20% to enhance the ability to capture dynamic features such as pupil diameter and fixation duration (matching the high activation requirement of the subsequent LeakyReLU slope of 0.3). For the region corresponding to the total initial respiratory features (the middle 7 dimensions of the input vector), the weight values ​​are positively correlated with the respiratory rate (AVRESP) feature. When the conditional encoding indicates a "high respiratory rate scenario," the corresponding weights are multiplied by a factor of 1.1 to obtain the final first target weight matrix.

[0085] Step a15: Expand the initial fused features based on the first target weight matrix to generate a target extended dimension feature vector.

[0086] Specifically, the electronic device uses the initial fusion features to right-multiply the first target weight matrix to generate a target extended dimension feature vector.

[0087] Step a16: Activate the target extended dimension feature vector based on the preset activation function and output the first hidden feature.

[0088] Specifically, the electronic device can determine the LeakyReLU activation function and, based on the different modalities corresponding to the sub-features in the target extended dimension feature vector, determine the slope in the LeakyReLU activation function, thereby activating the target extended dimension feature vector based on LeakyReLU activation functions with different slopes and outputting the first hidden feature.

[0089] The expression for the LeakyReLU activation function is: Where α is the slope parameter. Different slopes are used for different modal features, based on the physiological differences in ECG, respiration, and eye movement signals. The slope of the ECG feature neurons (generated from the first 12 physiological features in the initial physiological features) in the target extended dimension feature vector is set to 0.1 to suppress excessive amplification of high-frequency noise in the ECG signal. The slope of the eye movement feature neurons (generated from the last 5 physiological features in the initial physiological features) in the target extended dimension feature vector is set to 0.3 to retain key dynamic features such as pupillary fluctuations under high load. An adaptive slope is added for the respiration feature neurons (based on the corresponding middle 7 physiological features in the initial physiological features) in the target extended dimension feature vector. a. Extract AVRESP values ​​from the middle 7 features and determine if "AVRESP > 18 rpm" (rapid breathing state) is satisfied. b. If satisfied, apply LeakyReLU with α = 0.4 to the 7-dimensional respiration features to enhance sensitivity to negative features during rapid breathing (such as a brief decrease in respiratory depth). c. If the condition is not met (AVRESP≤18rpm), apply the default slope α=0.2 to balance the characteristic activation intensity under normal breathing conditions.

[0090] Step a2: Input the first hidden feature into the second hidden layer of the preset generation network, scale the first hidden feature, and generate the second hidden feature.

[0091] Specifically, step a2 may include the following steps: Step a21: Calculate the first mutual information value between each initial sub-fusion feature in the initial fusion feature.

[0092] Then, for each initial sub-fusion feature among the total initial ECG features, total initial respiratory features, initial eye movement features, and initial conditional features, local mutual information is calculated using a sliding window of a preset time length (the window size can be adjusted according to the feature dynamics, such as a 3-second window for physiological features and a 5-second window for conditional features). The average value is then taken to obtain the sub-feature mutual information value corresponding to each initial sub-fusion feature, thus avoiding the obscuring of dynamic correlations by a single global value.

[0093] Optionally, electronic devices can assign a 1.2x weight to high-load samples and a 0.8x weight to low-load samples, thereby strengthening the association of features related to cognitive load (such as the association between heart rate and pupil diameter under high load).

[0094] Then, the electronic device calculates the first mutual information value between each initial sub-fusion feature, using the following formula (after discretization): Here, X and Y represent two initial sub-fusion features to be analyzed (e.g., "heart rate" in the total initial ECG features and "respiratory rate" in the total initial respiratory features). P(X) and P(Y) represent the marginal probability distributions of X and Y, respectively, that is, the probability of a single initial sub-fusion feature taking all possible values ​​(e.g., the probability of a heart rate of 70 bpm and the probability of a respiratory rate of 15 rpm). P(X,Y) is the joint probability distribution of initial sub-fusion feature X and initial sub-fusion feature Y, representing the probability of both initial sub-fusion features taking a certain combination of values ​​(e.g., the probability of a heart rate of 70 bpm and a respiratory rate of 15 rpm).

[0095] Step a22: Construct a modal correlation matrix based on each first mutual information value.

[0096] Specifically, the electronic device can construct a modal correlation matrix using all initial sub-fusion features as rows and columns. Each element in the matrix corresponds to the first mutual information value of two initial sub-fusion features. This matrix clearly presents the degree of correlation between different modal sub-features (ECG, respiration, eye movement, and conditional features), such as the correlation strength between ECG features and conditional features, and the correlation strength between respiration features and eye movement features, providing a basic correlation basis for subsequent calculations.

[0097] For example, an electronic device can construct a 49×49 modal correlation matrix M based on the normalized first mutual information values, where M[i,j] represents the normalized mutual information value between the i-th initial sub-fusion feature and the j-th initial sub-fusion feature. For example, the correlation between ECG HR (E1) and respiratory AVRESP (R1) may be 0.65 (strong correlation); the correlation between ECG LF (E7) and eye movement O1 (pupil diameter) may be 0.2 (weak correlation).

[0098] Step a23: Mark each first hidden sub-feature according to the mapping relationship between each first hidden sub-feature in the first hidden features and each initial sub-fusion feature in the initial fusion features.

[0099] Specifically, the electronic device can determine the mapping relationship between each first hidden sub-feature in the first hidden features and each initial sub-fusion feature in the initial fusion features. By calculating the weight contribution of the first hidden sub-feature to each initial sub-fusion feature (such as the feature importance value obtained through gradient backpropagation), the type of initial sub-fusion feature it mainly depends on is determined.

[0100] Labeling rule: If a certain first hidden sub-feature has the highest total weight contribution to the initial ECG sub-fusion feature, it is labeled as "ECG dominant"; similarly, the first hidden sub-features of "respiration dominant", "eye movement dominant" and "conditional dominant" are labeled respectively.

[0101] Step a24: Based on the labeling results, calculate the mean vector corresponding to each first hidden sub-feature dominated by ECG, and use it as the first cluster center.

[0102] Specifically, the electronic device can determine the first hidden features dominated by ECG, the first hidden features dominated by respiration, the first hidden features dominated by eye movement, and the first hidden features dominated by condition based on the labeling results. Then, the electronic device calculates the mean vector corresponding to each first hidden feature dominated by ECG, which serves as the first cluster center.

[0103] Step a25: Calculate the mean vector of each first hidden sub-feature dominated by breathing, and use it as the second cluster center.

[0104] Specifically, the electronic device calculates the mean vector of each of the first hidden sub-features dominated by breathing, and uses it as the second cluster center.

[0105] Step a26: Calculate the mean vector of each first hidden sub-feature dominated by eye movement, and use it as the third cluster center.

[0106] Specifically, the electronic device calculates the mean vector of each of the first hidden sub-features dominated by eye movement, which serves as the third cluster center.

[0107] Step a27: Calculate the mean vector of each conditionally dominant first hidden sub-feature, and use it as the fourth cluster center.

[0108] Specifically, the mean vector of each of the first hidden sub-features dominated by the computational conditions of the electronic device is used as the fourth cluster center.

[0109] Step a28: Calculate the second mutual information value between each first hidden sub-feature based on the modal correlation matrix.

[0110] Specifically, the electronic device derives the second mutual information value between each first hidden sub-feature based on the constructed modal correlation matrix and the mapping relationship between the first hidden sub-feature and the initial sub-fusion feature.

[0111] For example, for two first hidden sub-features A and B, firstly, the core source tracing initial sub-fusion features corresponding to the first hidden sub-features A and B are determined. Then, the first mutual information values ​​of these two core source tracing initial sub-fusion features in the modal correlation matrix are calculated. The first mutual information values ​​of the secondary source tracing initial sub-fusion features corresponding to the first hidden sub-features A and B are also determined. Then, the proportions of the core source tracing initial sub-fusion features and secondary source tracing initial sub-fusion features in the first hidden sub-feature A of the electronic device, and the proportions of the core source tracing initial sub-fusion features and secondary source tracing initial sub-fusion features in the first hidden sub-feature B, are used to calculate the second mutual information value between the first hidden sub-features A and B.

[0112] For example, the core source fusion features of A (e.g., ECG SDNN, weighted at 80%) and the core source fusion features of B (e.g., respiratory AVRESP, weighted at 75%) are determined. Then, the electronic device extracts the mutual information values ​​of these two core source features from the modal correlation matrix (e.g., MI = 0.52 for SDNN and AVRESP). The mutual information of the secondary source features in A and B is weighted and summed (the weights are the contribution percentages during feature mapping), ultimately yielding MI(A,B). Example: If A has 70% weight from ECG HR (heart rate) and 30% from respiratory AVRESP; and B has 60% weight from respiratory AVRESP and 40% from eye movement Fixation Count, then MI(X,Y)=0.7×0.6×MI(HR,AVRESP)+0.7×0.4×MI(HR,FixationCount)+0.3×0.6×MI(AVRESP,AVRESP)+0.3×0.4×MI(AVRESP,Fixation Count) (where MI(AVRESP,AVRESP)=1, because the mutual information of the same first hidden sub-feature is 1).

[0113] Step a29: Based on the second mutual information value, cluster each first hidden sub-feature to obtain the ECG cluster group, the respiration cluster group, the eye movement cluster group, and the conditional cluster group.

[0114] Specifically, the electronic device can employ hierarchical clustering. Starting with four cluster centers, the similarity between each first hidden feature and each cluster center is calculated (similarity = 1 - Euclidean distance of the second mutual information value). Then, each first hidden feature is assigned to the cluster group with the highest similarity, forming ECG cluster, respiration cluster, eye movement cluster, and conditional cluster. For "mixed dominant" features, they are classified according to their maximum similarity with the four cluster centers. After classification, the electronic device can recalculate the mean vector of each cluster and update the cluster centers. The clustering process is then repeated until the change in the cluster centers is <0.01 (convergence).

[0115] Step a210: Output the second hidden feature based on the ECG cluster group, respiratory cluster group, eye movement cluster group, and conditional cluster group.

[0116] Specifically, step a210 may include the following steps: Step a2101: Based on the modal correlation matrix, calculate the average correlation degree between the ECG cluster group, the respiratory cluster group, the eye movement cluster group, and the conditional cluster group.

[0117] Specifically, the first mutual information value corresponding to the initial sub-fusion feature of each combination is extracted from the modal correlation matrix. Then, the arithmetic mean of the extracted first mutual information values ​​is calculated to obtain the average correlation degree between each two cluster groups, forming a 4×4 average correlation degree matrix.

[0118] Example: Correlation between ECG and respiratory groups (A) er ): Extract the intersection region (a 12×7 submatrix) of the modal correlation matrix M, which consists of the central electrical features (E1-E12) and the respiratory features (R1-R7). Calculate the average of all mutual information values ​​in this submatrix using the formula: A er =∑i / 12×7. If the sum of mutual information values ​​in the submatrix is ​​42, then A er =42 / (12×7)=0.5.

[0119] Step a2102: Determine the connection weights between the ECG cluster group, the respiratory cluster group, the eye movement cluster group, and the conditional cluster group based on the average correlation.

[0120] Specifically, electronic devices can determine the connection weights between ECG clusters, respiratory clusters, eye-tracking clusters, and conditional clusters based on the correspondence between average correlation and connection weights.

[0121] For example, the connection weight = average correlation × (1 + average correlation) to increase the weight of high correlation, with the weight range controlled within [0,2]. For example, if the average correlation between ECG and respiration is 0.6, then the connection weight = 0.6 × (1 + 0.6) = 0.96; if the average correlation between respiration and eye movement is 0.3, then the connection weight = 0.3 × (1 + 0.3) = 0.39.

[0122] Step a2103: Generate a second initial weight matrix based on the initial dimension corresponding to the initial fusion feature and the target extended dimension.

[0123] Specifically, the electronic device generates a second initial weight matrix (i.e., a 97×49 matrix) with dimensions of "target expanded dimension × initial dimension" based on the initial dimension of the initial fusion features (e.g., 49 dimensions, including 24 physiological features and 25 conditional features) and the target expanded dimension (e.g., 128 dimensions). The electronic device can use Kaiming initialization to make the matrix elements follow a normal distribution with a mean of 0 and a variance of 2 / initial dimension, adapting to the gradient characteristics of ReLU-type activation functions.

[0124] Step a2104: Divide the second initial weight matrix into intra-group and inter-group regions.

[0125] Specifically, the electronic device can define the region in the second initial weight matrix that corresponds to the same cluster group feature mapped to its own exclusive dimension as the intra-group region, and the region in the second initial weight matrix that corresponds to the cross-mapping of features from different cluster groups as the inter-group region.

[0126] Step a2105: Determine the first target weight value corresponding to the inter-group region in the second initial weight matrix based on the connection weights between the ECG cluster group, the respiratory cluster group, the eye movement cluster group, and the respiratory cluster group.

[0127] Specifically, the electronic device can calculate the first target weight value corresponding to the inter-group region in the second initial weight matrix by using the first target weight value = the element value of the second initial weight matrix × the connection weight.

[0128] Step a2106: Determine the second target weight value corresponding to the region within the group based on the criticality of each cluster sub-feature in the ECG cluster group, respiratory cluster group, eye movement cluster group, and conditional cluster group.

[0129] Specifically, electronic devices can determine the criticality of each cluster sub-feature by feature contribution (such as the feature importance score of random forest). The higher the criticality (e.g., ≥0.6), the greater the impact of the feature on cognitive load identification.

[0130] The weight calculation formula is: Second target weight value = Second initial weight matrix element value × (1 + criticality). That is, the sub-feature with a criticality of 0.8 will have its weight increased by 80% within the group (e.g., initial value 0.3 → 0.3 × 1.8 = 0.54). Among them, the criticality of the conditional clustering sub-features in the conditional clustering group is dynamically adjusted according to the scenario (e.g., the criticality of "flight phase coding" in the landing phase is set to 0.9).

[0131] Step a2107: Generate a second target weight matrix based on the first target weight and the second target weight values.

[0132] Specifically, the electronic device fills the first target weight value of the inter-group region and the second target weight value of the intra-group region into the corresponding positions, covering the original elements of the second initial weight matrix, and normalizes the integrated matrix so that the sum of the elements in each row is 1, so as to avoid the weight of a certain type of feature being too high, and finally obtains the second target weight matrix.

[0133] Step a2108: Multiply the first hidden feature by the second target weight matrix to obtain the second hidden feature.

[0134] Specifically, the first hidden feature is multiplied by the second target weight matrix to obtain the second hidden feature with the initial dimension.

[0135] Step a3: Activate the second hidden feature to generate noisy data.

[0136] Specifically, the electronic device splits the second hidden feature by modality and maps it to the target dimension: the first 12 dimensions: ECG noise features; the middle 7 dimensions: respiratory noise features; the last 5 dimensions: eye movement noise features; finally forming a 24-dimensional noise feature vector.

[0137] For ECG noise feature processing: OutputECG = Tanh(x) × 0.6 + 0.2, mapping the features to the [-0.4, 0.8] interval (matching the amplitude distribution of real ECG noise). For respiratory noise feature processing: OutputResp = Tanh(x) × 0.5 - 0.1, mapping the features to the [-0.6, 0.4] interval (consistent with the baseline offset characteristics of respiratory signals). For eye movement noise feature processing: directly using the Tanh(x) output, mapping to the [-1, 1] interval (eye movement signals have a wide dynamic range).

[0138] The electronic device checks whether each noise feature falls within a physiologically reasonable range (e.g., ECG noise amplitude ≤ 0.8mV, respiratory rate 12-30rpm). Then, it checks whether the correlation of noise data across modal features conforms to a pattern (e.g., when HR increases by > 10bpm, does the respiratory rate increase synchronously?). Finally, only noise data with a comprehensive score ≥ 80 points are retained, and abnormal samples that do not conform to physiological patterns are removed.

[0139] Step S3032: Input the noise data into the preset discrimination network in the preset data augmentation model to discriminate the noise data.

[0140] Specifically, step S3032 may include the following steps: Step b1: Calculate the rate of load change based on the initial ECG and initial eye movement data.

[0141] Specifically, electronic devices can extract heart rate (HR) data and RR interval standard deviation (SDNN) data from initial electrocardiogram (ECG) data, and extract pupil diameter change rate and saccade speed from initial eye movement (EMG) data.

[0142] For ECG load values: the formula ECG_load = 0.6 × (current HR - resting HR) / resting HR + 0.4 × (baseline SDNN - current SDNN) / baseline SDNN is used to quantify the load increase caused by increased heart rate and decreased heart rate variability (range 0-1). Here, resting HR and baseline SDNN can be obtained by testing the initial ECG data and initial eye movement data of the trained pilots.

[0143] For eye movement load (EOG) values: the formula EOG_load = 0.7 × (current pupil diameter - resting pupil diameter) / resting pupil diameter + 0.3 × current saccade velocity / maximum saccade velocity is used to reflect pupil dilation and visual scanning intensity (range 0-1). The resting pupil diameter and maximum saccade velocity can be obtained by testing trained pilots using initial ECG and initial eye movement data. The overall load value is fused: Overall Load = 0.5 × ECG_load + 0.5 × EOG_load, balancing the contributions of ECG and eye movement characteristics.

[0144] Load change rate calculation: with a time window of 10 seconds, the rate = (current total load - total load 10 seconds ago) / 10, the unit is "load units / second", a positive value indicates that the load is increasing, and a negative value indicates that it is decreasing.

[0145] Step b2 involves fusing noise data, training flight phases, training cognitive load levels, and load change rates to generate target fusion features.

[0146] Specifically, electronic devices can stitch together noise data, training flight phases, training cognitive load levels, and load change rates to generate target fusion features.

[0147] For example, the noise data is 24-dimensional, including 12-dimensional ECG, 7-dimensional respiration, and 5-dimensional eye-tracking noise features; the training flight phase is converted into a 3-dimensional vector through one-hot encoding, such as takeoff=[1,0,0], cruise=[0,1,0], landing=[0,0,1]; the training cognitive load level is a 1-dimensional numerical value, such as integer encoding of levels 0-5; and the load change rate is a 1-dimensional continuous value, output in step b1. The fusion method is to concatenate the data in the order of "noise data → flight phase → load level → change rate" to form a 24+3+1+1=29-dimensional target fusion feature, preserving the original correlation between each element.

[0148] Step b3: Based on the first branch, perform feature compression on the target fusion features to obtain the first branch features.

[0149] Specifically, electronic devices can construct a first weight matrix (e.g., a 50×32 matrix) and, through training, select features strongly correlated with the steady state of cognitive load. Higher weight values ​​indicate a stronger correlation between the feature and the steady state. Key stable features to retain include: ECG features: resting heart rate (average heart rate at baseline), baseline value of heart rate variability SDNN (standard deviation of normal sinus intervals); respiratory features: baseline respiratory rate (average respiratory rate per minute at rest), baseline tidal volume; eye movement features: resting pupil diameter, baseline blink rate. The fluctuation characteristics of stable features are as follows: under a steady cognitive load state (e.g., at rest or with low load), the fluctuations in physiological features are mostly small positive changes (e.g., a slight increase in heart rate near the baseline).

[0150] Matrix multiplication operation: Multiply the target fused features with the first weight matrix to obtain the first compressed intermediate features, thus completing the first feature compression.

[0151] Then, the electronic device constructs a second weight matrix (e.g., a 32×16 matrix) to compress the first compressed intermediate features. Compression objective: Focus on the most core stable physiological features, eliminate secondary correlated features, and improve feature discrimination efficiency. Core feature selection: ECG core features: SDNN baseline value, 24-hour mean resting heart rate, LF / HF baseline ratio (low-frequency to high-frequency power ratio); Respiratory core features: baseline values ​​of basic respiratory rate and respiratory depth coefficient of variation; Eye movement core features: resting pupil diameter level, fixation stability baseline value; Conditional correlation features: flight phase coding under low-load conditions, stable load level label.

[0152] Finally, through the mechanism of the ReLU activation function, expressed as f(x) = max(0,x), the input feature values ​​are thresholded, retaining only positive features and setting negative features directly to 0, thus obtaining the first branch features. Noise suppression indicates that negative fluctuations (such as sudden drops in heart rate or abnormal pupil constriction) are usually noise signals in an unstable state (such as measurement errors or sudden interference). ReLU can effectively filter these outliers and purify stable features.

[0153] Example: The baseline resting heart rate is 65 bpm, and the normal fluctuation range is 60-70 bpm (positive deviation). A sudden drop to 50 bpm with a negative deviation (-15) will be suppressed by ReLU to avoid interfering with the judgment of the steady state.

[0154] Step b4: Based on the second branch, perform feature compression on the target fusion features to obtain the second branch features.

[0155] Specifically, electronic devices can construct a third weight matrix (e.g., a 50×32 matrix), with the core objective of filtering features strongly correlated with dynamic changes in cognitive load. Feature types emphasized include: ECG dynamic features: heart rate increase (heart rate difference between adjacent 5-second intervals), LF / HF ratio change rate (sympathetic nerve activity indicator), RR interval shortening rate; Eye movement dynamic features: pupil dilation rate (diameter change per unit time), fixation duration shortening magnitude, scan rate increase; Conditional correlation dynamic features: task complexity increase rate, load level change slope. The filtering logic is that the matrix weight values ​​are positively correlated with the dynamic correlation of the features; for example, the weight of heart rate increase is higher than that of resting heart rate, ensuring that dynamic features are preserved after compression, resulting in 32-dimensional second compressed intermediate features.

[0156] In the 32-dimensional second compressed intermediate features output, the key focus is on retaining the detailed changes in high-load associated features: such as the instantaneous fluctuation of the LF / HF ratio rapidly increasing from 1.2 to 2.5 within 10 seconds; and the sudden change signal of fixation duration abruptly shortening from 0.5 seconds to 0.3 seconds. These details directly reflect the rapid increase in cognitive load and are the core basis for dynamic analysis.

[0157] Electronic devices can construct a 32×16 fourth weight matrix to further refine the changing trends of features rather than instantaneous values. Focus on trend types: ECG trends: 5-minute moving average slope of the LF / HF ratio, cumulative increment of heart rate increase; Eye movement trends: 30-second moving average of pupil diameter, deceleration rate of fixation duration; Comprehensive trends: synergy of multimodal feature changes (e.g., synchronicity between heart rate increase and pupil dilation).

[0158] Compression results: The 16-dimensional features fully focus on dynamic trends, such as regular signals like "continuous increase in the LF / HF ratio" and "accelerated pupil diameter dilation." Then, the compressed 16-dimensional features are activated using the ELU activation function to obtain the second branch features.

[0159] The expression for the ELU activation function is as follows: It directly preserves positive features (such as increased heart rate and pupil dilation, which are signals of increased workload), and retains a certain response to negative features (such as a slow decrease in heart rate and pupil constriction, which are signals of decreased workload), rather than completely suppressing them like ReLU. It can capture feature changes when the workload decreases (such as a negative change in heart rate from 100 bpm to 70 bpm after the task difficulty decreases). Such trends are as important as those when the workload increases in dynamic analysis (such as reflecting the recovery process after the task difficulty decreases).

[0160] Step b5: Divide the target fusion features into 4 specific groups according to physiological modality and functional attributes and label them.

[0161] The four dedicated groups are ECG group, respiratory group, eye movement group, and load change rate label group.

[0162] Specifically, for the ECG group (labeled "ECG"): Core features: 12 ECG features from 24-dimensional noisy data, including time-domain, frequency-domain, and nonlinear features such as heart rate (HR), RR interval standard deviation (SDNN), high-frequency power (HF), and low-frequency power (LF). Corresponding conditional encoding: The part of the 25-dimensional conditional encoding related to ECG features, mainly task complexity-assisted encoding related to changes in ECG features (such as encoding related to ECG feature weight adjustment in emergency scenarios). Labeling method: All feature vector elements in this group are labeled "ECG". In subsequent processing, features with this label will be given priority for ECG modality-specific verification (such as frequency band range verification for LF and HF).

[0163] For the respiratory group (labeled "RESP"): Core features: 7 respiratory features from the 24-dimensional noise data, including the average respiratory rate (AVRESP), standard deviation of respiratory peak interval (Std), and respiratory signal energy (Power). Corresponding conditional coding: The part of the 25-dimensional conditional coding related to respiratory features, such as the flight phase coding related to changes in respiratory rate (the coding related to changes in respiratory features during the fifth-side landing phase). Labeling method: A "RESP" label is added. Subsequent processing will focus on verifying the matching between respiratory rate and load level for this group of features (e.g., whether AVRESP > 16 rpm under high load).

[0164] For the eye-tracking group (labeled "EYE"): Core features: Eye-tracking features (5 items) in the 24-dimensional noisy data, including average pupil diameter (PupilMean), number of fixations (FixationCount), and average fixation duration (FixationMeanduration). Corresponding conditional encoding: The part of the 25-dimensional conditional encoding related to eye-tracking features, such as the encoding related to the weighting of eye-tracking features in visually intensive tasks (landing phase). Labeling method: Add the "EYE" label; subsequent verification will focus on the physiological laws unique to the eye-tracking modality, such as the negative correlation between pupil diameter and fixation duration.

[0165] The load change rate label group (labeled "RATE") features are constructed as follows: A separate 1D load change rate label is generated by calculating the average of the heart rate increase and pupil dilation rate over adjacent 5 seconds, ranging from -1 to 1. Labeling method: The "RATE" label is attached. Since it reflects the dynamic trend of cognitive load changes, it will participate as an independent feature in the triggering judgment of the "load difference amplifier" mechanism in subsequent processing (e.g., increasing the weight of sensitive path features when the load level changes from low to high).

[0166] Step b6: Verify the first branch feature and the second branch feature respectively, and obtain the ECG dimension score, respiratory dimension score and eye movement dimension score corresponding to the first branch feature and the second branch feature respectively.

[0167] Specifically, (1) For the calculation of ECG dimension scores: verification indicators and physiological range: LF (low frequency power): the effective physiological range is limited to 0.04-0.15Hz (reflecting sympathetic nerve activity), and exceeding this range (such as 0.03Hz or 0.16Hz) is considered abnormal; HF (high frequency power): the effective physiological range is limited to 0.15-0.4Hz (reflecting parasympathetic nerve activity), and exceeding this range (such as 0.14Hz or 0.41Hz) is considered data distortion.

[0168] The electronic device extracts LF and HF values ​​from the first branch feature (normal path, focusing on static features) and the second branch feature (sensitive path, focusing on dynamic features), respectively. If LF or HF exceeds the physiological range, the weight of the corresponding feature is reduced by "exceeding the limit by 0.5". For example: if HF = 0.45Hz (exceeding the upper limit by 0.05Hz), the weight is reduced by 0.05 × 0.5 = 0.025 times; if LF = 0.03Hz (below the lower limit by 0.01Hz), the weight is reduced by 0.01 × 0.5 = 0.005 times. ECG dimension score generation: Scoring rules: 100 points for both LF and HF within the physiological range; 50 points deducted for one exceeding the limit (50 points gained); 0 points for both exceeding the limit.

[0169] (2) Calculation of respiratory dimension score: Verification index and matching standard: AVRESP (average respiratory rate) must match the load level label and conform to physiological laws. For example, in a high load scenario (load level label is "high"), AVRESP is required to be >16 rpm (breathing speed increases under high load); in a low load scenario (load level label is "low"), AVRESP is required to be 12-16 rpm (breathing is stable).

[0170] The electronic device extracts AVRESP values ​​from the first and second branch features respectively, and combines them with the load level label to determine the matching. If AVRESP does not match the scene (e.g., AVRESP=15rpm under high load), it is marked as a "feature to be corrected", and a penalty weight of 1.2 times is added to the subsequent loss calculation (to strengthen the correction requirement).

[0171] Respiratory dimension score generation: Scoring rules: AVRESP matching the load level gets 100 points; mismatch is marked "to be corrected" and gets 50 points (leaving room for correction).

[0172] (3) Calculation of eye movement dimension score: Verification index and physiological relationship: The physiological effective range of pupil diameter is limited to 2-8mm. Data outside the range (such as 1.5mm or 8.5mm) are considered invalid. The correlation between pupil diameter and fixation duration should be negative (Pearson correlation coefficient < -0.2), that is, fixation duration is shortened when the pupil dilates (which is consistent with the rule that visual scanning speeds up under high load).

[0173] Pupil diameter and fixation duration are extracted from the first and second branch features, respectively, and their Pearson correlation coefficients are calculated. If the pupil diameter exceeds the limit, or the correlation coefficient is ≥-0.2 (if it is only -0.1, the negative correlation is insufficient), the connection weight of the corresponding feature is reduced by 30% (to weaken the influence of invalid features).

[0174] Eye movement dimension score generation: Scoring rules: 100 points are awarded if the pupil diameter is within the range and the correlation coefficient is <-0.2; 50 points are deducted if one item fails to meet the standard; 0 points are awarded if two items fail to meet the standard.

[0175] Finally, the initial ECG dimension scores, initial respiratory dimension scores, and initial eye movement dimension scores corresponding to the first-branch feature and the second-branch feature are standardized to the [0,1] interval (e.g., 100 points equals 1.0, 50 points equals 0.5, and 0 points equals 0.0), respectively, to obtain the ECG dimension scores, respiratory dimension scores, and eye movement dimension scores corresponding to the first-branch feature and the second-branch feature.

[0176] Step b7: The first branch features, the second branch features, and the scores of the ECG dimension, the respiratory dimension, and the eye movement dimension are fused to obtain the discriminative fusion features.

[0177] Optionally, the electronic device can concatenate the first branch features, the second branch features, and the scores of the electrocardiogram dimension, the respiratory dimension, and the eye movement dimension to obtain discriminative fusion features.

[0178] Optionally, the electronic device can also perform weighted fusion of the ECG dimension scores corresponding to the first branch features and the second branch features respectively to obtain the target ECG dimension score; similarly, the target respiratory dimension score and the target eye movement dimension score are obtained. Then, the first branch features, the second branch features, and the target ECG dimension score, the target respiratory dimension score, and the target eye movement dimension score are concatenated.

[0179] For example, the electronic device appends the standardized 3D compliance score (target ECG dimension score, target respiration dimension score, and target eye movement dimension score) to the 32-dimensional feature vector formed by the intersection of the first and second branch features, creating a 35-dimensional feature vector. This vector contains both original feature information and compliance indicators. For instance, the 32-dimensional features reflect the numerical characteristics of physiological signals, while the 3D score reflects whether these features conform to physiological laws and scenario logic, providing a comprehensive basis for subsequent authenticity judgment, load level assessment, and loss calculation.

[0180] Step b8: Input the discriminative fusion features into the authenticity discrimination branch and output the probability that the noisy data is real data.

[0181] Specifically, the authenticity discrimination branch employs a two-layer fully connected network (35-dimensional → 20-dimensional → 1-dimensional), with ReLU activation in the intermediate layers. The output layer processes the discriminative fusion features using a Sigmoid activation function, outputting the probability (P) that noisy data is real data. 真实 ), with a value range of 0-1.

[0182] Judgment criteria: Set the threshold to 0.5, when P 真实 > When a threshold is set, the noisy data is judged to be "close to reality," indicating that its feature distribution differs little from the real physiological characteristics in the original training data; when P 真实 If the value is less than or equal to the set threshold, it is considered "deviation from reality" and requires further analysis in conjunction with other indicators.

[0183] Step b9: Input the discriminative fusion features into the load level discriminative branch and output the load level probability corresponding to the noise data.

[0184] Specifically, the load level discrimination branch uses a two-layer fully connected network (35-dimensional → 24-dimensional → 6-dimensional), with Mish activation in the intermediate layers. The output layer outputs two probability values ​​(summing to 1) through the Softmax activation function, corresponding to the predicted probabilities of high cognitive load and low cognitive load.

[0185] Step b10: Based on the preset loss function, the noisy data is judged.

[0186] Specifically, step b10 may include the following steps: Step b101: Calculate the authenticity loss based on the probability that the noisy data is real data.

[0187] Specifically, the authenticity loss uses the binary classification cross-entropy loss function, with the following formula: in, For data type labels (1 represents real samples, 0 represents generated noise data); This represents the probability that noisy data is real data.

[0188] Step b102: Calculate the load level loss based on the load level probability corresponding to the noise data.

[0189] Specifically, the cross-entropy loss function is used to calculate the loss for low and high load levels, and the formula is as follows: in, and For load level labels (when under low load) , High load , ); and This refers to the probability values ​​for low load and high load in the "Probability of Load Level Corresponding to Noise Data" output in step b9.

[0190] Step b103: Calculate compliance loss based on ECG, respiration, and eye movement scores.

[0191] Specifically, compliance losses are calculated based on the following formula: Among them, s i For the i-th dimension score (e.g., ECG dimension score s1 = 80 points); the lower the score, the lower the s1 score. i The larger the value, the greater the compliance, thus penalizing noisy data that does not conform to physiological characteristics.

[0192] Step b104: Generate a preset loss function based on authenticity loss, load level loss, and compliance loss; Specifically, the preset loss function is: Step b105: Based on the preset loss function, the noisy data is judged.

[0193] Specifically, electronic devices can identify noisy data based on a preset loss function.

[0194] Step S3033: Based on the discrimination results, the initial training dataset is expanded to generate the target training dataset.

[0195] Specifically, the electronic device can supplement the initial training dataset with noise data that is judged to be true based on the discrimination result, thereby expanding the initial training dataset and generating the target training dataset.

[0196] The cognitive load identification model training method provided in this application determines the target expansion dimension based on the training flight stage and training cognitive load level, generates a first initial weight matrix, and corrects the first initial weight matrix to obtain a first target weight matrix. Based on the first target weight matrix, the initial fused features are expanded to obtain a target expansion dimension feature vector. This enables the target cognitive load identification model to adaptively adjust the feature dimension and weights according to the characteristics of different flight stages and load levels, highlighting key features, better adapting to dynamic changes in data, and improving the model's accuracy in assessing cognitive load in specific scenarios. The expanded target expansion dimension feature vector is activated by a preset activation function to obtain the first hidden feature, providing a pre-transformed feature representation more suitable for model learning for subsequent feature processing. This helps the model further extract high-level features and uncover potential patterns in the data.

[0197] Then, the first mutual information value of each sub-feature in the initial fused features is calculated to construct the modality association matrix. First mutual information measures the dependency between features, helping the model understand the correlation between features of different modalities, discover potential connections between features, and provide a basis for subsequent clustering and weight determination. This helps reduce feature redundancy and enables the model to utilize feature information more efficiently. Each first hidden sub-feature in the first hidden features is labeled and clustered to determine the average correlation and connection weights between each cluster group. Grouping related features through clustering allows the target cognitive load identification model to process features of different modalities and functions in a targeted manner, improving the compactness and discriminativeness of feature representation. Determining connection weights helps the model consider the interaction between different cluster groups and better capture the collaborative relationships of multimodal data.

[0198] Next, a second objective weight matrix is ​​generated based on the clustering results and the keyity of features. This second objective weight matrix is ​​then multiplied by the first hidden feature to obtain the second hidden feature. This second objective weight matrix comprehensively considers the inter-group correlation and the importance of features within groups, enabling the target cognitive load identification model to allocate weights more reasonably during feature transformation, highlighting important features and suppressing redundant features, thereby obtaining a more representative and discriminative second hidden feature.

[0199] Noise data is generated by activating the second hidden feature, and the rate of change of cognitive load is calculated. The generation of noise data can simulate noise or uncertainty in real-world data, enhancing the robustness of the model. The calculation of the rate of change of cognitive load reflects the dynamic changes in cognitive load, providing the model with more comprehensive load information and helping to more accurately assess the real-time state of cognitive load. The noise data, training flight phase, training cognitive load level, and rate of change of cognitive load are fused to generate the target fusion feature. Based on the first branch, the target fusion feature is compressed to obtain the first branch feature. Based on the second branch, the target fusion feature is compressed to obtain the second branch feature. The target fusion feature is then divided into four specific groups according to physiological modality and functional attributes and labeled. The first branch feature and the second branch feature are validated separately, obtaining the corresponding ECG dimension score, respiratory dimension score, and eye movement dimension score. Feature compression reduces data dimensionality and computational complexity while retaining key information. The validated dimensional scores can evaluate the rationality and effectiveness of the features from different modal perspectives, providing a more reliable basis for subsequent fusion. The features from the first and second branches, along with scores from the ECG, respiration, and eye-tracking dimensions, are fused to obtain discriminative fusion features. These features are then input into the authenticity discrimination branch, outputting the probability that noisy data is real data. Similarly, the discriminative fusion features are input into the load level discrimination branch, outputting the load level probability corresponding to the noisy data. This fusion method integrates multiple aspects of information, enabling the target cognitive load identification model to evaluate data from both authenticity and load level perspectives. This provides a more comprehensive judgment for cognitive load assessment, contributing to improved accuracy and reliability of the assessment results. Authenticity loss, load level loss, and compliance loss are calculated based on different probabilities and dimensional scores, generating a preset loss function to discriminate noisy data. Through the calculation of multi-dimensional loss functions, the model can measure the difference between predicted results and reality from multiple perspectives, guiding the model to learn more accurate feature representations and prediction rules, continuously optimizing model parameters, and improving the accuracy of the target cognitive load identification model in cognitive load assessment and its ability to discriminate noisy data.

[0200] This embodiment also provides a data expansion device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0201] This embodiment provides a data expansion device, such as... Figure 4 As shown, it includes: The acquisition module 401 is used to acquire the initial training dataset corresponding to the training pilot. The initial training dataset includes the initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight stage, and training cognitive load level corresponding to the training pilot. The generation module 402 is used to extract features from the initial training dataset and generate initial fusion features based on the extracted features; The expansion module 403 is used to input the initial fusion features into the preset data expansion model to expand the initial training dataset and generate the target training dataset.

[0202] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A data augmentation method, characterized in that, The method includes: Obtain the initial training dataset corresponding to the trainee pilot. The initial training dataset includes the initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight phase, and training cognitive load level corresponding to the trainee pilot. Feature extraction is performed on the initial training dataset, and initial fusion features are generated based on the extracted features; The initial fusion features are input into a preset data augmentation model to augment the initial training dataset and generate the target training dataset.

2. The method according to claim 1, characterized in that, The step of expanding the initial training dataset based on a preset data augmentation model to generate a target training dataset includes: Feature extraction is performed on the initial ECG data in the initial training dataset to obtain the total initial ECG features corresponding to the initial ECG data; the total initial ECG features include initial ECG time-domain features, initial ECG frequency-domain features, and initial ECG nonlinear features. Feature extraction is performed on the initial respiratory data in the initial training dataset to obtain the total initial respiratory features corresponding to the initial respiratory data; the total initial respiratory features include initial respiratory time-domain features and initial respiratory frequency-domain features. Feature extraction is performed on the initial eye-tracking data in the initial training dataset to obtain initial eye-tracking features; The initial electrocardiogram features, the initial respiratory features, and the initial eye movement features are fused to generate initial physiological features; Based on the operational elements of each stage of the five-plane flight, task complexity characteristics are generated. The initial condition features are generated by fusing the training flight phase, the training cognitive load level, and the task complexity features. The initial physiological features and the initial conditional features are fused to generate initial fused features; The initial fusion features are input into the preset generation network in the preset data augmentation model to generate noisy data corresponding to the initial training dataset. The noise data is input into the preset discrimination network in the preset data augmentation model to discriminate the noise data; Based on the discrimination results, the initial training dataset is expanded to generate the target training dataset.

3. The method according to claim 1, characterized in that, The step of inputting the initial fusion features into a preset data augmentation model to augment the initial training dataset and generate a target training dataset includes: The initial fusion features are input into the preset generation network in the preset data augmentation model to generate noisy data corresponding to the initial training dataset. The noise data is input into the preset discrimination network in the preset data augmentation model to discriminate the noise data; Based on the discrimination results, the initial training dataset is expanded to generate the target training dataset.

4. The method according to claim 3, characterized in that, The step of inputting the initial fusion features into a preset data augmentation model to augment the initial training dataset and generate a target training dataset includes: The initial fused features are input into the first hidden layer of the preset generation network, and the initial fused features are expanded to generate the first hidden features; The first hidden feature is input into the second hidden layer of the preset generation network, and the first hidden feature is scaled to generate the second hidden feature; The second hidden feature is activated to generate the noise data.

5. The method according to claim 4, characterized in that, The step of inputting the initial fused features into the first hidden layer of the preset generation network, and expanding the initial fused features to generate the first hidden features includes: The training flight phase and the training cognitive load level are determined from the initial fusion features; Based on the training flight phase and the training cognitive load level, determine the target expansion dimension corresponding to the initial fusion feature from a preset mapping table; A first initial weight matrix is ​​generated based on the initial dimension corresponding to the initial fusion feature and the target extended dimension; Based on the training flight phase and the training cognitive load level, the first initial weight matrix is ​​modified to obtain the first target weight matrix; Based on the first target weight matrix, the initial fusion features are expanded to generate a target extended dimension feature vector. The target extended dimension feature vector is activated based on a preset activation function, and the first hidden feature is output.

6. The method according to claim 4, characterized in that, The step of inputting the first hidden feature into the second hidden layer of the preset generation network, scaling the first hidden feature, and generating the second hidden feature includes: Calculate the first mutual information value among the initial sub-fusion features in the initial fusion feature; Based on each of the first mutual information values, construct a modal correlation matrix; Based on the mapping relationship between each first hidden sub-feature in the first hidden feature and each initial sub-fusion feature in the initial fusion feature, each first hidden sub-feature is marked; Based on the labeling results, the mean vectors corresponding to each of the first hidden sub-features dominated by ECG are calculated and used as the first cluster centers. Calculate the mean vector of each of the first hidden sub-features dominated by breathing, and use it as the second cluster center; Calculate the mean vector of each of the first hidden sub-features dominated by eye movement, and use it as the third cluster center; The mean vector of each of the first hidden sub-features dominated by the calculation conditions is used as the fourth cluster center; Based on the modal correlation matrix, calculate the second mutual information value between each of the first hidden sub-features; Based on the second mutual information value, each of the first hidden sub-features is clustered to obtain an ECG cluster group, a respiration cluster group, an eye movement cluster group, and a conditional cluster group; The second hidden feature is output based on the ECG cluster group, the respiratory cluster group, the eye movement cluster group, and the conditional cluster group.

7. The method according to claim 6, characterized in that, The step of outputting the second hidden feature based on the ECG cluster group, the respiration cluster group, the eye movement cluster group, and the conditional cluster group includes: Based on the modal correlation matrix, the average correlation degree among the ECG cluster group, the respiratory cluster group, the eye movement cluster group, and the conditional cluster group is calculated; Based on the average correlation degree, the connection weights between the ECG cluster group, the respiratory cluster group, the eye movement cluster group, and the conditional cluster group are determined; A second initial weight matrix is ​​generated based on the initial dimension corresponding to the initial fusion feature and the target extended dimension; The second initial weight matrix is ​​divided into intra-group and inter-group regions; Based on the connection weights between the ECG cluster group, the respiratory cluster group, the eye movement cluster group, and the respiratory cluster group, determine the first target weight value corresponding to the inter-group region in the second initial weight matrix; Based on the criticality of each cluster sub-feature in the ECG cluster group, the respiratory cluster group, the eye movement cluster group, and the conditional cluster group, the second target weight value corresponding to the region within the group is determined; Based on the first target weight and the second target weight value, a second target weight matrix is ​​generated; The second hidden feature is obtained by multiplying the first hidden feature by the second target weight matrix.

8. The method according to claim 3, characterized in that, The step of inputting the noise data into the preset discrimination network in the preset data augmentation model and discriminating the noise data includes: Based on the initial electrocardiogram data and the initial eye movement data, calculate the rate of load change; The noise data, the training flight phase, the training cognitive load level, and the load change rate are fused to generate target fusion features; Based on the first branch, the target fusion features are compressed to obtain the first branch features; Based on the second branch, the target fusion features are compressed to obtain the second branch features; The target fusion features are divided into four specific groups and labeled according to physiological modalities and functional attributes; the four specific groups are the electrocardiogram group, the respiratory group, the eye movement group, and the load change rate label group. The first branch feature and the second branch feature are verified respectively to obtain the ECG dimension score, respiratory dimension score and eye movement dimension score corresponding to the first branch feature and the second branch feature respectively; The first branch features, the second branch features, the ECG dimension score, the respiratory dimension score, and the eye movement dimension score are fused together to obtain the discriminative fusion features; The discriminative fusion features are input into the authenticity discrimination branch, and the probability that the noisy data is real data is output. The discriminative fusion features are input into the load level discrimination branch, and the load level probability corresponding to the noise data is output. The noise data is judged based on a preset loss function.

9. The method according to claim 8, characterized in that, The step of judging the noise data based on a preset loss function includes: Calculate the authenticity loss based on the probability that the noise data is real data; Based on the load level probability corresponding to the noise data, calculate the load level loss; Compliance loss is calculated based on the ECG score, the respiration score, and the eye movement score. Based on the authenticity loss, the load level loss, and the compliance loss, the preset loss function is generated; The noise data is judged based on the preset loss function.

10. A data expansion device, characterized in that, The device includes: The acquisition module is used to acquire the initial training dataset corresponding to the trainee pilot. The initial training dataset includes the initial electrocardiogram data, initial respiratory data, initial eye movement data, training flight stage, and training cognitive load level corresponding to the trainee pilot. The generation module is used to extract features from the initial training dataset and generate initial fusion features based on the extracted features; An expansion module is used to input the initial fusion features into a preset data expansion model to expand the initial training dataset and generate a target training dataset.

Images