An efficient simulation flight platform optimization method and system based on big data

Abstract

The application provides a high-efficiency simulation flight platform optimization method and system based on big data, which comprises the following steps: constructing a general data set and a field data set, collecting a target task data set, and containing eye movement data and operation data in each data set; building a multi-modal feature fusion network; performing first-stage pre-training on the multi-modal feature fusion network based on the general data set to obtain a general base model; performing second-stage pre-training on the base model based on the field data set to obtain a field base model; performing fine-tuning training on the field base model based on the target task data set to obtain a target task model, and realizing prediction of pilot operation deviation and optimization of the simulation flight platform based on the target task model. Through integration of eye movement and operation multi-modal data, the application realizes dynamic optimization of platform parameters and individualized assistance in the training process by using big data analysis and transfer learning technology.

Description

Technical Field

[0001] This invention belongs to the field of electronic digital data processing technology, specifically relating to an efficient simulation flight platform optimization method and system based on big data. Background Technology

[0002] Current flight simulation training platforms mainly rely on fixed scenario simulations and standardized operation assessments, resulting in low data utilization efficiency. Furthermore, a large amount of flight operation data and physiological feedback data are only recorded after the fact, failing to form a closed loop for dynamic optimization of platform performance. This makes it impossible to adjust in real time according to the different operational characteristics and skill levels of trainees, leading to low training efficiency and resource utilization.

[0003] With the development of big data and virtual simulation technologies, some simulation platforms have begun to incorporate data acquisition and analysis functions. Big data analytics and transfer learning technologies offer new solutions to the aforementioned problems. However, there is currently no systematic approach to applying these technologies to the dynamic optimization of flight simulation platforms, particularly lacking technical solutions that combine multimodal data to achieve self-adjustment of platform parameters and personalized auxiliary strategies. Therefore, there is an urgent need for an efficient optimization method and system for flight simulation platforms that can integrate multi-source data, reuse extensive experience, and dynamically optimize platform performance. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a method and system for optimizing a high-efficiency simulation flight platform based on big data. The aim is to integrate eye-tracking and operational multimodal data and utilize big data analysis and transfer learning techniques to achieve dynamic optimization of platform parameters and personalized assistance in the training process.

[0005] To achieve the above objectives, the present invention provides the following solution: A method for optimizing a high-efficiency flight simulation platform based on big data, the method comprising: S1. Construct a general dataset and a domain dataset, and collect target task datasets. Each dataset contains eye-tracking data and operational data. S2. Construct a multimodal feature fusion network; S3. Based on a general dataset, perform the first stage of pre-training on the multimodal feature fusion network to obtain a general basic model; S4. Perform a second-stage pre-training of the basic model based on the domain dataset to obtain the domain basic model; S5. Fine-tune and train the domain basic model based on the target task dataset to obtain the target task model. Based on the target task model, realize the prediction of pilot operation deviation and the optimization of the simulation flight platform.

[0006] Preferred, A general dataset, including eye-tracking data and operational data from various operating scenarios such as car driving and industrial equipment operation; Domain datasets include eye-tracking and operational data from senior pilots in different flight scenarios. Senior pilots are those with ≥3000 hours of flight time or who serve as pilot instructors. The target task dataset includes eye-tracking and operational data of pilots awaiting deployment in different flight scenarios.

[0007] Preferably, the multimodal feature fusion network includes: an eye-tracking feature extraction module, an operation feature extraction module, a cross-modal attention fusion module, and a domain discriminator; The eye-tracking feature extraction module is used to extract the temporal features of eye-tracking data to obtain eye-tracking features; The operation feature extraction module is used to extract the sequence features of the operation data to obtain the operation features; The cross-modal attention fusion module is used to calculate the mutual information entropy of eye-tracking features and operational features, dynamically allocate fusion weights, and fuse eye-tracking features and operational features based on the fusion weights to obtain fused features; A domain discriminator is used to determine the domain origin of the fused features.

[0008] Preferably, the method for dynamically allocating fusion weights based on the mutual information entropy of eye-tracking features and operational features includes: ; ; in, For eye movement feature weights, For the operation feature weights, This refers to the mutual information between eye-tracking features and operational features. This refers to the mutual information between operational features and eye-tracking features.

[0009] Preferably, the method for dynamically allocating fusion weights based on the mutual information entropy of eye-tracking features and operational features further includes: Introducing a temperature coefficient into the weighting formula: ; ; Where T is the temperature coefficient, which is dynamically adjusted according to the differences in modal importance in the target task.

[0010] Preferably, methods for fine-tuning and training a domain-based foundational model based on the target task dataset to obtain a target task model include: Based on the target task dataset, a hierarchical parameter transfer strategy is adopted to transfer the knowledge of the domain base model to the initial target task model; A progressive parameter unfreezing strategy was adopted to fine-tune the initial target task model and obtain the target task model.

[0011] The present invention also provides a high-efficiency simulation flight platform optimization system based on big data. The system is used to implement the aforementioned method and includes: a data acquisition module, a network construction module, a first training module, a second training module, and a fine-tuning module. The data acquisition module is used to build general datasets and domain datasets, and to collect target task datasets. Each dataset contains eye-tracking data and operational data. The network construction module is used to build multimodal feature fusion networks; The first training module is used to perform the first stage of pre-training on the multimodal feature fusion network based on a general dataset to obtain a general basic model; The second training module is used to perform a second-stage pre-training of the basic model based on the domain dataset to obtain the domain basic model. The fine-tuning module is used to fine-tune the domain base model based on the target task dataset to obtain the target task model. Based on the target task model, the prediction of pilot operation deviations and the optimization of the simulation flight platform are realized.

[0012] Preferred, A general dataset, including eye-tracking data and operational data from various operating scenarios such as car driving and industrial equipment operation; Domain datasets include eye-tracking and operational data from senior pilots in different flight scenarios. Senior pilots are those with ≥3000 hours of flight time or who serve as pilot instructors. The target task dataset includes eye-tracking and operational data of pilots awaiting deployment in different flight scenarios.

[0013] Preferably, the multimodal feature fusion network includes: an eye-tracking feature extraction module, an operation feature extraction module, a cross-modal attention fusion module, and a domain discriminator; The eye-tracking feature extraction module is used to extract the temporal features of eye-tracking data to obtain eye-tracking features; The operation feature extraction module is used to extract the sequence features of the operation data to obtain the operation features; The cross-modal attention fusion module is used to calculate the mutual information entropy of eye-tracking features and operational features, dynamically allocate fusion weights, and fuse eye-tracking features and operational features based on the fusion weights to obtain fused features; A domain discriminator is used to determine the domain origin of the fused features.

[0014] Preferably, the fine-tuning module includes: a migration unit and a fine-tuning unit; The transfer unit is used to transfer knowledge from the domain base model to the initial target task model based on the target task dataset using a hierarchical parameter transfer strategy. The fine-tuning unit is used to fine-tune the initial target task model using a progressive parameter unfreezing strategy to obtain the target task model.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a high-efficiency simulation flight platform optimization method and system based on big data. By using a general dataset, a domain dataset, and a target task dataset, a multimodal feature fusion network is built. The multimodal feature fusion network is pre-trained in the first stage based on the general dataset, and a general basic model is pre-trained in the second stage based on the domain dataset. Finally, the parameters of the domain basic model are transferred to the target task model through a parameter transfer strategy, and fine-tuning training is performed based on the target task dataset. This realizes the construction and adaptive dynamic adjustment of the network model based on multimodal data and big data, which can effectively improve the training efficiency and utilization of the simulation flight platform. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the optimization method for a high-efficiency simulation flight platform based on big data according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the construction and training process of a multimodal feature fusion network according to an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] Example 1 This invention provides a method for optimizing a high-efficiency flight simulation platform based on big data, comprising: S1. Construct a general dataset and a domain dataset, and collect target task datasets. Each dataset contains eye-tracking data and operational data. S2. Construct a multimodal feature fusion network; S3. Based on a general dataset, perform the first stage of pre-training on the multimodal feature fusion network to obtain a general basic model; S4. Perform a second-stage pre-training of the basic model based on the domain dataset to obtain the domain basic model; S5. Fine-tune and train the domain basic model based on the target task dataset to obtain the target task model. Based on the target task model, realize the prediction of pilot operation deviation and the optimization of the simulation flight platform.

[0021] like Figures 1-2 As shown, the specific implementation process of the present invention is as follows: S1. Construct a general dataset and a domain dataset, and collect target task datasets. Each dataset contains eye-tracking data and operational data, including: The general dataset includes eye-tracking data and operational data from various operation scenarios such as car driving and industrial equipment operation; the eye-tracking data includes gaze coordinates, pupil diameter, etc., and the operational data includes joystick position, throttle opening, etc.

[0022] Domain datasets include eye-tracking and operational data from senior pilots in different flight scenarios. Senior pilots are those with ≥3000 hours of flight time or who serve as pilot instructors. The target task dataset includes eye-tracking and operational data of pilots awaiting deployment in different flight scenarios.

[0023] Specifically, the method for constructing a general dataset in this invention is as follows: A dataset with high task matching degree and large data volume (task degree refers to content related to operational environment requirements, personnel requirements, and task presumptions, such as the publicly available vehicle dataset selected in this embodiment, which contains the same data types as this embodiment—eye-tracking data and operational data—and has a certain degree of scene adaptability—both are based on driving environments with human operation and have the same multi-scenario testing tasks, etc., showing a certain degree of matching in the tasks of the data source, thus making it suitable as a general dataset. In the prior art, general datasets for transfer learning are usually pre-trained using publicly available image datasets, but their feature sources and data types differ from the type studied in this invention. Therefore, this invention changes the selection of the general dataset and filters and constructs it according to the above method), extracting the required features (required features refer to the features required by the domain dataset, which should be included in the general dataset; in this embodiment, these can correspond to eye-tracking features and operational behavior data) to construct the dataset. This application uses SEED-VIG. This dataset is a multimodal dataset primarily used to study driving fatigue and alertness. Data was collected through simulated driving experiments and mainly includes electroencephalogram (EEG), electrooculogram (EOG), and eye-tracking data. Eye-tracking data was used to calculate the PERCLOS index (percentage of eyelid closure time) as a label for alertness. Specifically, in the experiments, vehicle movement was controlled via the steering wheel and accelerator pedal, and the scene was updated in real time based on the participants' actions, thus also including some operational data.

[0024] Specifically, the method for constructing the domain dataset in this invention is as follows: Attitude training is performed using the AW109 VR flight simulation program; a flight simulator is introduced for training to build a simulated flight platform; and multimodal data from experienced pilots under different flight scenarios is collected through eye-tracking devices and the simulated flight platform. The multimodal data includes eye-tracking feature data and operational behavior data. The eye-tracking feature data includes fixation point coordinates, fixation duration, saccade amplitude, and pupil change rate. The operational behavior data includes joystick displacement, throttle adjustment, avionics interaction frequency, and emergency operation response time. The collected multimodal data is cleaned, specifically by deleting invalid data. Then, to unify the dimensions of the multimodal data, it is normalized for subsequent model processing. Scene type labels and operational quality labels are labeled to form the domain dataset. Specifically, the scene type labels include takeoff, cruise, approach, landing, and special scenarios involving single engine failure. The operational quality labels are divided into three levels: excellent, good, and satisfactory, based on the operational error rate.

[0025] Specifically, the method for collecting the target task dataset in this invention is as follows: Attitude training is performed using the AW109 VR flight simulation program; a flight simulator is introduced for training to build a simulated flight platform; multimodal data of pilots about to be deployed (pilots awaiting deployment) under different flight scenarios are collected through eye-tracking devices and the simulated flight platform. The multimodal data includes eye-tracking feature data and operational behavior data. The eye-tracking feature data includes fixation point coordinates, fixation duration, saccade amplitude, and pupil change rate; the operational behavior data includes joystick displacement, throttle adjustment, avionics interaction frequency, and emergency operation response time. The collected multimodal data is cleaned, specifically by deleting invalid data. Then, to unify the dimensions of the multimodal data, it is normalized for subsequent model processing. Scene type labels and operational quality labels are labeled to form the target task dataset. Specifically, the scene type labels include takeoff, cruise, approach, landing, and special scenarios involving single engine failure; the operational quality labels are divided into three levels: excellent, good, and satisfactory based on the operational error rate.

[0026] S2. Construct a multimodal feature fusion network, including: The constructed multimodal feature fusion network includes: an eye-tracking feature extraction module, an operation feature extraction module, a cross-modal attention fusion module, and a domain discriminator; The eye-tracking feature extraction module is used to extract the temporal features of eye-tracking data to obtain eye-tracking features; The operation feature extraction module is used to extract the sequence features of the operation data to obtain the operation features; The cross-modal attention fusion module is used to calculate the mutual information entropy of eye-tracking features and operational features, dynamically allocate fusion weights, and fuse eye-tracking features and operational features based on the fusion weights to obtain fused features; A domain discriminator is used to determine the domain origin of the fused features.

[0027] Specifically, since eye-tracking features and operational features change along with the timeline of flight events, and operational features directly reflect the pilot's operations while eye-tracking features indirectly reflect the characteristics of the pilot's operations, time-series feature extraction is performed on both types of features to ensure the timeliness consistency of the two features and to maximize the representation of the information contained in the two features.

[0028] The eye-tracking feature extraction module uses a three-layer one-dimensional convolutional neural network with kernel sizes of 3, 5, and 3, and output channels of 64, 128, and 256, respectively, to extract the temporal features of the eye-tracking data. The temporal features of the extracted eye-tracking data are denoted as E.

[0029] The operation feature extraction module uses a two-layer bidirectional LSTM network (which can be referred to as the top layer and the bottom layer to facilitate subsequent layer-by-layer freezing training of the network model). Specifically, the number of hidden layer units is set to 256, and the dropout probability is 0.2, which is used to extract the sequence features of the operation data. The sequence features of the extracted operation data are denoted as O.

[0030] In the cross-modal attention fusion module, fusion weights are dynamically assigned by calculating the mutual information entropy of eye-tracking features and operational features. The eye-tracking features and operational features are then fused according to these weights, resulting in the following fused features: ; The mutual information entropy of eye-tracking features and operational features is dynamically allocated to the fusion weights as follows: ; ; in, For eye movement feature weights, For the operation feature weights, This refers to the mutual information between eye-tracking features and operational features. This refers to the mutual information between operational features and eye-tracking features.

[0031] The cross-modal attention fusion module can dynamically adjust the matching weights of the two features, thereby ensuring the optimal effectiveness of the features extracted by the model.

[0032] Furthermore, this invention introduces a temperature coefficient T into the weight allocation formula (T=1 in the domain-based model, with no temperature regulation), and the adjusted weight allocation formula is as follows: ; ; Specifically, the temperature coefficient T is dynamically adjusted based on the differences in modal importance in the target task. If operational features are more important than eye-tracking features in the target task (e.g., operational accuracy is prioritized in special situation handling), then T = 0.8 (reducing the temperature coefficient). The weighting of eye-tracking features is calculated; if eye-tracking features are more important (e.g., attention-guided operations in complex scenes), then T = 1.2 (increases the weighting). (weight).

[0033] This invention processes features by dynamically adjusting the weights of two features through the addition of a temperature coefficient to adapt to feature allocation at different flight stages, thereby enabling the model to achieve optimal results at each stage.

[0034] A fully connected neural network structure is used as a domain discriminator to determine whether the fused features come from the source domain or the target domain. Through adversarial training, the distribution of the fused features in the source domain and the target domain tends to be consistent. The general dataset is the source domain, and the domain dataset is the target domain.

[0035] Specifically, the steps for using a domain discriminator are as follows: Label the input fused features with "domain labels": features from the general dataset (source domain) are labeled with "1", and features from the domain dataset (target domain) are labeled with "0"; input the fused features F and the corresponding domain labels into the domain discriminator, calculate the discriminator loss, and here the binary cross-entropy loss function is used as the loss function for loss judgment, update the discriminator parameters, so that the discriminator has the ability to distinguish between source domain and target domain fused features (for example, it can be trained until the binary cross-entropy loss function does not decrease for 3 consecutive rounds on the validation set).

[0036] S3. Based on a general dataset, perform the first stage pre-training of the multimodal feature fusion network to obtain a general basic model. This enables the multimodal feature fusion network to learn cross-domain general rules, avoiding the low learning efficiency caused by the randomization of initial parameters. This includes: The general dataset is divided into training and validation sets in a 7:3 ratio.

[0037] The training set is input into a multimodal feature fusion network using the AdamW optimizer. The initial learning rate is set to 1e-4, weight decay to 1e-5, batch size to 32, first-order momentum coefficient β1 = 0.9, second-order momentum coefficient β2 = 0.999, and a numerical stability term ε = 1e-8 to prevent division by zero. The training consists of 80 epochs, with each epoch using forward propagation to update the parameters of all modules in the multimodal feature fusion network. After each epoch, an early stopping strategy is employed. The model performance is evaluated by calculating the validation set cross-entropy loss and the validation set classification accuracy, and the model parameters for the current epoch are saved. Training stops when the validation set loss function value does not decrease for 8 consecutive epochs, and the current model is saved as a general base model to avoid overfitting.

[0038] In this embodiment, the application of general data is continuously adjusted and changed to meet the training requirements of the present invention.

[0039] S4. Perform a second-stage pre-training of the base model based on the domain dataset to obtain the domain base model, including: This step optimizes the general basic model for domain adaptation. While retaining general knowledge, it allows the multimodal feature fusion network to learn professional features related to the flight operation domain. This enables the present invention to transfer to the pilot's flight missions (such as pilot emergency handling), thereby avoiding the problem of overfitting when the multimodal feature fusion network is directly adapted to domain data in transfer learning.

[0040] Specifically, since the lower-level convolutional learning of eye-tracking data shows strong generality and requires no domain adaptation, while the upper-level convolutional learning of pilot gaze patterns on the instrument panel needs updating and adaptation, the lower-level LSTM learning of the temporal correlation of operational actions shows strong generality, and the upper-level LSTM learning of the continuous adjustment patterns of the joystick during takeoff needs updating and adaptation. Therefore, we first freeze the parameters of the lower two convolutional kernels (the weights and biases of the first and second layers) in the eye-tracking feature extraction module (3 layers of 1D convolution) and the lower-level LSTM parameters (the hidden layer weights, input gate / forget gate / output gate weights) in the operational feature extraction module of the general base model. Only the parameters of the upper-level (3rd layer) convolutional kernel of the eye-tracking feature extraction module and all parameters of the upper-level LSTM of the operational feature extraction module, as well as the biases of the two LSTM layers, are retained and can be updated.

[0041] The domain dataset is divided into a training set and a validation set in a 7:3 ratio, and the training set is input into the general base model.

[0042] The AdamW optimizer is still used, with an initial learning rate of 5e-5, weight decay of 1e-5, batch size of 32, first-order momentum coefficient β1=0.9, second-order momentum coefficient β2=0.999, numerical stability term ε=1e-8 to prevent division by zero, and 50 training epochs. In each training epoch, the forward propagation method is used to update the parameters of all modules of the multimodal feature fusion network.

[0043] After each training round, an early stopping strategy is adopted. The model performance is evaluated by calculating the validation set cross-entropy loss and the validation set classification accuracy using the validation set. The model parameters of the current round are saved. Training stops when the validation set loss function value does not decrease for 5 consecutive rounds. The current model is saved as the domain base model. Thus, the pre-training of the model is completed.

[0044] S5. Fine-tune and train the domain-based model based on the target task dataset to obtain the target task model. Based on this model, predict pilot operational deviations and optimize the simulation flight platform, including: Methods for fine-tuning and training a domain-based foundational model based on a target task dataset to obtain a target task model include: Based on the target task dataset, a hierarchical parameter transfer strategy is adopted to transfer the knowledge of the domain base model to the initial target task model; A progressive parameter unfreezing strategy was adopted to fine-tune the initial target task model and obtain the target task model.

[0045] Specifically, the target task dataset is divided into a training set and a validation set in a 7:3 ratio, and the training set is input into the domain base model; A hierarchical parameter transfer strategy is adopted to transfer the knowledge of the domain-based model to the target task model (the fine-tuned domain-based model is the target task model). Due to the special nature of the data feature types collected in this application, in order to maximize the information content contained in different features, specific processing methods are adopted for different features in step S2 to build a multimodal feature fusion network. Then, the multimodal feature fusion network is pre-trained in the first stage based on a general dataset to obtain a general basic model. In this step, the method of fine-tuning the domain-based model requires separate hierarchical parameter transfer for the specifically designed parallel feature processing network, so as to ensure the accuracy of network adjustment during fine-tuning training and improve the target recognition effect of the final model. Specifically, firstly, direct transfer is performed, transferring the bottom two convolutional kernel parameters of the eye-tracking feature extraction module and the bottom LSTM parameters of the operational feature extraction module from the domain-based model to the target task model. Then, adjustment transfer is performed, inputting the training set of the target task dataset into the target task model, adjusting the top convolutional kernel parameters of the eye-tracking feature extraction module, all parameters of the top LSTM of the operational feature extraction module, and the bias parameters of the two LSTM layers, training these parameters, and then calculating the cross-entropy loss and classification accuracy of the target task model using validation set data, saving the model parameters for the current round, stopping when the loss function value does not decrease for 5 consecutive rounds and the classification accuracy is higher than 90%. Finally, framework preservation is performed, retaining the basic framework for calculating mutual information entropy in the cross-modal attention fusion module, retaining the mutual information entropy calculation framework and basic weight formula, adjusting the coefficients and temperature coefficient T in the weight allocation formula, and obtaining the initial target task model.

[0046] A progressive parameter unfreezing strategy is employed to fine-tune the initial target task model: Specifically, in the initial training phase (e.g., the first 10 rounds), direct transfer parameters and frame-preserving parameters are frozen, and only the parameters for adjusting the transfer are trained. The loss function after training is calculated, and the cross-entropy loss function is selected here. As training progresses (after the tenth round), the transfer parameters and frame-preserving parameters are gradually unfrozen, with each 10 rounds serving as a node. During this progressive parameter unfreezing process, the loss function after training is calculated, again using the cross-entropy loss function. Furthermore, to avoid overfitting and improve the model's generalization ability on the target task, regularization parameter constraints (e.g., L1 regularization) are applied until all parameters are unfrozen. Specifically, the total loss during backpropagation is: ; in, This represents the total loss during backpropagation of the model. The cross-entropy loss function; This is the regularization intensity coefficient; Parameters that have been thawed w The sum of squares, This is the set of parameters that have been thawed.

[0047] Train the model on the validation set of the target task. Training stops when the model fails to improve after 5 consecutive rounds, yielding the final model (the fine-tuned target task model). It's also important to note that when using parameter regularization constraints, regularization is only applied to unfrozen parameters; unfrozen parameters are not included in the regularization calculation.

[0048] In addition, big data can be used to periodically optimize the above models. This can be achieved by periodically collecting pilot data (e.g., collecting the same flight data from newly added pilots who are about to start their jobs each year, and from newly added senior pilots), test performance data, and platform operation logs, and updating them to the domain dataset or target task dataset. Incremental training can be used to iteratively update the platform optimization base model (domain base model or target task model) (updated data is input into the domain dataset or target task dataset), enabling the model to continuously learn new operating modes and optimization requirements, and ensuring that the platform performance continues to improve with the accumulation of data.

[0049] This invention utilizes transfer learning to pre-train a general basic model to obtain the overall model framework. Then, by using targeted domain data and newly input domain data, and through gradual unfreezing and parameter fine-tuning, the domain basic model and the target task model can be effectively adapted to avoid cold starts, allowing the existing model to quickly adapt to new tasks. Furthermore, considering the varying importance of collected operational and eye-tracking features under different flight conditions, this application sets a temperature coefficient to dynamically adjust the weight of modal importance in the target task. In this way, under different flight tasks and flight phases, different types of feature parameters can be effectively dynamically adjusted to maximize feature information and optimize the model, achieving corresponding personalized assistance results.

[0050] In this embodiment, the above method can effectively improve the accuracy of the final model in the task of predicting pilot operation deviations.

[0051] In summary, this invention provides an efficient optimization method for a flight simulation platform based on big data. By using a general dataset, a domain dataset, and a target task dataset, a multimodal feature fusion network is constructed. The multimodal feature fusion network is pre-trained in the first stage based on the general dataset, and a general basic model is pre-trained in the second stage based on the domain dataset. Finally, the parameters of the domain basic model are transferred to the target task model through a parameter transfer strategy, and fine-tuning training is performed based on the target task dataset. This method realizes the construction and adaptive dynamic adjustment of the network model based on multimodal data and big data, which can effectively improve the training efficiency and utilization of the flight simulation platform.

[0052] Example 2 Based on the same inventive concept, the present invention also provides a high-efficiency simulation flight platform optimization system based on big data, used to implement the method described in the foregoing embodiments. The system includes: a data acquisition module, a network construction module, a first training module, a second training module, and a fine-tuning module. The data acquisition module is used to build general datasets and domain datasets, and to collect target task datasets. Each dataset contains eye-tracking data and operational data. The network construction module is used to build multimodal feature fusion networks; The first training module is used to perform the first stage of pre-training on the multimodal feature fusion network based on a general dataset to obtain a general basic model; The second training module is used to perform a second-stage pre-training of the basic model based on the domain dataset to obtain the domain basic model. The fine-tuning module is used to fine-tune the domain base model based on the target task dataset to obtain the target task model. Based on the target task model, the prediction of pilot operation deviations and the optimization of the simulation flight platform are realized.

[0053] Furthermore, A general dataset, including eye-tracking data and operational data from various operating scenarios such as car driving and industrial equipment operation; Domain datasets include eye-tracking and operational data from senior pilots in different flight scenarios. Senior pilots are those with ≥3000 hours of flight time or who serve as pilot instructors. The target task dataset includes eye-tracking and operational data of pilots awaiting deployment in different flight scenarios.

[0054] Furthermore, the multimodal feature fusion network includes: an eye-tracking feature extraction module, an operation feature extraction module, a cross-modal attention fusion module, and a domain discriminator; The eye-tracking feature extraction module is used to extract the temporal features of eye-tracking data to obtain eye-tracking features; The operation feature extraction module is used to extract the sequence features of the operation data to obtain the operation features; The cross-modal attention fusion module is used to calculate the mutual information entropy of eye-tracking features and operational features, dynamically allocate fusion weights, and fuse eye-tracking features and operational features based on the fusion weights to obtain fused features; A domain discriminator is used to determine the domain origin of the fused features.

[0055] Furthermore, the fine-tuning module includes: a migration unit and a fine-tuning unit; The transfer unit is used to transfer knowledge from the domain base model to the initial target task model based on the target task dataset using a hierarchical parameter transfer strategy. The fine-tuning unit is used to fine-tune the initial target task model using a progressive parameter unfreezing strategy to obtain the target task model.

[0056] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for optimizing a high-efficiency simulation flight platform based on big data, characterized in that, The method includes: S1. Construct a general dataset and a domain dataset, and collect target task datasets. Each dataset contains eye-tracking data and operational data. S2. Construct a multimodal feature fusion network; S3. Based on a general dataset, perform the first stage of pre-training on the multimodal feature fusion network to obtain a general basic model; S4. Perform a second-stage pre-training of the basic model based on the domain dataset to obtain the domain basic model; S5. Fine-tune and train the domain basic model based on the target task dataset to obtain the target task model. Based on the target task model, realize the prediction of pilot operation deviation and the optimization of the simulation flight platform.

2. The method according to claim 1, characterized in that, A general dataset, including eye-tracking data and operational data from various operating scenarios such as car driving and industrial equipment operation; Domain datasets include eye-tracking and operational data from senior pilots in different flight scenarios. Senior pilots are those with ≥3000 hours of flight time or who serve as pilot instructors. The target task dataset includes eye-tracking and operational data of pilots awaiting deployment in different flight scenarios.

3. The method according to claim 1, characterized in that, The multimodal feature fusion network includes: an eye-tracking feature extraction module, an operation feature extraction module, a cross-modal attention fusion module, and a domain discriminator; The eye-tracking feature extraction module is used to extract the temporal features of eye-tracking data to obtain eye-tracking features; The operation feature extraction module is used to extract the sequence features of the operation data to obtain the operation features; The cross-modal attention fusion module is used to calculate the mutual information entropy of eye-tracking features and operational features, dynamically allocate fusion weights, and fuse eye-tracking features and operational features based on the fusion weights to obtain fused features; A domain discriminator is used to determine the domain origin of the fused features.

4. The method according to claim 3, characterized in that, Methods for dynamically allocating fusion weights based on the mutual information entropy of eye-tracking and operational features include: ； ； in, For eye movement feature weights, For the operation feature weights, For mutual information between eye-tracking features and operational features, This refers to the mutual information between operational features and eye-tracking features.

5. The method according to claim 4, characterized in that, Methods for dynamically allocating fusion weights based on the mutual information entropy of eye-tracking and operational features also include: Introducing a temperature coefficient into the weighting formula: ； ； Where T is the temperature coefficient, which is dynamically adjusted according to the differences in modal importance in the target task.

6. The method according to claim 1, characterized in that, Methods for fine-tuning and training a domain-based foundational model based on a target task dataset to obtain a target task model include: Based on the target task dataset, a hierarchical parameter transfer strategy is adopted to transfer the knowledge of the domain base model to the initial target task model; A progressive parameter unfreezing strategy was adopted to fine-tune the initial target task model and obtain the target task model.

7. A high-efficiency simulation flight platform optimization system based on big data, the system being used to implement the method described in any one of claims 1-6, characterized in that, The system includes: a data acquisition module, a network construction module, a first training module, a second training module, and a fine-tuning module; The data acquisition module is used to build general datasets and domain datasets, and to collect target task datasets. Each dataset contains eye-tracking data and operational data. The network construction module is used to build multimodal feature fusion networks; The first training module is used to perform the first stage of pre-training on the multimodal feature fusion network based on a general dataset to obtain a general basic model; The second training module is used to perform a second-stage pre-training of the basic model based on the domain dataset to obtain the domain basic model. The fine-tuning module is used to fine-tune the domain base model based on the target task dataset to obtain the target task model. Based on the target task model, the prediction of pilot operation deviations and the optimization of the simulation flight platform are realized.

8. The system according to claim 7, characterized in that, A general dataset, including eye-tracking data and operational data from various operating scenarios such as car driving and industrial equipment operation; Domain datasets include eye-tracking and operational data from senior pilots in different flight scenarios. Senior pilots are those with ≥3000 hours of flight time or who serve as pilot instructors. The target task dataset includes eye-tracking and operational data of pilots awaiting deployment in different flight scenarios.

9. The system according to claim 7, characterized in that, The multimodal feature fusion network includes: an eye-tracking feature extraction module, an operation feature extraction module, a cross-modal attention fusion module, and a domain discriminator; The eye-tracking feature extraction module is used to extract the temporal features of eye-tracking data to obtain eye-tracking features; The operation feature extraction module is used to extract the sequence features of the operation data to obtain the operation features; The cross-modal attention fusion module is used to calculate the mutual information entropy of eye-tracking features and operational features, dynamically allocate fusion weights, and fuse eye-tracking features and operational features based on the fusion weights to obtain fused features; A domain discriminator is used to determine the domain origin of the fused features.

10. The system according to claim 7, characterized in that, The fine-tuning module includes: a migration unit and a fine-tuning unit; The transfer unit is used to transfer knowledge from the domain base model to the initial target task model based on the target task dataset using a hierarchical parameter transfer strategy. The fine-tuning unit is used to fine-tune the initial target task model using a progressive parameter unfreezing strategy to obtain the target task model.

Images