Multi-task model construction method and device, equipment, medium and product

Abstract

The invention discloses a multi-task model construction method and device, equipment, a medium and a product, and relates to the technical field of model construction. The method comprises the following steps: acquiring historical operation data of a plurality of engines, and acquiring a target model; the target model comprises a backbone network and an output network, and the output network comprises at least one of a prediction output network, a classification output network and a regression output network; preprocessing the historical operation data to obtain a model input mark; the model input mark comprises a sequence mark, a prompt mark and a task mark; based on the sequence mark and the task mark, adjusting model parameters of the backbone network and the output network to obtain a target base model; and based on the prompt mark, adjusting a prompt embedding layer parameter of the target base model to obtain a multi-task model. The universality of the model can be improved.

Description

Technical Field

[0001] This application belongs to the field of model building technology, and in particular relates to a method, apparatus, device, medium and product for building a multi-task model. Background Technology

[0002] As a core component of aircraft, the performance stability and health status of aircraft engines are directly related to flight safety. Therefore, prognostics and health management (PHM) of aircraft engines is of paramount importance.

[0003] Currently, machine learning and deep learning algorithms are widely used in PHM (Prognostics and Health Management) tasks for aero-engines. By learning patterns and regularities in historical operational data, they enable the prediction, classification, and regression analysis of engine failures and remaining service life.

[0004] Then, most current PHM models are limited to a single specific task, such as only for fault prediction or only for fault classification. This means that models serving different PHM tasks need to be built separately, resulting in poor model versatility. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and product for constructing a multi-task model, which can improve the versatility of the model.

[0006] A first aspect of this application provides a method for constructing a multi-task model, the method comprising:

[0007] Acquire historical operating data from multiple engines and obtain a target model; the target model includes a backbone network and an output network, and the output network includes at least one of a prediction output network, a classification output network, and a regression output network;

[0008] Historical operational data is preprocessed to obtain model input tags; model input tags include sequence tags, prompt tags, and task tags;

[0009] Based on sequence labels and task labels, the model parameters of the backbone network and the output network are adjusted to obtain the target base model;

[0010] Based on the cue tags, the cue embedding layer parameters of the target base model are adjusted to obtain a multi-task model.

[0011] Furthermore, this application proposes adjusting the model parameters of the backbone network and the output network based on sequence labels and task labels to obtain the target base model, including:

[0012] The sequence markers are concatenated with the corresponding task markers to form the first training sample;

[0013] The first training sample is masked to obtain the masked training sample.

[0014] The masked training samples are input into the target model to obtain the prediction results; the prediction results include at least one of the prediction reconstruction results and the prediction classification results.

[0015] The masked training samples are compared with the prediction results to construct the target loss function; the target loss function includes at least one of the generation reconstruction loss and the classification reconstruction loss.

[0016] Based on the target loss function, the model parameters of the backbone network and the output network are adjusted to obtain the target base model.

[0017] Furthermore, this application also proposes to mask the first training sample to obtain a masked training sample, including:

[0018] The first training sample is masked to obtain the sample masking result; the masking process includes at least one of random masking and right-side masking.

[0019] Add a masked label to the masked position in the sample masking result to obtain the masked training sample.

[0020] Furthermore, this application also proposes that the backbone network of the target model includes a sequence multi-head attention module, a variable multi-head self-attention module, and a dynamic multilayer perceptron;

[0021] The masked training samples are input into the target model to obtain the prediction reconstruction results, including:

[0022] The masked training samples are input into the sequence multi-head attention module to obtain the first global feature of the masked training samples in the sequence dimension;

[0023] The masked training samples are input into the variable multi-head self-attention module to obtain the second global feature of the masked training samples in the variable dimension;

[0024] The masked training samples are input into a dynamic multilayer perceptron to obtain the sample detail features of the masked training samples;

[0025] The first global feature, the second global feature, and the sample detail features are fused to obtain the comprehensive sample features;

[0026] By integrating the sample features into the output network, the prediction and reconstruction results are obtained.

[0027] Furthermore, this application also proposes adjusting the cue embedding layer parameters of the target base model based on cue tags to obtain a multi-task model, including:

[0028] The cue markers are concatenated with the corresponding sequence markers and task markers to form the second training sample;

[0029] The second training sample is input into the target base model, and the corresponding task loss value is determined by a preset loss function.

[0030] Based on the task loss value, the parameters of the prompt embedding layer of the target base model are adjusted to obtain a multi-task model.

[0031] Furthermore, this application also proposes preprocessing historical operational data to obtain model input labels, including:

[0032] The historical operation data is divided according to a preset time step to construct the sequence label corresponding to the historical operation data;

[0033] Based on the task content corresponding to the historical operation data, prompt tags corresponding to the historical operation data are constructed.

[0034] Based on the task types corresponding to historical operation data, task tags corresponding to historical operation data are constructed.

[0035] A second aspect of this application provides an apparatus for constructing a multi-task model, the apparatus comprising:

[0036] The information acquisition module is used to acquire historical operating data of multiple engines and acquire the target model; the target model includes a backbone network and an output network, and the output network includes at least one of a prediction output network, a classification output network, and a regression output network;

[0037] The data processing module is used to preprocess historical running data to obtain model input tags; model input tags include sequence tags, prompt tags, and task tags.

[0038] The model training module is used to adjust the model parameters of the backbone network and the output network based on sequence labels and task labels to obtain the target base model;

[0039] The model fine-tuning module is used to adjust the cue embedding layer parameters of the target base model based on cue tags to obtain a multi-task model.

[0040] A third aspect of the embodiments of this application provides an electronic device, the device including: a memory and a program or instructions stored in the memory and executable on a processor, wherein when the program or instructions are executed by the processor, they implement a method for constructing a multitasking model as provided in any aspect of the embodiments of this application above.

[0041] A fourth aspect of the embodiments of this application provides a readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, a method for constructing a multitasking model as provided in any aspect of the embodiments of this application above is implemented.

[0042] A fifth aspect of the embodiments of this application provides a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform a method for constructing a multitasking model as provided in any of the embodiments of this application described above.

[0043] The multi-task model construction method provided in this application first preprocesses the historical operating data of multiple engines to obtain model input labels including sequence labels, cue labels, and task labels. Then, using the sequence labels and task labels, the model parameters of the backbone network and output network are pre-trained, allowing the target model to learn a general health state representation, thereby obtaining the target base model. Finally, based on the cue labels, the cue embedding layer parameters of the target base model are lightweightly fine-tuned for specific tasks. Thus, through this "pre-training + fine-tuning" approach, a multi-task model with multi-task learning capabilities can be efficiently developed. This model retains the general knowledge learned during pre-training while accurately adapting to specific needs, performing well in tasks such as fault prediction, fault classification, and remaining life analysis, thereby improving the model's versatility. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating a method for constructing a multi-task model according to an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of the structure of the target model provided in one embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the structure of a multi-task model construction apparatus provided in one embodiment of this application;

[0048] Figure 4 This is a schematic diagram of the structure of a multi-task model construction device provided in one embodiment of this application. Detailed Implementation

[0049] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0050] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0051] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application all comply with the relevant provisions of national laws and regulations.

[0052] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0053] In traditional aero-engine fault prediction and health management systems, the use of a single-task model architecture results in low model reusability and insufficient resource utilization. Since different PHM tasks require independently built models, the system needs to train and maintain independent network structures for fault prediction, fault classification, and remaining lifetime regression, leading to redundant computational resources and parameter redundancy between models. Furthermore, the potential correlations between different tasks cannot be transferred across tasks through joint training, affecting the model's generalization ability and prediction accuracy under complex operating conditions.

[0054] For example, a fault prediction and health management system for a certain type of turbofan engine needs to simultaneously perform three tasks: high-pressure compressor blade crack detection, fuel efficiency anomaly classification, and remaining service life prediction. Existing technical solutions require the construction of three independent models: a fault prediction model based on a long short-term memory network to process time-series vibration signals, a convolutional neural network to classify anomalies in thermal imaging images, and a random forest regressor to analyze performance degradation curves.

[0055] To address the aforementioned challenges, this application first considers how to integrate multi-task processing capabilities through a unified model architecture to eliminate parameter redundancy. Traditional independent model schemes lead to repetitive computation in the feature extraction layer. This application attempts to converge the feature learning process of different PHM tasks to a shared backbone network, retaining task-specific output heads and learning a general health status representation. Simultaneously, it further explores introducing task type identifiers on the input side to guide the network in dynamically adjusting the feature fusion method to adapt to the needs of different PHM tasks. To avoid gradient conflicts during multi-task training, a phased parameter adjustment strategy is designed: first, the backbone network is fixed to optimize the output layer parameters, and then knowledge transfer between tasks is achieved through prompting the embedding layer parameters.

[0056] In this regard, such as Figure 1 As shown, this application provides a flowchart illustrating a method for constructing a multi-task model. This method can be applied to devices with high-performance computing capabilities, large-capacity data storage, and dedicated machine learning software environments, such as high-performance computers, servers, or cloud computing platforms.

[0057] The method for constructing this multi-task model may include the following steps S101 to S104:

[0058] S101, acquire historical operating data of multiple engines and acquire the target model; the target model includes a backbone network and an output network, and the output network includes at least one of a prediction output network, a classification output network, and a regression output network.

[0059] In this step, historical operating data is used to characterize various data generated by the engine during its operation over a past period. This data reflects information such as the engine's operating status and performance. Specifically, historical operating data includes 21 performance degradation parameters, such as temperature, pressure, speed, and fuel flow, as well as three operating condition parameters: flight altitude, flight Mach number, and throttle lever angle.

[0060] The target model refers to the initial model architecture upon which a multi-task model is built. It includes a backbone network and output networks, providing a basic framework for subsequent model training and tuning. Taking a simple neural network as an example, the backbone network can be a multi-layer convolutional neural network or a recurrent neural network, used to extract features from engine operating data. The output networks are configured according to different tasks. For example, the prediction output network can be a fully connected layer for regression prediction, the classification output network can be a fully connected layer with a softmax activation function for classification, and the regression output network structure is similar to the prediction output network, used for different types of regression tasks.

[0061] The backbone network refers to the part of the target model responsible for extracting general features from the input data. These features are representative of multiple tasks. For example, if a multi-layer convolutional neural network is used as the backbone network to process engine image data (such as images of internal engine parts), features such as edges and textures in the image can be extracted through operations such as convolutional layers and pooling layers. These features can be used to determine different subsequent tasks, such as determining whether a part is worn (classification task) or predicting the remaining lifespan of a part (regression task).

[0062] The output network refers to the part of the target model that converts the features extracted by the backbone network into specific task outputs based on different task requirements. For example, a prediction output network: In an engine scenario, if we want to predict the temperature change trend of the engine in the next hour, the prediction output network can process the features extracted by the backbone network and output a temperature sequence over time; a classification output network: When it is necessary to determine whether the engine is in a faulty state, the classification output network receives the features from the backbone network and outputs a probability value, representing the probability that the engine belongs to the faulty or normal category; a regression output network: For example, to predict the remaining service life of the engine (in hours), the regression output network outputs a specific numerical value, representing the estimated value of the remaining service life.

[0063] Collecting historical operating data from multiple engines aims to obtain a sufficient number of samples, enabling the model to learn various operating modes and characteristics of the engines and improve its generalization ability. Determining the target model structure provides a framework for subsequent model training and task processing; different output networks correspond to different task types to meet the needs of multi-task processing.

[0064] Specifically, firstly, historical operating data over time, varying under different operating conditions, is collected from various sensors (such as temperature sensors, pressure sensors, and speed sensors) of multiple engines. This data covers multiple key engine indicators. Simultaneously, the structure of the target model is determined, consisting of a backbone network and an output network. The output network, depending on task requirements, may include at least one of the following: a predictive output network (for predicting future engine failure states), a classification output network (for classifying engine operating states, such as normal or faulty), and a regression output network (for regression analysis, such as predicting the engine's remaining lifespan).

[0065] S102, preprocess the historical operation data to obtain model input tags; the model input tags include sequence tags, prompt tags and task tags.

[0066] In this step, the model input tokens consist of sequence tokens, prompt tokens, and task tokens. These tokens provide the model with comprehensive information, including time-series data, task context, and task type, enabling the model to process different time-series tasks uniformly and predict them accurately. Specifically, sequence tokens can be generated using time window segmentation, prompt tokens can be generated using text templates, and task tokens can be generated using task encoding.

[0067] Sequence labeling is achieved by segmenting the input time series samples along the time dimension using non-overlapping blocks of a specified size, and then projecting each block onto a fixed dimension through a linear layer to obtain the label.

[0068] Cue tags contain specific contextual information related to the data domain and task. They function similarly to prompts in natural language processing, providing the model with contextual information about the task and guiding the model to complete the corresponding task. Each task has its own set of cue tags.

[0069] Task tokens are sets of learnable tags designed to distinguish different types of time series tasks. They are divided into three main types: FORE tokens, CLS tokens, and REG tokens. FORE tokens are suitable for generative tasks, covering scenarios such as prediction, interpolation, and anomaly detection. CLS tokens are mainly used for identification tasks, such as fault mode recognition. REG tokens are for regression tasks, such as remaining lifetime prediction.

[0070] The generation of sequence tags transforms time-series data into a fixed-dimensional format that the model can process while preserving temporal information. Cue tags provide contextual information to the model, similar to prompts in natural language processing, helping the model understand the requirements of the current task. Task tags clarify the specific task type the model needs to perform, enabling it to adopt different processing strategies for different tasks.

[0071] Specifically, the time series samples of the input historical operation data are segmented according to the time dimension using non-overlapping blocks of a specified size to obtain sequence labels; corresponding prompt labels are generated based on the task content corresponding to the historical operation data; and corresponding task labels are generated based on the task type corresponding to the historical operation data.

[0072] S103, based on sequence labels and task labels, adjust the model parameters of the backbone network and the output network to obtain the target base model.

[0073] In this step, model parameter tuning refers to optimizing network weights through gradient backpropagation. Specifically, a masked training strategy can be used to calculate and reconstruct the loss function, driving the backbone and output networks to learn general feature representations. This process enhances the model's ability to capture common patterns across multiple tasks.

[0074] The target base model refers to a model obtained by adjusting the model parameters of the backbone and output networks based on sequence and task labels. It has already learned a general health status representation. It is an intermediate model in the process of building a multi-task model and already has the basic ability to handle different tasks, but it has not yet been fine-tuned for the prompt information.

[0075] The sequence markers contain key features of the engine's historical operating data, while the task markers clearly define the task type. Together, they provide the model with comprehensive information. By adjusting the model parameters through backpropagation, the model learns the mapping relationship between input data and task objectives, enabling accurate prediction, classification, or regression analysis of new input data. The target-based model serves as the foundation for further optimization and adjustments.

[0076] Specifically, sequence labels and task labels are used as inputs and fed into the backbone and output networks of the target model. During training, an appropriate loss function is used to calculate the error between the model output and the true labels. Using backpropagation, the gradient is calculated based on the error, and the model parameters of the backbone and output networks, such as weight matrices and bias vectors, are updated in the opposite direction of the gradient. After multiple iterations of training, the model's performance reaches the expected metrics, resulting in the target base model.

[0077] S104. Based on the cue markers, adjust the cue embedding layer parameters of the target base model to obtain a multi-task model.

[0078] In this step, the cue embedding layer parameters are a set of learnable parameters used in the multi-task model construction process to map cue markers into continuous vector representations, thereby enabling the model to understand the contextual information of different tasks. When constructing a multi-task model based on historical operating data of multiple engines, these cue embedding layer parameters are specifically set and adjusted for cue markers. Through the backpropagation algorithm, they are updated only based on gradient information related to the cue markers, allowing the model to accurately capture task features based on different cue markers. This guides the model to adopt appropriate computational paths and strategies for different tasks in subsequent processing, ultimately achieving effective multi-task processing.

[0079] A multi-task model refers to the final model obtained by adjusting the cue embedding layer parameters of the target base model based on cue tags. It can simultaneously handle different tasks such as engine remaining life regression analysis, fault classification, and fault prediction.

[0080] The adjustment of the cue embedding layer parameters aims to enable the model to better utilize the task context information provided by the cue tags. By fixing other parameters and adjusting only the cue embedding layer parameters, the relationship between the cue tags and the model's task processing can be optimized more precisely. This allows the model to quickly adapt to different task requirements based on different cue tags, achieving efficient multi-task processing.

[0081] Specifically, the cue tags are input into the cue embedding layer of the target base model, which converts the cue tags into continuous embedding vector representations. During model training, all parameters of the target base model except for the cue embedding layer parameters are fixed. The loss function is calculated only based on the cue tags and model output, and the gradient of the cue embedding layer parameters is calculated using the backpropagation algorithm. Then, the cue embedding layer parameters are updated along the inverse direction of the gradient. After multiple iterations of training, the cue embedding layer can better convert the cue tags into learnable embedding vectors suitable for the model to process, thereby improving the model's understanding and processing capabilities for different tasks, ultimately resulting in a multi-task model.

[0082] This application utilizes a multi-layered embedding mechanism of sequence labels, task labels, and cue labels, combined with a two-stage training strategy, to achieve joint training and rapid adaptation of aero-engine multi-task models within a unified framework. Specifically, training and optimizing the backbone network extracts common features across tasks, while fine-tuning the cue labels achieves low-cost adaptation to different PHM tasks through learnable embedding vectors, effectively addressing the problem that a single model cannot adequately handle multi-task performance.

[0083] As an example, firstly, historical operating data from multiple turbofan engines is acquired, including vibration signals, temperature data, and pressure data. Simultaneously, a pre-designed target model is obtained, which includes a Transformer-structured backbone network and three output networks for fault prediction, anomaly classification, and life regression.

[0084] Secondly, historical operational data is preprocessed. Time-series data is divided into sequence tags according to fixed time windows. Hint tags are constructed based on the task content corresponding to the data, such as "predicting high-pressure compressor blade cracks". Task tags are constructed based on task type, such as "prediction task".

[0085] Furthermore, the target model is trained using sequence labels and task labels. The sequence labels and task labels are concatenated and input into the backbone network, and pre-training is performed through masking reconstruction to optimize the backbone network parameters and output network parameters, thus obtaining the target base model.

[0086] Finally, the target base model is fine-tuned based on the cue tags. The cue tags, sequence tags, and task tags are concatenated and input into the target base model. The parameters of the cue embedding layer are optimized using a loss function specific to the task, enabling the model to adapt to different tasks and obtaining the final multi-task model.

[0087] In this embodiment, historical operating data from multiple engines are first preprocessed to obtain model input tags, including sequence tags, cue tags, and task tags. Then, using the sequence tags and task tags, the model parameters of the backbone and output networks are pre-trained, allowing the target model to learn general health state representations, thus obtaining the target base model. Next, based on the cue tags, the cue embedding layer parameters of the target base model are lightweightly fine-tuned for specific tasks. Thus, through this "pre-training + fine-tuning" approach, a multi-task model with multi-task learning capabilities can be efficiently developed. This model retains the general knowledge learned during pre-training while accurately adapting to specific needs, performing well in tasks such as fault prediction, fault classification, and remaining life analysis, thereby improving the model's versatility.

[0088] In some of the schemes described above in this application, when adjusting the parameters of the backbone network and the output network, training is based solely on sequence labels and task labels, which may cause the model to fail to effectively capture complex patterns and cross-task commonalities in the data, thus affecting the model's generalization ability in diverse PHM tasks.

[0089] In this regard, this application further proposes that S103 may specifically include:

[0090] The sequence markers are concatenated with the corresponding task markers to form the first training sample;

[0091] The first training sample is masked to obtain the masked training sample.

[0092] The masked training samples are input into the target model to obtain the prediction results; the prediction results include at least one of the prediction reconstruction results and the prediction classification results.

[0093] The masked training samples are compared with the prediction results to construct the target loss function; the target loss function includes at least one of the generation reconstruction loss and the classification reconstruction loss.

[0094] Based on the target loss function, the model parameters of the backbone network and the output network are adjusted to obtain the target base model.

[0095] In this embodiment, the masking process employs random masking or right-side masking, randomly selecting masking positions based on probability distributions or masking the end portion of the sequence in a fixed direction; special labels are added to the masked positions to mark missing information. The prediction results include at least one of a prediction reconstruction result and a prediction classification result. The prediction reconstruction result characterizes the generation result of the time-series data of the masked portion in the masked training samples, while the prediction classification result characterizes the prediction result for the task type of the masked training samples. The objective loss function includes at least one of a generation reconstruction loss and a classification reconstruction loss, comprehensively considering the model's ability to reconstruct the masked portion of the time series under different features and processing methods.

[0096] Specifically, sequence labels and task labels are concatenated to form the first training sample, ensuring that the model simultaneously receives task type and data sequence information. After random or right-side masking of the samples, the target model needs to reconstruct the masked portion based on the remaining information and predict the corresponding task type. This forces the backbone network to learn the correlation between variables and the dynamic changes in the sequence, and forces the output network to learn the corresponding task type. By minimizing the generation reconstruction loss and the classification reconstruction loss, the model's ability to reconstruct the time series of the masked portion under different features and processing methods is comprehensively considered, improving the efficiency of extracting common features of multiple tasks. For example, in the remaining lifetime regression task, the target model learns the continuity features of the engine performance degradation curve through masking training, while capturing the distinguishing boundary between abnormal patterns and normal states in the fault classification task, enhancing the adaptability of the base model to diverse PHM tasks.

[0097] For example, the target loss function is shown in Equation 1 below:

[0098]

[0099] In formula 1, Used to characterize the target loss function, MSE(H) fore (e′ f),x) is used to characterize the generative reconstruction loss, MSE(H fore (H cls (e′ c ),x) are used to characterize the classification reconstruction loss.

[0100] Among them, H fore (e′ f The symbol ) represents the prediction reconstruction result obtained after inputting the masked training samples into the target model, x represents the first training sample, MSE represents the mean squared error, and H represents the mean squared error. cls (e′ c This is used to characterize the target model's predicted classification results for task types when classifying masked training samples.

[0101] As an example, the sequence labels are first concatenated with the corresponding task labels to form the first training sample. For instance, for historical operating data of an aero-engine, the sequence labels can be time series of engine parameters, such as temperature, pressure, and speed, while the task labels can be fault prediction, fault classification, or remaining life prediction.

[0102] The first training sample is then masked to obtain the masked training sample. Specifically, a random masking method can be used, randomly selecting a certain proportion of the input tags for masking. Alternatively, a right-side masking method can be used, continuously masking a certain number of tags starting from the right side of the sequence.

[0103] The masked training samples are then input into the target model to obtain prediction results, which include at least one of the predicted reconstruction results and predicted classification results. The target model includes a backbone network and an output network. The backbone network can be a Transformer-based encoder structure, and the output network can include output layers related to multiple tasks such as prediction, classification, and regression.

[0104] The masked training samples are then compared with the prediction results to construct the target loss function. The target loss function can include generation reconstruction loss and classification reconstruction loss.

[0105] Finally, based on the target loss function, the model parameters of the backbone network and the output network are adjusted to obtain the target base model. Backpropagation and gradient descent can be used to optimize the model parameters and minimize the target loss function.

[0106] This embodiment demonstrates the effective training of a multi-task model. Through masking and task reconstruction, the model learns the inherent structure and patterns of the input sequence. Simultaneously, joint multi-task training improves the model's generalization ability and adaptability to different tasks. This method facilitates the construction of a general aero-engine PHM (Prognostics and Manufacturing Management) platform model, laying the foundation for subsequent task-specific fine-tuning.

[0107] In some of the schemes mentioned above in this application, the single masking method may lead to insufficient ability of the model to capture features at different positions in the sequence, affecting the accuracy of the reconstruction results.

[0108] In response, this application further proposes to mask the first training sample to obtain a masked training sample, including:

[0109] The first training sample is masked to obtain the sample masking result; the masking process includes at least one of random masking and right-side masking.

[0110] Add a masked label to the masked position in the sample masking result to obtain the masked training sample.

[0111] In this embodiment, random masking covers data by randomly selecting portions of the sequence, while right-side masking continuously covers a preset length of data starting from the end of the sequence. Masked labels are embedded in the masking locations using special identifiers to guide the model to focus on the masked areas during training. The two masking methods can be applied individually or in combination. The proportion of random masking can be set to 15%-35% of the total sequence length, and the length of right-side masking can be set to 10%-25% of the total sequence length.

[0112] Specifically, when constructing masked training samples, if random masking is used, each data point at each time step has a 20% probability of being replaced with a masked label; if right-side masking is used, the last 30% of the data segments in the sequence are uniformly replaced. The masked labels use a specific encoding vector independent of the original data distribution, which participates in the attention weight calculation during training. By alternately applying the two masking strategies, the model can learn the contextual relationships of features at different locations. The masked labels act as location indicators, enabling the model to accurately identify the regions that need to be reconstructed, thereby improving the completeness of feature reconstruction.

[0113] As an example, the first training sample is first masked to obtain the masking results. The masking process includes at least one of random masking and right-side masking. Random masking involves randomly selecting a certain percentage of positions in the first training sample for masking, for example, selecting 15% of the positions for masking. Right-side masking starts from the right side of the first training sample and continuously masks a certain percentage of positions, for example, masking the rightmost 30% of the positions.

[0114] Next, masked labels are added to the masked positions in the sample masking results to obtain masked training samples. The masked labels can be specific markers, such as "[MASK]". In this way, the masked positions in the masked training samples will be replaced with the "[MASK]" marker.

[0115] This embodiment enables the generation of diverse masked training samples, enhancing the model's generalization ability. Random masking allows the model to learn contextual information at different locations, while right-side masking simulates a sequence prediction task, enabling the model to predict future states based on known information. Adding masked labels further clarifies the locations to be predicted, helping the model better learn the reconstruction task. This combination of masking strategies increases the diversity of training data, thereby enhancing the model's adaptability to different scenarios.

[0116] In some of the schemes described above in this application, when training the model by masking training samples, traditional methods only use a single-dimensional attention mechanism, which cannot simultaneously capture the temporal dependence between sequences and the interaction between variables, resulting in insufficient accuracy of the reconstructed prediction results in complex multidimensional data.

[0117] In response, this application further proposes that the backbone network of the target model includes a sequence multi-head attention module, a variable multi-head self-attention module, and a dynamic multilayer perceptron;

[0118] The masked training samples are input into the target model to obtain the prediction reconstruction results, including:

[0119] The masked training samples are input into the sequence multi-head attention module to obtain the first global feature of the masked training samples in the sequence dimension;

[0120] The masked training samples are input into the variable multi-head self-attention module to obtain the second global feature of the masked training samples in the variable dimension;

[0121] The masked training samples are input into a dynamic multilayer perceptron to obtain the sample detail features of the masked training samples;

[0122] The first global feature, the second global feature, and the sample detail features are fused to obtain the comprehensive sample features;

[0123] By integrating the sample features into the output network, the prediction and reconstruction results are obtained.

[0124] In this embodiment, the Sequence Multi-Head Attention Module (Sequence MHSA) employs multiple attention heads to capture long-term dependency patterns in the time dimension by computing the sequence position correlations in different subspaces in parallel, thus enabling information interaction between different time steps. The Variable Multi-Head Self-Attention Module (Variable MHSA) employs multiple attention heads to model the nonlinear relationships between different sensor variables within the same time step, capturing the relationships between different variables, and is suitable for multivariate time series analysis. The Dynamic Multilayer Perceptron (Dynamic MLP) contains three hidden layers, combining local detail and global relationship extraction. It uses a 3-kernel convolutional layer to capture local details and fuses the results of dynamic linear operations and ordinary linear layers through grouped feature processing to obtain richer feature representations.

[0125] Specifically, when masked training samples enter the sequence multi-head attention module, each attention head calculates the correlation weights between different time steps, such as identifying periodic fluctuation patterns in engine vibration signal sequences. Subsequently, the variable multi-head self-attention module performs cross-dimensional interactive calculations on multivariate data such as temperature, pressure, and speed, for example, discovering the correlation between sudden changes in speed and decreases in oil pressure. The dynamic multilayer perceptron uses a 3-kernel convolutional layer to capture local details and fuses the results of dynamic linear operations and ordinary linear layers through grouped feature processing to obtain richer feature representations. By fusing the first global features of the variable multi-head self-attention module, the second global features of the variable multi-head self-attention module, and the sample detail features of the dynamic multilayer perceptron, a comprehensive feature vector containing spatiotemporal correlation information is finally generated. After reconstruction by the output network, this vector can accurately recover the masked sensor data values.

[0126] The output of sequence attention in the sequence multi-head attention module can be determined by the following formula 2:

[0127]

[0128] In formula 2, Attention Seqiemce The output is used to characterize the sequence attention; Softmax is used to convert the attention score into a probability distribution; Q, K, and V are used to characterize the query, key, and value matrices, respectively, which are generated from the masked training samples of the output through linear transformation; d is used to characterize the scaling factor, usually the dimension of the key, to prevent the dot product result from being too large; T is used to characterize the transpose processing.

[0129] The first global feature obtained by the sequence multi-head attention module can be determined by the following formula 3:

[0130] SequenceMHSA(e in ) = Linear out(ConcatHeads(Attention Sequence )) Formula 3

[0131] In formula 3, SequenceMHSA(e in ) is used to characterize the output of the sequence multi-head attention module, i.e., the first global feature; Linear out Used to characterize linear transformations of the output; used to adjust the output dimension; ConcatHeads is used to concatenate the outputs of multiple heads together; Attention Sequence The output used to characterize sequence attention.

[0132] The second global feature finally obtained by the multi-head self-attention module can be determined by the following formula 4:

[0133]

[0134] In formula 4, VariableMHSA(e in This is used to characterize the output of the multi-head self-attention module for variables, i.e., the second global feature; These are used to represent the query and key matrices after averaging along the sequence dimension, respectively; d is used to represent the scaling factor, usually the dimension of the key, to prevent the dot product result from being too large; T is used to represent the transpose processing; V is used to represent the value matrix; and Softmax is used to convert the attention score into a probability distribution.

[0135] The Dynamic Multilayer Perceptron further comprises a Dynamic Feedforward Network (FFN), a Dynamic Linear Operator (Dy Linear), and a Gate Module. The Dynamic FFN is a dynamic feedforward network that combines local details and global relationships, dynamically adjusting its behavior based on the characteristics of the input data to extract detailed features from the masked training samples. The Dy Linear is an operator that dynamically adjusts the linear transformation weights to model the dense relationships between different time points in the masked training samples, adapting to changes in the length of the input masked training samples through methods such as bilinear interpolation. The Gate Module is a module used to adjust the feature scale, reducing interference between different tasks by generating scaling factors to ensure feature values ​​remain within a reasonable range.

[0136] The dynamic feedforward network first extracts local details by performing a one-dimensional convolution operation on the masked training samples to obtain local detail features; then, the local detail features are divided into two groups to obtain the first local detail features and the second local detail features; finally, the output of the dynamic feedforward network is obtained through the following formula 5:

[0137]

[0138] In Formula 5, Dynamic FFN(e in ) is used to characterize the output of a dynamic feedforward network. Used to characterize the first local detail features Used to represent the second local detail features; Dylinear is used to represent dynamic linear operations, used to transform the first local detail features; Concat is used to represent the concatenation operation, concatenating two sets of features together; Linear... out Used to characterize the linear transformation of the output.

[0139] The dynamic linear operator is used to model the dense relationships between different time points in a sequence. It adjusts the shape of the weight matrix through bilinear interpolation and then applies it to the input masked training samples. It can be processed by the following formula 6:

[0140] Dy Linear out =W interp Formula 6

[0141] In formula 6, Dy Linear out W is used to characterize the output of a dynamic linear operator. interp The weight matrix is ​​used to characterize the weight matrix, and X is used to characterize the features of the input dynamic linear operator.

[0142] The weight matrix can be determined using the following formula 7:

[0143] W interp =Interp bilinear (ω) Formula 7

[0144] In formula 7, W interp Used to represent the weight matrix; ω is used to represent the weight vector and to adjust the shape of the linear transformation; Interp bilinear Used to characterize linear interpolation functions, and used to adjust the shape of the weight matrix according to ω.

[0145] The gating module is used to adjust the feature scale and reduce interference between different tasks. It can be processed using the following formula 8:

[0146] Gate(e in =Sigmoid(Linear) gate (e in ))⊙e in Formula 8

[0147] In formula 8, Gate(e in ) is used to characterize the output of the gating module, e in Linear training samples used to characterize the input gateThe linear transformation used to characterize the scaling factor is ⊙, which is used to characterize the element-wise multiplication operation, and the sigmoid is used to convert the attention score into a probability distribution.

[0148] The output network includes three types: prediction output network, classification output network, and regression output network.

[0149] Specifically, the predicted output network can be processed using the following formula 9:

[0150] y fore =Proj(MLP(e′) f +Dy Linear(e′ f ))) Formula 9

[0151] In formula 9, y fore Used to characterize the output of the predictive output network, e f The FORE token is used to represent the features extracted from the comprehensive sample. Dy Linear is used to represent the dynamic linear transformation. MLP is used to represent the multilayer perceptron. Proj is used to represent the projection layer and is used to map the output of MLP to the target space.

[0152] The classification output network can be processed using the following formula 10:

[0153]

[0154] In formula 10, y cls e is used to characterize the output of the classification output network. c_final Used to characterize the features of a comprehensive sample The embedding information is used to characterize the category, and argmin is used to characterize the function that finds the index of the minimum value.

[0155] The regression output network can be processed using the following formula 11:

[0156] y reg =MLP(GRU(e′) r )) Formula 11

[0157] In formula 11, e′ r REG tokens are used to characterize features extracted from comprehensive samples; GUR tokens are used to characterize gated recurrent units, which process sequential data and capture temporal dependencies; and MLP tokens are used to characterize multilayer perceptrons.

[0158] like Figure 2The diagram illustrates the structure of a target model. This target model is centered around a sequence multi-head attention module 210, which captures long-term dependencies and complex patterns in the data. Through the combination of a variable multi-head self-attention module 220 and a dynamic multilayer perceptron 230, the target model further enhances its feature extraction and representation learning capabilities. The design of the gating module 240 contributes to the selective transmission of information and the stability of the model.

[0159] At the output of the target model, three different output modules are set up to meet diverse task requirements: the prediction output module 250 is responsible for predicting the future of the time series, the classification output module 260 focuses on the identification of failure modes, and the regression output module 270 is used for predicting remaining lifetime. These three output modules work in parallel, sharing the underlying feature representation, but each is optimized for a specific task, thereby achieving efficient multi-task learning.

[0160] As an example, the masked training samples are first fed into a sequence multi-head attention module. This sequence multi-head attention module captures the long-term dependencies of the samples in the sequence dimension through a multi-head self-attention mechanism, thereby obtaining the first global features of the masked training samples in the sequence dimension.

[0161] Secondly, the masked training samples are input into the variable multi-head self-attention module. This module uses a multi-head self-attention mechanism to model the samples in the variable dimension, obtaining the second global feature of the masked training samples in the variable dimension.

[0162] Next, the masked training samples are input into a dynamic multilayer perceptron, which uses a 3-kernel convolutional layer to capture local details and obtain sample detail features.

[0163] Then, the first global feature, the second global feature, and the sample detail features are fused together, and the information of the sequence dimension and the variable dimension is comprehensively processed to generate a comprehensive sample feature that masks the training samples.

[0164] Finally, the combined sample features are input into the output network to obtain the prediction and reconstruction results. The output network can be one or more combinations of prediction, classification, or regression output networks, selected according to the specific task requirements.

[0165] This embodiment achieves comprehensive feature extraction of masked training samples in both sequence and variable dimensions. The sequence multi-head attention module and the variable multi-head self-attention module capture key information from the samples across different dimensions, while the dynamic multilayer perceptron further extracts richer detailed features, thus generating comprehensive sample features. This multi-dimensional, multi-level feature extraction and fusion mechanism improves the model's understanding of complex data patterns, thereby enhancing the accuracy and robustness of the prediction and reconstruction results. This design enables the model to better adapt to different types of aero-engine PHM tasks, improving its versatility and practical value.

[0166] In some of the solutions described above in this application, the correlation between cue tags and task tags was not effectively utilized when adjusting the parameters of the backbone network and the output network, resulting in insufficient adaptability of the model in specific task scenarios.

[0167] In this regard, this application further proposes that S104 may specifically include:

[0168] The cue markers are concatenated with the corresponding sequence markers and task markers to form the second training sample;

[0169] The second training sample is input into the target base model, and the corresponding task loss value is determined by a preset loss function.

[0170] Based on the task loss value, the parameters of the prompt embedding layer of the target base model are adjusted to obtain a multi-task model.

[0171] In this embodiment, the second training sample is constructed through a concatenation operation, where the cue tag is concatenated before the sequence tag, and the task tag is appended to the end of the sample in the form of a category code. The preset loss function is either the cross-entropy loss function or the mean squared error loss function, dynamically selected according to the output network type. The cue embedding layer parameters are located in the input layer of the backbone network, and their dimension is consistent with the embedding space of the sequence tag; gradient updates are performed using the backpropagation algorithm.

[0172] Specifically, the concatenated second training samples undergo a sequence multi-head attention module to extract temporal features, a variable multi-head self-attention module to capture cross-variable correlations, and a dynamic multilayer perceptron to extract richer sample detail features. Finally, these three are fused to generate comprehensive sample features. The output network calculates the difference between the predicted result and the true label based on the task type. The loss function is cross-entropy for classification tasks and mean squared error for prediction and regression tasks. During backpropagation, the target base model only updates its cue embedding layer parameters, while the rest of the backbone network remains frozen. This allows the target base model to retain general features while rapidly adapting to different PHM tasks through fine-tuning of the cue embedding layer parameters. For example, in fault classification tasks, adjusting the cue embedding layer parameters guides the model to focus on high-frequency components in vibration signals; in remaining lifetime prediction tasks, the cue embedding layer parameters guide the model to focus on long-term trend changes in temperature sensors.

[0173] As an example, the cue markers are concatenated with their corresponding sequence markers and task markers to form a second training sample. For instance, for an aero-engine fault prediction task, the cue marker could be "predict whether a fault will occur within the next 5 hours," the sequence marker could be the engine's sensor data sequence from the past 24 hours, and the task marker could be "fault prediction." These markers are concatenated into a complete input sequence.

[0174] The second training sample is input into the target base model. The target base model consists of a backbone network and an output network. The backbone network can be an encoder based on the Transformer architecture, and the output network can be a prediction head for different tasks.

[0175] Then, the corresponding task loss value is determined by a preset loss function. For prediction tasks, the mean squared error can be used to calculate the difference between the predicted result and the true label; for classification tasks, the cross-entropy loss function can be used to calculate the difference between the predicted category and the true category; for regression tasks, the mean squared error can be used to calculate the difference between the predicted result and the true label.

[0176] Based on the task loss value, the cue embedding layer parameters of the target base model are adjusted to obtain the multi-task model. Specifically, gradient descent can be used to update the cue embedding layer parameters corresponding to the cue markers in the target base model to minimize the task loss.

[0177] This embodiment demonstrates a flexible multi-task model construction method. By introducing cue tags and fine-tuning the cue embedding layer parameters, a single model can adapt to various different PHM task requirements, improving the model's versatility and adaptability. Simultaneously, this method fully leverages knowledge transfer between different tasks, enhancing the model's performance across various tasks.

[0178] In some of the solutions described above in this application, the process of preprocessing historical running data to generate model input labels lacks a clear structured method, which makes it impossible for the input labels to effectively distinguish sequence features, task content and task type, thus affecting the training effect of multi-task models.

[0179] In this regard, this application further proposes that S102 may include:

[0180] The historical operation data is divided according to a preset time step to construct the sequence label corresponding to the historical operation data;

[0181] Based on the task content corresponding to the historical operation data, prompt tags corresponding to the historical operation data are constructed.

[0182] Based on the task types corresponding to historical operation data, task tags corresponding to historical operation data are constructed.

[0183] In this embodiment, the preset time step is set according to the engine operating cycle or data acquisition frequency, and the segmented sequence labels retain the continuity characteristics in the time dimension. Task content is generated as a prompt label through natural language description or encoding; for example, the prompt label for a fault prediction task is "predict future fault states." Task types are generated as task labels through classification tags or vector encoding; for example, a classification task is labeled "classification," and a regression task is labeled "regression."

[0184] Specifically, the pre-defined time step segmentation ensures that each sequence label covers data within a fixed time window, enabling the model to learn periodic or trend features. Task-related cue labels are concatenated with the sequence labels using text or vector embedding, guiding the model to focus on data patterns relevant to specific tasks. Task labels, combined with classification encoding, define the activation method of the model's output network. For example, when the task label is classification, the classification output network is activated; when the task label is regression, the regression output network is activated. Through structured segmentation and label generation, the model can handle different task types simultaneously, avoiding the problem of repeatedly building single-task models.

[0185] As an example, the historical operating data is first divided according to a preset time step, and the corresponding sequence tags are constructed. For example, the operating data of an engine for one month can be divided into one time step per hour, resulting in sequence tags for 720 time steps.

[0186] Secondly, based on the task content corresponding to the historical operating data, prompt tags corresponding to the historical operating data are constructed. For example, for a fault prediction task, "predict whether a fault will occur at the next moment" can be constructed as a prompt tag; for a remaining life prediction task, "predict the remaining life of the engine" can be constructed as a prompt tag.

[0187] Finally, based on the task types corresponding to the historical operational data, task labels corresponding to the historical operational data are constructed. For example, the fault prediction task can be labeled as a "classification task", and the remaining life prediction task can be labeled as a "regression task".

[0188] This embodiment demonstrates effective preprocessing of historical operational data, transforming the raw data into a standardized format suitable for multi-task model input. This preprocessing method preserves the temporal characteristics, task semantics, and task type information of the data, providing structured input for subsequent model training and improving the model's learning efficiency and generalization ability. Simultaneously, this standardized data processing method enhances the model's adaptability to different task types, laying the foundation for building a general-purpose multi-task model.

[0189] Based on the method for constructing a multi-task model provided in this application, correspondingly, this application also provides specific embodiments of a device for constructing a multi-task model.

[0190] like Figure 3 As shown, the multi-task model construction device 300 provided in this application embodiment includes an information acquisition module 310, a data processing module 320, a model training module 330, and a model fine-tuning module 340.

[0191] The information acquisition module 310 is used to acquire historical operating data of multiple engines and acquire a target model; the target model includes a backbone network and an output network, and the output network includes at least one of a prediction output network, a classification output network, and a regression output network;

[0192] The data processing module 320 is used to preprocess historical running data to obtain model input tags; the model input tags include sequence tags, prompt tags, and task tags;

[0193] The model training module 330 is used to adjust the model parameters of the backbone network and the output network based on sequence labels and task labels to obtain the target base model.

[0194] The model fine-tuning module 340 is used to adjust the cue embedding layer parameters of the target base model based on cue tags to obtain a multi-task model.

[0195] In the multi-task model construction apparatus provided in this application embodiment, historical operating data of multiple engines are first preprocessed to obtain model input tags, including sequence tags, cue tags, and task tags. Then, using the sequence tags and task tags, the model parameters of the backbone network and output network are pre-trained, allowing the target model to learn general health state representations, thereby obtaining the target base model. Finally, based on the cue tags, the cue embedding layer parameters of the target base model are lightweightly fine-tuned for specific tasks. Thus, through this "pre-training + fine-tuning" approach, a multi-task model with multi-task learning capabilities can be efficiently developed. This model retains the general knowledge learned during pre-training while accurately adapting to specific needs, performing well in tasks such as fault prediction, fault classification, and remaining life analysis, thereby improving the model's versatility.

[0196] Furthermore, this application also proposes a model training module 330, comprising:

[0197] The sample construction unit is used to concatenate the sequence label with the corresponding task label to form the first training sample;

[0198] The sample masking unit is used to mask the first training sample to obtain the masked training sample;

[0199] The sample reconstruction unit is used to input masked training samples into the target model to obtain the prediction reconstruction result;

[0200] The loss construction unit is used to compare the masked training samples with the predicted reconstruction results to construct the target loss function; the target loss function includes at least one of the generative reconstruction loss and the classification reconstruction loss.

[0201] The parameter adjustment unit is used to adjust the model parameters of the backbone network and the output network based on the target loss function to obtain the target base model.

[0202] Furthermore, this application also proposes a sample masking unit for:

[0203] The first training sample is masked to obtain the sample masking result; the masking process includes at least one of random masking and right-side masking.

[0204] Add a masked label to the masked position in the sample masking result to obtain the masked training sample.

[0205] Furthermore, this application also proposes that the backbone network of the target model includes a sequence multi-head attention module, a variable multi-head self-attention module, and a dynamic multilayer perceptron;

[0206] Sample reconstruction unit, used for:

[0207] The masked training samples are input into the sequence multi-head attention module to obtain the first global feature of the masked training samples in the sequence dimension;

[0208] The masked training samples are input into the variable multi-head self-attention module to obtain the second global feature of the masked training samples in the variable dimension;

[0209] The masked training samples are input into a dynamic multilayer perceptron to obtain the sample detail features of the masked training samples;

[0210] The first global feature, the second global feature, and the sample detail features are fused to obtain the comprehensive sample features;

[0211] By integrating the sample features into the output network, the prediction and reconstruction results are obtained.

[0212] Furthermore, this application also proposes a model fine-tuning module 340 for:

[0213] The cue markers are concatenated with the corresponding sequence markers and task markers to form the second training sample;

[0214] The second training sample is input into the target base model, and the corresponding task loss value is determined by a preset loss function.

[0215] Based on the task loss value, the parameters of the prompt embedding layer of the target base model are adjusted to obtain a multi-task model.

[0216] Furthermore, this application also proposes a data processing module 320 for:

[0217] The historical operation data is divided according to a preset time step to construct the sequence label corresponding to the historical operation data;

[0218] Based on the task content corresponding to the historical operation data, prompt tags corresponding to the historical operation data are constructed.

[0219] Based on the task types corresponding to historical operation data, task tags corresponding to historical operation data are constructed.

[0220] Based on the method for constructing a multi-task model provided in this application, correspondingly, this application also provides specific embodiments of a device for constructing a multi-task model.

[0221] Figure 4 A schematic diagram of the hardware structure of the multi-task model construction device provided in an embodiment of this application is shown.

[0222] The device for building a multitasking model may include a processor 401 and a memory 402 storing computer program instructions.

[0223] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0224] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal to an integrated gateway disaster recovery device. In a particular embodiment, memory 402 is non-volatile solid-state memory.

[0225] The processor 401 reads and executes computer program instructions stored in the memory 402 to implement any of the multi-task model construction methods in the above embodiments.

[0226] In one example, the device for building a multi-task model may further include a communication interface 403 and a bus 410. Wherein, as Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0227] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0228] Bus 410 includes hardware, software, or both, that couples components of a multitasking model building device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 410 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0229] Furthermore, in conjunction with the multi-task model construction method in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the multi-task model construction methods in the above embodiments.

[0230] In addition, in conjunction with the multi-task model construction method in the above embodiments, this application embodiment can provide a computer program product for implementation. When the instructions in the computer program product are executed by the processor of an electronic device, the electronic device executes the multi-task model construction method provided by any aspect of the above embodiments of this application.

[0231] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0232] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0233] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0234] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0235] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for constructing a multi-task model, characterized in that, The method includes: Historical operating data of multiple engines are acquired, and a target model is obtained; the target model includes a backbone network and an output network, and the output network includes at least one of a prediction output network, a classification output network, and a regression output network; The historical operation data is preprocessed to obtain model input tags; the model input tags include sequence tags, prompt tags, and task tags. Based on the sequence labels and the task labels, the model parameters of the backbone network and the output network are adjusted to obtain the target base model; Based on the aforementioned prompt markers, the prompt embedding layer parameters of the target base model are adjusted to obtain a multi-task model.

2. The method according to claim 1, characterized in that, The step of adjusting the model parameters of the backbone network and the output network based on the sequence labels and the task labels to obtain the target base model includes: The sequence markers are concatenated with the corresponding task markers to form the first training sample; The first training sample is masked to obtain a masked training sample. The masked training samples are input into the target model to obtain prediction results; the prediction results include at least one of prediction reconstruction results and prediction classification results; The masked training samples are compared with the prediction results to construct a target loss function; the target loss function includes at least one of generation reconstruction loss and classification reconstruction loss. Based on the target loss function, the model parameters of the backbone network and the output network are adjusted to obtain the target base model.

3. The method according to claim 2, characterized in that, The process of masking the first training sample to obtain a masked training sample includes: The first training sample is masked to obtain the sample masking result; the masking process includes at least one of random masking and right-side masking; Add a masked label to the masked position in the sample masking result to obtain the masking training sample.

4. The method according to claim 2, characterized in that, The backbone network of the target model includes a sequence multi-head attention module, a variable multi-head self-attention module, and a dynamic multilayer perceptron. The step of inputting the masked training samples into the target model to obtain the prediction reconstruction result includes: The masked training sample is input into the sequence multi-head attention module to obtain the first global feature of the masked training sample in the sequence dimension; The masked training sample is input into the variable multi-head self-attention module to obtain the second global feature of the masked training sample in the variable dimension; The masked training samples are input into the dynamic multilayer perceptron to obtain the sample detail features of the masked training samples; The first global feature, the second global feature, and the sample detail feature are fused to obtain a comprehensive sample feature; The comprehensive sample features are input into the output network to obtain the prediction reconstruction result.

5. The method according to any one of claims 1-4, characterized in that, The step of adjusting the prompt embedding layer parameters of the target base model based on the prompt marker to obtain a multi-task model includes: The prompt marker is concatenated with the corresponding sequence marker and the task marker to form a second training sample; The second training sample is input into the target base model, and the corresponding task loss value is determined by a preset loss function. Based on the task loss value, the prompt embedding layer parameters of the target base model are adjusted to obtain the multi-task model.

6. The method according to any one of claims 1-4, characterized in that, The preprocessing of the historical operational data to obtain model input tags includes: The historical operation data is divided according to a preset time step to construct the sequence markers corresponding to the historical operation data; Based on the task content corresponding to the historical operation data, a prompt marker corresponding to the historical operation data is constructed; Based on the task type corresponding to the historical operation data, a task tag corresponding to the historical operation data is constructed.

7. A device for constructing a multi-task model, characterized in that, The device includes: The information acquisition module is used to acquire historical operating data of multiple engines and acquire a target model; the target model includes a backbone network and an output network, and the output network includes at least one of a prediction output network, a classification output network, and a regression output network. The data processing module is used to preprocess the historical running data to obtain model input tags; the model input tags include sequence tags, prompt tags, and task tags; The model training module is used to adjust the model parameters of the backbone network and the output network based on the sequence labels and the task labels to obtain the target base model; The model fine-tuning module is used to adjust the prompt embedding layer parameters of the target base model based on the prompt markers to obtain a multi-task model.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method for constructing a multi-task model according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for constructing a multi-task model according to any one of claims 1 to 6.

10. A computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the method for constructing a multitasking model according to any one of claims 1 to 6.

Images