Time sequence large model hierarchical fine-tuning method and device for thermal power generating unit based on gradient variance analysis

Abstract

The application discloses a thermal power unit time sequence large model layered fine-tuning method and device based on gradient variance analysis, belongs to the technical field of intelligent monitoring of thermal power unit equipment and time sequence model optimization, and realizes lightweight migration adaptation of the time sequence large model for the thermal power industry scene through an adaptive dynamic layer selection training and a multi-loss joint constraint optimization idea, controls training computing power overhead, reduces engineering landing threshold, effectively solves the technical problem that early degradation of equipment under similar working conditions and hidden fault characteristics are difficult to identify, and balances model training efficiency, application cost and fault diagnosis precision.

Description

Technical Field

[0001] This invention belongs to the field of intelligent monitoring and time series model optimization technology for thermal power unit equipment, specifically a hierarchical fine-tuning method and device for a large time series model of thermal power units based on gradient variance analysis. Background Technology

[0002] In intelligent diagnostic scenarios for thermal power equipment, large-scale time-series models rely on massive amounts of operational data to predict equipment faults, assess conditions, and locate potential hazards, serving as the core technological support for intelligent operation and maintenance in thermal power plants. General pre-trained large-scale time-series models need to be fine-tuned and adapted to the specific operating conditions of thermal power units to accurately match the actual business needs of equipment monitoring and diagnosis. Currently, the industry's fine-tuning methods for large-scale time-series models in thermal power plants are mainly divided into two categories: full-parameter fine-tuning and fixed-level fine-tuning. Both have significant technical shortcomings, making it difficult to balance fine-tuning efficiency, computing cost, and model diagnostic accuracy.

[0003] Full-parameter fine-tuning requires unlocking all network parameters of the model for iterative training. Thermal power equipment runtime data is characterized by high sampling frequency, massive data volume, and complex temporal dimensions, resulting in a large number of parameters for the corresponding time-series models. The full-parameter iteration process consumes extremely high computing resources, leading to high hardware load and long training cycles, significantly increasing the economic cost of model deployment and failing to meet the lightweight and low-cost application requirements of power plants. Fixed-level fine-tuning, on the other hand, relies on manual experience to pre-define the model's frozen and training layers. The layer division criteria are subjective and rigid, making it impossible to dynamically adjust the optimization range based on the differences in time-series data characteristics of different thermal power equipment, operating conditions, and fault types.

[0004] The actual operating conditions of thermal power units fluctuate dynamically. Under scenarios such as deep peak shaving, unit start-up and shutdown, and load fluctuations, the distribution of time-series data for normal operation, performance degradation, and sudden faults varies significantly. Fixed-level fine-tuning is prone to problems such as key feature learning layers not participating in optimization and ineffective network layers iterating repeatedly. This results in insufficient feature extraction capabilities for early weak faults and hidden dangers in thermal power equipment, making it difficult to improve the accuracy of fault early warning and diagnosis. At the same time, traditional fine-tuning methods only use a single regression loss function to optimize the model, lacking constrained learning of the differentiated features between time-series samples, and cannot effectively distinguish between normal and abnormal time-series signals under similar operating conditions. In summary, how to achieve efficient adaptation of large-scale time-series models to the specific diagnostic tasks of thermal power equipment with minimal computational cost is a technical pain point that urgently needs to be overcome in this field. Summary of the Invention

[0005] This invention addresses the problems of high computational cost of full-scale training, poor adaptability of manually fixed layers, and limited diagnostic accuracy due to weak ability to distinguish weak fault features in the fine-tuning of large-scale time-series models of thermal power plants. It proposes a layered fine-tuning method and apparatus for large-scale time-series models of thermal power plants based on gradient variance analysis. This method abandons the inherent pattern of subjective experience in dividing layers, and adaptively selects key training layers of the model through gradient variance quantification, accurately identifying the network layers that need iterative optimization, significantly reducing the computational cost and training time caused by iterating with invalid parameters. Simultaneously, a time-series comparison loss mechanism is introduced to enhance the ability to distinguish weak fault time-series features under similar operating conditions, effectively improving the accuracy of fault early warning and the ability to identify early hidden dangers in thermal power equipment.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis, comprising the following steps: Collect on-site time-series monitoring data of thermal power units, and obtain steady-state time-series raw data and thermal power equipment-specific time-series labeled data respectively; configure parameter permissions for the pre-trained time-series large model, lock the parameter update permissions of the bottom feature embedding layer and intermediate time-series coding layer of the model backbone network, and only open the training permissions of the top-level adaptation structure of the model; Steady-state time-series raw data is used to iteratively train the top-level adaptation structure separately. The backbone network parameters are fixed throughout the process. The convergence of the adaptation structure parameters is achieved through feature mapping constraints, resulting in a pre-trained model adapted to the basic operating conditions of thermal power plants. Input the specific time-series labeled data of thermal power equipment into the pre-trained model adapted to the basic working conditions of thermal power, extract the time-series features through forward propagation, combine the real labels to solve the gradient information of each intermediate time-series coding layer in reverse, quantify the sensitivity of each coding layer to fault diagnosis task, and dynamically open the parameter training permissions of the highly sensitive intermediate time-series coding layer. Using the unlocked, highly sensitive intermediate temporal coding layer and the converged top-level adaptation structure as the trainable entities, and relying on the exclusive temporal annotation data of thermal power equipment, a fusion loss function is constructed to iteratively optimize the parameters until the loss converges. This completes the fine-tuning of the large temporal model and yields a fault diagnosis model for thermal power units under all operating conditions.

[0007] Furthermore, the steady-state time-series raw data is field-collected data under stable unit load, without operating disturbances, and without equipment anomalies, requiring no manual annotation or refined preprocessing; the thermal power equipment-specific time-series annotation data covers the full operating condition time-series data of unit load change, deep peak shaving, start-up and shutdown fluctuations, equipment steady-state operation, performance degradation, and fault anomalies. The full operating condition time-series data is subjected to outlier removal, missing data completion, time-series sampling alignment, sliding window segmentation, and precise manual annotation to obtain the thermal power equipment-specific time-series annotation data.

[0008] Furthermore, the model parameter permission configuration method is as follows: disable the gradient backpropagation function of the model's bottom feature embedding layer and all intermediate temporal coding layers, set the corresponding network layer parameters to read-only state, clear the learning rate of the corresponding network layer, fix the pre-trained general temporal features throughout the process, and only open the parameter iteration update channel of the top-level adaptation structure.

[0009] Furthermore, the top-level adaptation structure is a lightweight temporal adapter specifically for thermal power plants. It adopts a lightweight architecture that combines one-dimensional causal convolution, temporal pooling, and dimensional mapping to map the general temporal features output by the model backbone to the specific temporal features of thermal power equipment. The distribution deviation between the general temporal features and the thermal power adaptation features is quantified by the L2 feature mapping loss. The adapter weight parameters are updated in a gradual iteration with a small learning rate. When the feature mapping loss value is stable and there is no obvious downward trend after multiple iterations, the convergence training of the top-level adaptation structure is completed.

[0010] Furthermore, the task sensitivity of each intermediate temporal coding layer is quantified by calculating the gradient mean and gradient variance of each intermediate temporal coding layer. A preset gradient variance threshold is used as a screening criterion. Only the parameter freezing state of high-sensitivity coding layers with gradient variance greater than the preset threshold is released, while the parameters of the remaining low-sensitivity coding layers remain fixed throughout the process.

[0011] Furthermore, the fusion loss function is a weighted joint loss function, which is composed of time-series regression loss and time-series comparison loss fused together with a fixed weight ratio; the time-series comparison loss is constructed based on positive and negative samples of the thermal power equipment-specific time-series labeled data.

[0012] This invention also provides a hierarchical fine-tuning system for a large-scale time-series model of thermal power units based on gradient variance analysis, characterized in that it includes the following components for executing the aforementioned hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis: The data acquisition module is used to collect on-site time-series monitoring data of thermal power units, and to obtain both steady-state time-series raw data and thermal power equipment-specific time-series labeled data. The model configuration module is used to configure parameter permissions for pre-trained temporal large models, locking the parameter update permissions of the bottom feature embedding layer and intermediate temporal coding layer of the model backbone network, and only opening the training permissions of the top-level adaptation structure of the model. The adaptation training module is used to iteratively train the top-level adaptation structure using steady-state time-series raw data. The parameters of the backbone network are fixed throughout the process, and the parameters of the adaptation structure are converged through feature mapping constraints to obtain a pre-trained model adapted to the basic working conditions of thermal power plants. The coding training module is used to input the specific time-series labeled data of thermal power equipment into the pre-trained model adapted to the basic working conditions of thermal power. It extracts time-series features through forward propagation, combines real labels to solve the gradient information of each intermediate time-series coding layer in reverse, quantifies the sensitivity of each coding layer to fault diagnosis tasks, and dynamically opens the parameter training permissions of highly sensitive intermediate time-series coding layers. The loss training module is used to construct a fusion loss function based on the unlocked highly sensitive intermediate temporal coding layer and the converged top-level adaptation structure as the trainable main body, and to iteratively optimize the parameters until the loss converges, thereby completing the fine-tuning of the large temporal model and obtaining a fault diagnosis model for the entire operating condition of the thermal power unit.

[0013] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the hierarchical fine-tuning method for a large time series model of thermal power units based on gradient variance analysis as described above.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described hierarchical fine-tuning method for a large-scale time series model of thermal power units based on gradient variance analysis.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-mentioned hierarchical fine-tuning method for a large-scale time series model of thermal power units based on gradient variance analysis.

[0016] Compared with the prior art, the present invention has at least the following beneficial effects: This invention provides a hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis. It employs a hierarchical, progressive model fine-tuning architecture, relying on two types of time-series data with independent applications to complete iterative model optimization. Combined with a closed-loop technical system of backbone network parameter fixing, top-level structure pre-adaptation, gradient variance adaptive hierarchical selection, and weighted joint loss fine-tuning, it effectively overcomes the inherent drawbacks of traditional fine-tuning methods. Traditional full-parameter fine-tuning suffers from high computational investment, long training time, and high deployment costs. Manual fixed-layer fine-tuning relies on subjective experience and cannot adapt to dynamic operating conditions of thermal power units, resulting in fixed training layers and poor adaptability. This invention, through adaptive dynamic layer selection training and multi-loss joint constraint optimization, achieves lightweight transfer and adaptation of the large-scale time-series model to thermal power industry scenarios. While controlling training computational costs and lowering the engineering implementation threshold, it effectively overcomes the technical challenges of early equipment degradation and difficulty in identifying latent fault characteristics under similar operating conditions, balancing model training efficiency, application cost, and fault diagnosis accuracy.

[0017] Furthermore, this invention differentiates the collection scope and processing specifications for two types of time-series data. Steady-state raw data requires no complex preprocessing or manual annotation and can be directly used for early-stage scenario adaptation training, simplifying the data preparation process and reducing early-stage training investment. The dedicated labeled data for all operating conditions covers unit load variations, deep peak shaving, start-stop fluctuations, and the entire operating state of the equipment. After standardized cleaning, completion, alignment, slicing, and precise annotation, it provides stable and complete data support for model gradient calculation, hierarchical selection, and fault feature learning. By solidifying the parameters of the model's bottom-level feature embedding layer and intermediate time-series coding layer, the general time-series representation capabilities learned by the pre-trained model are fully preserved, avoiding the destruction of core basic features by multiple iterations of training, ensuring the stability and controllability of the entire training process, and eliminating the waste of computing power caused by invalid parameter iterations.

[0018] Furthermore, this invention configures a lightweight temporal adapter to complete the mapping and transformation from general features to thermal power-specific temporal features. It avoids the problem of temporal information leakage by relying on a causal convolutional structure and achieves rapid convergence with specific loss constraints, realizing scenario adaptation with extremely low parameter increments. Simultaneously, this invention uses gradient mean and gradient variance to quantify the task sensitivity of each network layer, replacing the traditional manual fixed layering mode. Only highly sensitive network layers are selected to participate in parameter updates, accurately focusing on effective feature learning and significantly improving the utilization rate of training resources. Based on this, a weighted joint loss function is constructed by fusing temporal regression loss and temporal contrast loss. This ensures the accuracy of the model's quantitative assessment of equipment status and distinguishes subtle temporal differences between normal fluctuations in operating conditions and minor faults through positive and negative sample feature constraints. This significantly improves the model's ability to identify latent anomalies and early degradation, reduces the probability of missed or false fault detections, and effectively improves the model's generalization performance and diagnostic reliability under complex actual operating conditions in thermal power plants.

[0019] Furthermore, this invention enables fully automated execution of the entire process, including automatic normalization of thermal power time-series data, standardized configuration of model parameters, lightweight scenario adaptation, adaptive hierarchical filtering, and refined loss optimization. This reduces manual intervention and effectively improves the standardization and operational efficiency of the model fine-tuning process. The overall architecture is adapted to the large-scale deployment needs of thermal power plants, stably completing scenario adaptation and fault diagnosis optimization of the time-series model. It offers stronger engineering feasibility and ease of operation and maintenance, making it well-suited for practical application scenarios of intelligent operation and maintenance in thermal power plants. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0021] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0022] like Figure 1As shown, this invention provides a hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis. It employs a two-stage progressive fine-tuning architecture, utilizes gradient variance quantization to filter and dynamically unfreeze network layers during training, and combines this with time-series comparative joint loss to achieve refined model optimization. Specifically, it includes the following technical steps: Step S1: Dataset preprocessing and model initialization.

[0023] This invention uses two types of on-site time-series data from thermal power plants. All data are taken from actual measured data collected by the DCS system and online status monitoring system of thermal power plants.

[0024] The first category is the raw data of conventional steady-state time series. The data collected under pure steady-state conditions with stable load operation, no disturbance operation, and no equipment abnormalities are selected. Only conventional steady-state time series signals such as temperature, vibration, pressure, and speed are retained. This type of data does not require manual annotation or fine preprocessing. The second category is a time-series dataset specifically for thermal power equipment. This dataset is a standard dataset dedicated to the refined training of this invention. The data structure covers disturbance conditions such as unit load changes, deep peak shaving, and start-up / shutdown fluctuations, as well as time-series data for all scenarios including steady-state operation, minor performance degradation, latent anomalies, and explicit faults. Specifically, it includes time-series sequences of core measurement points such as temperature, vibration, pressure, speed, and medium flow of boilers, steam turbines, and various auxiliary machines. Standardized preprocessing is performed on this type of raw data, including outlier removal, sensor missing value completion, time-series sampling alignment, sliding window sequence segmentation, and accurate labeling of operating conditions and fault categories, ultimately forming a standardized dataset with unified format, time-series alignment, and complete labels. Select a pre-trained general temporal transformer (TST) model that has been trained, and perform parameter locking and freezing operations on the model. Specifically, the gradient backpropagation permissions of the model's bottom feature embedding layer and all intermediate temporal coding layers are disabled through program configuration. The parameters of the above network layers are set to read-only state and the learning rate is set to zero. During the training process, the original pre-trained weights are always retained, and no parameter iteration updates occur. Only the parameter training and update permissions of the top-level temporal adapter are enabled.

[0025] Step S2, First Stage Lightweight Adaptation Training This phase involves the first stage of lightweight scenario adaptation training for the model. This invention adds a lightweight time-series adapter specifically for thermal power plants to the top layer of the pre-trained temporal Transformer model. This structure is specifically designed to address the industrial characteristics of thermal power units, such as strong noise in time-series data, variable time spans, and easy data distribution shifts. The adapter adopts a minimalist and lightweight architecture combining one-dimensional causal convolution, temporal pooling, and dimensional mapping, abandoning redundant fully connected layer stacking. The overall parameter count is far lower than the model's backbone network, enabling scenario feature adaptation with extremely low computational power consumption.

[0026] This invention fixes the core structural parameters of the adapter, sets the convolution kernel size to k=3, and the dilation coefficient to d=2, uniformly mapping the output feature dimension to 256 dimensions; relying on the unidirectional operation characteristics of causal convolution, it strictly ensures that time series data completes feature calculation only based on historical and current information, eliminates the problem of future information leakage, and meets the causal constraint requirements of industrial time series diagnostic tasks.

[0027] The adapter is responsible for adapting the general time-series features output from the model backbone network to specific time-series features that fit the operating patterns of thermal power equipment. The overall computation process can be quantitatively expressed by formulas:

[0028] In the formula, This represents the general temporal feature vector output by the backbone network of the TST model. This represents a one-dimensional causal convolution operation with corresponding kernel size and dilation coefficient. ReLU is a non-linear activation function used to introduce non-linear feature representation capabilities. The final output is a thermal power-specific timing feature with a fixed output dimension of 256.

[0029] The training data used in this stage is the conventional steady-state time-series raw data obtained in step S1. This type of data does not require manual annotation or fine preprocessing, mainly consists of steady-state operation data of the unit, has low noise, stable data distribution, and is suitable for the early-stage scenario adaptation training requirements of the model. The parameter freezing strategy from step S1 is continued throughout the training process, continuously locking the backbone parameters of the model's bottom-level feature embedding layer and all intermediate time-series coding layers. Gradient backpropagation and parameter update permissions of the backbone network are disabled, and the backbone weights always retain the general time-series features learned during pre-training. This effectively avoids the backbone features being destroyed by minor iterations, while also avoiding the waste of computational power caused by training all parameters. This stage only involves parameter iteration updates for the newly added lightweight time-series adapter, relying on massive steady-state time-series samples to complete the model scenario pre-adaptation, accurately achieving the transfer and adaptation of general time-series features to thermal power industry time-series features.

[0030] During training, the adapter weight parameters are continuously optimized and updated through iterations. The formula for weight iteration and update is as follows:

[0031] In the formula, These are the weight parameters for the current iteration of the adapter. These are the newly updated weight parameters. This phase utilizes a lightweight design with a small learning rate to ensure smooth parameter iteration and avoid oscillations and non-convergence issues. This is the gradient of the feature mapping loss function with respect to the adapter weight parameters, used to guide the reverse parameter update direction.

[0032] To constrain the feature adaptation effect, this stage uses only feature mapping loss as the single optimization objective. By quantifying the distribution deviation between general features and thermal power adaptation features, the stability and accuracy of adapter feature mapping are ensured. The loss calculation formula is as follows:

[0033] In the formula, This represents the number of steady-state time-series samples in a single batch of input models. This represents the general temporal feature of a single sample output by the model's backbone network. This refers to the specific timing characteristics of the thermal power plant output corresponding to the adapter. This is a 2-norm operation used to quantify the spatial distance between two sets of feature vectors, accurately representing the magnitude of the feature mapping deviation.

[0034] This stage employs a lightweight training mode with a small learning rate and gradual iterations, using the convergence of the feature mapping loss as the criterion for training termination. During multiple consecutive iterations... When the loss value no longer decreases and the overall fluctuation tends to stabilize, it can be determined that the adapter weight parameters have stabilized, the feature mapping effect has reached its optimal level and no longer improves, and the iterative training is then terminated. Through this stage of training, the model can fully learn the basic distribution patterns of the steady-state operation data of thermal power units, complete the initial adaptation from general time-series features to thermal power industry time-series features, and complete the preparatory work for scene migration with extremely low computing power consumption. This lays a solid parameter foundation for subsequent steps S3 and S4, which rely on finely labeled datasets to carry out fault feature learning and adaptive hierarchical fine-tuning.

[0035] Step S3, Second-stage adaptive hierarchical fine-tuning of gradient variance After the lightweight adapter adaptation convergence in the first stage of step S2, the process moves to the second stage: fine-tuning of layered parameters. This stage differs from the traditional manual fixed-level training model by employing a data-driven gradient variance adaptive selection mechanism. It automatically identifies effective network layers in the intermediate coding layers of the model that are highly sensitive to thermal power fault diagnosis tasks, completing fine-tuning of key parameters iteratively while strictly controlling computational overhead. This stage uniformly uses the thermal power equipment-specific time-series dataset constructed in step S1, which has undergone standardized preprocessing and precise fault labeling, as the sole training data source. This dataset contains finely labeled samples with multiple operating conditions, degradation levels, and fault types, accurately reflecting the differences in the abnormal time-series characteristics of thermal power equipment and providing reliable data support for the quantitative evaluation of network layer sensitivity. The overall data flow in this stage is as follows: the thermal power equipment-specific time-series dataset is input into the complete model that has completed steady-state adaptation; after the model's forward time-series features are extracted, the prediction results are output; combined with real fault labels, error backpropagation is performed, generating network parameter gradients layer by layer; and then, gradient variance quantization is used to complete the layer selection, ultimately determining the effective training layers that can participate in iterative updates.

[0036] In the early stages of training, the parameter freezing strategy described earlier is continued, continuously locking the parameters of the model's bottom-level feature embedding layers and only unfreezing some highly sensitive intermediate layers to achieve on-demand local training. First, a specific time-series dataset for thermal power equipment is input into the pre-adapted time-series Transformer model. Through forward propagation, full-dimensional time-series feature extraction and equipment status prediction are completed. Then, based on the backpropagation of the error loss between the predicted and true labels, the parameter gradient information of all intermediate Transformer time-series encoding layers is solved layer by layer. Gradient values ​​can intuitively reflect the update trend and optimization potential of network parameters for the current fault task. To further quantify the task sensitivity of each layer, the gradient mean and gradient variance are calculated batch by batch and layer by layer to accurately characterize the fluctuation update amplitude of each network layer parameter in a single iteration. The corresponding calculation formulas are as follows:

[0037]

[0038] In the formula, This represents the total number of trainable parameters for the corresponding network layer within a single batch. For the first The first intermediate Transformer coding layer The gradient values ​​of each parameter. For the first The mean gradient of the layer parameters is used to characterize the overall gradient update bias of that layer. For the first Layer parameter gradient variance is used to characterize the task-sensitive variability of layer parameters.

[0039] The magnitude of the gradient variance directly corresponds to the adaptability of the network layer to the fault diagnosis task in thermal power plants: a larger gradient variance indicates a more significant update magnitude of the layer's parameters after being stimulated by fault time-series data, and a higher sensitivity to learning features such as minor faults, operating condition deviations, and equipment degradation; a smaller gradient variance indicates that the fitting of the general time-series features at that layer has approached saturation, and the optimization space for thermal power-specific fault tasks is extremely small, requiring no further iterative updates. This invention, considering the inherent characteristics of strong noise, hidden fault features, and frequent operating condition fluctuations in thermal power time-series data, specifically presets a gradient variance screening threshold. Establish a standardized hierarchical thawing determination mechanism.

[0040] The specific rules for hierarchical selection are as follows: compare the gradient variance of each intermediate layer with a preset threshold; if the threshold is met... If the intermediate layer is deemed a highly sensitive and effective feature learning layer, its parameter freeze state is automatically lifted, and gradient backpropagation and parameter update permissions are enabled. For low-sensitivity layers with gradient variance less than or equal to a threshold, the parameters remain frozen with read-only access and a zero learning rate to prevent invalid parameter iteration. During model training, joint parameter iteration updates are performed only on the dynamically unfrozen highly sensitive intermediate encoding layers and the top-level lightweight temporal adapter that has converged in step S2. The pre-trained weights of the remaining network layers are kept fixed throughout the process, precisely focusing on optimizing effective parameters. This mechanism avoids the limitations of redundant computational power in full-scale fine-tuning and the adaptation limitations of manual fixed-layer fine-tuning.

[0041] Step S4: Time-series comparison joint loss optimization training Building upon the adaptive hierarchical unfreezing and locking of effective training layers in step S3, this stage establishes a weighted joint loss optimization system specific to thermal power time series models. This system replaces the traditional single mean square error loss constraint method for time series models, simultaneously considering both the model's quantitative prediction accuracy and fault feature discrimination capability. It addresses the technical challenge of identifying subtle faults and latent degradation features under similar operating conditions. This stage utilizes the unified data source and data flow path from S3, continuously relying on the thermal power equipment-specific time series dataset constructed in step S1 for iterative optimization. By leveraging the complete positive and negative sample pairs of normal, deteriorated, and faulty components within the dataset, clustering and sparsity constraints of time series features are achieved, further improving the model's refined diagnostic performance. The specific data flow in this stage is as follows: the thermal power equipment-specific time series dataset is fed into the model structure with determined training permissions. Feature depth extraction is performed using the highly sensitive intermediate layer and convergence adapter selected in S3. After generating time series prediction results, regression loss and contrast loss are calculated separately. The total loss is obtained through weighted fusion and then backpropagated using the total loss to guide the iterative update of trainable parameters, completing the model's refined convergence optimization.

[0042] To balance the model's basic fitting performance with its ability to identify differences in operating conditions, this invention integrates time-series regression loss and time-series comparison loss to construct a weighted joint loss function adaptable to the diagnostic tasks of thermal power plants under all operating conditions. The overall formula is as follows:

[0043] In the formula, To optimize the loss of the model as a whole, The weighted balancing coefficient is uniformly set to 0.7 in this invention. By using a fixed weight ratio, the fitting accuracy of the model's quantitative diagnosis is guaranteed first. At the same time, the constraint weights of the time series comparison loss are reasonably allocated to enhance the ability to identify subtle differences in working conditions.

[0044] Among them, time-series regression loss It performs the basic fitting constraint function, which is used to reduce the deviation between the model's predicted output and the actual state label of the equipment, and to ensure the accuracy of the model's quantitative assessment of the degree of equipment degradation and fault state. Its calculation formula is as follows:

[0045] In the formula, This represents the total number of labeled samples entered in a single batch. Labels indicating the actual operating status and fault categories of thermal power equipment. The model's basic diagnostic accuracy converges by using mean square error constraints to achieve the predicted output results for the corresponding time series samples.

[0046] Time-series comparison loss This design optimizes losses specifically for thermal power generation, addressing the issues of misjudgment and missed detection caused by similar loads, operating conditions, and subtle fault characteristics in thermal power units. The core functionality involves aggregating similar fault characteristics and separating dissimilar operating condition characteristics, enhancing the model's ability to capture subtle time-series differences such as latent faults and early degradation. The calculation formula is as follows:

[0047] In the formula, This is a cosine similarity metric function used to quantify the spatial distance between features of samples from different time periods; The feature vector of the baseline time series sample within a single batch; Features of positive samples that are the same as the baseline samples in terms of equipment, operating conditions, and fault types are used for aggregation constraints of similar features. Features from normal steady-state operating conditions or negative samples of different fault types are used for heterogeneous feature isolation constraints. This loss constraint can effectively isolate minute hourly fluctuation differences under similar operating conditions, thoroughly improving the shortcomings of traditional models such as feature homogenization and insufficient ability to identify minor faults.

[0048] During this training iteration phase, the joint total loss is used. The convergence state is used as the criterion for terminating model training, and the weight parameters of the unfrozen highly sensitive intermediate layer and top-level adapter are continuously updated in reverse iteration. When the joint loss value no longer decreases in multiple consecutive iterations, the overall fluctuation tends to be stable, and the model prediction accuracy does not improve significantly, the model parameters are considered to have converged and fine-tuned. In this stage, through dual-loss weighted fusion optimization, combined with a pre-adaptive hierarchical screening mechanism, the time-series large model is finally able to accurately adapt to the refined fault diagnosis and condition assessment tasks of multiple devices and all operating conditions of thermal power units under the premise of low computing power consumption.

[0049] To fully implement the aforementioned two-stage progressive hierarchical fine-tuning method of this invention and address the issues of high computational cost, rigid hierarchy, and insufficient accuracy in identifying minor faults when adapting traditional time-series models for thermal power equipment diagnosis, this invention provides a dedicated modular functional architecture. Each functional module strictly corresponds to all implementation steps S1 to S4. The modules collaborate to complete the entire process of data preprocessing, model initialization, lightweight scenario adaptation, adaptive hierarchy selection, and refined loss optimization, perfectly matching the core technical logic of the method. The specific functions and working principles of each module are as follows: 1. Thermal power time-series dual-source data preprocessing module.

[0050] This module serves as the foundational support for the overall method of this invention. Its core function is to construct two types of thermal power time-series data sources. The module connects in real-time to the DCS control system and online equipment condition monitoring system of the thermal power plant, collecting batches of time-series measurement data from all dimensions of the unit's boiler, turbine, and various auxiliary equipment, including temperature, vibration, pressure, speed, and medium flow rate. On one hand, it filters raw operating data from units with stable, undisturbed operation and no equipment anomalies, obtaining unlabeled, non-processable conventional steady-state time-series raw data specifically for the lightweight adapter adaptation training in step S2. On the other hand, it performs standardized and refined processing on the collected multi-condition raw data, sequentially completing outlier removal, sensor data completion, time-series sampling alignment, sliding window sequence segmentation, and accurate labeling of operating conditions and faults. This constructs a standardized, time-aligned, and fully labeled time-series dataset specifically for thermal power equipment, providing standardized, high-quality data support for the gradient variance hierarchy screening in step S3 and the refined iterative optimization of the joint loss in step S4.

[0051] 2. Model parameter freezing and lightweight timing adaptation module.

[0052] This module corresponds to step S1 (model initialization) and step S2 (lightweight adaptation process in the first stage), and is the core module for achieving migration in low-computing-power scenarios. In the initial stage, following the rules set in step S1, the module locks the parameter update permissions of the bottom-level feature embedding layer and all intermediate temporal coding layers of the temporal Transformer pre-trained model through programmatically fixed permission configuration. This fixes the model's general temporal feature extraction capabilities, retaining only the training permissions of the top-level adapter. The module incorporates a lightweight temporal adapter specific to thermal power plants, employing a minimalist architecture of one-dimensional causal convolution with k=3 kernels and d=2 dilation coefficients, combined with temporal pooling and dimensionality mapping. This allows the general temporal features output by the model backbone to be mapped to 256-dimensional thermal power-specific temporal features without leaking temporal information. Iterative convergence of the adapter is completed using steady-state temporal data, with feature mapping loss constraining the adaptation effect. Throughout the process, only the adapter parameters are updated, without modifying the backbone network, achieving the initial adaptation and migration from general temporal features to thermal power industry temporal features with extremely low computational overhead.

[0053] 3. Gradient variance adaptive hierarchical filtering module.

[0054] This module corresponds to the core logic of the second stage of fine-tuning in step S3, addressing the industry pain points of poor adaptability and high inefficient computational consumption in traditional manual fixed-layer training. The module utilizes a time-series dataset specific to thermal power equipment to complete forward feature extraction and backward gradient calculation. It calculates the mean and variance of the gradients of each intermediate encoding layer in batches and layer by layer, accurately quantifying the sensitivity of each network layer to thermal power fault diagnosis tasks. Considering the characteristics of thermal power data, such as strong noise and hidden fault features, a standardized layer-based judgment mechanism is constructed based on a preset gradient variance threshold. This automatically selects high-sensitivity feature learning layers with gradient variance greater than the threshold and releases their parameter freeze, while keeping the parameters of other low-sensitivity layers locked. Finally, the effective training layers that can participate in iterative optimization are determined. Joint training is conducted only by linking the unfrozen intermediate layers and the convergence adapter, abandoning the subjective judgment mode based on human experience, and achieving data-driven, precise layer selection training, reducing inefficient computational iteration overhead from the source.

[0055] 4. Time series comparison joint loss optimization module.

[0056] This module corresponds to step S4, the refined convergence training process, and is a key core module for improving the accuracy of the model in identifying subtle faults. Based on the effective training levels and fixed data flow links determined in step S3, the module performs iterative optimization using a time-series dataset specifically designed for thermal power equipment with precise labels. The module incorporates a weighted joint loss function, using a balance coefficient of 0.7 to fuse the mean squared error regression loss and the time-series comparison loss. The regression loss ensures the accuracy of the model's quantitative assessment of equipment status, while the time-series comparison loss aggregates features of similar faults and separates features of dissimilar operating conditions, effectively distinguishing subtle time-series differences under similar operating conditions in thermal power plants, and solving the problems of misjudgment and missed detection of early degradation and latent faults. The module uses the convergence state of the joint total loss as the training termination criterion, continuously optimizing trainable parameters, and ultimately completing the fine-tuning of the model to achieve low-computing-power, high-precision diagnostic results specifically for thermal power equipment.

[0057] In another embodiment of the present invention, a terminal device is also provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can implement the operation of a hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis.

[0058] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis in the above embodiments.

[0059] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0060] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] To fully verify the practical engineering application effect of the two-stage hierarchical fine-tuning method of this invention and highlight its dual advantages in computing power loss control and fault diagnosis accuracy, three typical types of equipment in the thermal power industry—auxiliary rotating equipment, main shaft equipment, and static pressure-bearing equipment—were selected for field testing. All embodiments strictly followed the complete method flow from steps S1 to S4, unifying basic parameters such as hardware operating environment, iteration rounds, learning rate, and loss weight. Two conventional schemes—traditional full-scale fine-tuning and manual fixed-level fine-tuning—were set as control groups. All experimental data came from real power plant operating conditions, and the experimental results are reproducible and traceable, as detailed below: Example 1: This embodiment addresses the fault diagnosis scenario of a 350MW subcritical unit boiler induced draft fan. It employs a large-scale TST time-series model with 12 million parameters. Following step S1, two types of datasets are collected and constructed. A massive amount of steady-state time-series raw data is selected for backup. Simultaneously, a dedicated time-series dataset for thermal power equipment is constructed, containing 6200 samples. These samples include 4200 steady-state operation data points and 2000 data points related to bearing wear and fan stall faults, covering both normal equipment operation and typical abnormal conditions. Training is strictly implemented according to the method steps of this invention. Step S1 completes the global freezing of the model's backbone parameters and dataset regularization. Step S2 iterates through 30 rounds using the steady-state raw data to achieve lightweight adapter convergence, realizing preliminary adaptation to thermal power time-series characteristics. Step S3 sets the gradient variance threshold to 0.02, adaptively selecting 5 highly sensitive intermediate encoding layers through gradient variance calculation and unfreezing them for training. Step S4 uses a joint loss function with a weight coefficient of 0.7 for 25 rounds of iteration to complete the model's refined parameter optimization and convergence.

[0064] The experimental results show that, compared with the traditional full-scale fine-tuning method, the method of the present invention reduces computing power consumption by 61.2%, shortens training time by 58%, and achieves an overall wind turbine fault early warning accuracy of 92.3%. The control group's manual fixed 3-layer fine-tuning scheme has a diagnostic accuracy of only 84.1%, and the computing power consumption of full-scale fine-tuning is 2.57 times that of the present invention. This fully verifies that the present invention can significantly reduce training costs while ensuring diagnostic accuracy.

[0065] Example 2 This embodiment addresses the scenario of diagnosing shaft vibration faults in a 660MW ultra-supercritical turbine unit. Such equipment exhibits subtle fault characteristics and high similarity in operating conditions, making it a challenging piece of equipment for thermal power plant diagnosis. Following the data acquisition specifications in step S1, the dataset was organized to obtain 8500 time-series samples covering all operating conditions, including 5800 samples of normal unit operation and 2700 samples of bearing wear, shaft misalignment, and abnormal bearing vibration faults. The training process fully follows the two-stage workflow of this invention: Step S1 completes data classification and model parameter freezing initialization; Step S2 utilizes steady-state data for 35 iterations to achieve adapter feature adaptation convergence; Step S3, considering the time-series characteristics of precision turbine equipment, sets a gradient variance threshold of 0.018 and adaptively selects 6 high-value intermediate encoding layers to grant training access; Step S4 relies on a weighted joint loss function to complete the differential constraints of subtle fault features and parameter iterative convergence.

[0066] Experimental tests show that the training computational cost of the method of this invention is reduced by 63.5% compared with full fine-tuning, and the accuracy rate of shaft fault early warning reaches 93.1%. The accuracy rate of the control group's manual fixed 5-layer fine-tuning scheme is only 85.6%, and it cannot effectively identify early hidden vibration faults, with obvious omission problems. The adaptive layered fine-tuning method of this invention significantly improves the adaptation and optimization effect on weak faults of precision rotating equipment.

[0067] Example 3 This embodiment addresses the scenario of predicting slow degradation risks such as wear, corrosion, and leakage in the four tubes of a boiler. Such equipment has a long degradation cycle, gradual temporal changes, and abnormal characteristics are easily masked by operating noise. Based on the data construction standards in step S1, core measurement point data such as tube wall temperature, medium pressure, and flue gas temperature are selected, resulting in 5100 full-dimensional time-series samples covering normal operating conditions, minor wear, severe corrosion, and early leakage. The fine-tuning method of this invention is fully implemented: step S1 completes the construction of the dual datasets and model initialization and freezing; step S2 iterates 32 times to complete the lightweight adapter adaptation; step S3, combined with the slow degradation time-series pattern of the boiler, sets a gradient variance threshold of 0.022 and filters and unfreezes four highly sensitive network layers for optimization; step S4 enhances the feature discrimination ability of samples with different degradation levels through temporal comparison and joint loss, achieving accurate model convergence.

[0068] The final test results show that, compared with the traditional full-scale fine-tuning scheme, the computing power cost of the method of this invention is reduced by 58.7%, and the accuracy rate of early warning of potential leakage in the four tubes of the boiler can reach 91.7%. The traditional manual fixed-level fine-tuning method cannot adapt to changes in operating conditions and has difficulty distinguishing the similar timing characteristics of slight wear of equipment and normal load fluctuations of the unit. The diagnostic accuracy rate is only 82.9%, and the probability of missing early deterioration risks is extremely high.

[0069] The embodiments of this invention fully verify that, for thermal power equipment with slow degradation, the combination mechanism of adaptive layer selection training and time-series feature differentiation constraint can effectively capture hidden and weak abnormal features. Under the premise of strictly controlling the training computing power consumption, it can significantly improve the early fault prediction capability of the equipment and has strong engineering practical value.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the present invention.

Claims

1. A hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis, characterized in that, Includes the following steps: Collect on-site time-series monitoring data of thermal power units, and obtain both steady-state time-series raw data and thermal power equipment-specific time-series labeled data. Configure parameter permissions for the pre-trained temporal large model, lock the parameter update permissions of the bottom feature embedding layer and the intermediate temporal coding layer of the model backbone network, and only open the training permissions of the top-level adaptation structure of the model. Steady-state time-series raw data is used to iteratively train the top-level adaptation structure separately. The backbone network parameters are fixed throughout the process. The convergence of the adaptation structure parameters is achieved through feature mapping constraints, resulting in a pre-trained model adapted to the basic operating conditions of thermal power plants. Input the specific time-series labeled data of thermal power equipment into the pre-trained model adapted to the basic working conditions of thermal power, extract the time-series features through forward propagation, combine the real labels to solve the gradient information of each intermediate time-series coding layer in reverse, quantify the sensitivity of each coding layer to fault diagnosis task, and dynamically open the parameter training permissions of the highly sensitive intermediate time-series coding layer. Using the unlocked, highly sensitive intermediate temporal coding layer and the converged top-level adaptation structure as the trainable entities, and relying on the exclusive temporal annotation data of thermal power equipment, a fusion loss function is constructed to iteratively optimize the parameters until the loss converges. This completes the fine-tuning of the large temporal model and yields a fault diagnosis model for thermal power units under all operating conditions.

2. The hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis according to claim 1, characterized in that, The steady-state time-series raw data is field-collected data under stable unit load, no operating condition disturbance, and no equipment anomaly, requiring no manual annotation or fine preprocessing. The thermal power equipment-specific time-series annotation data covers the full operating condition time-series data of unit load change, deep peak shaving, start-up and shutdown fluctuations, equipment steady-state operation, performance degradation, and fault anomalies. The full operating condition time-series data is subjected to outlier removal, missing data completion, time-series sampling alignment, sliding window segmentation, and precise manual annotation to obtain the thermal power equipment-specific time-series annotation data.

3. The hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis according to claim 1, characterized in that, The model parameter permission configuration method is as follows: disable the gradient backpropagation function of the model's bottom feature embedding layer and all intermediate temporal coding layers, set the corresponding network layer parameters to read-only state, clear the learning rate of the corresponding network layer, fix the pre-trained general temporal features throughout the process, and only open the parameter iteration update channel of the top-level adaptation structure.

4. The hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis according to claim 1, characterized in that, The top-level adaptation structure is a lightweight time-series adapter specifically for thermal power plants. It adopts a lightweight architecture that combines one-dimensional causal convolution, temporal pooling, and dimensional mapping to map the general time-series features output by the model backbone to time-series features specific to thermal power equipment. The distribution deviation between general temporal features and thermal power adaptation features is quantified by using L2 feature mapping loss. The adapter weight parameters are updated by a gradual iteration method with a small learning rate. When the feature mapping loss value is stable and there is no obvious downward trend after multiple rounds of iteration, the top-level adaptation structure convergence training is completed.

5. The hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis according to claim 1, characterized in that, The task sensitivity of each intermediate temporal coding layer is quantified by calculating the gradient mean and gradient variance of each intermediate coding layer. A preset gradient variance threshold is used as a screening criterion. Only the parameter freezing state of high-sensitivity coding layers with gradient variance greater than the preset threshold is released, while the parameters of the remaining low-sensitivity coding layers remain fixed throughout the process.

6. The hierarchical fine-tuning method for a large-scale time-series model of thermal power units based on gradient variance analysis according to claim 1, characterized in that, The fusion loss function is a weighted joint loss function, which is composed of time-series regression loss and time-series comparison loss fused together with a fixed weight ratio; the time-series comparison loss is constructed based on positive and negative samples of thermal power equipment-specific time-series labeled data.

7. A hierarchical fine-tuning system for a large-scale time-series model of a thermal power unit based on gradient variance analysis, characterized in that, A method for implementing a hierarchical fine-tuning of a large-scale time-series model of thermal power units based on gradient variance analysis as described in any one of claims 1 to 6, comprising: The data acquisition module is used to collect on-site time-series monitoring data of thermal power units, and to obtain both steady-state time-series raw data and thermal power equipment-specific time-series labeled data. The model configuration module is used to configure parameter permissions for pre-trained temporal large models, locking the parameter update permissions of the bottom feature embedding layer and intermediate temporal coding layer of the model backbone network, and only opening the training permissions of the top-level adaptation structure of the model. The adaptation training module is used to iteratively train the top-level adaptation structure using steady-state time-series raw data. The parameters of the backbone network are fixed throughout the process, and the parameters of the adaptation structure are converged through feature mapping constraints to obtain a pre-trained model adapted to the basic working conditions of thermal power plants. The coding training module is used to input the specific time-series labeled data of thermal power equipment into the pre-trained model adapted to the basic working conditions of thermal power. It extracts time-series features through forward propagation, combines real labels to solve the gradient information of each intermediate time-series coding layer in reverse, quantifies the sensitivity of each coding layer to fault diagnosis tasks, and dynamically opens the parameter training permissions of highly sensitive intermediate time-series coding layers. The loss training module is used to construct a fusion loss function based on the unlocked highly sensitive intermediate temporal coding layer and the converged top-level adaptation structure as the trainable main body, and to iteratively optimize the parameters until the loss converges, thereby completing the fine-tuning of the large temporal model and obtaining a fault diagnosis model for the entire operating condition of the thermal power unit.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the hierarchical fine-tuning method for a large time series model of thermal power units based on gradient variance analysis as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the hierarchical fine-tuning method for a large time series model of thermal power units based on gradient variance analysis as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the hierarchical fine-tuning method for a large time series model of thermal power units based on gradient variance analysis as described in any one of claims 1 to 6.

Images