A method and apparatus for predicting remaining lifetime based on a two-stage trained large language model

By employing a two-stage training method for large language models, combined with a Token-Channel Mixing module and a masked autoregressive mechanism, the problems of insufficient time-series data representation and generalization ability under small sample conditions in equipment remaining lifetime prediction are solved, achieving high-precision and stable lifetime prediction.

CN123068948APending Publication Date: 2026-11-10UNIV OF SCI & TECH BEIJING
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202610982454.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-02
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing technologies have insufficient time-series data representation capabilities, inadequate utilization of unlabeled data, and limited generalization ability under small sample conditions in equipment remaining life prediction, resulting in insufficient prediction accuracy and stability.

Method used

A two-stage training method for large language models is adopted. Pre-training is performed by introducing a Token-Channel Mixing module and a masked autoregressive mechanism. Feature learning is carried out using unlabeled time series data. In the supervised fine-tuning stage, only the lightweight Shared-LoRA module and prediction head are updated to achieve efficient modeling and accurate prediction of device operating status.

Benefits of technology

It improves the accuracy and stability of equipment remaining life prediction, especially maintaining good generalization ability and robustness to complex degradation patterns under small sample conditions, reducing the number of training parameters and avoiding overfitting and catastrophic forgetting problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068948A_ABST
    Figure CN123068948A_ABST
Patent Text Reader

Abstract

This invention relates to the field of intelligent operation and maintenance and health management technology for equipment, and particularly to a method and apparatus for predicting remaining lifetime based on a two-stage trained large language model. It includes: constructing a two-stage learning framework of "continued pre-training + supervised fine-tuning"; in the pre-training stage, fully utilizing unlabeled time-series data for feature learning, enabling the model to maintain good degradation modeling capabilities even under small sample conditions, alleviating the problem of scarce labeled data in RUL prediction; by introducing a masked autoregressive mechanism, the model can perform sequence prediction under partial observation conditions, improving robustness to missing information and complex degradation patterns; in the supervised fine-tuning stage, only the introduced Shared-LoRA module and prediction head are trained, freezing the backbone parameters of the large language model, which significantly reduces the number of trainable parameters and avoids overfitting or catastrophic forgetting problems in large models under small sample conditions, improving training stability and generalization ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance and health management technology for equipment, and in particular to a method and apparatus for predicting remaining lifetime based on a two-stage trained large language model. Background Technology

[0002] As industrial equipment becomes increasingly intelligent and complex, it is susceptible to environmental changes, operational disturbances, and material aging during long-term operation, leading to gradual performance degradation and even malfunctions. Therefore, accurate prediction of the remaining useful life (RUL) is crucial for predictive maintenance, reducing operating costs, and improving system safety. Traditional RUL prediction methods mainly include physical model-based and data-driven approaches. Physical model methods rely on mechanistic analysis but struggle to adapt to complex systems, while data-driven methods (such as RNNs, CNNs, and Transformers) perform well with large-scale labeled data but are prone to overfitting and insufficient generalization in small-sample scenarios.

[0003] Existing research on small-sample RUL prediction mainly focuses on data augmentation, transfer learning, and meta-learning methods, but these methods still have certain limitations in terms of modeling assumptions, generalization ability, or engineering applicability [1]. Recent studies have shown that pre-trained large language models have excellent capabilities in modeling complex sequence dependencies, and can maintain good prediction performance even in small-sample scenarios. Inspired by this, this invention attempts to introduce pre-trained large language models to improve the accuracy and generalization ability of device remaining life RUL prediction under small-sample conditions. However, pre-trained large language models are usually trained based on large-scale text data, and their input data modalities are significantly different from the device operation time series data. At the same time, their pre-training objectives are mainly language modeling tasks, which are inconsistent with the RUL prediction regression task in terms of objective form. Therefore, pre-trained large language models are difficult to apply directly to the RUL prediction problem and need to be specifically modified and optimized at the levels of input representation, task adaptation, and model structure.

[0004] Patent [202510853706.0] – A method for intelligent prediction of remaining service life of mechanical equipment based on fine-tuning of a large language model. This method addresses existing methods for predicting remaining service life of equipment based on fine-tuning of a large language model, which typically convert numerical time-series data into structured text representations and then combine this with low-rank adaptation (LoRA) for efficient parameter fine-tuning. While this method reduces training costs, discretizing continuous time-series data into text form leads to information loss and weakens the ability to express fine-grained degradation features. Furthermore, the single-stage fine-tuning strategy is insufficient for unlabeled data, has limited generalization ability in complex degradation scenarios, and may still suffer from unstable adaptation under small sample conditions.

[0005] Patent [202410785933.X] – A method for predicting the remaining useful life of mechanical equipment based on fine-tuning a large language model. Addressing existing two-stage fine-tuning methods, this method combines self-supervised pre-training with supervised fine-tuning, freezes the Transformer's main parameters, and introduces an adapter for task learning. While this method can utilize unlabeled data, the discrepancy between the pre-training task and the downstream RUL prediction target affects representation consistency. Furthermore, freezing the attention layer and feedforward network limits the model's ability to model complex degradation patterns and long-range dependencies, and the adapter structure's expressive power is limited, making it difficult to fully characterize the multi-stage, multi-scale equipment life evolution process. Summary of the Invention

[0006] To address the technical problems of existing equipment remaining lifetime prediction methods, such as insufficient time-series data representation capabilities, inadequate utilization of unlabeled data, and limited generalization ability under small sample conditions, this invention provides a remaining lifetime prediction method and apparatus based on a two-stage trained large language model. The technical solution is as follows: On the one hand, a method for predicting remaining lifespan based on a two-stage training large language model is provided, the method including: S1. Improve the structure of the large language model and pre-train it to obtain an improved large language model; S2. Acquire multi-channel condition monitoring data of mechanical equipment and standardize the data to obtain standardized multi-channel time series data; divide the standardized multi-channel time series data based on a sliding window to obtain sub-sequence samples of multiple equipment operating states; S3. Based on the sub-sequence samples of multiple device operating states, each time series sub-sequence is divided into patches and mapped to a time series token representation; S4. Using unlabeled time series data, the input time series tokens are randomly masked while retaining the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions, and the improved large language model is trained in the first stage. S5. Adapt the large language model trained in the first stage to a task using a small number of samples with remaining lifetime RUL tags, train the RUL prediction task in the second stage, and update the introduced lightweight Shared-LoRA module and prediction head. S6. Based on the two-stage training model, construct a mechanical equipment remaining life prediction system, and use the mechanical equipment remaining life prediction system to assess and predict equipment life.

[0007] Optionally, in S1, structural improvements are made to the large language model, including: Based on the pre-trained LLM backbone, the structure is extended by introducing a TCM module between the input embedding layer and the Transformer encoding layer to model the inter-channel correlation and local temporal dependency in multivariate time series.

[0008] Optionally, in S2, the data is standardized to obtain standardized multi-channel time series data. The standardized multi-channel time series data is then divided using a sliding window to obtain sub-sequence samples of multiple device operating states, including: For the original sequence any channel in Its standardized value The calculation is performed using the following formula:

[0009] in, For the first The passage is The original value at time, and These represent the mean and standard deviation of the channel over the observation period, respectively; T represents the mean and standard deviation; C represents the standard deviation. No. i The subsequence samples are represented as follows: , Wherein, the sliding window length is W, the sliding step size is S, and the start time is... ; By using a sliding window, the long sequence data is transformed into M equal-length subsequence samples of device operating status, where .

[0010] Optionally, in S3, based on subsequence samples of multiple device operating states, each time series subsequence is divided into patches, including: For each device operating state subsequence sample The patch is divided into non-overlapping patches of a fixed length, resulting in a patch sequence. , in, This indicates the total number of patches.

[0011] Optionally, in S4, the input time-series tokens are randomly masked using unlabeled time-series data while retaining the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions. This allows for the first stage of training of the improved large language model, including: Using unlabeled time series data, the first N patches are mapped to token representations through the input embedding layer and then input into the TCM module for feature fusion. A masked autoregressive mechanism is introduced to randomly mask some patch tokens and replace them with learnable mask tokens, enabling the model to perform sequence modeling under partially observable conditions. The continuous pre-training process uses an autoregressive approach to predict the next patch, and its loss function is defined as: , Where N represents the number of patches. This indicates a real patch. This indicates a model prediction patch.

[0012] Optionally, in S5, the large language model trained in Stage I is adapted for the task using a small number of RUL-labeled samples, and then trained for the RUL prediction task in Stage II. This updates the introduced lightweight Shared-LoRA module and prediction head, including: The model is adapted to the task using a small number of labeled samples. Based on the model that has been pre-trained, a lightweight Shared-LoRA module is introduced and replaced with a task output header for RUL prediction. The input processing method is consistent with that of Stage I, that is, the time series patches are mapped to embedded representations in sequence and then input into the frozen large language model backbone network; During training, only the Shared-LoRA module and prediction head parameters are updated, while the backbone network parameters remain unchanged. The RUL prediction head is used to output the predicted remaining lifespan of the machinery. and with genuine RUL labels The supervised loss is calculated, and its loss function is defined as: .

[0013] Optionally, in S6, a mechanical equipment remaining life prediction system is constructed based on the two-stage trained model. Based on this system, equipment lifespan assessment and prediction are performed, including: A system for predicting the remaining life of mechanical equipment is constructed using a model that has completed two stages of training. The system collects real-time status monitoring data from multiple channels, performs patching and embedding mapping based on the mechanical equipment remaining life prediction system, inputs the data into the model, and outputs the corresponding remaining life prediction results through the prediction head.

[0014] On the other hand, a remaining lifetime prediction device based on a two-stage trained large language model is provided. This remaining lifetime prediction device based on a two-stage trained large language model is used to implement the aforementioned remaining lifetime prediction method based on a two-stage trained large language model. The system comprises: The model improvement module is used to improve the structure of the large language model and pre-train it to obtain an improved large language model. The data processing module is used to acquire multi-channel condition monitoring data of mechanical equipment, and to standardize the data to obtain standardized multi-channel time series data; the standardized multi-channel time series data is divided based on a sliding window to obtain sub-series samples of multiple equipment operating states; The data partitioning and mapping module is used to partition each time series subsequence based on subsequence samples of multiple device operating states and map them into time series token representations. The first-stage training module is used to randomly mask the input time-series tokens using unlabeled time-series data while retaining the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions and pre-training the improved large language model. The second-stage training module is used to adapt the pre-trained large language model to the task using a small number of samples with RUL labels, train the RUL prediction task, and update the introduced lightweight Shared-LoRA module and prediction head. The assessment and prediction module is used to build a mechanical equipment remaining life prediction system based on a two-stage trained model, and to assess and predict equipment life based on the mechanical equipment remaining life prediction system.

[0015] On the other hand, a remaining lifetime prediction device based on a two-stage trained large language model is provided. The remaining lifetime prediction device based on a two-stage trained large language model includes: a processor; a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the processor, any one of the remaining lifetime prediction methods based on a two-stage trained large language model described above is implemented.

[0016] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for predicting remaining lifetime based on a two-stage trained large language model.

[0017] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention proposes a masked autoregressive pre-training mechanism that introduces a random mask while maintaining autoregressive modeling. This enables the model to perform time series modeling under partially observable conditions and uses patches as the prediction unit, thereby simultaneously improving the modeling ability for degradation processes and robustness to missing data.

[0018] This invention introduces the Token-Channel Mixing (TCM) module into the large language model, and achieves collaborative feature extraction of multivariate time series by jointly modeling time dimension dependencies and inter-channel correlations, which has stronger representation capabilities compared with traditional single-dimensional modeling methods.

[0019] This invention proposes a parameter-efficient fine-tuning strategy based on Shared-LoRA. By sharing low-rank adaptation parameters and freezing the backbone network, only a small number of newly added parameters are updated, achieving efficient model adaptation and stable training under small sample conditions.

[0020] This invention constructs a two-stage learning framework that combines the above methods, realizing unified modeling of unlabeled pre-training and small sample fine-tuning, and can still achieve high-precision RUL prediction under complex working conditions. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of the remaining lifetime prediction method based on a two-stage training large language model provided in an embodiment of the present invention; Figure 2 A schematic diagram of the TCM module provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the Shared-LoRA module provided in an embodiment of the present invention; Figure 4 This is a graph showing the RUL prediction results provided in an embodiment of the present invention; Figure 5 This is a block diagram of the remaining lifetime prediction device based on a two-stage trained large language model provided in an embodiment of the present invention. Figure 6 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0024] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0025] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0026] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0027] This invention provides a method for predicting remaining lifetime based on a two-stage trained large language model. This method can be implemented by a remaining lifetime prediction device based on a two-stage trained large language model, which can be a terminal or a server. Figure 1 The flowchart shown is for a remaining life prediction method based on a two-stage training large language model. The method's processing flow may include the following steps: S1. Improve the structure of the large language model and pre-train it to obtain an improved large language model; In one feasible implementation, S1 involves structural improvements to the large language model, including: Based on the pre-trained LLM backbone, the structure is extended by introducing a Token-Channel Mixing (TCM) module between the input embedding layer and the Transformer encoding layer to model inter-channel correlations and local temporal dependencies in multivariate time series. The TCM module structure is as follows: Figure 2 As shown.

[0028] S2. Acquire multi-channel condition monitoring data of mechanical equipment and standardize the data to obtain standardized multi-channel time series data; divide the standardized multi-channel time series data based on a sliding window to obtain sub-sequence samples of multiple equipment operating states; In one feasible implementation, in step S2, the data is standardized to obtain standardized multi-channel time series data. The standardized multi-channel time series data is then divided using a sliding window to obtain sub-sequence samples of multiple device operating states, including: For the original sequence any channel in Its standardized value The calculation is performed using the following formula:

[0029] in, For the first The passage is The original value at time, and These represent the mean and standard deviation of the channel over the observation period, respectively; T represents the mean; C represents the standard deviation; thus eliminating dimensional differences.

[0030] No. i The subsequence samples are represented as follows: , Wherein, the sliding window length is W, the sliding step size is S, and the start time is... By using a sliding window, the long sequence data is transformed into M equal-length subsequence samples of device operating status, where... .

[0031] S3. Based on the sub-sequence samples of multiple device operating states, each time series sub-sequence is divided into patches and mapped to a time series token representation; In one feasible implementation, in S3, based on sub-sequence samples of multiple device operating states, each time series sub-sequence is divided into patches, including: For each device operating state subsequence sample The patch is divided into non-overlapping patches of a fixed length, resulting in a patch sequence. , in, This indicates the total number of patches.

[0032] S4. Using unlabeled time series data, the input time series tokens are randomly masked while retaining the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions, and the improved large language model is trained in the first stage. In one feasible implementation, in S4, the input time-series tokens are randomly masked using unlabeled time-series data while retaining the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions. The improved large language model then undergoes Phase I training, including: Using unlabeled time series data, the first N patches are mapped to token representations through the input embedding layer and then input into the TCM module for feature fusion. A masked autoregressive mechanism is introduced to randomly mask some patch tokens and replace them with learnable mask tokens, enabling the model to perform sequence modeling under partially observable conditions. The continuous pre-training process uses an autoregressive approach to predict the next patch, and its loss function is defined as: , Where N represents the number of patches. This indicates a real patch. This indicates a model prediction patch.

[0033] S5. Adapt the large language model trained in the first stage to a task using a small number of samples with RUL labels, train the RUL prediction task in the second stage, and update the introduced lightweight Shared-LoRA module and prediction head. In one feasible implementation, in S5, the large language model trained in the first stage is adapted for the task using a small number of RUL-labeled samples, and the second stage of RUL prediction task training is performed. This involves updating the introduced lightweight Shared-LoRA module and prediction head, including: The model is task-adapted using a small number of labeled samples. Based on the model after further pre-training, a lightweight Shared-LoRA module is introduced and replaced with a task output header geared towards RUL prediction; for example... Figure 3 The Shared-LoRA module provided in this embodiment of the invention; The input processing method is consistent with that of Stage I, that is, the time series patches are mapped to embedded representations in sequence and then input into the frozen large language model backbone network; During training, only newly added parameters such as the Shared-LoRA module and prediction head are updated, while the backbone network parameters remain unchanged. This improves parameter utilization efficiency while avoiding damage to the learned degradation feature representation capabilities. The RUL prediction head is used to output the predicted remaining lifespan of mechanical equipment. and with genuine RUL labels The supervised loss is calculated, and its loss function is defined as: .

[0034] S6. Based on the two-stage training model, construct a mechanical equipment remaining life prediction system, and use the mechanical equipment remaining life prediction system to assess and predict equipment life.

[0035] In one feasible implementation, in step S6, a mechanical equipment remaining life prediction system is constructed based on a two-stage trained model. Based on this system, equipment lifespan assessment and prediction are performed, including: A system for predicting the remaining life of mechanical equipment is constructed using a model that has completed two stages of training. Real-time status monitoring data from multiple channels is collected, patched and embedded based on the mechanical equipment remaining life prediction system, and then input into the model. The corresponding remaining life prediction results are output through the prediction head, such as... Figure 4 The image shown is a graph of the RUL prediction results.

[0036] In this embodiment of the invention, I. Through a two-stage learning framework of "continued pre-training + supervised fine-tuning", the unlabeled time series data is fully utilized for feature learning during the pre-training stage, so that the model can still have good degradation modeling ability under small sample conditions, thereby effectively alleviating the problem of scarce labeled data in RUL prediction. II. By introducing a masked autoregressive mechanism, the model can perform sequence prediction under partial observation conditions, which not only retains the causal modeling ability, but also improves the robustness to missing information and complex degradation patterns, which is conducive to capturing the long-term evolution law of the equipment. III. In the supervised fine-tuning stage, only the introduced Shared-LoRA module and prediction head are trained, and the backbone parameters of the large language model are frozen. This not only significantly reduces the number of trainable parameters, but also avoids the problem of overfitting or catastrophic forgetting in large models under small sample conditions, thereby improving training stability and generalization ability.

[0037] Figure 5 This is a block diagram of a remaining lifetime prediction apparatus 500 based on a two-stage trained large language model, according to an exemplary embodiment. The system 500 is used for a remaining lifetime prediction method based on a two-stage trained large language model. (Refer to...) Figure 5 The system includes a model improvement module 510, a data processing module 520, a data partitioning and mapping module 530, a first-stage training module 540, a second-stage training module 550, and an evaluation and prediction module 560. Among them: Model improvement module 510 is used to improve the structure of the large language model and pre-train it to obtain an improved large language model. The data processing module 520 is used to acquire multi-channel condition monitoring data of mechanical equipment, and to standardize the data to obtain standardized multi-channel time series data; the standardized multi-channel time series data is divided based on a sliding window to obtain sub-series samples of multiple equipment operating states; The data partitioning and mapping module 530 is used to partition each time series subsequence based on subsequence samples of multiple device operating states and map them into time series token representations. Phase I training module 540 is used to randomly mask the input time series tokens using unlabeled time series data while retaining the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions and pre-training the improved large language model. Phase II training module 550 is used to adapt the pre-trained large language model to the task using a small number of samples with RUL labels, train the RUL prediction task, and update the introduced lightweight Shared-LoRA module and prediction head. The evaluation and prediction module 560 is used to build a mechanical equipment remaining life prediction system based on a two-stage trained model, and to evaluate and predict equipment life based on the mechanical equipment remaining life prediction system.

[0038] Optionally, the model improvement module 510 is used to extend the structure based on the pre-trained LLM backbone by introducing a TCM module between the input embedding layer and the Transformer encoding layer to model the inter-channel correlation and local temporal dependency in multivariate time series.

[0039] Optionally, the data processing module 520 is used for processing the original sequence. any channel in Its standardized value The calculation is performed using the following formula:

[0040] in, For the first The passage is The original value at time, and These represent the mean and standard deviation of the channel over the observation period, respectively; T represents the mean and standard deviation; C represents the standard deviation. No. i The subsequence samples are represented as follows: , Wherein, the sliding window length is W, the sliding step size is S, and the start time is... ; By using a sliding window, the long sequence data is transformed into M equal-length subsequence samples of device operating status, where .

[0041] Optionally, the data partitioning and mapping module 530 is used to partition the subsequence samples of each device's operating state. The patch is divided into non-overlapping patches of a fixed length, resulting in a patch sequence. , in, This indicates the total number of patches.

[0042] Optionally, the first-stage training module 540 is used to map the first N patches to token representations through the input embedding layer using unlabeled time series data, and input them into the TCM module for feature fusion; A masked autoregressive mechanism is introduced to randomly mask some patch tokens and replace them with learnable mask tokens, enabling the model to perform sequence modeling under partially observable conditions. The continuous pre-training process uses an autoregressive approach to predict the next patch, and its loss function is defined as: , Where N represents the number of patches. This indicates a real patch. This indicates a model prediction patch.

[0043] Optionally, the second-stage training module 550 is used to adapt the model to the task using a small number of labeled samples. Based on the model that has completed further pre-training, a lightweight Shared-LoRA module is introduced and replaced with a task output head for RUL prediction. The input processing method is consistent with that of Stage I, that is, the time series patches are mapped to embedded representations in sequence and then input into the frozen large language model backbone network; During training, only the Shared-LoRA module and prediction head parameters are updated, while the backbone network parameters remain unchanged. The RUL prediction head is used to output the predicted remaining lifespan of the machinery. and with genuine RUL labels The supervised loss is calculated, and its loss function is defined as: .

[0044] Optionally, the evaluation prediction module 560 is used to build a mechanical equipment remaining life prediction system using the model after the two-stage training is completed; The system collects real-time status monitoring data from multiple channels, performs patching and embedding mapping based on the mechanical equipment remaining life prediction system, inputs the data into the model, and outputs the corresponding remaining life prediction results through the prediction head.

[0045] In this embodiment of the invention, I. Through a two-stage learning framework of "continued pre-training + supervised fine-tuning", the unlabeled time series data is fully utilized for feature learning during the pre-training stage, so that the model can still have good degradation modeling ability under small sample conditions, thereby effectively alleviating the problem of scarce labeled data in RUL prediction. II. By introducing a masked autoregressive mechanism, the model can perform sequence prediction under partial observation conditions, which not only retains the causal modeling ability, but also improves the robustness to missing information and complex degradation patterns, which is conducive to capturing the long-term evolution law of the equipment. III. In the supervised fine-tuning stage, only the introduced Shared-LoRA module and prediction head are trained, and the backbone parameters of the large language model are frozen. This not only significantly reduces the number of trainable parameters, but also avoids the problem of overfitting or catastrophic forgetting in large models under small sample conditions, thereby improving training stability and generalization ability.

[0046] Figure 6 This is a schematic diagram of the structure of a remaining lifetime prediction device based on a two-stage trained large language model provided in an embodiment of the present invention, as shown below. Figure 6 As shown, a remaining life prediction device based on a two-stage trained large language model may include the above-mentioned... Figure 5 The illustrated device is a remaining lifetime prediction apparatus based on a two-stage trained large language model. Optionally, a remaining lifetime prediction apparatus 410 based on a two-stage trained large language model may include a first processor 2001.

[0047] Optionally, a remaining lifetime prediction device 410 based on a two-stage trained large language model may further include a memory 2002 and a transceiver 2003.

[0048] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0049] The following is combined with Figure 6 The components of a remaining lifetime prediction device 410 based on a two-stage trained large language model are described in detail below: The first processor 2001 is the control center of a remaining lifetime prediction device 410 based on a two-stage trained large language model. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0050] Optionally, the first processor 2001 can perform various functions of a remaining lifetime prediction device 410 based on a two-stage trained large language model by running or executing software programs stored in memory 2002 and calling data stored in memory 2002.

[0051] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 6 CPU0 and CPU1 are shown in the diagram.

[0052] In a specific implementation, as one example, a remaining lifetime prediction device 410 based on a two-stage trained large language model may also include multiple processors, for example... Figure 6 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0053] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0054] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via an interface circuit of a two-stage training large language model-based remaining lifetime prediction device 410. Figure 6 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0055] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0056] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 6 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0057] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and is connected to an interface circuit of a remaining lifetime prediction device 410 based on a two-stage trained large language model. Figure 6 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0058] It should be noted that, Figure 6 The structure of a remaining lifetime prediction device 410 based on a two-stage trained large language model shown in the figure does not constitute a limitation on the router. Actual knowledge structure recognition devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0059] Furthermore, the technical effect of a remaining lifetime prediction device 410 based on a two-stage trained large language model can be referred to the technical effect of a remaining lifetime prediction method based on a two-stage trained large language model described in the above method embodiments, and will not be repeated here.

[0060] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0061] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0062] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable sensors. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0063] It should be understood that the term "and / or" in this invention is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Furthermore, the character " / " in this invention generally indicates an "or" relationship between the preceding and following related objects, but it may also indicate an "and / or" relationship. Please refer to the preceding and following text for a more detailed understanding.

[0064] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0065] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0066] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0067] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0068] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0069] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for predicting remaining lifespan based on a two-stage trained large language model, characterized in that, The method includes: S1. Improve the structure of the large language model and pre-train it to obtain an improved large language model; S2. Acquire multi-channel condition monitoring data of mechanical equipment and standardize the data to obtain standardized multi-channel time series data; divide the standardized multi-channel time series data based on a sliding window to obtain sub-sequence samples of multiple equipment operating states; S3. Based on the subsequence samples of multiple device operating states, each time series subsequence in the subsequence sample is divided into patches and mapped to a time series token representation. S4. Using unlabeled time series data, the input time series tokens are randomly masked while retaining the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions, and the improved large language model is trained in the first stage. S5. Adapt the large language model trained in the first stage to a task using a small number of samples with remaining lifetime RUL tags, train the RUL prediction task in the second stage, and update the introduced lightweight Shared-LoRA module and prediction head. S6. Based on the two-stage training model, construct a mechanical equipment remaining life prediction system, and use the mechanical equipment remaining life prediction system to assess and predict equipment life.

2. The remaining lifetime prediction method based on a two-stage trained large language model according to claim 1, characterized in that, In S1, structural improvements are made to the large language model, including: Based on the LLM backbone, the structure is extended by introducing a TCM module between the input embedding layer and the Transformer encoding layer to model the inter-channel correlation and local temporal dependency in multivariate time series.

3. The remaining lifetime prediction method based on a two-stage trained large language model according to claim 2, characterized in that, In step S2, the data is standardized to obtain standardized multi-channel time series data. The standardized multi-channel time series data is then divided using a sliding window to obtain sub-sequence samples of multiple device operating states, including: For the original sequence any channel in Its standardized value The calculation is performed using the following formula: in, For the first The passage is The original value at time, and These represent the mean and standard deviation of the channel over the observation period, respectively; T represents the mean and standard deviation; C represents the standard deviation. No. i The subsequence samples are represented as follows: , Wherein, the sliding window length is W, the sliding step size is S, and the start time is... ; By using a sliding window, the long sequence data is transformed into M equal-length subsequence samples of device operating status, where .

4. The remaining lifetime prediction method based on a two-stage trained large language model according to claim 3, characterized in that, In step S3, based on sub-sequence samples of multiple device operating states, each time series sub-sequence is divided into patches, including: For each device operating state subsequence sample The patch is divided into non-overlapping patches of a fixed length, resulting in a patch sequence. , in, This indicates the total number of patches.

5. The remaining lifetime prediction method based on a two-stage trained large language model according to claim 4, characterized in that, In step S4, unlabeled time series data is used to randomly mask the input time-series tokens while preserving the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions. The improved large language model undergoes Phase I training, including: Using unlabeled time series data, the first N patches are mapped to token representations through the input embedding layer and then input into the TCM module for feature fusion. A masked autoregressive mechanism is introduced to randomly mask some patch tokens and replace them with learnable mask tokens, enabling the model to perform sequence modeling under partially observable conditions. The continuous pre-training process uses an autoregressive approach to predict the next patch, and its loss function is defined as: , Where N represents the number of patches. This indicates a real patch. This indicates a model prediction patch.

6. The remaining lifetime prediction method based on a two-stage trained large language model according to claim 5, characterized in that, In step S5, a small number of samples with remaining lifetime RUL tags are used to adapt the large language model trained in stage I to the task, and then the RUL prediction task is trained in stage II. This involves updating the introduced lightweight Shared-LoRA module and prediction head, including: The model is adapted to the task using a small number of labeled samples. Based on the model that has been pre-trained, a lightweight Shared-LoRA module is introduced and replaced with a task output header for RUL prediction. The input data processing method is consistent with that of Stage I, that is, the time series patches are mapped sequentially to embedded representations and then input into the frozen large language model backbone network; During training, only the Shared-LoRA module and prediction head parameters are updated, while the backbone network parameters remain unchanged. The RUL prediction head is used to output the predicted remaining lifespan of the machinery. and with genuine RUL labels The supervised loss is calculated, and its loss function is defined as: 。 7. The remaining lifetime prediction method based on a two-stage trained large language model according to claim 5, characterized in that, In step S6, a mechanical equipment remaining life prediction system is constructed based on a two-stage trained model. Based on this system, equipment lifespan assessment and prediction are performed, including: A system for predicting the remaining life of mechanical equipment is constructed using a model that has completed two stages of training. The system collects real-time status monitoring data from multiple channels, performs patching and embedding mapping based on the mechanical equipment remaining life prediction system, inputs the data into the model, and outputs the corresponding remaining life prediction results through the prediction head.

8. A device for predicting remaining lifetime based on a two-stage trained large language model, wherein the device is used to implement the remaining lifetime prediction method based on a two-stage trained large language model as described in any one of claims 1-7, characterized in that, The system includes: The model improvement module is used to improve the structure of the large language model and pre-train it to obtain an improved large language model. The data processing module is used to acquire multi-channel condition monitoring data of mechanical equipment, and to standardize the data to obtain standardized multi-channel time series data; the standardized multi-channel time series data is divided based on a sliding window to obtain sub-series samples of multiple equipment operating states; The data partitioning and mapping module is used to partition each time series subsequence based on subsequence samples of multiple device operating states, and map them into time series token representations. The first-stage training module is used to randomly mask the input time-series tokens using unlabeled time-series data while retaining the autoregressive mechanism, enabling the model to perform sequence modeling under partially observable conditions and pre-training the improved large language model. The Phase II training module is used to adapt the pre-trained large language model to the task using a small number of samples with remaining lifetime RUL labels, to train the RUL prediction task, and to update the introduced lightweight Shared-LoRA module and prediction head. The assessment and prediction module is used to build a mechanical equipment remaining life prediction system based on a two-stage trained model, and to assess and predict equipment life based on the mechanical equipment remaining life prediction system.

9. A device for predicting remaining lifespan based on a two-stage trained large language model, characterized in that, The remaining lifetime prediction device based on a two-stage trained large language model includes: Processor; memory storing computer-readable instructions that, when executed by the processor, implement any one of the remaining lifetime prediction methods based on a two-stage trained large language model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to implement any one of the remaining lifetime prediction methods based on a two-stage trained large language model as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Method for predicting residual life of mechanical equipment based on fine-tuning large language model

    CN118709837A

  • Mechanical equipment residual life intelligent prediction method based on large language model fine tuning

    CN120744432A