Industrial fault diagnosis model modeling method

Abstract

The invention discloses a modeling method of an industrial fault diagnosis model. The modeling method comprises the following steps: S1, preprocessing original data; s2, establishing an automatic sample generation labeling mechanism based on the thinking chain; s3, establishing a state transition mapping layer; s4, establishing a fault sample filtering mechanism; s5, fault diagnosis modeling based on LLMs is carried out; and S6, evaluating the method. The general industrial intelligent fault diagnosis model established by the invention is not only suitable for cold start zero sample scenes of various environmental conditions, but also can overcome the limitation of system scale, environmental conditions and topological structures on model applicability, solves the problems that samples cannot be collected in advance during industrial field cold start, the model precision is low and the generalization is poor, and improves the fault diagnosis efficiency. The method is of great significance in avoiding economic loss and major safety accidents caused by the fact that equipment faults are not found in time.

Description

Technical Field

[0001] The present invention mainly relates to the field of large models, and in particular to a modeling method for industrial fault diagnosis models. Background Art

[0002] Continuous advances in science and technology have led to a significant increase in system complexity and cost within the industrial sector. This has led to increased reliability and safety requirements for industrial systems. Initially, industrial systems often continue to operate despite a fault. However, over time, unnoticed faults can lead to potential equipment failure, loss of profitability, and even safety incidents. Therefore, monitoring the health of industrial systems and quickly and accurately diagnosing the fault type after a fault occurs are crucial. These measures not only facilitate timely maintenance and system restoration, but also effectively reduce the risk of personnel safety incidents.

[0003] Over the past few decades, numerous methods have been proposed for industrial equipment fault diagnosis. Data-driven methods are considered one of the most effective due to their powerful automatic feature extraction capabilities and end-to-end learning advantages. With the rapid development of artificial intelligence (AI), deep learning, such as convolutional neural networks (CNN) and residual networks (ResNet), has played a significant role in data-driven modeling.

[0004] While these methods can achieve high diagnostic performance, they rely heavily on a large number of labeled samples. However, due to the diversity of environmental conditions and fault types, as well as the low frequency and significant differences in faults, collecting sufficient labeled fault samples is both expensive and difficult. Furthermore, during the actual operation of industrial field equipment, it is impractical to manually adjust the system to failure modes and collect a large number of fault samples to train data-driven models, which can lead to power generation losses and safety hazards.

[0005] Recently, to address the problem of fault diagnosis in industrial field scenarios with limited sample sizes, numerous data-driven methods based on few-shot learning have been proposed, attempting to improve the problem of limited fault or labeled samples. These methods include federated learning (FL), generative adversarial networks (GANs), meta-learning (MeL), and siamese networks (SNs). Although these data-driven techniques have demonstrated remarkable effectiveness in fault diagnosis under specific circumstances, their application in real-world industrial field equipment still faces several unresolved challenges. First, model performance depends on the quality and quantity of initial samples. In cold-start scenarios with very few samples, existing techniques often struggle to achieve satisfactory results due to the lack of prior access to real-world samples of the fault type. Second, because industrial equipment data is significantly affected by environmental factors such as temperature and radiation, the applicability and accuracy of the models are largely limited by the sample richness. Third, model parameters are trained based on the numerical characteristics of specific data rather than knowledge of the fault mechanism, meaning the models are not universal. Once deployed to other systems and equipment, data collection in the new operating environment and equipment is required to retrain the models. Fourth, the electrical characteristics of industrial equipment are often overlooked. However, leveraging mechanistic prior knowledge can facilitate sample generation and assist fault diagnosis models.

[0006] Today, pre-trained large language models (LLMs) have demonstrated remarkable performance, becoming a powerful tool for natural language processing and generation tasks. For example, OpenAI's pre-trained generative transformers (GPTs) have trained billions of parameters on a large corpus and are able to deeply understand language semantics. LLMs technology has been applied to a variety of fields, including autonomous driving, code generation, industrial anomaly detection, aerospace assembly functional testing, assisted medical care, and personalized recommendations. Unlike data-driven methods that train networks by extracting features from samples, LLMs analyze and approach problems more from a semantic perspective rather than a numerical analysis perspective, that is, they are knowledge-driven.

[0007] As the scale of models increases and cost considerations arise, retraining large models in the field becomes difficult to achieve. Therefore, how to effectively utilize open source LLM to complete specific tasks becomes a key issue. One approach is comprehensive fine-tuning, which involves subsequently modifying all parameters and creating different models for various tasks. However, this technique often proves to be infeasible in practical applications. In addition to the costs associated with training, this field often lacks a large number of high-quality datasets, and general dataset training will lead to problems such as catastrophic forgetting of large models. Another approach is incremental adjustment, which involves updating only a small number of parameters of the LLM and freezing most of the other parameters, such as low-rank adaptation (LoRA). The outstanding feature of LoRA is branch fine-tuning, which can achieve fully fine-tuned performance in various tasks without changing the original model parameters. Summary of the Invention

[0008] In view of the above-mentioned defects of the prior art, the technical problems to be solved by the present invention include:

[0009] A new modeling paradigm is constructed by combining data-driven and knowledge-driven methods to comprehensively utilize the numerical and semantic features of samples to establish a general industrial intelligent fault diagnosis model.

[0010] To achieve the above object, the present invention provides an industrial fault diagnosis model modeling method, comprising the steps of:

[0011] S1, raw data preprocessing;

[0012] S2, establish a sample automatic generation and annotation mechanism based on the thinking chain;

[0013] S3, establish the state transition mapping layer;

[0014] S4, establish a fault sample filtering mechanism;

[0015] S5, fault diagnosis modeling based on LLMs;

[0016] S6, Method evaluation.

[0017] Furthermore, the raw data preprocessing includes: actual data collection and simulation model construction;

[0018] The actual data collection specifically refers to collecting real-time monitoring data through industrial equipment sensors, including but not limited to time series data such as vibration signals, temperature, pressure, current, voltage, and image data of equipment operating status;

[0019] The simulation model construction specifically involves building a high-fidelity simulation model based on the physical model and operating characteristics of the equipment to generate simulated monitoring data when the actual data set is insufficient.

[0020] Furthermore, the simulation model construction includes the steps of:

[0021] Remove noise by selecting low-pass, high-pass, or band-pass filtering technologies based on the frequency characteristics of key features of data from different industrial field equipment to reduce the impact of high-frequency noise;

[0022] Outlier elimination: using statistical methods to identify and process outliers in historical samples;

[0023] Missing data interpolation: For missing data, use the mean, median, or predicted value based on similar data to fill in the missing value;

[0024] Dimension reconstruction reduces the number of features through dimensionality reduction techniques while retaining the information of the original data as much as possible.

[0025] Furthermore, the establishment of a sample automatic generation and annotation mechanism based on thought chain includes full-scale model knowledge distillation and thought chain construction and generation and optimization of thought chain training samples.

[0026] Furthermore, the full-scale model knowledge distillation and thought chain construction includes:

[0027] Extract key features from device status data;

[0028] Based on the extracted features and combined with the domain knowledge base, logical reasoning is performed to generate a description text of the device status;

[0029] The generated description text is combined with the device status data to form automatically labeled training samples;

[0030] The generation and optimization of the thought chain training samples include:

[0031] Integrate multimodal information such as image data and time series data into the thought chain to ensure that the generated description text can fully reflect the device status;

[0032] By introducing noise and data enhancement technologies, we generate diverse device status data and combine it with thought chains to generate corresponding description text, ensuring the diversity of training samples and avoiding model overfitting.

[0033] An expert review mechanism is introduced to conduct quality assessment of the generated samples and optimize the logical reasoning process of the thinking chain based on feedback.

[0034] Furthermore, the establishment of the state transition mapping layer is specifically to further embed a state transition mapping layer on the basis of the existing multimodal large model; the state transition mapping layer represents the multivariate continuous function as a two-layer nested superposition of single-variable continuous functions, and improves the performance and interpretability of the network by replacing the weight parameters with a learnable single-variable function.

[0035] Furthermore, the fault sample filtering mechanism is established by designing three similarity evaluation indicators based on the mean, standard deviation and covariance between samples.

[0036] Furthermore, the establishment of the fault sample filtering mechanism is specifically as follows:

[0037] When the device collects a new sample X′, it first converts the Nor s Convert to the same working condition, expressed as Nor′, and define the IV vectors corresponding to samples X′ and Nor′ as p and q respectively;

[0038] Next, define the mean comparison function m(p, q) of the sample pair as follows:

[0039]

[0040] Among them, μ p and μ p is the average value of samples p and q, and C1 is a predefined constant value used to limit the function value range when the mean is close to zero;

[0041] The standard deviation comparison function s(p, q) of the sample pair is defined as follows:

[0042]

[0043] Among them, σ p and σ q is the standard deviation of samples p and q. Similar to C1, C2 is a predefined constant value used to limit the range of function values ​​when the mean is close to zero.

[0044] Next, the present invention defines the covariance comparison function c(p, q) of the sample pair as follows:

[0045]

[0046] By combining these three comparison functions, we can get the final similarity comparison function g:

[0047]

[0048] Among them, 1 indicates a fault, and 0 indicates that the system is operating normally;

[0049] When the similarity is lower than the threshold, the sample will be regarded as a fault type, and then the corresponding sample will be uploaded to the cloud server for further diagnosis.

[0050] Furthermore, the LLMs-based fault diagnosis modeling is specifically as follows: first, a LLMs training dataset is constructed based on previously generated virtual samples, and semantic templates are designed in combination with prior knowledge of fault characteristics; then, low-rank decomposition technology is introduced to achieve efficient fine-tuning of LLMs parameters based on supervised learning; finally, a universal industrial equipment fault diagnosis model is obtained.

[0051] Furthermore, the method evaluation includes,

[0052] The proposed method is comprehensively evaluated using the accuracy, macro-precision, macro-recall and macro-F1-Score indicators, which are calculated as follows:

[0053]

[0054] Among them, T p Indicates the number of samples that are actually positive among the samples predicted to be positive, F p Indicates the number of samples that are actually positive among the samples whose predicted results are negative; T N Indicates the number of samples that are actually negative among the samples predicted to be positive, F N It represents the number of samples that are actually negative among the samples predicted to be negative; in addition, the variable n represents the total count of different classification categories.

[0055] Compared with the existing technical solutions, the technical effects of the present invention are:

[0056] Most of the existing artificial intelligence-based fault diagnosis technologies are data-driven, which means that the model parameters are adjusted based on the numerical characteristics of specific data, ignoring the semantic characteristics of the samples and the knowledge of the fault mechanism, which makes the model non-universal. Once the model needs to be deployed on a new system or device, data collection and model training need to be carried out again, which consumes a lot of manpower, material resources and time costs. The present invention designs a new paradigm for fault diagnosis that combines data-driven and knowledge-driven, and proposes a general industrial intelligent fault diagnosis method. Utilizing the powerful natural language processing, generalization and image processing capabilities of the multimodal large model, the semantic information and numerical features are comprehensively utilized to establish a diagnostic model. This makes it no longer necessary for industrial sites to carry out tedious fault sample simulation collection work, and zero-sample modeling can be achieved. When the model is applied to other systems and equipment, there is no need to re-collect data and retrain the model, and it can be directly migrated because the model is universal.

[0057] There are many data modalities collected from industrial objects, and it is difficult and costly to accurately characterize the features of different types of data. Existing large-model training samples rely on artificial natural language descriptions to label samples. The effectiveness and construction efficiency of training samples are low, and there is a lack of effective large-model training mechanisms. The present invention proposes a new large-model training mechanism that combines knowledge distillation sample generation with feature enhancement RAG retrieval and correction. Since the number of parameters of the full-scale large model is large, the direct training cost is very high, so its general capabilities are used to extract and analyze the key features of multimodal data. Through the knowledge distillation of the full-scale large model, a thinking chain training sample that combines equipment status data such as images and time series data with natural language is constructed. The thinking chain ensures the correctness and effectiveness of the feature extraction logic by imitating the thinking mode of human thinking about problems. The automatic generation and annotation of large-model training samples is realized, the construction efficiency and accuracy of training samples are greatly improved, and the generalization and accuracy of the model after training are significantly improved.

[0058] Due to the complex working conditions at industrial sites, it is difficult to collect samples under various circumstances. The samples to be identified lack reference comparison under standard working conditions, which limits the accuracy and generalization of model diagnosis. The present invention proposes a new large-model multimodal data feature encoding layer, which enables it to achieve pan-working-condition multimodal adaptive feature fusion. On the basis of the existing multimodal large model encoder, a sample state correction mechanism is further embedded to ensure that the data samples of industrial equipment can convert the working conditions to the standard state at the same time as multimodal fusion, thereby overcoming the problem of insufficient sample richness. This significantly improves the generalization of the model.

[0059] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is the overall flow chart of the present invention.

[0061] Figure 2 It is the technical roadmap of large-scale fault diagnosis modeling of the present invention.

[0062] Figure 3 This is the framework diagram of the vertical field fault diagnosis large model training of the present invention. DETAILED DESCRIPTION

[0063] The following describes several preferred embodiments of the present invention with reference to the accompanying drawings to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0064] The technical solution of the present invention mainly includes six steps: S1 data preprocessing, S2 establishing a sample automatic generation and annotation mechanism based on the thinking chain, S3 establishing a state transition mapping layer, S4 establishing a fault sample filtering mechanism, S5 fault diagnosis modeling based on LLMs, and S6 method evaluation.

[0065] As attached Figure 1 The general industrial intelligent fault diagnosis method proposed in this invention adopts a large model technology route, such as Figure 2 To facilitate actual deployment and implementation, the present invention adopts an edge-cloud collaborative framework, as shown in Figure 3 As shown. Among them, the end side is responsible for real-time data collection and preprocessing, the edge side is responsible for virtual sample generation and fault sample filtering, and the cloud side is responsible for LLMs fine-tuning and fault diagnosis. The specific implementation steps of the present invention are described below:

[0066] S1: Raw data preprocessing

[0067] In industrial equipment monitoring scenarios, obtaining actual data sets is often difficult, especially due to the scarcity of fault samples. Furthermore, considering that data collection from actual industrial field equipment is subject to multiple influences, such as noise, packet loss and missing data, sampling frequency differences, and environmental factors, it is difficult to directly use the original samples in historical data for model training. Therefore, it is necessary to first design an effective data preprocessing method to perform data cleaning, outlier removal, missing data interpolation, dimensionality reconstruction, and data set partitioning on historical data. At the same time, it is deployed on the device side to process real-time data to facilitate subsequent algorithm execution. This mainly includes the following steps:

[0068] Data collection: The present invention proposes the following two data acquisition methods:

[0069] (1) Actual data collection: Real-time monitoring data is collected through industrial equipment sensors, including but not limited to vibration signals, temperature, pressure, current, voltage and other time series data, as well as image data of equipment operating status.

[0070] (2) Simulation model construction: When the actual data set is insufficient, a high-fidelity simulation model is constructed based on the physical model and operating characteristics of the equipment to generate simulated monitoring data. The simulation model must fully consider the dynamic behavior of the equipment under different operating conditions to ensure the diversity and authenticity of the generated data.

[0071] Remove noise by selecting low-pass, high-pass, or band-pass filtering technologies based on the frequency characteristics of key features of data from different industrial field equipment to reduce the impact of high-frequency noise;

[0072] Outlier elimination: using statistical methods such as the IQR (interquartile range) method to identify and process outliers in historical samples. For example, setting lower and upper limits for normal operating conditions for key equipment parameters, and values ​​outside this range will be replaced or deleted.

[0073] Missing data interpolation: For missing data, use the mean, median, or predicted value based on similar data to fill the missing value, thereby avoiding abnormal errors in the numerical calculation process.

[0074] Dimensionality reconstruction: Considering that there are too many features in the dataset or there is a high correlation between some features, dimensionality reduction techniques such as principal component analysis (PCA) are used to reduce the number of features while retaining the information of the original data as much as possible.

[0075] It's worth noting that since the input and output dimensions of the subsequent neural network model are fixed, the dimensions of the reconstructed samples need to remain consistent. Specifically, the dimensions of the cleaned raw data can be adjusted through methods such as resampling and bilinear interpolation. This also further eliminates redundant information from the samples, saving computing resources and improving model accuracy without compromising fault characteristics.

[0076] For devices that need to collect current and voltage, the present invention defines Nor s To specify the standard conditions, N represents the total number. Then, construct sample pairs, each sample pair is defined as {Nor s , Nor i}, i = 1, 2, ..., N, and i ≠ s. Each sample O in the dataset i will be rebuilt as:

[0077] O i =[V s ,I s ,T s ,T i ,L],

[0078] L=[I i , V i ],

[0079] Among them, V s , I s 、T s Respectively represent the voltage, current, and environmental vector of industrial equipment under specified standard conditions. i Represents the environment vector corresponding to the target sample i. The voltage and current target values ​​of the industrial equipment sample after state transition mapping are defined as L.

[0080] S2: Establish a sample automatic generation and annotation mechanism based on thought chain

[0081] Full-scale model knowledge distillation and thought chain construction

[0082] In order to combine the equipment status data with the natural language description, the present invention uses a full-scale large model (LLM) for knowledge distillation to construct a thinking chain training sample. First, a pre-trained full-scale large model (such as GPT, BERT, etc.) is used to extract key knowledge from text data such as professional field literature, equipment maintenance manuals, fault diagnosis reports, etc. to build a domain knowledge base. Through distillation technology, the knowledge of the large model is transferred to a small model in the vertical field to ensure the model's accurate understanding of the domain terminology and logical relationships. Afterwards, based on the thinking chain, the logical chain that imitates humans thinking about problems is imitated, and the equipment status data is combined with the natural language description.

[0083] Specifically, the thought chain is generated through the following steps:

[0084] Feature extraction: Extract key features from equipment status data, such as vibration spectrum, temperature change trend, etc.

[0085] Logical reasoning: Based on the extracted features and combined with the domain knowledge base, logical reasoning is performed to generate a descriptive text about the equipment status. For example, if an abnormally high vibration frequency is detected, the thought chain will generate a description such as "the equipment may have bearing wear or imbalance faults."

[0086] Annotation generation: The generated description text is combined with the device status data to form automatically annotated training samples. Each sample includes the device status data (image or time series data) and its corresponding natural language description.

[0087] Generation and optimization of thought chain training samples

[0088] In order to ensure the quality and diversity of thought chain generation samples, the present invention proposes the following optimization strategy:

[0089] Multimodal data fusion: Integrating multimodal information such as image data and time series data into thought chains ensures that the generated description text fully reflects the equipment status. For example, combining vibration signals and temperature data can generate a description such as "The equipment vibrates abnormally in a high-temperature environment, which may indicate poor lubrication or bearing wear."

[0090] Enhanced sample diversity: By introducing noise, data enhancement and other technologies, diverse device status data is generated, and corresponding descriptive text is generated in combination with thought chains to ensure the diversity of training samples and avoid model overfitting.

[0091] Quality Control and Feedback Mechanism: An expert review mechanism is introduced to evaluate the quality of generated samples and optimize the logical reasoning process of the thought chain based on feedback. In addition, automated evaluation metrics (such as BLEU and ROUGE) are used to quantitatively evaluate the generated description text to ensure its accuracy and readability.

[0092] S3: Establishing a state transition mapping layer

[0093] In the cold start zero-sample diagnosis scenario studied in the present invention, the industrial field historical data contains a large number of normal samples, but lacks fault type samples. In order to ensure that the final fault diagnosis model is applicable to any working conditions, for the labeled samples generated in S2, it is necessary to further construct an efficient and accurate working condition adaptive multimodal feature extraction and fusion mechanism based on the existing multimodal large model, so as to learn the influence of factors such as the environment on the sample fault characteristics and improve the applicability of the model. To this end, the present invention further embeds a state transition mapping layer on the basis of the existing multimodal large model VisionTransformer (ViT). The constructed state transition layer is a new type of neural network architecture, which improves the performance and interpretability of the network by replacing the weight parameters with learnable single-variable functions. Specifically, the network has the following characteristics:

[0094] A learnable activation function that is located on the edges (weights) of the network, not on the nodes.

[0095] (2) Parameterized univariate function: This network does not use traditional linear weights, but replaces each weight parameter with a parameterized univariate function, which is represented by a B-spline function in this invention.

[0096] This design change makes the constructed state transition layer superior to the traditional multilayer perceptron (MLP) in terms of accuracy and interpretability, especially in data fitting and partial differential equation solving tasks.

[0097] Specifically, the state transition layer proposed in the present invention represents a multivariate continuous function as two layers of nested superposition of single variable continuous functions. That is, any multidimensional function f(x) = f(x1, x2, ..., x n ) can be decomposed into a combination of multiple univariate functions:

[0098]

[0099] in, and

[0100] For the sample x generated in step 2, the output of the state transition layer is defined as:

[0101] y=(Φ L-1 °Φ L-2 °…°Φ1°Φ0)(x)

[0102] Where, ° represents Φ l By continuous multiplication of , we can further obtain:

[0103]

[0104] Among them, φ represents the activation function, l represents the number of network layers, and n l Indicates the width of the l-th layer network.

[0105] Furthermore, each φ is described as a parameterized univariate activation function:

[0106] φ(x)=c r r(x)+c B B(x)

[0107] in, represents the sigmoid activation function, represents the k-order B-spline activation function.

[0108] In the training paradigm of the state transition layer, the adjustable parameters represented by θ include c r 、c B and c i Given data {x i ,y i}, the goal is to reduce the deviation between the predicted results and the actual target value. The mean squared error loss function is used to evaluate the model performance:

[0109]

[0110] Where K is the total number of training data. Vector V i and I i Represent the true voltage and current values ​​of the target sample, respectively. and Represents the output of the state transition layer.

[0111] In order to further utilize the prior information that the predicted current and voltage values ​​must be non-negative, an additional negative logarithmic regularization term is introduced:

[0112]

[0113]

[0114] Among them, V j and I j Represents the prediction vector V i and I in predicted values ​​of voltage and current.

[0115] Therefore, the overall loss function is defined as:

[0116]

[0117] Here, λ is a parameter used to balance the model prediction accuracy and physical constraints.

[0118] It is worth noting that the Vit module of the traditional multimodal large model is only used to realize the numerical feature extraction of samples. It is unable to process data of systems of different scales and equipment operating in different working conditions, and the scope of application of the model will be limited by the limited training set samples. The present invention, by embedding a state transformation layer in the existing ViT module, can first realize the working condition migration of the input samples before multimodal feature extraction, and convert samples under any working condition to standard working conditions. Since the samples are all in the same working condition at this time, feature extraction and fusion will no longer be limited by the diversity of samples, thereby greatly improving the accuracy and generalization of the model.

[0119] S4: Establish a fault sample filtering mechanism

[0120] Considering that in the actual deployment and application of the method, the proportion of fault samples in the real-time data of industrial equipment is much lower than that of normal samples, that is, the fault frequency is generally low. In this case, frequently starting LLMs for algorithm inference will consume a lot of computing resources and also cause unnecessary time costs. To this end, the present invention further designs a fault sample filtering mechanism that can quickly distinguish fault samples from normal samples and achieve accurate binary classification of real-time data. This part is trained and deployed on the edge computing device.

[0121] Specifically, considering that most industrial field equipment is discrete data of time series or scattered sampling, the present invention designs three similarity evaluation indicators based on the mean, standard deviation and covariance between samples. Specifically, when the device collects a new sample X', first, based on the KAN model trained in step 2, the Nor s Convert it to the same working condition and express it as Nor′. Define the IV vectors corresponding to samples X′ and Nor′ as p and q respectively.

[0122] Next, the present invention defines the mean comparison function m(p, q) of the sample pair as follows:

[0123]

[0124]

[0125] Among them, μ p and μ pis the average value of samples p and q, and C1 is a predefined constant value used to limit the function value range when the mean is close to zero.

[0126] Furthermore, the present invention defines the standard deviation comparison function s(p, q) of a sample pair as follows:

[0127]

[0128] Among them, σ p and σ q is the standard deviation of samples p and q. Similar to C1, C2 is a predefined constant value used to limit the range of function values ​​when the mean is close to zero.

[0129] Next, the present invention defines the covariance comparison function c(p, q) of the sample pair as follows:

[0130]

[0131] By combining these three comparison functions, we can get the final similarity comparison function g:

[0132]

[0133] Here, 1 indicates a fault, and 0 indicates normal system operation. Obviously, when the similarity falls below the threshold, the sample is considered a fault type. The corresponding sample is then uploaded to the cloud server for further diagnosis.

[0134] S5: Fault diagnosis modeling based on LLMs

[0135] In the proposed LLMs fault diagnosis method, a training dataset for LLMs is first constructed based on previously generated virtual samples. Semantic templates are then designed based on prior knowledge of fault characteristics. Low-Rank Adaptation (LoRA) technology is then introduced to efficiently fine-tune LLMs parameters based on supervised learning. Finally, a universal industrial equipment fault diagnosis model is obtained. This part of the model training and deployment is completed on a cloud server. The specific implementation steps are as follows:

[0136] Remove the Prompt project to build a large model fine-tuning dataset:

[0137] In traditional data-driven fault diagnosis methods, features are usually automatically extracted by inputting data into machine learning models such as neural networks, and finally a diagnostic result is output. However, this process often only involves the numerical features of the data, while ignoring the prior knowledge of various fault modes. This results in the diagnostic model being applicable only to the industrial equipment system where the data source is located. Once it is applied to equipment in other systems, it will fail due to large numerical differences, and universal applicability cannot be achieved. However, multimodal LLMs that have been fine-tuned in vertical domains have prior knowledge of new domains and powerful image numerical analysis, semantic understanding, and generalization capabilities. This provides a new paradigm for establishing a universal industrial equipment fault diagnosis model with strong generalization.

[0138] The first step in enabling industrial equipment fault diagnosis with LLMs is to construct a data set from samples that are automatically generated and annotated by the thought chain according to a specific template. It is worth noting that traditional LLMs often require manual annotation of samples when fine-tuning large models, and multiple rounds of attempts are required to determine the final Prompt (this step is called Prompt engineering in the prior art). The Prompt engineering marks the professional domain knowledge of industrial equipment to samples through semantic descriptions. However, the existing technical route requires a lot of manpower and material resources when annotating samples, and the accuracy of sample annotation is difficult to guarantee. However, the present invention does not require additional Prompt engineering. This is because the large model distillation technology proposed in step two has completed the automatic generation and annotation of samples, and combined with the state transformation layer in step three, it can automatically and efficiently integrate the industrial equipment fault diagnosis task, sample multimodal fault feature information and auxiliary information.

[0139] Specifically, the prompt word template used for fine-tuning LLMs in the present invention consists of three parts: instructions, users, and assistants. First, the instruction part is used to describe contextual information, specifically the tasks to be performed and the user's expected behavior of the model. Second, the user part includes the input samples to be diagnosed and the semantic logic guidance, which corresponds to the output result of the sample automatically generated based on the thinking chain in step 2 after passing through the state transition layer in step 3. Third, the assistant part is the expected output / label for model fine-tuning, which corresponds to the sample annotation automatically generated based on the thinking chain in step 2.

[0140] LoRA-based LLMs vertical domain training:

[0141] Considering that developing new LLMs for industrial equipment fault diagnosis tasks requires a large amount of data and computing resources, this paper uses existing high-performance training technology LLMs for fine-tuning. LoRA is a method to optimize the fine-tuning of large pre-trained models by injecting trainable low-rank matrices into Transformer blocks, reducing the number of training parameters and GPU memory usage while maintaining inference speed. For the pre-trained weight matrix W0∈R d×k , can be updated through low-rank decomposition as:

[0142] W0+ΔW=W0+BA,

[0143] Where W0 is the frozen weight. B∈R d×r and A∈R r×k Both have learnable parameters, and r << min(d, k). The product of matrices A and B forms a low-rank matrix ΔW, which is added to the original weight matrix W0 to fine-tune the model. For the original forward transfer function h, the fine-tuning changes are as follows:

[0144] h=W0x+ΔWx=W0x+BAx

[0145] It is worth noting that before fine-tuning, matrix A is initialized with a random Gaussian and matrix B is initialized with zero. Therefore, training starts with W = 0 to ensure that the original output does not change when fine-tuning begins.

[0146] Specifically, the present invention fine-tunes LLMs through LoRA, and the main steps include:

[0147] Prepare the model and dataset: First, download an open-source multimodal large model from platforms such as Hugging Face and load it locally. Use the pip install command to install necessary libraries to resolve dependency issues, such as transformers, accelerate, tiktoken, einops, and scipy. These dependencies are used to load and test the model.

[0148] Register the dataset to LLaMFActory: Place the dataset in the data directory and edit the dataset_info.json file in the directory to register the dataset. This file contains all the dataset information, where the key is the dataset name and the value is the dataset configuration.

[0149] LoRA fine-tuning: Specify parameters such as model path, fine-tuning type (LoRA), template, data set, and execute the training command. During the fine-tuning process, analyze the loss curve to view the training status and decide whether to modify the parameters and re-fine-tune. During fine-tuning, create a LoRA configuration, specify the task type, target module, training mode, LoRA rank, LoRA scaling parameters, dropout probability of the LoRA layer, etc. In the present invention, the key parameters involved in LoRA fine-tuning are as follows:

[0150] 1) Logging steps = 10;

[0151] 2) Save steps = 500;

[0152] 3)Per device train batch size=1;

[0153] 4)Gradient accumulation steps=8;

[0154] 5) Learning rate = 1.0e-4;

[0155] 6) Num train epochs = 12.0;

[0156] 7)Lr scheduler type=cosine;

[0157] 8) Warmup ratio = 0.1;

[0158] 9) Eval steps = 500;

[0159] 10) Ddp timeout = 180000000;

[0160] 11) Val size = 0.1;

[0161] 12)Per device eval batch size=1.

[0162] LoRA weight merging: Merge the LoRA weights into the original model to obtain a new LLMs model after fine-tuning. This step is the key to LoRA fine-tuning, because LoRA trains "modified weights" on the basic model, so the final model needs to merge the pre-trained model and adapter weights into one model.

[0163] Model export and deployment: Export the fine-tuned LoRA weights and merge them with the original LLMs weights to form a new LLMs weight. After the merger is complete, use the fused large model for inference to verify the fine-tuning effect.

[0164] S6: Method evaluation

[0165] In this step, the present invention first introduces the method evaluation criteria and then introduces the detailed experimental configuration.

[0166] Method evaluation criteria:

[0167] Considering that the training of the present invention is a regression problem, the root mean square error (RMSE) is selected for verification.

[0168]

[0169] Where m represents the total number of test samples. i represents the real value vector of current and voltage under standard conditions. The output vector of the KAN model is defined as

[0170] For the fault diagnosis problem enabled by LLMs, this paper uses the accuracy (Acc), macro-precision (mP), macro-recall (mR), and macro-F1-Score (mF) indicators to comprehensively evaluate the proposed method. The calculation formula is as follows:

[0171]

[0172] Among them, T p Indicates the number of samples that are actually positive among the samples predicted to be positive, F p Indicates the number of samples that are actually positive among the samples predicted to be negative. N Indicates the number of samples that are actually negative among the samples predicted to be positive, F N Indicates the number of samples that are actually negative among the samples predicted to be negative. In addition, the variable n represents the total count of different classification categories.

[0173] Taking photovoltaic fault diagnosis as an example, we selected mainstream machine learning methods such as generative adversarial networks (GANs), convolutional neural networks (CNNs), residual networks (ResNets), extreme learning machines (ELMs), and Siamese networks (SNs) to compare with the proposed modeling method on the aforementioned evaluation metrics. It is worth noting that the method comparison includes two parts: a comparison of diagnostic results using simulated data and a comparison of diagnostic results using real data. In each comparison, all methods were tested under the same data conditions. The detailed comparison results are shown in Table 1:

[0174]

[0175]

[0176] Specific experimental configuration: In order to verify the method proposed in the present invention, we built a physical end-edge-cloud collaborative platform. Among them, the device end is a Raspberry Pi, which is used to simulate the industrial equipment end in the actual field, responsible for data preprocessing and transmitting test data to the edge end. The edge end is served by NVIDIA Jetson NX, which is responsible for sample semantic feature format conversion and fault sample screening. The cloud is supported by Tencent computing server GN10Xp.2XLARGE40 (equipped with Intel Xeon Cascade Lake 8255C CPU, 40GB memory and NVIDIA V10032G GPU), which is responsible for performing subsequent fine-tuning and reasoning operations of LLMs. Taking into account that the industrial equipment diagnosis task involves image analysis, numerical processing and semantic understanding, the present invention uses the Qwen2-VL-7B open source multimodal large model for verification.

[0177] The present invention relates to the industrial field and the fault diagnosis field, and in particular to a fault diagnosis method under the circumstances where industrial field fault samples are scarce and there are zero samples for cold start. The main purpose is to design a new fault diagnosis paradigm that combines data-driven and knowledge-driven approaches, and to propose a general industrial intelligent fault diagnosis method. In the first stage, relevant monitoring data of industrial equipment is collected (when there is a lack of actual data sets, a simulation model is selected as an alternative) and professional domain knowledge. Through the full-scale large-scale model knowledge distillation, a thinking chain training sample combining equipment status data such as images and time series data with natural language is constructed. The thinking chain ensures the correctness and effectiveness of the feature extraction logic by imitating the thinking mode of human thinking about problems, and realizes the automatic generation and annotation of samples and the construction of data sets for training vertical multimodal large language models (LLMs).

[0178] In the second stage, based on the existing multimodal large model encoder, a sample state correction mechanism is further embedded to ensure that the data samples of industrial equipment can be converted to the standard state during multimodal fusion, so as to facilitate the subsequent fine-tuning of LLMs and overcome the problem of insufficient sample richness.

[0179] In the third stage, a sample similarity evaluation index is designed to filter faulty samples and normal samples to improve the execution efficiency of the method and save computing and communication costs.

[0180] In the fourth phase, a large-scale model fine-tuning mechanism was designed to achieve efficient vertical domain model training. Learning maps image samples to semantic features, providing interpretable diagnostic results. The established universal industrial intelligent fault diagnosis model is not only applicable to cold start zero-sample scenarios in various environmental conditions, but also overcomes limitations imposed by system scale, environmental conditions, and topology. This addresses the limitations of cold start scenarios in industrial sites, such as the lack of pre-collected samples and the resulting low model accuracy and generalization. This is crucial for preventing economic losses and major safety incidents caused by untimely equipment failure detection.

[0181] This invention addresses the problem that the scarcity of fault samples in the extremely small sample size and cold start zero sample conditions in actual industrial sites seriously restricts the application of intelligent fault diagnosis methods, as well as the problems of low model accuracy, poor generalization, difficulty in application to complex environmental conditions, lack of interpretability, and lack of universality. A new paradigm for industrial equipment fault diagnosis that combines data-driven and knowledge-driven approaches is designed, and a universal industrial intelligent fault diagnosis method is proposed. In this invention, the historical data of industrial field equipment in the initial state contains only normal samples, while all fault types have no real samples. The modeling methods of the existing technology are unusable in this cold start zero sample situation. At the same time, although the existing technology has zero sample fault diagnosis solutions, the existing technology still relies on numerical feature extraction and ignores the generalization problem of the model, and cannot achieve high accuracy under complex and changing working conditions. In addition, the existing technology cannot synergistically utilize the operating mechanism of industrial equipment, sample numerical features, and semantic attributes to achieve universal fault diagnosis.

[0182] The universal fault diagnosis technology proposed in this paper can directly apply the model to systems of varying sizes, topologies, and device models, eliminating the need to repeatedly collect sample data and train fault diagnosis models for different application scenarios. This will significantly improve the efficiency of actual industrial field diagnosis. Furthermore, existing zero-sample fault diagnosis methods lack interpretability. The model simply outputs the diagnostic results of a sample without sufficient explanation for the underlying causes. This makes it difficult for on-site maintenance personnel to determine the validity of the diagnostic results, hindering subsequent maintenance.

[0183] To this end, the present invention first collects relevant monitoring data of industrial equipment (when there is a lack of actual data sets, a simulation model is constructed instead) and professional domain knowledge, and uses the full-scale large-scale model knowledge distillation to construct a thinking chain training sample that combines equipment status data such as images and time series data with natural language. The thinking chain ensures the correctness and effectiveness of the feature extraction logic by imitating the thinking mode of human thinking, and realizes the automatic generation and annotation of samples and the construction of data sets for training vertical field multimodal large language models (LLMs). Then, based on the existing multimodal large model encoder, a sample state correction mechanism is further embedded to ensure that the data samples of industrial equipment can convert the working conditions to the standard state while multimodally fusing, so as to fine-tune the LLMs and overcome the problem of insufficient sample richness. Afterwards, in order to save computing and communication resources, the present invention establishes a sample similarity evaluation index based on the mean, standard deviation and covariance between samples, and designs a screening mechanism for distinguishing fault samples from normal samples to improve the reasoning efficiency of LLMs. Finally, by carefully designing the prompt word engineering, LLMs can better learn the numerical and semantic features of samples, and through feature enhancement encoding, a plug-in knowledge vector library is constructed to achieve rapid iterative updates of industrial equipment knowledge, avoiding repeated training of the model and making the diagnosis results interpretable.

[0184] Therefore, based on the above analysis, in actual industrial sites, the general industrial intelligent zero-sample fault diagnosis method of the multimodal large model proposed in the present invention is not only applicable to cold start zero-sample scenarios under various environmental conditions, but also can overcome the limitations of system scale, environmental conditions and topological structure on model applicability, and solve the problem that samples cannot be collected in advance during cold start of industrial sites, and the model has low accuracy and poor generalization. It is of great significance for avoiding economic losses and major safety accidents caused by untimely detection of equipment failures.

[0185] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0186] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0187] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0188] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A modeling method for an industrial fault diagnosis model, characterized in that: Including steps: S1, raw data preprocessing; S2, establish a sample automatic generation and annotation mechanism based on the thinking chain; S3, establish the state transition mapping layer; S4, establish a fault sample filtering mechanism; S5, fault diagnosis modeling based on LLMs; S6, Method evaluation.

2. The industrial fault diagnosis modeling method according to claim 1, characterized in that: The raw data preprocessing includes: actual data collection and simulation model construction; The actual data collection specifically refers to collecting real-time monitoring data through industrial equipment sensors, including but not limited to time series data such as vibration signals, temperature, pressure, current, voltage, and image data of equipment operating status; The simulation model construction specifically involves building a high-fidelity simulation model based on the physical model and operating characteristics of the equipment to generate simulated monitoring data when the actual data set is insufficient.

3. The industrial fault diagnosis modeling method according to claim 1, characterized in that: The simulation model construction includes the following steps: Remove noise by selecting low-pass, high-pass, or band-pass filtering technologies based on the frequency characteristics of key features of data from different industrial field equipment to reduce the impact of high-frequency noise; Outlier elimination: using statistical methods to identify and process outliers in historical samples; Missing data interpolation: For missing data, use the mean, median, or predicted value based on similar data to fill in the missing value; Dimension reconstruction reduces the number of features through dimensionality reduction techniques while retaining the information of the original data as much as possible.

4. The industrial fault diagnosis modeling method according to claim 1, characterized in that: The establishment of a sample automatic generation and annotation mechanism based on thought chain includes full-scale model knowledge distillation and thought chain construction and the generation and optimization of thought chain training samples.

5. The industrial fault diagnosis modeling method according to claim 4, characterized in that: The full-scale model knowledge distillation and thinking chain construction include: Extract key features from device status data; Based on the extracted features and combined with the domain knowledge base, logical reasoning is performed to generate a description text of the device status; The generated description text is combined with the device status data to form automatically labeled training samples; The generation and optimization of the thought chain training samples include: Integrate multimodal information such as image data and time series data into the thought chain to ensure that the generated description text can fully reflect the device status; By introducing noise and data enhancement technologies, we generate diverse device status data and combine it with thought chains to generate corresponding description text, ensuring the diversity of training samples and avoiding model overfitting. An expert review mechanism is introduced to conduct quality assessment of the generated samples and optimize the logical reasoning process of the thinking chain based on feedback.

6. The industrial fault diagnosis modeling method according to claim 1, characterized in that: Specifically, the state transition mapping layer is established by further embedding a state transition mapping layer on the basis of the existing multimodal large model; the state transition mapping layer represents a multivariate continuous function as a two-layer nested superposition of a single-variable continuous function, and improves the performance and interpretability of the network by replacing the weight parameters with a learnable single-variable function.

7. The industrial fault diagnosis modeling method according to claim 1, characterized in that: The fault sample filtering mechanism is established by designing three similarity evaluation indicators based on the mean, standard deviation and covariance between samples.

8. The industrial fault diagnosis modeling method according to claim 7, characterized in that: The specific steps of establishing a fault sample filtering mechanism are as follows: When the device collects a new sample X′, it first converts the Nor s Convert to the same working condition, expressed as Nor′, and define the IV vectors corresponding to samples X′ and Nor′ as p and q respectively; Next, define the mean comparison function m(p,q) of the sample pair as follows: Among them, μ p and μ p is the average value of samples p and q, and c1 is a predefined constant value used to limit the function value range when the mean is close to zero; The standard deviation comparison function s(p, q) of the sample pair is defined as follows: Among them, σ p and σ q is the standard deviation of samples p and q. Similar to c1, c2 is a predefined constant value used to limit the range of function values ​​when the mean is close to zero. Next, the present invention defines the covariance comparison function c(p, q) of the sample pair as follows: By combining these three comparison functions, we can get the final similarity comparison function g: Among them, 1 indicates a fault, and 0 indicates that the system is operating normally; When the similarity is lower than the threshold, the sample will be regarded as a fault type, and then the corresponding sample will be uploaded to the cloud server for further diagnosis.

9. The industrial fault diagnosis modeling method according to claim 1, characterized in that: The LLMs-based fault diagnosis modeling method specifically involves first constructing a LLMs training dataset based on previously generated virtual samples, and designing semantic templates based on prior knowledge of fault characteristics. Then, low-rank decomposition technology is introduced to achieve efficient fine-tuning of LLMs parameters based on supervised learning. Finally, a universal industrial equipment fault diagnosis model is obtained.

10. The industrial fault diagnosis model building method according to claim 1, characterized in that: The method evaluation includes, The proposed method is comprehensively evaluated using the accuracy, macro-precision, macro-recall and macro-F1-Score indicators, which are calculated as follows: Among them, T P Indicates the number of samples that are actually positive among the samples predicted to be positive, F P Indicates the number of samples that are actually positive among the samples whose predicted results are negative; T N Indicates the number of samples that are actually negative among the samples predicted to be positive, F N It represents the number of samples that are actually negative among the samples predicted to be negative; in addition, the variable n represents the total count of different classification categories.

Images