An intelligent diagnosis method for large model knowledge annotation

Abstract

The present invention discloses an intelligent diagnosis method for large model knowledge annotation, which includes: knowledge injection into the large model to obtain formatted semantic knowledge of each fault category; using all fault data to build a self-supervised pre-trained network to train the encoder to extract class-independent features; building and training a healthy feature comparison and semantic knowledge prediction network; using the trained encoder and the healthy feature comparison and semantic knowledge prediction network to obtain the semantic knowledge of unseen category fault samples, adjusting the predicted semantic knowledge based on a dual-view dynamic semantic knowledge learning strategy, dividing the test data into high-probability samples and low-probability samples, adjusting the semantic knowledge of low-probability samples using the spatial distribution of high-probability samples, and collaboratively diagnosing the fault. The present invention can use professional documents to perform semantic knowledge annotation when fault data is limited, and adjust the spatial distribution of semantic knowledge to improve the accuracy and credibility of fault diagnosis.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial zero-sample fault diagnosis, and in particular to an intelligent diagnosis method for large model knowledge annotation. Background Art

[0002] With the continuous deepening of industrial automation and intelligence, fault diagnosis technology has become the key to ensuring the stable operation of industrial systems. Data-driven methods have shown remarkable performance in diagnosing visible faults, but these methods rely on a large number of labeled training samples. Collecting these labeled fault samples requires a lot of manpower and time resources. Since industrial machines are in a healthy state for most of their operating time, failures are relatively rare, especially in critical infrastructure and high-risk industries, which makes it difficult to obtain training data for specific faults, thus limiting the performance of traditional data-driven fault diagnosis methods.

[0003] Zero-shot learning, as an innovative learning paradigm, aims to enable the model to recognize new categories without visible samples. This paradigm shows great potential in the field of fault diagnosis, which can use the attributes and semantic information between categories to identify unseen fault modes. The goal of zero-shot fault diagnosis is to diagnose unseen faults when no training samples are available. Specifically, experts in various industrial fields describe and label known and unseen fault categories from multiple perspectives. This process provides rich semantic knowledge, such as fault attributes and fault descriptions. The mapping model between the semantic knowledge and samples obtained by training on known fault samples can be applied to unseen fault categories. Fault diagnosis is performed by comparing the knowledge predicted by the model with the semantic knowledge of unseen faults. This semantic knowledge learning method can provide strong support for intelligent maintenance of industrial systems.

[0004] As a powerful tool for natural language processing, large language models have made significant progress in understanding and generating natural language. These models are able to capture and model the complex structure and semantics of language, providing new possibilities for semantic knowledge learning. With the exponential growth of model parameters and the accumulation of large amounts of training data, large models are able to encode massive amounts of text information, thereby enhancing their ability to understand and generate human-like text based on specific requirements. However, for many domain-specific tasks, large models sometimes produce hallucinations, that is, incorrect and factually contradictory results. This is due to the limited domain-specific expertise in the training corpus of large models, which affects the reliability of large models in highly specialized vertical fields.

[0005] In order to improve the reliability of large models in specific domain tasks, the models need to be adapted and fine-tuned. This may involve further training the model with data from a specific domain, or developing new algorithms to identify and correct the hallucinations produced by the model. Through these methods, the application effect of large models in key areas such as fault diagnosis can be enhanced, and their practicality and accuracy in actual industrial environments can be improved. Summary of the invention

[0006] The purpose of the present invention is to provide an intelligent diagnosis method with large model knowledge annotation to solve the problems of lack of training data, high cost of manual annotation and poor effect of zero-sample fault diagnosis in industrial scene fault detection in the prior art. The present invention can use a large language model to annotate fault samples and use a dynamic semantic knowledge learning strategy to improve the accuracy of the zero-sample fault diagnosis model in the industrial process.

[0007] The object of the present invention is achieved through the following technical solution: an intelligent diagnosis method for large model knowledge annotation, comprising the following steps:

[0008] (1) Collect expert knowledge documents and write contextual prompts for the large language model to inject knowledge into the large language model and obtain formatted semantic knowledge corresponding to each fault category to construct training and test sets;

[0009] (2) Construct a self-supervised pre-training network, which includes an encoder, a decoder, a predictor, a classifier, and a gradient reversal layer, and use all fault samples in the training set and the test set for training. Take minimizing the first total loss function as the optimization goal, optimize the parameters of the encoder, decoder, predictor, classifier, and gradient reversal layer to obtain a trained encoder;

[0010] (3) Constructing a health feature comparison and semantic knowledge prediction network, and using the trained encoder to train it using the training set, with minimizing the second total loss function as the optimization goal, optimizing the parameters of the health feature comparison and semantic knowledge prediction network to obtain a trained health feature comparison and semantic knowledge prediction network;

[0011] (4) For the test samples in the test set, the trained health features are compared with the semantic knowledge prediction network to predict the corresponding semantic knowledge, and the predicted semantic knowledge is adjusted based on the dual-view dynamic semantic knowledge learning strategy to obtain the final classification result of fault diagnosis;

[0012] (5) When applied, a test sample of a fault category not seen in the test set is input into the trained encoder to obtain class-independent features. At the same time, the test sample and any healthy sample are input into the trained healthy feature comparison and semantic knowledge prediction network to obtain the predicted semantic knowledge corresponding to the test sample; and the predicted semantic knowledge is adjusted according to the dual-view dynamic semantic knowledge learning strategy to obtain the final classification result of the fault diagnosis corresponding to the test sample.

[0013] Furthermore, the step (1) includes the following sub-steps:

[0014] (1.1) Collect expert knowledge documents and write contextual prompts for the large language model based on the expert knowledge documents. The contextual prompts include professional documents, task requirements, and output demonstrations;

[0015] (1.2) Input contextual prompts into the large language model to inject knowledge into the large language model and obtain formatted semantic knowledge corresponding to each fault category output by the large language model, namely, the fault category-attribute matrix and the text description of the fault;

[0016] (1.3) Expert review of the formatted semantic knowledge corresponding to each fault category to remove unreasonable formatted semantic knowledge and adjust the contextual prompts to regain the corresponding formatted semantic knowledge;

[0017] (1.4) Use the pre-trained BERT model to convert the text description of the fault into a fault description text embedding vector of the same length;

[0018] (1.5) Construct training and test sets based on the fault category-attribute matrix, fault description text embedding vector, and collected data samples.

[0019] Furthermore, the construction of the training set and the test set specifically includes:

[0020] Denote the visible fault categories used for training as ,in Indicates the first visible fault category, represents the pth visible fault category, p is the total number of visible fault categories; the unseen fault category is represented as ,in Indicates the first unseen fault category, represents the qth unseen fault category, q is the total number of unseen fault categories, and ;

[0021] Each row of the fault category-attribute matrix represents the fault category, each column represents the fault attribute, and the value of each cell is used to indicate whether the fault category has the corresponding fault attribute; the category-attribute matrix of all L fault categories is expressed as ,in represents the fault attribute matrix of all visible fault categories S, represents the fault attribute matrix of all unseen fault categories, L=p+q, M represents the number of fault attributes;

[0022] The fault description text embedding vectors of all L fault categories are represented as ,in A matrix of fault description text embedding vectors representing all visible fault categories, represents the matrix composed of the fault description text embedding vectors of all unseen fault categories, and d represents the length of the fault description text embedding vector;

[0023] Fault class labels to be used for training Respectively with the category-attribute matrix and the matrix consisting of the fault description text embedding vector Merge to get the corresponding fault attribute label and fault description label ,in Represents the total number of original training samples;

[0024] For each original training sample of length t, a window length of The time sliding window with a sliding step of 1 divides it into multiple training samples, and finally obtains training samples, denoted as ; and health data samples Included in the training set;

[0025] The training set is represented as ,in, represents a data sample, represents the set of all fault samples after the time sliding window processing, represents the collection of healthy samples, represents the total number of all training samples after the time sliding window processing, and k represents the dimension of the sample features; Represents data samples The corresponding fault category, represents the set of all fault category labels used for training, Indicates health category label; Represents data samples The corresponding fault attribute label, Represents the set of all fault attribute labels; Represents data samples The corresponding fault description label, represents the set of all fault description labels; the test set is represented by ,in, represents the set of test samples containing unseen fault categories, Represents the total number of all test samples after the time sliding window processing; Represents the set of classification labels for all unseen fault categories.

[0026] Furthermore, the self-supervised pre-training network includes an encoder for extracting class-independent features, a decoder for reconstructing samples, a classifier and a gradient reversal layer for predicting data of the next time step of the input sample time window, and for determining whether the sample belongs to a visible fault category or an unseen fault category, wherein the encoder includes three layers of long short-term memory network layers, and the decoder, predictor and classifier all use a multi-layer perceptron model;

[0027] First, a random mask operation is performed on each fault sample to obtain a masked fault sample; then the masked fault sample is input into the encoder to obtain the encoded class-independent features; secondly, the encoded class-independent features are input into the decoder to obtain the decoded reconstructed samples; the encoded class-independent features are input into the predictor to obtain the predicted data of the next time step; the encoded class-independent features are input into the gradient reversal layer and then into the classifier to obtain the classification result of whether the fault sample belongs to the visible fault category or the unseen fault category.

[0028] Furthermore, the training is performed using all fault samples in the training set and the test set, with minimizing the first total loss function as the optimization goal, and optimizing the parameters of the encoder, decoder, predictor, classifier, and gradient reversal layer to obtain a trained encoder, specifically including:

[0029] Each fault sample in the training set and the test set Perform random masking operation to obtain masked fault samples;

[0030] The masked fault samples are input into the self-supervised pre-trained network to obtain the decoded reconstructed samples, the predicted data of the next time step, and the classification result of whether the fault samples belong to the visible fault category or the unseen fault category;

[0031] The loss function of the reconstruction task is calculated based on the decoded reconstructed samples and the corresponding original fault samples in the training set and test set, which is expressed as:

[0032]

[0033] in, represents the loss function of the reconstruction task, represents the expectation of the faulty samples, represents the original fault samples in the training set and the test set, represents the L2 norm, represents the masked fault samples, represents the encoder, represents a decoder;

[0034] The loss function of the prediction task is calculated based on the predicted data of the next time step and the corresponding real data of the next time step in the training set and the test set, which is expressed as:

[0035]

[0036] in, represents the loss function of the prediction task, represents the actual next time step data in the training set and the test set, represents the predictor;

[0037] The loss function of adversarial classification is calculated based on the classification result of whether the fault sample belongs to the visible fault category or the unseen fault category and the true label of whether the fault sample belongs to the visible fault category or the unseen fault category, which is expressed as:

[0038]

[0039] in, represents the loss function of adversarial classification, represents the cross entropy loss, represents the classifier, represents the gradient reversal layer, represents the gradient reversal factor, The true label indicating whether the fault sample belongs to a seen fault category or an unseen fault category;

[0040] The first total loss function is calculated according to the loss function of the reconstruction task, the loss function of the prediction task, and the loss function of the adversarial classification, which is expressed as:

[0041]

[0042] in, represents the first total loss function; taking minimizing the first total loss function as the optimization goal, the parameters of the encoder, decoder, predictor, classifier, and gradient reversal layer are optimized until a preset training round is reached or the first total loss function converges to obtain a trained encoder.

[0043] Furthermore, the health feature comparison and semantic knowledge prediction network includes a feature extractor, a time window normalization module, a fault attribute predictor and a fault description predictor, wherein the fault attribute predictor and the fault description predictor are both multi-layer perceptrons;

[0044] The fault samples are input into the trained encoder to extract the class-independent features of the fault samples; the fault samples and healthy samples are respectively input into the same feature extractor to obtain the corresponding fault sample features and healthy sample features; the fault sample features and healthy sample features are respectively input into the time window normalization module to obtain the aligned fault features and healthy features; the aligned fault features are subtracted from the healthy features to obtain the contrast features; the class-independent features, contrast features and aligned fault features are fused to obtain the fused features; the fused features are respectively input into the fault attribute predictor and the fault description predictor to predict the semantic knowledge corresponding to the fault samples, that is, to predict the corresponding fault attributes and fault description embedding vectors.

[0045] Further, the trained encoder is trained using a training set, with minimization of the second total loss function as the optimization goal, and the parameters of the health feature comparison and semantic knowledge prediction network are optimized to obtain a trained health feature comparison and semantic knowledge prediction network, specifically including:

[0046] The fault samples in the training set are input into the trained encoder, and the fault samples in the training set and the healthy samples in the training set are input into the health feature comparison and semantic knowledge prediction network to predict the corresponding fault attributes and fault description embedding vectors;

[0047] The fault attribute prediction loss is calculated based on the predicted fault attributes and the corresponding fault attribute labels in the training set, which is expressed as:

[0048]

[0049] in, represents the fault attribute prediction loss, represents the expectation of the data sample x in the training set, represents the fault attribute predictor, Represents the fault attribute label corresponding to sample x in the training set The kth value of ;

[0050] The dot product similarity between the predicted fault description embedding vector and the fault description text embedding vector of all fault categories is calculated as the predicted probability of each fault category. The fault description prediction loss is calculated based on the predicted probability of each fault category and the corresponding fault category label in the training set, which is expressed as:

[0051]

[0052] in, represents the fault description prediction loss, represents the fault description predictor, D is the fault description text embedding vector of all fault categories, and y represents the fault category label corresponding to sample x in the training set;

[0053] The second total loss function is calculated based on the fault attribute prediction loss and the fault description prediction loss, which is expressed as:

[0054]

[0055] in, represents the second total loss function; taking minimizing the second total loss function as the optimization goal, optimizing the parameters of the feature extractor, the time window normalization module, the fault attribute predictor and the fault description predictor until the preset training rounds are reached or the second total loss function converges, so as to obtain the trained health feature comparison and semantic knowledge prediction network.

[0056] Furthermore, the step (4) includes the following sub-steps:

[0057] (4.1) For the fault attribute prediction branch in the trained health feature comparison and semantic knowledge prediction network, the classification results of the test samples are calculated by The distance is determined by applying The classification probability of all unseen fault categories is obtained by calculation, and the specific calculation formula is:

[0058]

[0059]

[0060] in, represents the fusion features of the test sample, represents the fault attribute predictor, Fault attribute matrix representing unseen fault categories The fault attribute corresponding to the i-th fault category in is, Represents the relationship between the predicted fault attribute corresponding to the test sample and the fault attribute vector of the i-th unseen fault category distance; exp is the exponential function, j represents the sequence number of the fault attribute vector of the unseen fault category, It represents the probability that the predicted fault attribute corresponding to the test sample is the fault attribute of the i-th unseen fault category, Represents the probability distribution of test samples on the fault attribute prediction branch;

[0061] (4.2) For the fault description prediction branch in the trained health feature comparison and semantic knowledge prediction network, based on dot product similarity, by applying The probability of each test sample belonging to each unseen fault category is obtained by calculation. The specific calculation formula is:

[0062]

[0063]

[0064] in, represents the fault description predictor, Fault description matrix representing unseen fault categories The fault description text embedding vector corresponding to the i-th fault category in , represents the dot product similarity between the predicted fault description embedding vector and the fault description text embedding vector of the i-th unseen fault category; represents the probability that the predicted fault description embedding vector corresponding to the test sample is the fault description text embedding vector of the i-th unseen fault category, Represents the probability distribution of test samples on the fault description prediction branch;

[0065] (4.3) For each test sample in the test set , if the predicted categories from the fault attribute prediction branch and the fault description prediction branch are the same, and both probabilities are greater than the preset probability threshold , then the predicted category is the final classification result of the test sample, and the test sample is called a high-probability test sample; the set of all high-probability test samples is expressed as , the remaining test samples in the test set are called low-probability test samples, and their set is expressed as ;

[0066] (4.4) Using high-probability test samples as references to adjust the semantic knowledge of low-probability test samples to obtain adjusted fault attributes and fault description embedding vectors;

[0067] (4.5) Using the adjusted fault attribute and fault description embedding vectors, repeat steps (4.1) and (4.2) to recalculate the probabilities of low-probability test samples on the fault attribute prediction branch and the fault description prediction branch to obtain the final classification result of fault diagnosis.

[0068] Furthermore, the use of high-probability test samples as references to adjust the semantic knowledge of low-probability test samples specifically includes:

[0069] Based on the predicted categories of high-probability test samples, the average value of the predicted fault attributes of all high-probability test samples in each unseen fault category is calculated as the predicted semantic knowledge center of the fault attribute, and the average value of the predicted fault description embedding vector of all high-probability test samples in each unseen fault category is calculated as the predicted semantic knowledge center of the fault description embedding vector. The calculation formulas are:

[0070]

[0071]

[0072] in, The predictive semantic knowledge center representing fault attributes, represents the predictive semantic knowledge center of the fault description embedding vector, Indicates a high probability test sample The fusion characteristics of Indicates A high probability test sample, Represents the total number of all high-probability test samples;

[0073] The predicted semantic knowledge center of the fault attribute and the predicted semantic knowledge center of the fault description embedding vector are used to adjust the predicted fault attribute and fault description embedding vector corresponding to the low probability test sample to obtain the adjusted fault attribute and fault description embedding vector, which is expressed as:

[0074]

[0075]

[0076] in, represents the adjusted fault attribute, represents the adjusted fault description embedding vector, Represents a low probability test sample of the fault attribute predictor output The corresponding predicted fault attributes, Represents a low probability test sample of the fault description predictor output The corresponding predicted fault description embedding vector, Indicates A low probability test sample.

[0077] Furthermore, the step (5) specifically includes:

[0078] When applied, the test sample of the unseen fault category in the test set is input into the trained encoder to obtain class-independent features; the test sample of the unseen fault category and any healthy sample are respectively input into the trained feature extractor and time window normalization module and then subtracted to obtain contrast features; the class-independent features, contrast features and aligned fault features are fused to obtain fused features; the fused features are respectively input into the trained fault attribute predictor and fault description predictor to obtain the predicted fault attributes and fault description embedding vectors corresponding to the test sample; the predicted fault attributes and fault description embedding vectors corresponding to the test sample are adjusted based on the dual-view dynamic semantic knowledge learning strategy to obtain the adjusted fault attributes and fault description embedding vectors; and the final prediction result of the fault attributes is calculated based on the adjusted fault attributes, which is expressed as:

[0079]

[0080]

[0081] in, is the final prediction result of the fault attribute; the final prediction result of the fault description embedding vector is calculated according to the adjusted fault description embedding vector, which is expressed as:

[0082]

[0083] in, is the final prediction result of the fault description embedding vector; if the final prediction result of the fault attribute Final prediction result with fault description embedding vector If the results are the same, the final classification result of the fault diagnosis corresponding to the test sample is directly output; if the final prediction result of the fault attribute is Final prediction result with fault description embedding vector If different, take and The category corresponding to the maximum probability value in is the final classification result of the fault diagnosis corresponding to the test sample.

[0084] Compared with the prior art, the present invention has the following beneficial effects:

[0085] (1) To address the high cost of experts labeling semantic knowledge information for unseen fault categories, the present invention designs a semantic knowledge labeling method driven by a large language model. By injecting knowledge into the large language model through professional documents, it assists in generating semantic knowledge of fault categories, which helps to reduce labeling costs.

[0086] (2) To address the problem of lack of unseen fault samples and the model's tendency to overfit the seen fault categories, the present invention designs a class-independent feature extraction method based on a self-supervised pre-trained network. Class-independent features are extracted in parallel through three tasks: sample reconstruction, time series prediction, and adversarial discrimination, so that the model can better model fault characteristics.

[0087] (3) The present invention constructs a health feature comparison and semantic knowledge prediction network, fully utilizes the complementary features of healthy samples and fault samples, and obtains better semantic knowledge prediction effects through feature fusion.

[0088] (4) To address the problem that the model's prediction results for unseen categories deviate from the actual labels, the present invention designs a dynamic semantic knowledge learning strategy, which divides the data into high-probability samples and low-probability samples from two aspects: attribute prediction and fault description text embedding. The semantic knowledge distribution of high-probability samples is used to adjust the classification results of low-probability samples.

[0089] (5) The present invention can use a large language model to annotate fault samples, use professional documents to annotate semantic knowledge when fault data is limited, and adjust the spatial distribution of semantic knowledge based on the prediction results. It uses a dynamic semantic knowledge learning strategy to improve the accuracy of the zero-sample fault diagnosis model in the industrial process, effectively improving the accuracy and reliability of fault diagnosis, providing support for safe production, and achieving better practical results. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 A flow chart of the intelligent diagnosis method for large model knowledge annotation of the present invention;

[0091] Figure 2 A flowchart of the semantic knowledge annotation and knowledge embedding method driven by a large language model of the present invention;

[0092] Figure 3 is a flow chart of the architecture of the self-supervised pre-training network of the present invention;

[0093] Figure 4 The architecture flow chart of the health feature comparison and semantic knowledge prediction network of the present invention;

[0094] Figure 5 is a flow chart of the dual-view dynamic semantic knowledge learning strategy of the present invention;

[0095] Figure 6 This is an architectural flow chart of the intelligent diagnosis method annotated with large model knowledge of the present invention when applied to a specific scenario. DETAILED DESCRIPTION

[0096] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0097] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0098] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0099] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0100] See also Figure 1 The intelligent diagnosis method of large model knowledge annotation of the present invention specifically includes the following steps:

[0101] (1) Collect expert knowledge documents and write contextual prompts for the large language model (abbreviated as the large model) to inject knowledge into the large language model and obtain formatted semantic knowledge corresponding to each fault category to construct training sets and test sets.

[0102] Specifically, this embodiment designs a semantic knowledge annotation and knowledge embedding method driven by a large language model to implement the above process, such as Figure 2 As shown, it specifically includes the following sub-steps:

[0103] (1.1) Collect expert knowledge documents and write contextual prompts for the large language model based on the expert knowledge documents. The contextual prompts include professional documents, task requirements and output demonstrations. For example, "You are an expert in ***. Please label the possible fault attributes and fault descriptions for the industrial process according to the relevant documents. The specific task requirements are as follows: <Task requirements>, and the output example is <Output example>". Among them, professional documents contain knowledge accumulated from industrial processes, such as fault symptoms and maintenance strategies. This knowledge is used to provide knowledge for the large language model to solve the problem of large model hallucination; task requirements specify formatted semantic knowledge for zero-shot diagnosis, and use natural language to describe the specific format and content prompts of annotations to unify the output of the large language model; the output demonstration is the reference content provided to the large language model based on the expert documents, which includes which fault attributes the experts are more interested in, standard output formats and language specification examples, etc.

[0104] (1.2) Inputting contextual cues into the large language model to inject knowledge into the large language model can eliminate the large model hallucination problem. The large language model can annotate formatted semantic knowledge for each fault category based on unstructured professional documents, and obtain the formatted semantic knowledge corresponding to each fault category output by the large language model, that is, the fault category-attribute matrix and the fault text description. The fault category-attribute matrix and the fault text description are used as semantic knowledge to assist zero-sample fault diagnosis.

[0105] (1.3) Perform expert review on the formatted semantic knowledge corresponding to each fault category to remove unreasonable formatted semantic knowledge and adjust the contextual prompts to regain the corresponding formatted semantic knowledge.

[0106] (1.4) Use the pre-trained BERT (Bidirectional Encoder Representations from Transformer) model to convert the text description of the fault into a fault description text embedding vector of the same length.

[0107] It should be understood that this embodiment uses an open source pre-trained BERT model to convert the text description of the fault into a fault description text embedding vector of the same length.

[0108] (1.5) Construct training and test sets based on the fault category-attribute matrix, fault description text embedding vector, and collected data samples.

[0109] Specifically, the visible fault categories used for training are represented as ,in Indicates the first visible fault category, represents the pth visible fault category, p is the total number of visible fault categories; the unseen fault category is represented as ,in Indicates the first unseen fault category, represents the qth unseen fault category, q is the total number of unseen fault categories, and Each row of the fault category-attribute matrix represents the fault category, each column represents the fault attribute such as the fault cause or symptom, and the value of each cell is used to indicate whether the fault category has the corresponding fault attribute. The category-attribute matrix of all L fault categories is expressed as ,in represents the fault attribute matrix of all visible fault categories S, represents the fault attribute matrix of all unseen fault categories, L=p+q, and M represents the number of fault attributes. The text description of the fault is a natural language description of each fault category, including the fault mechanism, the impact caused by the fault, and the maintenance strategy. In this embodiment, the pre-trained BERT model is used to convert the text description of the fault into a fault description text embedding vector of the same length d. The fault description text embedding vectors of all L fault categories are represented as ,in A matrix of fault description text embedding vectors representing all visible fault categories, represents the matrix composed of the fault description text embedding vectors of all unseen fault categories, and d represents the length of the fault description text embedding vector. The fault category labels used for training With the category-attribute matrix and the matrix consisting of the fault description text embedding vector Merge to get the corresponding fault attribute label and fault description label ,in Represents the total number of original training samples. For each original training sample of length t, a window length of The time sliding window with a sliding step of 1 divides it into multiple training samples, and finally obtains training samples, denoted as In addition, since the device is often running in a healthy state, this embodiment also includes health data samples Incorporated into the model training process, its number is equivalent to the average number of samples of visible fault categories. In summary, the training set is represented as ,in, represents a data sample, represents the set of all fault samples after the time sliding window processing, represents the collection of healthy samples, represents the total number of all training samples after the time sliding window processing, and k represents the dimension of the sample features; Represents data samples The corresponding fault category, represents the set of all fault category labels used for training, Indicates health category label; Represents data samples The corresponding fault attribute label, Represents the set of all fault attribute labels; Represents data samples The corresponding fault description label, represents the set of all fault description labels. The test set is represented by ,in, represents the set of test samples containing unseen fault categories, Represents the total number of all test samples after the time sliding window processing; Represents the set of classification labels for all unseen fault categories.

[0110] (2) Construct a self-supervised pre-trained network, which includes an encoder, a decoder, a predictor, a classifier, and a gradient reversal layer. Use all fault samples in the training set and the test set for training. Take minimizing the first total loss function as the optimization goal, optimize the parameters of the encoder, decoder, predictor, classifier, and gradient reversal layer to obtain a trained encoder.

[0111] In this embodiment, the self-supervised pre-training network includes an encoder for extracting class-independent features, a decoder for reconstructing samples, a classifier and a gradient reversal layer for predicting the data of the next time step of the input sample time window, and for determining whether the sample belongs to a visible fault category or an unseen fault category, such as Figure 3 As shown in the figure, the encoder includes three long short-term memory (LSTM) layers, and the decoder, predictor, and classifier all use a multi-layer perceptron model (MLP).

[0112] Specifically, for each time window of each fault sample , first perform a random mask operation on it, recorded as , specifically, for the two-dimensional tensor matrix Randomly select 20% of the elements in the image and assign them to 0. Perform the same operation on all time windows to obtain the masked fault samples, which are expressed as:

[0113]

[0114] in, represents the masked fault samples, represents the fault samples in the training set and the test set, Represents a random mask operation.

[0115] The masked fault samples are then input into the encoder to obtain the encoded class-independent features, expressed as:

[0116]

[0117] Among them, h represents the encoded class-independent features, represents the encoder, represents the parameters of the encoder, and H represents the dimension of the hidden state of the LSTM layer. Afterwards, the encoded class-independent features are used for time series reconstruction, time series prediction, and adversarial classification tasks, respectively, and class-independent features can be learned.

[0118] Based on the encoded class-independent features, the time series reconstruction task is performed. Specifically, the encoded class-independent features are input into the decoder to obtain a reconstructed sample sequence with the same scale as the input sample time series, expressed as:

[0119]

[0120] in, represents the reconstructed samples after decoding, Represents a decoder, Represents the parameters of the decoder.

[0121] Based on the encoded class-independent features, the time series prediction task is performed so that the features learned by the encoder can model the long-term dependency of the time series. Specifically, the encoded class-independent features are input into the predictor, and the encoded class-independent features of the current time series are used to predict the data of the next time step, and the predicted data of the next time step is obtained, which is expressed as:

[0122]

[0123] in, represents the data of the next time step to be predicted, represents the predictor, Represents the parameters of the predictor. Since only the data of the next time step is predicted, The dimension of is consistent with the dimension of the input sample, which is k.

[0124] Based on the encoded class-independent features, an adversarial classification task is performed. Specifically, the gradient reversal layer and the classifier are regarded as adversarial classification networks, and the features of the visible fault category and the features of the unseen fault category are regarded as two different distributions. The adversarial classification network is designed so that the encoder can extract consistent features in different situations where the fault is visible or not. The gradient reversal layer is used to prevent overfitting of visible faults. The calculation formula is:

[0125]

[0126] in, represents the gradient reversal layer, Represents the gradient reversal factor, which is set to 0.5 in this embodiment, and I represents the unit matrix. Specifically, in the forward propagation, i.e., model inference process, the feature input remains unchanged, and its gradient is reversed when the model parameters are updated in the back propagation. In this process, the optimization goal of the encoder is opposite to that of the classifier. The gradient reversal layer enables the classifier to classify the extracted features. The training goal of the encoder is to make it difficult for the classifier to classify the visible fault category, so that the encoder confuses the visible fault category input with the unseen fault category input, learns class-independent features, and eliminates the feature distribution differences between the visible fault category and the unseen fault category. The encoded class-independent features are input into the gradient reversal layer and then into the classifier for classification to obtain the classification result of whether the fault sample belongs to the visible fault category or the unseen fault category.

[0127] It should be understood that, in this embodiment, all fault samples in the training set and the test set, including visible fault categories and unseen fault categories, are used to perform self-supervised training on the self-supervised pre-trained network, so that the self-supervised pre-trained network can perform unified class-independent feature learning on visible fault categories and unseen fault categories, thereby reducing the performance degradation caused by overfitting of visible faults.

[0128] In this embodiment, all fault samples in the training set and the test set are used for training, with minimizing the first total loss function as the optimization goal, and the parameters of the encoder, decoder, predictor, classifier and gradient reversal layer are optimized to obtain a trained encoder, specifically including: each fault sample in the training set and the test set is Perform random masking operations to obtain masked fault samples; input the masked fault samples into the self-supervised pre-training network to obtain decoded reconstructed samples, predicted data for the next time step, and classification results of whether the fault samples belong to the visible fault category or the unseen fault category. The loss function of the reconstruction task is calculated based on the decoded reconstructed samples and the corresponding original fault samples in the training set and test set, which is expressed as:

[0129]

[0130] in, represents the loss function of the reconstruction task, represents the expectation of the faulty samples, represents the original fault samples in the training set and the test set, Represents the L2 norm. The loss function of the prediction task is calculated based on the predicted data of the next time step and the corresponding real data of the next time step in the training set and the test set, which is expressed as:

[0131]

[0132] in, represents the loss function of the prediction task, Represents the real next time step data in the training set and the test set. The loss function of adversarial classification is calculated based on the classification result of whether the fault sample belongs to the visible fault category or the unseen fault category and the real label of whether the fault sample belongs to the visible fault category or the unseen fault category, which is expressed as:

[0133]

[0134] in, represents the loss function of adversarial classification, represents the cross entropy loss, represents the classifier, The true label indicating whether the fault sample belongs to the visible fault category or the unseen fault category is determined based on whether the fault sample comes from the training set or the test set. The first total loss function is calculated based on the loss function of the reconstruction task, the loss function of the prediction task, and the loss function of the adversarial classification, and is expressed as:

[0135]

[0136] in, represents the first total loss function; taking minimizing the first total loss function as the optimization goal, the parameters of the encoder, decoder, predictor, classifier, and gradient reversal layer are optimized until a preset training round is reached or the first total loss function converges to obtain a trained encoder.

[0137] (3) Construct a health feature comparison and semantic knowledge prediction network, and use the trained encoder to train it using the training set, with the minimization of the second total loss function as the optimization goal, optimize the parameters of the health feature comparison and semantic knowledge prediction network to obtain a trained health feature comparison and semantic knowledge prediction network.

[0138] In this embodiment, the health feature comparison and semantic knowledge prediction network includes a feature extractor, a time window normalization module, a fault attribute predictor and a fault description predictor. Figure 4 As shown, both the fault attribute predictor and the fault description predictor use multi-layer perceptrons, and their specific parameters are different.

[0139] Specifically, the fault sample is input into the encoder trained in step (2) to extract the class-independent features of the fault sample; the fault sample and the healthy sample are respectively input into the same feature extractor to obtain the corresponding fault sample features and healthy sample features; the fault sample features and the healthy sample features are respectively input into the time window normalization module to align the fault sample features and the healthy sample features with the class-independent features, and obtain the aligned fault features and healthy features, so that the features extracted by the feature extractor conform to the class-independent feature distribution; the aligned fault features and the healthy features are subtracted to obtain the contrast features; the class-independent features, the contrast features and the aligned fault features are fused to obtain the fused features, which are expressed as:

[0140]

[0141] in, represents the fusion feature, represents feature concatenation, represents the class-independent features encoded by the encoder, Represents contrast characteristics, represents the fault characteristics after alignment, represents the fault sample characteristics, represents the feature extractor; the fused features are respectively input into the fault attribute predictor and the fault description predictor to predict the semantic knowledge corresponding to the fault sample, that is, the corresponding fault attribute and fault description embedding vector are predicted; wherein the fault attribute is predicted from the feature perspective.

[0142] It should be noted that the encoder trained in step (2) can be used to obtain the class-independent features of the fault samples and calculate the data distribution of the class-independent fault features.

[0143] Furthermore, the time window normalization module can be used to align the data distribution of the features extracted by the feature extractor and the class-independent features encoded by the encoder. Specifically, the fault sample features are input into the time window normalization module so that the fault sample features extracted by the feature extractor conform to the class-independent feature distribution. Correspondingly, the calculation formula of the time window normalization module is:

[0144]

[0145] in, represents the time window normalization module, represents the fault sample features extracted by the feature extractor, represents the feature extractor, represents a fault sample, represents the class-independent features encoded by the encoder, represents the encoder, and They represent the standard deviation and mean calculated along the time window dimension of each sample, and their calculation formulas are:

[0146]

[0147]

[0148] in, is the length of the time window. Similarly, the healthy sample features extracted from the healthy samples are also processed by the time window normalization module, that is, the healthy sample features are input into the time window normalization module, and its calculation formula is expressed as:

[0149]

[0150] in, Represents a healthy sample. Then, in the feature dimension, the aligned fault feature and the healthy feature obtained after the time window normalization module are subtracted to obtain the comparison feature .

[0151] In this embodiment, the trained encoder is used to train the encoder using the training set, and the parameters of the health feature comparison and semantic knowledge prediction network are optimized with the optimization goal of minimizing the second total loss function to obtain the trained health feature comparison and semantic knowledge prediction network, specifically including: inputting the fault samples in the training set into the trained encoder, and at the same time inputting the fault samples in the training set and the healthy samples in the training set into the health feature comparison and semantic knowledge prediction network to predict the corresponding fault attributes and fault description embedding vectors. The fault attribute prediction loss is calculated based on the predicted fault attributes and the corresponding fault attribute labels in the training set, which is expressed as:

[0152]

[0153] in, represents the fault attribute prediction loss, represents the expectation of the data sample x in the training set, represents the fault attribute predictor, Represents the fault attribute label corresponding to sample x in the training set The kth value of . The dot product similarity between the predicted fault description embedding vector and the fault description text embedding vector of all fault categories is calculated as the predicted probability of each fault category. The fault description prediction loss is calculated based on the predicted probability of each fault category and the corresponding fault category label in the training set, which is expressed as:

[0154]

[0155] in, represents the fault description prediction loss, represents the fault description predictor, D is the fault description text embedding vector of all fault categories, and y represents the fault category label corresponding to sample x in the training set. The second total loss function is calculated based on the fault attribute prediction loss and the fault description prediction loss, which is expressed as:

[0156]

[0157] in, Represents the second total loss function. Taking minimizing the second total loss function as the optimization goal, the parameters of the feature extractor, time window normalization module, fault attribute predictor, and fault description predictor are optimized until the preset training rounds are reached or the second total loss function converges, so as to obtain the trained health feature comparison and semantic knowledge prediction network.

[0158] In addition, during the testing phase, for each test sample On the one hand, based on the fault attribute prediction branch, its attribute can be predicted , and calculate the attribute matrix of the unseen fault categories of rows distance, Zhongyu The fault category of the closest row is the predicted classification result of the fault attribute prediction branch. On the other hand, based on the fault description prediction branch, the predicted fault description text embedding vector can be calculated. Fault text description text embedding vectors with all unseen fault categories The dot product similarity between Zhongyu The fault category corresponding to the row with the largest dot product similarity is the classification result corresponding to the fault description prediction branch. Unseen fault categories and Fault attributes, prediction attributes With Unseen Fault Types The L1 distance is:

[0159]

[0160] Predicted fault description text embedding vector With Unseen Fault Types The dot product similarity is:

[0161]

[0162] in, The dimension of the fault description text embedding vector.

[0163] (4) For the test samples in the test set, the trained health features are compared with the semantic knowledge corresponding to the prediction of the semantic knowledge prediction network, and the predicted semantic knowledge is adjusted based on the dual-view dynamic semantic knowledge learning strategy to obtain the final classification result of fault diagnosis.

[0164] In this embodiment, during the training process of the above step (3), there is a lack of training samples of unseen fault categories, and the health feature comparison and semantic knowledge prediction network only learns from the training samples of visible fault categories, resulting in poor zero-sample fault diagnosis performance. Therefore, in the current step, it is necessary to use the unseen fault information in the unlabeled test samples to perform semantic knowledge adjustment.

[0165] like Figure 5 As shown, it specifically includes the following sub-steps:

[0166] (4.1) For the fault attribute prediction branch in the trained health feature comparison and semantic knowledge prediction network, the classification results of the test samples are calculated by The distance is determined by applying The classification probability of all unseen fault categories is obtained by calculation, and the specific calculation formula is:

[0167]

[0168]

[0169] in, represents the fusion features of the test sample, represents the fault attribute predictor, Fault attribute matrix representing unseen fault categories The fault attribute corresponding to the i-th fault category in is, Represents the relationship between the predicted fault attribute corresponding to the test sample and the fault attribute vector of the i-th unseen fault category distance; exp is the exponential function, j represents the serial number of the fault attribute vector of the unseen fault category, It represents the probability that the predicted fault attribute corresponding to the test sample is the fault attribute of the i-th unseen fault category, Represents the probability distribution of test samples on the fault attribute prediction branch.

[0170] (4.2) For the fault description prediction branch in the trained health feature comparison and semantic knowledge prediction network, based on dot product similarity, by applying The probability of each test sample belonging to each unseen fault category is obtained by calculation. The specific calculation formula is:

[0171]

[0172]

[0173] in, represents the fault description predictor, The fault description matrix representing the unseen fault categories (i.e., the matrix composed of the fault description text embedding vectors of the unseen fault categories) The fault description text embedding vector corresponding to the i-th fault category in , represents the dot product similarity between the predicted fault description embedding vector and the fault description text embedding vector of the i-th unseen fault category; represents the probability that the predicted fault description embedding vector corresponding to the test sample is the fault description text embedding vector of the i-th unseen fault category, Represents the probability distribution of test samples on the fault description prediction branch.

[0174] (4.3) For each test sample in the test set , if the predicted categories from the fault attribute prediction branch and the fault description prediction branch are the same, and both probabilities are greater than the preset probability threshold , then the classification result is considered to be relatively reliable, the predicted category is the final classification result of the test sample, and the test sample is called a high-probability test sample; the set of all high-probability test samples is expressed as , the remaining test samples in the test set are called low-probability test samples, and their set is expressed as .

[0175] (4.4) Use the high-probability test samples as reference to adjust the semantic knowledge of the low-probability test samples (i.e., fault attributes and fault description embedding vectors) to obtain the adjusted fault attributes and fault description embedding vectors.

[0176] It should be noted that for low probability test samples , their predicted categories are determined based on the adjusted dual-view semantic knowledge. Specifically, for high-probability samples, the predicted fault attributes and predicted fault description vectors reflect the performance of the predictor for unseen fault categories. The predictor is trained on visible faults. For visible faults, the model has been gradually fitted during the training process, and the predicted semantic knowledge from the samples can accurately match the annotated semantic knowledge from the large language model. However, due to the inconsistent feature distributions of visible faults and unseen faults, the predicted semantic knowledge for unseen faults often deviates from the annotated semantic knowledge. High-probability samples are used as references to adjust the semantic knowledge to make it more consistent with the actual output distribution of unseen faults in the attribute and description prediction branches.

[0177] Furthermore, the semantic knowledge of the low-probability test samples is adjusted using the high-probability test samples as a reference, specifically including: based on the predicted category of the high-probability test samples, the average value of the predicted fault attributes of all high-probability test samples in each unseen fault category is calculated as the predicted semantic knowledge center of the fault attribute, and the average value of the predicted fault description embedding vectors of all high-probability test samples in each unseen fault category is calculated as the predicted semantic knowledge center of the fault description embedding vector. The calculation formulas are:

[0178]

[0179]

[0180] in, The predictive semantic knowledge center representing fault attributes, represents the predictive semantic knowledge center of the fault description embedding vector, Indicates a high probability test sample The fusion characteristics of Indicates A high probability test sample, Represents the total number of all high-probability test samples. The predicted semantic knowledge center of the fault attribute and the predicted semantic knowledge center of the fault description embedding vector are used to adjust the predicted fault attribute and fault description embedding vector corresponding to the low-probability test sample to obtain the adjusted fault attribute and fault description embedding vector, which is expressed as:

[0181]

[0182]

[0183] in, represents the adjusted fault attribute, represents the adjusted fault description embedding vector, Represents a low probability test sample of the fault attribute predictor output The corresponding predicted fault attributes, Represents a low probability test sample of the fault description predictor output The corresponding predicted fault description embedding vector, Indicates A low probability test sample.

[0184] (4.5) Using the adjusted fault attribute and fault description embedding vectors, repeat steps (4.1) and (4.2) to recalculate the probabilities of low-probability test samples on the fault attribute prediction branch and the fault description prediction branch to obtain the final classification result of fault diagnosis.

[0185] Specifically, using the adjusted semantic knowledge, i.e., the adjusted fault attributes and fault description embedding vectors, we calculate Distance and dot product similarity, the category with the smallest distance and the highest similarity is taken as the classification result. Specifically, repeat steps (4.1) and (4.2) to recalculate the classification probability of all unseen fault categories of low-probability test samples on the fault attribute prediction branch and the fault description prediction branch; for the fault attribute prediction branch, select The category corresponding to the maximum probability value of is taken as the final prediction result of the fault attribute; for the fault description prediction branch, select The category corresponding to the maximum probability value is taken as the final prediction result of the fault description. In the case that the final prediction results based on the two knowledge views are inconsistent, the result with a higher prediction probability is given priority as the final classification result of the fault diagnosis. This dual-view dynamic semantic knowledge learning strategy has the lowest computational overhead and only requires additional calculation of distance and similarity without repeating the forward pass or back propagation of the health feature comparison and semantic knowledge prediction network.

[0186] (5) When applied, a test sample of a fault category not seen in the test set is input into the trained encoder to obtain class-independent features. At the same time, the test sample and any healthy sample are input into the trained healthy feature comparison and semantic knowledge prediction network to obtain the predicted semantic knowledge corresponding to the test sample; and the predicted semantic knowledge is adjusted according to the dual-view dynamic semantic knowledge learning strategy to obtain the final classification result of the fault diagnosis corresponding to the test sample.

[0187] Specifically, when applied to specific scenarios, such as Figure 6As shown, firstly, a large language model is used to annotate the fault samples with semantic knowledge, and then self-supervised pre-training is performed through the fault samples to obtain an encoder that can extract class-independent features. The test samples of the unseen fault categories in the test set are input into the trained encoder to obtain class-independent features. The test samples of the unseen fault categories and any healthy samples are respectively input into the trained feature extractor for feature extraction, and then respectively pass through the time window normalization module to align with the class-independent features encoded by the encoder to obtain aligned fault features and health features; then the aligned fault features and health features are subtracted to obtain contrast features; then the class-independent features, contrast features and aligned fault features are feature fused to obtain fused features; finally, the fused features are respectively input into the trained fault attribute predictor and fault description predictor for prediction to obtain the predicted fault attributes and fault description embedding vectors corresponding to the test sample. Then, the predicted fault attributes and fault description embedding vectors corresponding to the test sample are adjusted based on the dual-view dynamic semantic knowledge learning strategy to obtain the adjusted fault attributes and the adjusted fault description embedding vector Then, the final prediction result of the fault attribute is calculated based on the adjusted fault attribute, which is expressed as:

[0188]

[0189]

[0190] in, is the final prediction result of the fault attribute. The final prediction result of the fault description embedding vector is calculated based on the adjusted fault description embedding vector, which is expressed as:

[0191]

[0192] in, is the final prediction result of the fault description embedding vector. If the final prediction result of the fault attribute Final prediction result with fault description embedding vector If the results are the same, the final classification result of the fault diagnosis corresponding to the test sample is directly output; if the final prediction result of the fault attribute is Final prediction result with fault description embedding vector If different, take and The category corresponding to the maximum probability value in is the final classification result of the fault diagnosis corresponding to the test sample.

[0193] The embodiment of the present invention lists two typical examples, namely: a fault diagnosis task in a thermal power generation process and a fault diagnosis task in a chemical product synthesis process. The method of the present invention can be used for any industrial process lacking relevant fault data and having a fault diagnosis demand in the fields of energy, chemical industry, machinery, etc., wherein the annotation of semantic knowledge, feature extraction of faults and intelligent diagnosis can all be processed by the method of the present invention.

[0194] Example 1: Fault diagnosis task in thermal power generation process

[0195] This embodiment verifies the effectiveness of the method described in the present invention with the test results on the thermal power data set. The data set used is from an actual 1000MW supercritical thermal power plant. The power plant generates electricity by burning coal in the furnace to vaporize water and drive the turbine connected to the generator. The data set includes 430 process variables from different components, including equipment such as coal mills and blowers, and captured signals such as temperature and pressure. Fault data is mainly concentrated on equipment such as coal mills, primary fans and induced draft fans. Fault data was collected according to the monitoring system of the power plant, and 9 types of faults were obtained, of which 6 types of faults were randomly selected as visible faults, and the remaining 3 types were used as unseen faults in each experimental group. For each type of fault, 15 fault attributes were annotated by the Wenxin Yiyanda model, as shown in Table 1; the fault description was also annotated by the Wenxin Yiyanda model and converted into text embedding using the pre-trained BERT model. Since fault analysis of the thermal power generation process is not easy to obtain on the Internet. The present invention uses the fault diagnosis reports of these faults as the knowledge source of the large language model. The fault diagnosis report contains relevant information such as fault measurement point, handling record, root cause analysis and remedial measures.

[0196] Table 1: Fault attributes of thermal power generation process

[0197]

[0198] Here, three classic zero-shot learning methods and two industrial zero-shot fault diagnosis methods are selected for comparative experiments with the method described in the present invention. The selected zero-shot learning methods are ESZSL (Embarrassingly SimpleZero-Shot Learning), ALE (Attribute Label Embedding) and SJE (Structured Joint Embedding), and the industrial zero-shot fault diagnosis methods are FDAT (Fault Description based Attribute Transfer) and SCE (Semantic-Consistent Embedding). The average accuracy of unseen fault categories is used as the evaluation index, and its expression is:

[0199]

[0200] in, represents the average accuracy of the unseen fault category, represents the total number of samples without failures, Indicates the number of correctly classified unseen fault samples. Three types of faults were randomly selected as unseen fault categories for experiments. A total of four groups of parallel experiments were conducted. The experimental results are shown in Table 2.

[0201] Table 2: Experimental results of different methods in thermal power dataset

[0202]

[0203] It can be seen from the experimental results in Table 2 that, compared with other methods, the average accuracy of the method of the present invention is improved by 6.87% in the unseen fault category, which proves the feasibility and effectiveness of the method of the present invention.

[0204] In addition, in order to verify the versatility and effectiveness of the method of the present invention in other fields, the following brief examples are given.

[0205] Example 2: Fault diagnosis task in the process of chemical product synthesis

[0206] Taking a chemical process in a chemical plant that uses gaseous reactants to generate two liquid products and their by-products as an example, the fault attributes and fault descriptions annotated by the large language model are shown below. A total of 15 fault categories are annotated, and two of them are taken as examples here. The first category is: "D Feed Temperature Step Change: Sudden changes in D feed temperature may affect the rate of chemical reactions, either accelerating or slowing down the reaction. This may cause changes in product quality, and even equipment failure if the temperature change is extreme. Fixing this fault usually involves adjusting the temperature control system to maintain a stable feed temperature"; the second category is: "Reactor cooling water inlet temperature step change: Changes in the reactor cooling water inlet temperature may affect the reaction temperature, possibly leading to unsafe conditions or reducing reaction efficiency. Fixing this fault requires adjusting the cooling water system to maintain a constant inlet temperature."

[0207] Three types of faults are randomly selected as unseen faults and 12 types of faults are randomly selected as seen faults for experiments. The experimental results are shown in Table 3.

[0208] Table 3: Experimental results of different methods in chemical processes

[0209]

[0210] It can be seen from the experimental results in Table 3 that, compared with the traditional manually labeled zero-sample fault diagnosis method, the method of the present invention improves the average accuracy of unseen fault detection by 11.34%, which is a significant increase.

[0211] In summary, experiments on thermal power data and chemical data have proved that the method of the present invention can be applied to multiple fields and is superior to other methods.

[0212] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent diagnosis method for large model knowledge annotation, characterized in that: The following steps are involved: (1) Collect expert knowledge documents and write contextual prompts for the large language model to inject knowledge into the large language model and obtain formatted semantic knowledge corresponding to each fault category to build training sets and test sets; (2) constructing a self-supervised pre-training network, including an encoder, a decoder, a predictor, a classifier, and a gradient reversal layer, and using all fault samples in the training set and the test set for training, with minimizing the first total loss function as the optimization goal, optimizing the parameters of the encoder, decoder, predictor, classifier, and gradient reversal layer to obtain a trained encoder; (3) constructing a health feature comparison and semantic knowledge prediction network, and using the trained encoder to train it using the training set, with minimizing the second total loss function as the optimization goal, optimizing the parameters of the health feature comparison and semantic knowledge prediction network to obtain a trained health feature comparison and semantic knowledge prediction network; (4) For the test samples in the test set, the trained health features are compared with the semantic knowledge predicted by the semantic knowledge prediction network, and the predicted semantic knowledge is adjusted based on the dual-view dynamic semantic knowledge learning strategy to obtain the final classification result of fault diagnosis; (5) When applied, a test sample of a fault category not seen in the test set is input into the trained encoder to obtain a class-independent feature, and at the same time, the test sample and any healthy sample are input into the trained healthy feature comparison and semantic knowledge prediction network to obtain the predicted semantic knowledge corresponding to the test sample; The predicted semantic knowledge is adjusted according to the dual-view dynamic semantic knowledge learning strategy to obtain the final classification result of the fault diagnosis corresponding to the test sample.

2. The intelligent diagnosis method for large model knowledge annotation according to claim 1 is characterized in that: The step (1) comprises the following sub-steps: (1.1) Collect expert knowledge documents and write contextual prompts for the large language model based on the expert knowledge documents. The contextual prompts include professional documents, task requirements, and output demonstrations; (1.2) Input contextual prompts into the large language model to perform knowledge injection into the large language model and obtain formatted semantic knowledge corresponding to each fault category output by the large language model, namely, the fault category-attribute matrix and the text description of the fault; (1.3) Expert review of the formatted semantic knowledge corresponding to each fault category to remove unreasonable formatted semantic knowledge and adjust the contextual prompts to regain the corresponding formatted semantic knowledge; (1.4) Use the pre-trained BERT model to convert the text description of the fault into a fault description text embedding vector of the same length; (1.5) Construct training and test sets based on the fault category-attribute matrix, fault description text embedding vector, and collected data samples.

3. The intelligent diagnosis method for large model knowledge annotation according to claim 1 or 2, characterized in that: The construction of the training set and the test set specifically includes: The visible fault categories used for training are represented as S = {s1,…,s p }, where s1 represents the first visible fault category, s p represents the pth visible fault category, p is the total number of visible fault categories; the unseen fault category is represented as U = {u1,…,u q }, where u1 represents the first unseen fault category, u q represents the qth unseen fault category, q is the total number of unseen fault categories, and Each row of the fault category-attribute matrix represents the fault category, each column represents the fault attribute, and the value of each cell is used to indicate whether the fault category has the corresponding fault attribute; the category-attribute matrix of all L fault categories is expressed as in represents the fault attribute matrix of all visible fault categories S, represents the fault attribute matrix of all unseen fault categories, L = p + q, M represents the number of fault attributes; The fault description text embedding vectors of all L fault categories are represented as in A matrix of fault description text embedding vectors representing all visible fault categories, represents the matrix composed of the fault description text embedding vectors of all unseen fault categories, and d represents the length of the fault description text embedding vector; The fault class labels Y that will be used for training s Respectively with the category-attribute matrix A s and the matrix D consisting of the fault description text embedding vector s Merge to get the corresponding fault attribute label and fault description label Where N s Represents the total number of original training samples; For each original training sample of length t, a window of length d is used. w The time sliding window with a sliding step size of 1 divides it into multiple training samples, and finally obtains N s ×(td w +1) training samples, denoted as X s ; and the health data sample x∈X h Included in the training set; The training set is represented as Among them, x represents the data sample, X s represents the set of all fault samples after the time sliding window processing, X h represents the set of healthy samples, T s represents the total number of all training samples after the time sliding window processing, k represents the dimension of the sample feature; y represents the fault category corresponding to the data sample x, Y s represents the set of all fault category labels used for training, Y h Indicates the health category label; z a represents the fault attribute label corresponding to the data sample x, represents the set of all fault attribute labels; z d represents the fault description label corresponding to the data sample x, represents the set of all fault description labels; the test set is represented by Among them, X t represents the set of test samples containing unseen fault categories, T t Represents the total number of all test samples after the time sliding window processing; Y t Represents the set of classification labels for all unseen fault categories.

4. The intelligent diagnosis method for large model knowledge annotation according to claim 1 is characterized in that: The self-supervised pre-training network includes an encoder for extracting class-independent features, a decoder for reconstructing samples, a classifier and a gradient reversal layer for predicting data of the next time step of the input sample time window and for determining whether the sample belongs to a visible fault category or an unseen fault category, wherein the encoder includes three layers of long short-term memory network layers, and the decoder, predictor and classifier all use a multi-layer perceptron model; First, a random mask operation is performed on each fault sample to obtain a masked fault sample; then the masked fault sample is input into the encoder to obtain the encoded class-independent features; secondly, the encoded class-independent features are input into the decoder to obtain the decoded reconstructed samples; the encoded class-independent features are input into the predictor to obtain the predicted data of the next time step; the encoded class-independent features are input into the gradient reversal layer and then into the classifier to obtain the classification result of whether the fault sample belongs to the visible fault category or the unseen fault category.

5. The intelligent diagnosis method for large model knowledge annotation according to claim 1 is characterized in that: The method uses all fault samples in the training set and the test set for training, takes minimizing the first total loss function as the optimization goal, and optimizes the parameters of the encoder, decoder, predictor, classifier, and gradient reversal layer to obtain a trained encoder, specifically including: Perform a random mask operation on each fault sample x in the training set and the test set to obtain a masked fault sample; The masked fault samples are input into the self-supervised pre-trained network to obtain the decoded reconstructed samples, the predicted data of the next time step, and the classification result of whether the fault samples belong to the visible fault category or the unseen fault category; The loss function of the reconstruction task is calculated based on the decoded reconstructed samples and the corresponding original fault samples in the training set and test set, which is expressed as: in, represents the loss function of the reconstruction task, represents the expectation of the fault sample, x∈X s ∪X t represents the original fault samples in the training set and the test set, ∥·∥2 represents the L2 norm, represents the masked fault sample, Enc() represents the encoder, and Dec() represents the decoder; The loss function of the prediction task is calculated based on the predicted data of the next time step and the corresponding real data of the next time step in the training set and the test set, which is expressed as: in, represents the loss function of the prediction task, x t+1 Represents the actual next time step data in the training set and test set, and Pre() represents the predictor; The loss function of adversarial classification is calculated based on the classification result of whether the fault sample belongs to the visible fault category or the unseen fault category and the true label of whether the fault sample belongs to the visible fault category or the unseen fault category, which is expressed as: in, represents the loss function of adversarial classification, CE() represents cross entropy loss, Cls() represents classifier, R λ () represents the gradient reversal layer, λ represents the gradient reversal factor, and y represents the true label of whether the fault sample belongs to the visible fault category or the unseen fault category; The first total loss function is calculated according to the loss function of the reconstruction task, the loss function of the prediction task, and the loss function of the adversarial classification, which is expressed as: in, represents the first total loss function; taking minimizing the first total loss function as the optimization goal, the parameters of the encoder, decoder, predictor, classifier, and gradient reversal layer are optimized until a preset training round is reached or the first total loss function converges to obtain a trained encoder.

6. The intelligent diagnosis method for large model knowledge annotation according to claim 1 is characterized in that: The health feature comparison and semantic knowledge prediction network includes a feature extractor, a time window normalization module, a fault attribute predictor and a fault description predictor, wherein the fault attribute predictor and the fault description predictor are both multi-layer perceptrons; The fault samples are input into the trained encoder to extract the class-independent features of the fault samples; the fault samples and healthy samples are respectively input into the same feature extractor to obtain the corresponding fault sample features and healthy sample features; the fault sample features and healthy sample features are respectively input into the time window normalization module to obtain the aligned fault features and healthy features; the aligned fault features are subtracted from the healthy features to obtain the contrast features; the class-independent features, contrast features and aligned fault features are fused to obtain the fused features; the fused features are respectively input into the fault attribute predictor and the fault description predictor to predict the semantic knowledge corresponding to the fault samples, that is, to predict the corresponding fault attributes and fault description embedding vectors.

7. The intelligent diagnosis method for large model knowledge annotation according to claim 1 is characterized in that: The method uses the trained encoder and the training set to train the encoder, takes minimizing the second total loss function as the optimization goal, and optimizes the parameters of the health feature comparison and semantic knowledge prediction network to obtain the trained health feature comparison and semantic knowledge prediction network, specifically including: The fault samples in the training set are input into the trained encoder, and the fault samples in the training set and the healthy samples in the training set are input into the health feature comparison and semantic knowledge prediction network to predict the corresponding fault attributes and fault description embedding vectors; The fault attribute prediction loss is calculated based on the predicted fault attributes and the corresponding fault attribute labels in the training set, which is expressed as: in, represents the fault attribute prediction loss, represents the expectation of data sample x in the training set, AP() represents the fault attribute predictor, represents the fusion feature, Represents the fault attribute label z corresponding to sample x in the training set a The kth value of ; The dot product similarity between the predicted fault description embedding vector and the fault description text embedding vector of all fault categories is calculated as the predicted probability of each fault category. The fault description prediction loss is calculated based on the predicted probability of each fault category and the corresponding fault category label in the training set, which is expressed as: in, represents the fault description prediction loss, DP() represents the fault description predictor, D is the fault description text embedding vector of all fault categories, and y represents the fault category label corresponding to sample x in the training set; The second total loss function is calculated based on the fault attribute prediction loss and the fault description prediction loss, which is expressed as: in, represents the second total loss function; taking minimizing the second total loss function as the optimization goal, optimizing the parameters of the feature extractor, the time window normalization module, the fault attribute predictor and the fault description predictor until the preset training rounds are reached or the second total loss function converges, so as to obtain the trained health feature comparison and semantic knowledge prediction network.

8. The intelligent diagnosis method for large model knowledge annotation according to claim 1 is characterized in that: The step (4) comprises the following sub-steps: (4.1) For the fault attribute prediction branch in the trained health feature comparison and semantic knowledge prediction network, the classification result of the test sample is determined by calculating the L1 distance, and the classification probability of all unseen fault categories is obtained by applying the softmax operation. The specific calculation formula is: in, represents the fusion feature of the test sample, AP() represents the fault attribute predictor, Fault attribute matrix A representing unseen fault categories u The fault attribute corresponding to the i-th fault category in d i represents the L1 distance between the predicted fault attribute corresponding to the test sample and the fault attribute vector of the i-th unseen fault category; exp is an exponential function, j represents the sequence number of the fault attribute vector of the unseen fault category, represents the probability that the predicted fault attribute corresponding to the test sample is the fault attribute of the i-th unseen fault category, P att () represents the probability distribution of the test sample on the fault attribute prediction branch; (4.2) For the fault description prediction branch in the trained health feature comparison and semantic knowledge prediction network, based on the dot product similarity, the probability of each test sample belonging to each unseen fault category is obtained by applying the softmax operation. The specific calculation formula is: Among them, DP() represents the fault description predictor, The fault description matrix D represents the unseen fault category u The fault description text embedding vector corresponding to the i-th fault category in s i represents the dot product similarity between the predicted fault description embedding vector and the fault description text embedding vector of the i-th unseen fault category; represents the probability that the predicted fault description embedding vector corresponding to the test sample is the fault description text embedding vector of the i-th unseen fault category, P des (·) represents the probability distribution of the test sample on the fault description prediction branch; (4.3) For each test sample x in the test set, if the predicted categories from the fault attribute prediction branch and the fault description prediction branch are the same, and both probabilities are greater than the preset probability threshold α, then the predicted category is the final classification result of the test sample, and the test sample is called a high-probability test sample; where the set of all high-probability test samples is represented as The remaining test samples in the test set are called low-probability test samples, and their set is expressed as (4.4) Using high-probability test samples as references to adjust the semantic knowledge of low-probability test samples to obtain adjusted fault attributes and fault description embedding vectors; (4.5) Using the adjusted fault attribute and fault description embedding vectors, repeat steps (4.1) and (4.2) to recalculate the probability of low-probability test samples on the fault attribute prediction branch and the fault description prediction branch to obtain the final classification result of fault diagnosis.

9. The intelligent diagnosis method for large model knowledge annotation according to claim 8 is characterized in that: The method of using the high probability test sample as a reference to adjust the semantic knowledge of the low probability test sample specifically includes: Based on the predicted categories of high-probability test samples, the average value of the predicted fault attributes of all high-probability test samples in each unseen fault category is calculated as the predicted semantic knowledge center of the fault attribute, and the average value of the predicted fault description embedding vector of all high-probability test samples in each unseen fault category is calculated as the predicted semantic knowledge center of the fault description embedding vector. The calculation formulas are: in, The predictive semantic knowledge center representing fault attributes, represents the predictive semantic knowledge center of the fault description embedding vector, Represents a high probability test sample x hp The fusion characteristics of represents the hpth high probability test sample, N u Represents the total number of all high-probability test samples; The predicted semantic knowledge center of the fault attribute and the predicted semantic knowledge center of the fault description embedding vector are used to adjust the predicted fault attribute and fault description embedding vector corresponding to the low probability test sample to obtain the adjusted fault attribute and fault description embedding vector, which is expressed as: in, represents the adjusted fault attribute, represents the adjusted fault description embedding vector, Represents a low probability test sample of the fault attribute predictor output The corresponding predicted fault attributes, Represents a low probability test sample of the fault description predictor output The corresponding predicted fault description embedding vector, Represents the lpth low probability test sample.

10. The intelligent diagnosis method for large model knowledge annotation according to claim 9 is characterized in that: The step (5) specifically comprises: When applied, the test sample of the unseen fault category in the test set is input into the trained encoder to obtain class-independent features; the test sample of the unseen fault category and any healthy sample are respectively input into the trained feature extractor and time window normalization module and then subtracted to obtain contrast features; the class-independent features, contrast features and aligned fault features are fused to obtain fused features; the fused features are respectively input into the trained fault attribute predictor and fault description predictor to obtain the predicted fault attributes and fault description embedding vectors corresponding to the test sample; the predicted fault attributes and fault description embedding vectors corresponding to the test sample are adjusted based on the dual-view dynamic semantic knowledge learning strategy to obtain the adjusted fault attributes and fault description embedding vectors; and the final prediction result of the fault attributes is calculated based on the adjusted fault attributes, which is expressed as: Among them, y att is the final prediction result of the fault attribute; the final prediction result of the fault description embedding vector is calculated according to the adjusted fault description embedding vector, which is expressed as: Among them, y des is the final prediction result of the fault description embedding vector; if the final prediction result of the fault attribute y att The final prediction result y with the fault description embedding vector des If the results are the same, the final classification result of the fault diagnosis corresponding to the test sample is directly output; if the final prediction result y of the fault attribute is att The final prediction result y with the fault description embedding vector des If they are different, then take y att With y des The category corresponding to the maximum probability value in is the final classification result of the fault diagnosis corresponding to the test sample.

Images