A large-scale intelligent decision-making method for industrial operation and maintenance

Abstract

The present invention discloses a large-model intelligent decision-making method in the field of industrial operation and maintenance, which includes: collecting and arranging industrial operation and maintenance text materials, injecting industrial operation and maintenance knowledge into the large model based on prompt word engineering and constructing large language model training text pairs, and obtaining a large model in the industrial field; collecting sensor time series data, designing a cross-modal conversion module to convert the time series data into the same feature space as the natural language text, and through a cross-modal knowledge alignment strategy, using the information learned in the text space to supervise the training of the cross-modal conversion module; on this basis, constructing a task prompt text through a prompt word engineering to guide the large model to output the industrial equipment status classification result. The present invention can give the classification result of the industrial operation and maintenance task based on the equipment status text and the time series modal data collected by the sensor, based on the large model, thereby providing support for intelligent decision-making of industrial operation and maintenance.

Description

Technical Field

[0001] The present invention relates to the field of industrial equipment operation and maintenance classification technology, and in particular to a large-model intelligent decision-making method in the field of industrial operation and maintenance. Background Art

[0002] The existing industrial operation and maintenance methods have problems such as inaccurate fault analysis, complex and difficult to understand alarm reports, time-consuming fault report writing, and difficulty in locating the root cause, which seriously restrict the continuity and stability of industrial production. As a major breakthrough in the field of artificial intelligence in recent years, big model technology has powerful text analysis, speech processing and multimodal data processing capabilities, which can theoretically provide new ideas for solving the above industrial operation and maintenance problems.

[0003] However, despite the great potential shown by big models, their practical application in the industrial field still faces many obstacles. First, the knowledge of big models is broad but not specialized enough, and their performance in various industrial verticals is not outstanding. The variability of industrial environments and the diversity of equipment require big models to have strong generalization and adaptability so that they can maintain high performance in different industrial scenarios. Therefore, high-quality industrial data is needed to train and fine-tune big models. However, high-quality data is lacking, resulting in poor performance of big models on specific industrial tasks and inability to accurately understand and predict complex situations in industrial environments. Secondly, industrial data often contains multiple modalities, such as text, voice, image, and sensor data. The differences in these data types require big models to have strong cross-modal analysis capabilities. However, current big models have difficulties in processing these multimodal data. They are often limited to single-modal data processing, unable to fully utilize the rich multimodal data generated in industrial environments, and difficult to achieve effective data fusion and knowledge extraction. This limits the big model's ability to understand and predict complex industrial scenarios, affecting the accuracy of fault diagnosis and the efficiency of operation and maintenance decisions.

[0004] In order to address the problem that large models in the industrial field are not professional enough and cannot process time series data across modalities, there is an urgent need for a model with cross-modal alignment capabilities for industrial equipment operation and maintenance tasks. Summary of the invention

[0005] The purpose of the present invention is to provide a large model intelligent decision-making method in the field of industrial operation and maintenance to address the problem in the prior art that large language models in industrial scenarios are difficult to effectively utilize multiple modal industrial data such as time series data and industrial operation and maintenance texts. The present invention uses a cross-modal knowledge alignment strategy to align the time series data modality and the text modality, and injects professional knowledge through instruction fine-tuning to overcome the hallucination problem caused by the lack of professionalism of the large model (i.e., Large Language Model, LLM), and activate the fault analysis and diagnosis capabilities of the large model for time series data in industrial operation and maintenance scenarios.

[0006] The objective of the present invention is achieved through the following technical solution: A large-scale intelligent decision-making method in the field of industrial operation and maintenance, comprising the following steps:

[0007] (1) Collecting operation and maintenance text materials of industrial objects, and using the operation and maintenance text materials to construct a structured question-answering text dataset for large model training based on a prompt word engineering method; wherein the question-answering text data is composed of question-answering text pairs, and the question-answering text pairs include a question part and an answer part;

[0008] (2) Use the question-answering text dataset to fine-tune the pre-trained large model to obtain an industrial domain large model for industrial equipment operation and maintenance classification tasks;

[0009] (3) constructing a cross-modal conversion module to convert the collected sensor time series data in the industrial process into textual spatial features. The cross-modal conversion module includes a time series feature encoder, a time series query, a large model segmenter, a large model encoder, a text query, a cross-modal attention weighter, and a modality alignment encoder;

[0010] (4) Based on the prompt word engineering method, the task prompt text is constructed. After being processed by the large model word segmenter and the large model encoder, the features of the aligned time series data corresponding to the sensor time series data and the text encoding features of the operation and maintenance text materials are spliced ​​together, and the spliced ​​time series features and text features are used as the input of the industrial field large model;

[0011] (5) Using sensor time series data and operation and maintenance text materials as training set inputs and their corresponding industrial equipment operation and maintenance classification results as training set labels, the cross-modal conversion module, reconstructor, and classifier are trained. During the training process, the optimization goal is to minimize the total training loss function and adjust the parameters of the cross-modal conversion module, reconstructor, and classifier to obtain the trained cross-modal conversion module, reconstructor, and classifier;

[0012] (6) During application, the trained large model segmenter, large model encoder, cross-modal conversion module, reconstructor, industrial domain large model and classifier are used to obtain the encoding features of the operation and maintenance decision of the sensor time series data to be tested and the classification results of the industrial equipment operation and maintenance classification task.

[0013] Furthermore, the question part of the question-answer text pair includes role definition, task requirements and fault characterization, and the answer part of the question-answer text pair includes fault characterization summary, fault cause analysis and fault prevention or handling measures.

[0014] Furthermore, the question and answer text data is obtained specifically by the following method:

[0015] The text descriptions of fault cases in the operation and maintenance text materials of industrial objects in the industrial process are structured and sorted, the descriptions related to the operation and maintenance faults are extracted, and question-answer text pairs are constructed; wherein the question part of the question-answer text pair is composed of prompt words and fault representations, the prompt words include an instruction part and a content part, and the prompt words are constructed by a prompt word engineering method based on role-playing and demand prompts; the answer part of the question-answer text pair is a description related to the fault, and its format is specified in advance, and the answer part provides a structured natural language text response template for the classification results of the equipment operation and maintenance tasks, including three parts of operation and maintenance decision information: fault representation summary, fault cause analysis, and fault prevention or handling measures.

[0016] Furthermore, the step (2) specifically includes:

[0017] Based on the question-answer text dataset, the question part of the question-answer text pair is input into the pre-trained big model. Under the guidance of the prompt words in the question part, the big model extracts information from the question-answer text data to fit the mapping relationship between the fault representation text and the fault cause and fault prevention or treatment measures text, and obtains the answer part output by the big model;

[0018] The total cross entropy loss function of the entire sequence is calculated based on the answer part output by the large model and the corresponding true answer part in the question-answer text pair. The weight parameters of the large model are fine-tuned with the optimization goal of minimizing the total cross entropy loss function of the entire sequence. Finally, an industrial field large model for industrial equipment operation and maintenance classification tasks is obtained.

[0019] Furthermore, the calculation formula of the total cross entropy loss function of the entire sequence is:

[0020]

[0021]

[0022] Among them, L represents the total cross entropy loss function of the entire sequence, represents the cross entropy loss function at time step t, where T is the total time step length of the sequence, represents the i-th element in the one-hot encoding of the target word at time step t, Represents the probability of the i-th word predicted by the large model at time step t.

[0023] Furthermore, the cross-modal conversion module specifically includes:

[0024] The sensor time series data collected in the industrial process is input into the cross-modal conversion module. First, the large model word segmenter and the large model encoder are applied to the operation and maintenance text material. The large model word segmenter is used to segment a section of operation and maintenance text material into word sequence, and then the large model encoder is used to convert the word sequence into text encoding features. The calculation method is expressed as follows:

[0025]

[0026] in, Represents text encoding features, represents the large model encoder, Represents the large model tokenizer, Represents a string of input operation and maintenance text materials;

[0027] Then the text encoding features Input into the text query to obtain the operation and maintenance text query features ;

[0028] Secondly, a sensor time series data sample is selected and input into the time series feature encoder to obtain the time series encoding feature vector, whose calculation method is expressed as:

[0029]

[0030] in, represents the temporal encoding feature vector, represents the temporal feature encoder, Represents sensor time series data samples;

[0031] Then the time series encoding feature vector Input into the time series query to obtain time series data query features ;

[0032] Secondly, for each operation and maintenance text material, the corresponding operation and maintenance text query features can be obtained , and then get the operation and maintenance text query matrix , where each line corresponds to the operation and maintenance text query feature of a section of operation and maintenance text material , query features for time series data using queries and keys and operation text query features Do a dot product operation to get the attention score vector, expressed as:

[0033]

[0034] in, represents the attention score vector, represents the activation function, is the key-query space dimension;

[0035] Then, the cross-modal attention weighter is to convert the attention score vector As the similarity between each time series data query feature and the operation and maintenance text query feature, based on the similarity, the operation and maintenance text query matrix Weighted summation is performed and the summation result is output as the weighted operation and maintenance text query feature. The calculation formula is as follows:

[0036]

[0037] in, Represents the weighted operation and maintenance text query features;

[0038] Finally, the weighted operation and maintenance text query features Input to the modality alignment encoder to obtain the features of aligned time series data , expressed as:

[0039]

[0040] in, Represents the modality alignment encoder.

[0041] Furthermore, the task prompt text is not the question part of a complete question-answer text pair, but only contains a prompt word text, wherein the fault representation part is blank. The question part of the complete question-answer text pair is restored by splicing the coding features of the task prompt text with the text coding features of the operation and maintenance text material or the features of the aligned time series data corresponding to the sensor time series data to obtain the text features of the question part of the complete question-answer text pair.

[0042] Furthermore, the step (4) specifically includes:

[0043] First, the text of the natural language mode of the operation and maintenance text material is input into the large model word segmenter and the large model encoder to obtain the text encoding features of the operation and maintenance text. The calculation formula is:

[0044]

[0045] in, Represents the text encoding features of the operation and maintenance text, A fault characterization text string indicating a certain fault;

[0046] Then, the task prompt text is input into the large model word segmenter and the large model encoder to obtain the encoding features of the task prompt text, and the calculation formula is:

[0047]

[0048] in, represents the encoding features of the task prompt text, Represents the task prompt text string;

[0049] Secondly, the encoding features of the task prompt text are concatenated with the text encoding features of the operation and maintenance text or the features of the aligned time series data to obtain the concatenated text features or time series features as the text features of the question part in the complete question-answer text pair. The calculation formula is:

[0050]

[0051] or,

[0052]

[0053] in, Represents the question-part feature in the question-answer text pair corresponding to the concatenated text feature, Represents the question-part features in the question-answer text pair corresponding to the concatenated temporal features. It means concatenating two feature vectors.

[0054] Furthermore, the step (5) specifically includes:

[0055] The sensor time series data and operation and maintenance text materials are used as the training set inputs, and the industrial equipment operation and maintenance classification results corresponding to the sensor time series data and operation and maintenance text materials are used as the training set labels. The cross-modal conversion module, reconstructor and classifier are trained. During the training process, the parameters of the large model segmenter, large model encoder and industrial field large model are frozen and not updated.

[0056] Input the sensor time series data and operation and maintenance text materials into the cross-modal conversion module to obtain the features of the aligned time series data , the characteristics of the aligned time series data Input into the reconstructor to obtain the reconstructed time series data; calculate the reconstruction error based on the reconstructed time series data and the original sensor time series data, and the calculation formula is:

[0057]

[0058] in, represents the reconstruction loss of a single sample, represents the value of the i-th original sensor time series data at time t, represents the value of the i-th reconstructed time series data at time t, and n is the total number of elements of the sensor time series data samples;

[0059] Then, based on the sensor time series data and operation and maintenance text materials, the encoding features of the task prompt text are The text encoding features of the operation and maintenance text and characteristics of aligned time series data After splicing, we get the corresponding problem features and , the problem features and Input them into the industrial field model respectively, and obtain the text features of the answer part corresponding to the operation and maintenance text respectively. Text features of the answer part corresponding to the time series data , and then pass through the classifier to obtain their respective classification results and ; Based on the classification results when the splicing objects are operation and maintenance text and time series data and Calculate the cross-modal knowledge alignment loss, and its calculation formula is:

[0060]

[0061] in, represents the cross-modal knowledge alignment loss of a single sample, Indicates the classification result when the splicing object is operation and maintenance text. Indicates the classification result when the splicing object is time series data;

[0062] Secondly, the cross entropy loss is calculated based on the classification results output by the classifier and the corresponding training set labels. As the classification loss, its calculation formula is:

[0063]

[0064] in, represents the classification loss of a single sample, represents the training set label of a single sample, Represents the classification result output by the classifier;

[0065] Finally, the total training loss function is calculated based on the reconstruction error, cross-modal knowledge alignment loss and classification loss, which is expressed as:

[0066]

[0067] in, represents the total training loss function, N is the total number of sensor time series data samples; taking minimizing the total training loss function as the optimization goal, the parameters of the cross-modal conversion module, reconstructor and classifier are adjusted to obtain the trained cross-modal conversion module, reconstructor and classifier.

[0068] Furthermore, the step (6) specifically includes:

[0069] Input the sensor time series data to be tested into the trained cross-modal conversion module to obtain the features of the time series data after modality alignment;

[0070] At the same time, the task prompt text is passed through the large model word segmenter and the large model encoder to obtain the encoding features of the task prompt text. The encoding features of the task prompt text are spliced ​​with the features of the time series data after modal alignment and input into the industrial field large model to output the encoding features of the corresponding operation and maintenance decision. ;

[0071] The text description consisting of the summary of fault characterization, fault cause analysis and fault prevention or treatment measures of all categories is obtained through the large model encoder to obtain the text features of each category, where the text feature of the i-th category is recorded as ; Assume there are M categories, there is a set , using the nearest neighbor pair For classification, the calculation formula is:

[0072]

[0073] in, It is the classification result of industrial equipment operation and maintenance classification tasks.

[0074] The beneficial effects of the present invention are as follows: the present invention can solve the problem of insufficient professionalism of general large language models, by constructing industrial equipment operation and maintenance question and answer text pairs, using parameter efficient fine-tuning technology to fine-tune instructions, injecting industrial operation and maintenance knowledge into the large language model, and utilizing the powerful language comprehension and cultural background knowledge of the large language model to reveal the hidden association between the health status of industrial equipment and operation and maintenance decisions; for the time series data collected by the industrial equipment operation and maintenance sensors, the present invention designs a cross-modal conversion module to convert the time series sensor data features into the text feature space, and supervises the learning of the cross-modal conversion module through a cross-modal knowledge alignment strategy; under the guidance of the task prompt text, the present invention can output the corresponding industrial equipment status classification results and related operation and maintenance decisions for the test time series data, providing support for intelligent decision-making in industrial operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 It is a flow chart of the large-scale intelligent decision-making method in the field of industrial operation and maintenance of the present invention;

[0076] Figure 2 is a flow chart of the architecture of the classification decision method of the present invention;

[0077] Figure 3 is a flow chart of the architecture of the cross-modal conversion module of the present invention;

[0078] Figure 4This is a schematic diagram of a large-scale model intelligent decision-making process obtained in an embodiment of the present invention in a real thermal power plant;

[0079] Figure 5 This is a diagram showing the effectiveness of the zero-sample fault diagnosis task performed by the present invention on a real thermal power plant embodiment;

[0080] Figure 6 A t-SNE visualization of the summary features of the large model output for a zero-sample fault diagnosis task in a real thermal power plant embodiment of the present invention;

[0081] Figure 7 This is a schematic diagram of a large-scale model intelligent decision-making process obtained in an embodiment of the present invention in a real steelmaking plant;

[0082] Figure 8 The diagram is a schematic diagram of the large-scale intelligent decision-making process obtained by the present invention in an embodiment of a real organic synthetic material chemical plant. DETAILED DESCRIPTION

[0083] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0084] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0085] It should be understood that although the terms first, second, third, etc. may be used in the present invention to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0086] The present invention is described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the features of the following embodiments and implementations can be combined with each other.

[0087] See also Figure 1 The large-scale intelligent decision-making method for industrial operation and maintenance of the present invention specifically includes the following steps:

[0088] (1) Collect the operation and maintenance text materials of industrial objects, and use them to construct a structured question-answering text dataset for large model training based on the prompt word engineering method; the question-answering text data consists of question-answering text pairs, and the question-answering text pairs include a question part and an answer part.

[0089] It should be understood that industrial objects refer to industrial processes, not limited to equipment, such as the operation and maintenance of chemical processes. The prompt engineering method is an existing well-known technology, which is the process of designing prompt words to ask questions to large models. Using the prompt engineering method to design prompt words is a routine and necessary process for applying large models.

[0090] Furthermore, the question part of the question-answer text pair includes three parts: role definition, task requirements and fault characterization, and the answer part of the question-answer text pair includes three parts: fault characterization summary, fault cause analysis and fault prevention or handling measures.

[0091] Furthermore, the question-answer text data is obtained by the following method: the text description of the fault case in the operation and maintenance text material of the industrial object in a real industrial process is structured and sorted, and the operation and maintenance fault related descriptions such as fault representation, fault cause and fault prevention or treatment measures are extracted, and the question-answer text pair is constructed. Among them, the question part of the question-answer text pair is composed of prompt words and fault representation. The prompt words include the instruction part and the content part. The prompt words are constructed by the prompt word engineering method based on role-playing and demand prompts. Specifically, firstly, the professional background is set based on the role-playing strategy, such as "You are an expert who has mastered the knowledge related to the xx process (industrial process in a certain field). At present, the xx factory (a real factory in a certain field) has an equipment failure. Next, the representation of the fault analyzed from the time series data collected by the sensor will be provided"; secondly, the task requirements are clarified based on the demand prompt strategy, such as "Please analyze and reason the current fault representation based on your professional knowledge, and give the cause and treatment measures of the fault." The answer part of the question-answer text pair is a description of the fault, and its format is specified in advance. The answer part provides a structured natural language text response template for the classification result of the equipment operation and maintenance task, including operation and maintenance decision information such as fault characterization summary, fault cause analysis, and fault prevention or treatment measures, which is used for subsequent fault analysis, fault diagnosis, and equipment health status analysis tasks.

[0092] Furthermore, the text of the fault representation in the question part can be augmented by designing prompt words, and synonyms can be modified to replace some fault descriptions, etc., and task descriptions can be added to the prompt words, etc., which can enhance the robustness of the question-answering effect and avoid overfitting in subsequent fine-tuning. The prompt words are constructed by the prompt word engineering method based on role-playing and demand prompts. Specifically, first, the professional background is set based on the role-playing strategy, such as "You are an expert who has mastered the relevant knowledge of xx process (industrial process in a certain field)"; secondly, based on the demand prompt strategy, the task requirements are clarified, and the advantages of the large model in natural language text processing are used to perform text augmentation and instruct the large model to perform operations, such as "delete or appropriately modify some time and date, slightly change the specific data values ​​mentioned in the specific, adjust the word order of the sentence, use synonyms, etc."

[0093] (2) Use the question-answer text dataset to fine-tune the pre-trained large model to obtain an industrial domain large model for industrial equipment operation and maintenance classification tasks.

[0094] Furthermore, the pre-trained large models include but are not limited to: ChatGLM3, GLM4, Qwen2.5 and other large models, and the fine-tuning methods include but are not limited to: LoRA, Adapter, P-Tuning, P-Tuning v2, Prompt Tuning and other full parameter fine-tuning and partial parameter fine-tuning methods. In this embodiment, the pre-trained ChatGLM3 large model is selected. On the basis of the pre-trained ChatGLM3 large model, LoRA is used to implement supervised instruction fine-tuning. A set of question-and-answer examples containing instructions and expected outputs are used for supervised training to make the large model more in line with the requirements of the target task. LoRA is a low-rank adaptation technology for fine-tuning large language models. It fine-tunes the model by only training low-rank matrices and then injecting these parameters into the original model. This method reduces computing requirements.

[0095] Specifically, based on the question-answer text dataset, the question part of the question-answer text pair is input into the pre-trained large model. Under the guidance of the prompt words in the question part, the large model extracts information from the question-answer text data to fit the mapping relationship between the fault characterization text and the fault cause and fault prevention or treatment measures text, and obtains the answer part output by the large model. The total cross entropy loss function of the entire sequence is calculated based on the answer part output by the large model and the corresponding true answer part in the question-answer text pair. The weight parameters of the large model are fine-tuned with the minimization of the total cross entropy loss function of the entire sequence as the optimization goal, and finally the industrial field large model for industrial equipment operation and maintenance classification tasks is obtained.

[0096] In this embodiment, the question-answer text pairs in the question-answer text dataset are used for instruction fine-tuning. A set of question-answer examples containing instructions and expected outputs are used for supervised training to make the large model more in line with the requirements of the target task. The fine-tuning technology uses LoRA to efficiently fine-tune the ChatGLM3 large model. By adding the weight matrix of the pre-trained large model Insert a new weight matrix in parallel with it , and further Split into a reduced dimension matrix and the dimension-increasing matrix , which can be expressed as:

[0097]

[0098] in, is the hidden layer state, i.e. the output of the large model; is the input of the large model; d is the dimension of the input feature, k is the dimension of the output feature, and r is the intermediate dimension in the process of dimensionality reduction and dimensionality increase. In the forward reasoning process, and will multiply the same input , and then add them together. During the training of the large model, the original pre-trained large model parameters are fixed and only the dimension reduction matrix is ​​trained and the dimension-increasing matrix LoRA allows indirect training of some dense layers in a large model by optimizing the rank decomposition matrix of the dense layer changes during the adaptation process, while keeping the pre-trained weights unchanged. The number of newly added low-rank matrix parameters is very small, so that only very small parameters need to be trained to obtain the corresponding results.

[0099] The loss function for supervised training uses cross entropy loss. In the autoregressive generative model for natural language processing, cross entropy loss is calculated step by step. For each position in the sequence, the large model predicts a probability distribution that represents the next possible word. Then, the cross entropy loss is calculated between this predicted distribution and the distribution of the actual next word. This process is performed on the entire sequence, and the loss at each position is accumulated to form the final loss value. Assume is the large model at time step Given all the previous words The next word predicted under the condition The probability distribution of is the one-hot encoding of the actual next word. Cross-extraction loss At time step It can be expressed as:

[0100]

[0101] in, is the target word at time step The one-hot encoding of elements, It is the first words at time step The total cross entropy loss function of the entire sequence is is the sum of the losses over all time steps and is expressed as:

[0102]

[0103] in, is the total time step length of the sequence. Taking minimizing the total cross entropy loss function of the entire sequence as the optimization goal, the weight parameters of the large model are fine-tuned, and finally an industrial domain large model that can output the classification results of industrial equipment operation and maintenance classification tasks is obtained. Further, the model layer parameters of the industrial domain large model are frozen (i.e. ) for subsequent steps.

[0104] (3) Construct a cross-modal conversion module to convert the collected sensor time series data in the industrial process into textual spatial features. The cross-modal conversion module includes a time series feature encoder, a time series query Q, a large model segmenter, a large model encoder, a text query K, a cross-modal attention weighter, and a modality alignment encoder, as shown in Figure 3 shown.

[0105] In this embodiment, the sensor time series data collected in the industrial process is input into the cross-modal conversion module, and the features of the time series can be aligned to the semantic feature space, thereby activating the knowledge in the large model layer (i.e., the pre-trained large model body).

[0106] Specifically, Figure 3 As shown, the large model word segmenter and the large model encoder are first applied to the operation and maintenance text material. The large model word segmenter is first used to segment a section of operation and maintenance text material into a token sequence, and then the large model encoder is used to convert the token sequence into text encoding features. The calculation method is expressed as follows:

[0107]

[0108] in, Represents text encoding features, represents the large model encoder, Represents the large model tokenizer, Represents an input string of operation and maintenance text materials. The large model tokenizer and the large model encoder follow the network model settings of the industrial large model. The function of the large model tokenizer is to split the operation and maintenance text material string into tokens, and the function of the large model encoder is to convert the output of the tokenizer into numerical vectors, which capture the semantic information of the text.

[0109] Then the text encoding features Input into the text query machine K to obtain the operation and maintenance text query features In this embodiment, the text query K is a text encoding feature Left multiply the weight matrix The text query unit K includes but is not limited to learnable parameter modules such as linear layer networks. Therefore, in this embodiment, the calculation formula of the text query unit K is:

[0110]

[0111] in, It is the weight matrix corresponding to the key in the attention mechanism. It is a learnable weight matrix used to convert the text encoding feature vector into a key vector of equal length. The key in the attention calculation contains the information of each position in the input sequence, and the dot product result of the key and the query will be used to calculate the attention distribution.

[0112] Secondly, a sensor time series data sample is selected and input into the time series feature encoder to obtain the time series encoding feature vector, whose calculation method is expressed as:

[0113]

[0114] in, represents the temporal encoding feature vector, represents the temporal feature encoder, Represents a sensor time series data sample.

[0115] Then the time series encoding feature vector Input into the time series query device Q to obtain the time series data query features In this embodiment, the time series query device Q is a time series encoding feature vector Left multiply the weight matrix To achieve, but not limited to this, the time series query device Q can be composed of linear or nonlinear operation modules including but not limited to linear layer networks, kernel functions and other learnable parameters. Therefore, in this embodiment, the calculation in the time series query device Q is as follows:

[0116]

[0117] in, It is the weight matrix corresponding to the query in the attention mechanism, which is a learnable weight matrix. The query in the attention calculation represents the information that the current position or element needs to pay attention to.

[0118] Secondly, for each operation and maintenance text material, the corresponding operation and maintenance text query features can be obtained , and then get the operation and maintenance text query matrix , where each line corresponds to the operation and maintenance text query feature of a section of operation and maintenance text material , query features for time series data using queries and keys and operation text query features Do a dot product operation to get the attention score vector, expressed as:

[0119]

[0120] in, represents the attention score vector, represents the activation function, The key-query space dimension is used to control the size of the gradient and ensure numerical stability.

[0121] Then, the cross-modal attention weighter is to convert the attention score vector As the similarity between each time series data query feature and the operation and maintenance text query feature, based on the similarity, the operation and maintenance text query matrix Weighted summation is performed and the summation result is output as the weighted operation and maintenance text query feature. The calculation formula is as follows:

[0122]

[0123] in, Represents the weighted operation and maintenance text query features. Through attention weighting, the transformation from the time series data feature space to the text data feature space is completed.

[0124] Finally, the weighted operation and maintenance text query features Input to the modality alignment encoder to obtain the features of aligned time series data , It is the feature of the aligned time series data obtained after the cross-modal attention weighter conversion. This feature is the corresponding representation of the time series data in the feature space where the text is located, and is used to input the industrial field model in the subsequent steps. In this embodiment, the modality alignment encoder is composed of a linear layer and a LeakyReLU activation function, but the specific network implementation is not limited to this. The calculation formula of the modality alignment encoder is expressed as:

[0125]

[0126] in, Represents the modality alignment encoder.

[0127] (4) Based on the prompt word engineering method, the task prompt text is constructed. After being processed by the large model segmenter and the large model encoder, it is spliced ​​with the features of the aligned time series data corresponding to the sensor time series data and the text encoding features of the operation and maintenance text materials, and the spliced ​​time series features and text features are used as the input of the industrial field big model.

[0128] Furthermore, the task prompt text is not the question part of a complete question-answer text pair, but only contains the prompt word text, in which the fault representation part is missing. The question part of the complete question-answer text pair is restored by splicing the encoding features of the task prompt text with the text encoding features of the operation and maintenance text material or the features of the aligned time series data corresponding to the sensor time series data to obtain the text features of the question part of the complete question-answer text pair.

[0129] Specifically, real industrial process equipment including various fault representation fault tickets are used as operation and maintenance text materials, but not limited to this, any operation and maintenance text description related to industrial equipment operation and maintenance classification tasks (such as operation and maintenance logs, industry manuals, etc.) can be used as operation and maintenance text materials here. Figure 2 As shown, first, the text of the natural language mode of the operation and maintenance text material is input into the large model word segmenter and the large model encoder to obtain the text encoding features of the operation and maintenance text. The calculation formula is:

[0130]

[0131] in, Represents the text encoding features of the operation and maintenance text, A fault characterization text string representing a certain fault. The fault characterization text string is first passed through the large model word segmenter and large model encoder Processing to obtain the text encoding features of the operation and maintenance text .

[0132] Then, for the sensor time series data, the cross-modal conversion module is used to obtain the features of the aligned time series data. .

[0133] Then input the task prompt text into the large model word segmenter and the large model encoder to obtain the encoding features of the task prompt text, and the calculation formula is:

[0134]

[0135] in, represents the encoding features of the task prompt text, Represents the task prompt text string. The task prompt text string is first passed through the large model tokenizer and large model encoder Processing to obtain the encoding features of the task prompt text .

[0136] Secondly, the encoding features of the task prompt text are concatenated with the text encoding features of the operation and maintenance text or the features of the aligned time series data to obtain the concatenated text features or time series features as the text features of the question part in the complete question-answer text pair. The calculation formula is:

[0137]

[0138] or,

[0139]

[0140] in, Represents the question-part feature in the question-answer text pair corresponding to the concatenated text feature, Represents the question-part features in the question-answer text pair corresponding to the concatenated temporal features. represents the concatenation of two feature vectors. As a training set Input a large industrial model and output the text features of the answer part of the question-answer text pair ,Right now Figure 2 The summary characteristics of the large model output for the industry shown are This includes the fault characterization summary, fault cause analysis and fault prevention or handling measures. Input the classifier and get the classification result.

[0141] (5) Using sensor time series data and operation and maintenance text materials as training set inputs and their corresponding industrial equipment operation and maintenance classification results as training set labels, the cross-modal conversion module, reconstructor, and classifier are trained. During the training process, the optimization goal is to minimize the total training loss function and adjust the parameters of the cross-modal conversion module, reconstructor, and classifier to obtain the trained cross-modal conversion module, reconstructor, and classifier, such as Figure 2 shown.

[0142] Furthermore, the industrial equipment operation and maintenance classification result includes, but is not limited to, the fault type, the equipment health status level, etc. In this embodiment, the industrial equipment operation and maintenance classification result is the fault category.

[0143] Specifically, sensor time series data and operation and maintenance text materials are used as training set inputs, and the industrial equipment operation and maintenance classification results corresponding to the sensor time series data and operation and maintenance text materials are used as training set labels. The cross-modal conversion module, reconstructor and classifier are trained. During the training process, the parameters of the large model segmenter, large model encoder and industrial field large model are frozen and not updated. Figure 2 First, the sensor time series data and operation and maintenance text materials are input into the cross-modal conversion module, and the features of the aligned time series data are obtained through the process shown in step (3). , the characteristics of the aligned time series data Input into the reconstructor to obtain the reconstructed time series data, expressed as:

[0144]

[0145] in, represents the reconstructed time series data, Represents a reconstructor, and its implementation includes but is not limited to network structures such as multilayer perceptrons and autoencoders. In this embodiment, the reconstructor is composed of a linear layer and a LeakyReLU activation function. The reconstruction error is calculated based on the reconstructed time series data and the original sensor time series data. In this embodiment, the mean square error (MSE) is selected as the reconstruction error, and its calculation formula is:

[0146]

[0147] in, represents the reconstruction loss of a single sample, represents the value of the i-th original sensor time series data at time t, represents the value of the i-th reconstructed time series data at time t, and n is the sensor time series data sample (If it is a multivariate time series, then The reconstruction loss can be used to verify whether the temporal feature encoder in the cross-modal conversion module effectively captures the key features of the input data.

[0148] Then, based on the sensor time series data and operation and maintenance text materials, the encoding features of the task prompt text are converted into The text encoding features of the operation and maintenance text and characteristics of aligned time series data After splicing, we get the corresponding problem features and , the problem features and Input them into the industrial field model respectively, and obtain the text features of the answer part corresponding to the operation and maintenance text respectively. Text features of the answer part corresponding to the time series data , and then pass through the classifier to obtain their respective classification results and The present invention designs a cross-modal knowledge alignment strategy, proposes that after the alignment of different modal conversions of the same category, the classification results obtained after the classifier should be as similar as possible, and the cross-modal conversion module in step (3) is supervised by the similarity of the classification results obtained by the natural text modality and the time series data modality. and The difference in prediction probabilities of these two categories describes the difference between the text modality and the time series data modality for the operation and maintenance classification task after the cross-modal conversion module performs modal alignment. The cross-modal knowledge alignment loss quantifies this loss, and its calculation formula is:

[0149]

[0150] in, represents the cross-modal knowledge alignment loss of a single sample, Indicates the classification result when the splicing object is operation and maintenance text. Indicates the classification result when the splicing object is time series data. The cross-modal knowledge alignment loss is used to update the cross-modal conversion module in a targeted manner to ensure the modality alignment effect.

[0151] Secondly, the summary features output by the industrial field large model are input into the classifier to obtain the corresponding classification results. In this embodiment, the classifier is implemented by a linear layer, the activation function uses LeakyReLU, and finally inputs the Softmax layer to obtain the classification result, which is expressed as:

[0152]

[0153] in, The vector represents the classification result output by the classifier, which is the probability of belonging to each fault in this embodiment; is the network part before the Softmax layer in the classifier. In this embodiment, It is composed of linear layers and LeakyReLU activation functions, but is not limited to them; The function is an activation function commonly used in machine learning and deep learning. The function is usually used in the output layer of the model. It converts a vector or a set of real numbers into a probability distribution, which represents the predicted probability of each category. The model can output a category with the highest probability as the prediction result for each input sample. The cross entropy loss is calculated based on the classification result output by the classifier and its corresponding training set label as the classification loss. The calculation formula is:

[0154]

[0155] in, represents the classification loss of a single sample, Represents the training set label of a single example.

[0156] Finally, the total training loss function is calculated based on the reconstruction error, cross-modal knowledge alignment loss and classification loss, which is expressed as:

[0157]

[0158] in, Represents the total training loss function, and N is the total number of sensor time series data samples. Taking minimizing the total training loss function as the optimization goal, the parameters of the cross-modal conversion module, reconstructor, and classifier are adjusted to obtain the trained cross-modal conversion module, reconstructor, and classifier.

[0159] (6) During application, the trained large model segmenter, large model encoder, cross-modal conversion module, reconstructor, industrial domain large model and classifier are used to obtain the encoding features of the operation and maintenance decision of the sensor time series data to be tested and the classification results of the industrial equipment operation and maintenance classification task.

[0160] Specifically, the sensor time series data to be tested is input into the trained cross-modal conversion module to obtain the features of the time series data in the same feature space as the natural language text after modal alignment; at the same time, the task prompt text is passed through the large model word segmenter and the large model encoder to obtain the encoding features of the task prompt text, and the encoding features of the task prompt text are spliced ​​with the features of the time series data after modal alignment and input into the industrial field big model to output the corresponding encoding features of the operation and maintenance decision. ; Aggregate features of large model outputs in the industrial field is the coding feature of the operation and maintenance decision corresponding to the sample to be tested. In this embodiment, the text description consisting of the fault characterization summary, fault cause analysis and fault prevention or treatment measures of all fault categories is obtained through the large model encoder, where the text feature of the i-th category is recorded as . Assume there are M categories, and there is a set , using the nearest neighbor pair For classification, the calculation formula is:

[0161]

[0162] in, It is the classification result of industrial equipment operation and maintenance classification tasks.

[0163] It should be understood that since the industrial domain big model has been fully trained with the question-answer text data obtained in step (1), the industrial domain big model will output the encoding features of the operation and maintenance decision corresponding to the time series data sample of the sensor to be tested. Among them, the operation and maintenance decision refers to the text description consisting of the fault characterization summary, fault cause analysis and fault prevention or treatment measures. The operation and maintenance decision is further used to perform the industrial equipment operation and maintenance classification task.

[0164] The present invention is applicable to industrial fields including but not limited to energy, metallurgy, and chemical industry, and can be used for any industrial production process with industrial operation and maintenance needs. The present invention lists three examples including fault diagnosis of a real thermal power plant, operation and maintenance tasks of converter equipment in a metallurgical plant, and operation and maintenance tasks of equipment in an organic synthetic material chemical plant to illustrate the steps of the present invention and verify its effectiveness. Among them, any classification tasks related to industrial operation and maintenance, such as fault diagnosis and equipment health status classification, can be intelligently analyzed and decided using the method described in the present invention. The following embodiments use the fault sheet containing various fault representations of real industrial process equipment as the operation and maintenance text material, but are not limited to this. Any operation and maintenance text description related to the classification task of industrial equipment operation and maintenance (such as operation and maintenance logs, industry manuals, etc.) can be used as the operation and maintenance text material mentioned in the present invention.

[0165] Example 1: Intelligent decision-making of large-scale operation and maintenance model of thermal power generation equipment

[0166] This embodiment takes a fault diagnosis task based on measured data of a real thermal power plant as an example to illustrate the steps of the present invention and verify its effectiveness.

[0167] The production process of a 1000MW supercritical thermal power unit in a thermal power plant is a highly complex industrial system that exhibits significant time-varying, dynamic and non-stationary characteristics. During the production process, the chemical energy of the fuel is converted into thermal energy in the furnace, which heats the water in the boiler and converts it into steam. The steam generated by the boiler enters the steam turbine, which drives the turbine to rotate, and the mechanical energy generated by the rotation of the turbine drives the generator to generate electricity. For this production process, the plant sets 430 process variables, covering a variety of signal data from multiple key equipment such as primary fans, coal mills, induced draft fans, air preheaters, condensers, steam-driven feed water pumps and forced draft fans. These signal data include physical parameters such as temperature, pressure, water level, flow rate, opening and vibration. In order to enhance the real-time monitoring of the unit status, the power plant has installed sensor measurement points and data acquisition systems in multiple key locations in the early stage of the transformation, and set a sampling frequency of every 2 minutes. As an extremely complex and large-scale industrial production system, the operation of thermal power generating units under abnormal conditions may cause serious economic losses and potential equipment damage. Therefore, it is particularly important to carry out effective fault diagnosis, which can not only prevent potential failures, but also reduce equipment maintenance costs and improve system reliability, thereby ensuring the safe and stable operation of the generator set.

[0168] First, the text descriptions of fault cases in the operation and maintenance documents of a real thermal power plant are structured and organized, and the text is augmented as described in step (1). Finally, a question-answer text pair dataset in the natural language text mode is obtained. An example is as follows:

[0169] Question part: "You are a senior engineer who is proficient in the professional knowledge of thermal power generation. Currently, an equipment failure has occurred in a thermal power plant. Next, you will provide the characterization of the fault analyzed from the time series data collected by the sensor. Based on your professional knowledge, please analyze and infer the current fault characterization and give the cause of the fault and treatment measures. The fault characterization is as follows:" (prompt words) + "The speed change device of the gas turbine has shaken significantly, and the speed change box of the second gas turbine has triggered the vibration warning system" (fault characterization).

[0170] Answer part: "The speed change device shook and the second gas turbine speed change box vibration warning" (fault manifestation summary) + "The problem was caused by the loosening of the oil seal guard plate of the speed change gear box close to the gas turbine and the friction and wear of the coupling in contact with it" (fault cause) + "After investigating the characteristics and causes of the fault, the following maintenance and prevention measures were proposed: suspend water addition operations, reduce loads, clear the hardened material accumulated on the oil grid seal on one side of the speed change gear box, and readjust the clearance between the fast-rotating oil seal blocks" (fault handling measures).

[0171] Furthermore, the above question-answer text dataset is used to fine-tune the thermal power domain big model, and the cross-modal conversion module is trained according to steps (2) to (5). This enables the domain big model to output intelligent operation and maintenance decisions such as fault diagnosis, fault cause analysis, and fault handling measures after the user inputs the industrial field sensor time series data and prompt words. The big model intelligent decision-making process in this real thermal power plant embodiment is as follows: Figure 4 shown.

[0172] Furthermore, the effectiveness of step (6) of the present invention is illustrated by the fault diagnosis results under the zero-sample setting. The known information of the zero-sample setting problem is the fault characterization of the seen fault categories and the unseen fault categories, the fault cause analysis and the fault handling measures, as well as the sensor time series data of the seen categories. The model is trained to input the sensor time series data of the unseen categories, and the text embedding of the operation and maintenance decisions such as the fault cause and fault handling measures of the unseen fault categories can be obtained, that is, Figure 2 In the "aggregate features of large model output" section, we can further describe the Figure 5 As shown in the figure, the operation and maintenance decisions such as the fault causes and fault handling measures predicted by the large model are compared with the operation and maintenance records in the fault ticket, and the nearest neighbor classification in the text embedding space is performed to realize the classification of unseen fault categories.

[0173] The trained industrial model can output the summary features of the operation and maintenance decisions of unseen categories and seen categories with high discrimination, based on which the relevant operation and maintenance decision classification tasks can be performed. Figure 2 The t-SNE visualization of the summary features of the industrial domain large model output is shown in Figure 6 As shown in the figure, thanks to the powerful language ability and zero-sample generalization of the large model, the feature space of text embedding is highly distinguishable, compact within the class, and separated between classes, which can improve the classification accuracy.

[0174] The effect of the method of the present invention is measured by the zero-sample average accuracy. Here, ESZSL (Embarrassingly Simple Zero-Shot Learning), Attribute Label Embedding (ALE), Fault Description based Attribute Transfer (FDAT), and Semantic-Consistent Embedding for Zero-Shot Fault Diagnosis (SCE) methods are selected for experimental comparison, and the results are shown in Table 1.

[0175] Table 1: Comparison of experimental results of different methods (average accuracy)

[0176]

[0177] From the comparison results shown in Table 1, it can be seen that the average accuracy of the method of the present invention under four different zero-sample experimental settings is 0.7802, 0.7467, 0.6862, and 0.6647, respectively. It can be found that the method of the present invention has a significant improvement over the existing zero-sample method in the four groups of experiments. The experiment shows that the zero-sample reasoning ability based on the knowledge transfer of the large model in the industrial field can reveal the hidden associations between data and improve the accuracy of fault detection. Because the large model has an in-depth understanding of language and cultural background knowledge, the mapping relationship from the fault representation text to the fault cause and measure text has zero-sample generalization ability. That is, in the face of sensor time series data that does not exist in the training set, it can still be extrapolated and mapped to the relevant fault causes and fault handling measures. The above proves the feasibility and effectiveness of the method proposed in the present invention.

[0178] Example 2: Intelligent decision-making of large-scale model for operation and maintenance of metallurgical converter equipment

[0179] This embodiment takes the fault diagnosis task of measured data of converter equipment in a real steel plant as an example to illustrate the steps of the present invention and verify its effectiveness.

[0180] The main mechanical equipment in the steel industry is various transmission equipment and hydraulic equipment, such as machines, conveyor belts, various fans, etc. The molten iron produced by the blast furnace is mixed into the converter after the composition and temperature are homogenized in the mixing furnace, and scrap steel is added according to a predetermined ratio. Subsequently, oxygen blowing smelting is carried out through a water-cooled oxygen lance with a specific oxygen supply, lance position and slag making system. When the smelting reaches the predetermined end point, the lance is lifted and the furnace is poured, and the temperature is measured and the composition is analyzed. If the temperature and composition of the molten steel meet the target value, the steel tapping operation is carried out; if not, the oxygen blowing smelting continues. During the steel tapping process, deoxidizers and ferroalloys are added to the ladle to achieve deoxidation and alloying. Finally, the molten steel is sent to the mold casting site or continuous casting workshop for ingot casting. The field of steel smelting has attracted much attention due to the extreme nature of its operating environment, which includes extremely high temperature, high pressure, fast production rhythm and high requirements for output. In this industry, the reliability of the production line plays a decisive role in maintaining production efficiency, ensuring product quality and minimizing downtime. Potential failures of equipment pose a significant threat to the reliability of the production line, which may lead to production interruptions and economic losses. Therefore, it is urgent to build an efficient fault diagnosis mechanism to achieve real-time monitoring and fault warning of steel production lines, thereby ensuring the continuity and reliability of production.

[0181] The method of the present invention is applied to the operation and maintenance text and time series data of a real steel plant, such as Figure 7 As shown. By using the cross-modal conversion module described in step (3), the similarity between the time series data query feature and the embedded operation and maintenance text query feature corresponding to the texts such as "converter tilting motor overheating", "residual current alarm", "oxygen gun emergency stop alarm", "abnormal furnace pressure fluctuation", and "converter control response delay" is calculated. Based on the similarity, the weighted sum of the operation and maintenance text query features is performed to obtain the text embedding corresponding to the time series data, thereby realizing the conversion of time series sensor data into text space features.

[0182] An example of an operation and maintenance decision output by a fine-tuned large model for the steelmaking field is: "The steelmaking converter tilting system experiences abnormal jitter" (fault characterization summary) + "The problem is caused by severe wear of the converter trunnion bearings, which causes the converter to be unstable during the tilting process, resulting in jitter and noise. The coupling between the tilting motor and the reducer is damaged or loose, resulting in uneven power transmission" (fault cause) + "After investigating the fault characteristics and causes, the following maintenance and prevention measures are proposed: immediately stop the converter operation, inspect and replace the trunnion bearings, and inspect the coupling. If damaged, replace it; if loose, tighten it" (fault handling measures).

[0183] The operation and maintenance decisions output by large models in the steel smelting field can reduce the occurrence of system failures, and quickly respond to resume production when equipment failures occur, thereby improving operation and maintenance efficiency and equipment reliability, and providing decision support for production efficiency and cost control of production lines.

[0184] Example 3: Intelligent decision-making of large-scale model for chemical equipment operation and maintenance

[0185] This embodiment takes the fault diagnosis task of measured data of a real organic synthetic material chemical plant equipment as an example to illustrate the steps of the present invention and verify its effectiveness.

[0186] The chemical process mainly includes multiple operating units such as reactors, condensers, circulating compressors, separators and stripping towers. The reaction materials and catalysts are put into the reactor in proportion, heated to the set temperature by high-pressure steam introduced into the jacket, and maintained at a constant temperature to promote the exothermic reaction. Subsequently, the reaction products enter the condenser for cooling to control the reaction heat. Then, the condensed materials are circulated through the circulating compressor to maintain the material balance in the system. The separator is used to separate different components, while the stripping tower further purifies the product. The entire process ensures product quality and output through precise temperature and pressure control. In such a complex system, the system scale is large and the process variables are highly coupled, and mathematical mechanism modeling is difficult, which brings challenges to fault diagnosis.

[0187] The method of the present invention is applied to the operation and maintenance text and time series data of a real organic synthetic material chemical plant. An example of the operation and maintenance decision output by the fine-tuned large model in the field of iron and steel smelting is: "The temperature of the reactor rises abnormally" (fault characterization summary) + "The problem is caused by damage to the agitator blades, bearing wear or motor failure, or blockage of the cooling water pipe, failure of the cooling water pump, decreased heat exchange effect of the condenser, etc., which affect the uniformity of the reaction and cause local reaction overheating" (fault cause) + "After investigating the characteristics and causes of the fault, the following maintenance and prevention measures are proposed: emergency stop, immediately stop the feeding and heating of the reactor, and start the emergency cooling system, such as spray cooling or introduction of inert gas to reduce temperature and pressure" (fault handling measures).

[0188] like Figure 8 The figure shows the large-scale intelligent decision-making process of applying the present invention to an embodiment of a real organic synthetic material chemical plant. Compared with the intelligent operation and maintenance decision after fine-tuning, the large model without fine-tuning may misjudge the operation and maintenance status due to hallucination problems, resulting in incorrect fault diagnosis results, and then output wrong fault causes and treatment measures, which may seriously affect industrial production; even if the fault diagnosis results are close to the correct fault category, there is a problem that the output operation and maintenance analysis is not deep enough and comprehensive, and it is difficult to meet industrial needs. The intelligent operation and maintenance decision result obtained by the method of the present invention identifies the real fault category, and provides a reasonable fault cause analysis and effective fault treatment measures.

[0189] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large-scale intelligent decision-making method in the field of industrial operation and maintenance, characterized in that: The following steps are involved: (1) collecting operation and maintenance text materials of industrial objects, and using the operation and maintenance text materials to construct a structured question-answering text dataset for large model training based on a prompt word engineering method; wherein the question-answering text data is composed of question-answering text pairs, and the question-answering text pairs include a question part and an answer part; (2) Use the question-answer text dataset to fine-tune the pre-trained large model to obtain an industrial domain large model for industrial equipment operation and maintenance classification tasks; (3) constructing a cross-modal conversion module to convert the collected sensor time series data in the industrial process into features of aligned time series data, the cross-modal conversion module comprising a time series feature encoder, a time series query, a large model word segmenter, a large model encoder, a text query, a cross-modal attention weighter, and a modality alignment encoder; (4) Based on the prompt word engineering method, the task prompt text is constructed, and after being processed by the large model word segmenter and the large model encoder, the features of the aligned time series data corresponding to the sensor time series data and the text encoding features of the operation and maintenance text materials are spliced, and the spliced ​​time series features and text features are used as the input of the industrial field large model; (5) Using sensor time series data and operation and maintenance text materials as training set inputs and their corresponding industrial equipment operation and maintenance classification results as training set labels, the cross-modal conversion module, reconstructor, and classifier are trained. During the training process, the optimization goal is to minimize the total training loss function, and the parameters of the cross-modal conversion module, reconstructor, and classifier are adjusted to obtain the trained cross-modal conversion module, reconstructor, and classifier; (6) When applied, the trained large model segmenter, large model encoder, cross-modal conversion module, reconstructor, industrial domain large model and classifier are used to obtain the encoding features of the operation and maintenance decision of the sensor time series data to be tested and the classification results of the industrial equipment operation and maintenance classification task; The step (6) specifically comprises: Input the sensor time series data to be tested into the trained cross-modal conversion module to obtain the features of the time series data after modality alignment; At the same time, the task prompt text is passed through the large model word segmenter and the large model encoder to obtain the encoding features of the task prompt text. The encoding features of the task prompt text are spliced ​​with the features of the time series data after modal alignment and input into the industrial field large model to output the encoding features v of the corresponding operation and maintenance decision. answer ; The text description consisting of the summary of fault characterization, fault cause analysis and fault prevention or treatment measures of all categories is obtained through the large model encoder to obtain the text features of each category, where the text feature of the i-th category is recorded as Assume there are M categories, and there is a set Using the nearest neighbor pair v answer For classification, the calculation formula is: in, It is the classification result of industrial equipment operation and maintenance classification tasks.

2. The large-scale intelligent decision-making method in the field of industrial operation and maintenance according to claim 1 is characterized in that: The question part of the question-answer text pair includes role definition, task requirements and fault characterization, and the answer part of the question-answer text pair includes fault characterization summary, fault cause analysis and fault prevention or handling measures.

3. The large-scale intelligent decision-making method in the field of industrial operation and maintenance according to claim 1 is characterized in that: The question-answer text data is obtained specifically by the following method: The text descriptions of fault cases in the operation and maintenance text materials of industrial objects in the industrial process are structured and sorted, the descriptions related to the operation and maintenance faults are extracted, and question-answer text pairs are constructed; wherein the question part of the question-answer text pair is composed of prompt words and fault representations, the prompt words include an instruction part and a content part, and the prompt words are constructed by a prompt word engineering method based on role-playing and demand prompts; the answer part of the question-answer text pair is a description related to the fault, and its format is specified in advance, and the answer part provides a structured natural language text response template for the classification results of the equipment operation and maintenance tasks, including three parts of operation and maintenance decision information: fault representation summary, fault cause analysis, and fault prevention or handling measures.

4. The large-scale intelligent decision-making method for industrial operation and maintenance according to claim 1 is characterized in that: The step (2) specifically comprises: Based on the question-answer text dataset, the question part of the question-answer text pair is input into the pre-trained big model. Under the guidance of the prompt words in the question part, the big model extracts information from the question-answer text data to fit the mapping relationship between the fault representation text and the fault cause and fault prevention or treatment measures text, and obtains the answer part output by the big model; The total cross entropy loss function of the entire sequence is calculated based on the answer part output by the large model and the corresponding true answer part in the question-answer text pair. The weight parameters of the large model are fine-tuned with the optimization goal of minimizing the total cross entropy loss function of the entire sequence. Finally, an industrial field large model for industrial equipment operation and maintenance classification tasks is obtained.

5. The large-scale intelligent decision-making method for industrial operation and maintenance according to claim 4 is characterized in that: The calculation formula of the total cross entropy loss function of the entire sequence is: Among them, L represents the total cross entropy loss function of the entire sequence, L t represents the cross entropy loss function at time step t, T is the total time step length of the sequence, y i,t represents the i-th element in the one-hot encoding of the target word at time step t, p i,t Represents the probability of the i-th word predicted by the large model at time step t.

6. The large-scale intelligent decision-making method in the field of industrial operation and maintenance according to claim 1 is characterized in that: The cross-modal conversion module specifically includes: The sensor time series data collected in the industrial process is input into the cross-modal conversion module. First, the large model word segmenter and the large model encoder are applied to the operation and maintenance text material. The large model word segmenter is used to segment a section of operation and maintenance text material into word sequence, and then the large model encoder is used to convert the word sequence into text encoding features. The calculation method is expressed as follows: v = Embedding(Tokenizer(s)) Where v represents the text encoding feature, Embedding(·) represents the large model encoder, Tokenizer(·) represents the large model tokenizer, and s represents a string of input operation and maintenance text materials; Then the text encoding feature v is input into the text query to obtain the operation text query feature v w ; Secondly, a sensor time series data sample is selected and input into the time series feature encoder to obtain the time series encoding feature vector, whose calculation method is expressed as: h time =Encoder time (x) Among them, h time Represents the temporal encoding feature vector, Encoder time (·) represents the time series feature encoder, and x represents the sensor time series data sample; Then the time series encoding feature vector h time Input into the time series query device to obtain the time series data query feature v t ; Secondly, for each operation and maintenance text material, the corresponding operation and maintenance text query feature v can be obtained w , and then get the operation and maintenance text query matrix V w , where each line corresponds to the operation and maintenance text query feature of a section of operation and maintenance text material Query features v using query and key for time series data t and operation text query features v w Do a dot product operation to get the attention score vector, expressed as: Where ρ represents the attention score vector, Softmax(·) represents the activation function, and d k is the key-query space dimension; Then, the cross-modal attention weighter takes the attention score vector ρ as the similarity between each time series data query feature and the operation text query feature. Based on this similarity, the operation text query matrix V w Weighted summation is performed and the summation result is output as the weighted operation and maintenance text query feature. The calculation formula is as follows: v m′ =Sum(ρV w ) Among them, v m′ Represents the weighted operation and maintenance text query features; Finally, the weighted operation and maintenance text query feature v m′ Input to the modality alignment encoder to obtain the features v of the aligned time series data m , expressed as: v m =Encoder mode (v m′ ) Among them, Encoder mode (·) denotes the modality-aligned encoder.

7. The large-scale intelligent decision-making method in the field of industrial operation and maintenance according to claim 1 is characterized in that: The task prompt text is not the question part of a complete question-answer text pair, but only contains a prompt word text, wherein the fault representation part is missing. The question part of the complete question-answer text pair is restored by splicing the coding features of the task prompt text with the text coding features of the operation and maintenance text material or the features of the aligned time series data corresponding to the sensor time series data, thereby obtaining the text features of the question part of the complete question-answer text pair.

8. The large-scale intelligent decision-making method in the field of industrial operation and maintenance according to claim 1 is characterized in that: The step (4) specifically comprises: First, the text of the natural language mode of the operation and maintenance text material is input into the large model word segmenter and the large model encoder to obtain the text encoding features of the operation and maintenance text. The calculation formula is: v describe =Embedding(Tokenizer(s describe )) Among them, v describe Represents the text encoding features of the operation and maintenance text, s describe A fault characterization text string indicating a certain fault; Then, the task prompt text is input into the large model word segmenter and the large model encoder to obtain the encoding features of the task prompt text, and the calculation formula is: v task =Embedding(Tokenizer(s task )) Among them, v task Represents the encoding features of the task prompt text, s task Represents the task prompt text string; Secondly, the encoding features of the task prompt text are concatenated with the text encoding features of the operation and maintenance text or the features of the aligned time series data to obtain the concatenated text features or time series features as the text features of the question part in the complete question-answer text pair. The calculation formula is: or, in, Represents the question-part feature in the question-answer text pair corresponding to the concatenated text feature, It represents the question feature in the question-answer text pair corresponding to the concatenated temporal features. Concat(·) represents the concatenation of two feature vectors.

9. The large-scale intelligent decision-making method in the field of industrial operation and maintenance according to claim 1 is characterized in that: The step (5) specifically comprises: The sensor time series data and operation and maintenance text materials are used as the training set inputs, and the industrial equipment operation and maintenance classification results corresponding to the sensor time series data and operation and maintenance text materials are used as the training set labels. The cross-modal conversion module, reconstructor and classifier are trained. During the training process, the parameters of the large model segmenter, large model encoder and industrial field large model are frozen and not updated. Input the sensor time series data and operation and maintenance text materials into the cross-modal conversion module to obtain the features v of the aligned time series data m , the features v of the aligned time series data m Input into the reconstructor to obtain the reconstructed time series data; calculate the reconstruction error based on the reconstructed time series data and the original sensor time series data, and the calculation formula is: Among them, L r represents the reconstruction loss of a single sample, x i,t represents the value of the i-th original sensor time series data at time t, represents the value of the i-th reconstructed time series data at time t, and n is the total number of elements of the sensor time series data samples; Then, based on the sensor time series data and operation and maintenance text materials, the encoding features v of the task prompt text are task The text encoding features v of the operation and maintenance text are describe and the features v of the aligned time series data m After splicing, we get the corresponding problem features and Characterize some of the problems and Input them into the industrial field model respectively, and obtain the text features of the answer part corresponding to the operation and maintenance text respectively. Text features of the answer part corresponding to the time series data After passing through the classifier, the respective classification results prob are obtained. d and prob m ; According to the classification results when the splicing objects are operation and maintenance text and time series data, prob d and prob m Calculate the cross-modal knowledge alignment loss, and its calculation formula is: L m =‖prob d -prob m ‖2 Among them, L m represents the cross-modal knowledge alignment loss of a single sample, prob d Indicates the classification result when the splicing object is operation and maintenance text, prob m Indicates the classification result when the splicing object is time series data; Secondly, the cross entropy loss is calculated based on the classification results output by the classifier and the corresponding training set labels. As the classification loss, its calculation formula is: L c =CELoss(y,prob) Among them, L c represents the classification loss of a single sample, y represents the training set label of a single sample, and prob represents the classification result output by the classifier; Finally, the total training loss function is calculated based on the reconstruction error, cross-modal knowledge alignment loss and classification loss, which is expressed as: Among them, L loss represents the total training loss function, N is the total number of sensor time series data samples; taking minimizing the total training loss function as the optimization goal, the parameters of the cross-modal conversion module, reconstructor and classifier are adjusted to obtain the trained cross-modal conversion module, reconstructor and classifier.

Images