Steel production equipment fault diagnosis system based on large language model

Abstract

The invention belongs to the technical field of steel production equipment fault diagnosis, and particularly relates to a steel production equipment fault diagnosis system based on a large language model. The system comprises a data processing module, a large language model training module and a retrieval enhancement generation module. Through domain data processing and model training, an intelligent diagnosis platform specially used for steel production equipment is constructed. The data processing module is responsible for performing feature extraction on the sampled data and constructing a vector database to ensure that the model can perform efficient semantic matching; the training module performs field adaptive adjustment on the large language model to improve the reasoning ability of the large language model in a specific field; the retrieval enhancement generation module realizes fault analysis and technical guidance with strong pertinence and high accuracy by integrating a vector database and a generative answer mode; the system runs locally or on a privatized server, all data processing, model calculation and generation processes are ensured to be completed in a controlled environment, and the privacy of user data is fully protected.

Description

Technical Field

[0001] The invention belongs to the technical field of fault diagnosis of steel production equipment, and in particular relates to a fault diagnosis system for steel production equipment based on a large language model. Background Art

[0002] In recent years, with the rapid development of sensor technology and data collection methods, fault diagnosis of steel production equipment has gradually relied on a large amount of real-time data. However, how to extract effective information from massive sensor data and make accurate fault predictions is still a technical challenge. At the same time, fault diagnosis is not only a data-based problem, but also involves how to understand and process various forms of information during equipment operation, including logs, error messages, maintenance records, etc. Most existing systems lack the ability to effectively process these unstructured text data.

[0003] As an important branch of artificial intelligence, natural language processing has been widely used in the understanding and processing of various text data, including speech recognition, sentiment analysis, machine translation and other fields. In fault diagnosis, large language models can be used to parse text information such as equipment fault logs, maintenance records, user feedback, etc., to help the system better understand the type, location and severity of equipment faults. However, the fault diagnosis system based on large language models still faces some challenges: first, how to deal with the complex semantics and contextual information contained in equipment fault logs; second, how to automatically extract valuable information for diagnosis from a large number of equipment maintenance and operation records; finally, how to effectively integrate these text data from different sources with the equipment operation data to improve the accuracy and timeliness of diagnosis. Summary of the invention

[0004] The purpose of the present invention is to provide a steel production equipment fault diagnosis system based on a large language model with high accuracy and strong timeliness.

[0005] The steel production equipment fault diagnosis system based on the large language model proposed in the present invention mainly includes a data processing module suitable for retrieval enhancement generation, a large language model training module, and a large language model use module after training, namely, a retrieval enhancement generation (RAG) module. The system constructs an intelligent diagnosis platform dedicated to steel production equipment by processing domain data and training models; wherein the data processing module is responsible for feature extraction of sampled data and construction of a vector database to ensure that the model can perform efficient semantic matching; the training module performs domain adaptability adjustment on the large language model to enhance its reasoning ability in specific fields; the retrieval enhancement generation module realizes highly targeted and accurate fault analysis and technical guidance by integrating vector databases and generative answers. Specifically:

[0006] (1) The data processing module specifically includes:

[0007] (1) Collecting raw data: First, collect raw data from various data sources such as fault reports of steel production equipment, equipment operation manuals, maintenance records, etc.

[0008] (2) Data preprocessing: In order to ensure that the data can be used for vectorized processing, multiple preprocessing is required, including data cleaning, format normalization, and field labeling. Among them:

[0009] Data cleaning includes removing irrelevant information from the original data, such as numbers, timestamps, and other irrelevant descriptions, to improve the training and retrieval efficiency of the model.

[0010] Format normalization converts the cleaned data into a unified format, such as JSON or CSV, to ensure consistent formatting in further processing. This process also removes redundant spaces, symbols, and noise data, retaining only key fields that are useful for fault analysis.

[0011] Field labeling is to add labels or classification identifiers to each data field in a standardized format, such as "fault type", "operation steps", etc., so as to extract data features during vectorization.

[0012] (3) Data vectorization: After data preprocessing, a large language model (such as Qwen-2.5-32b) is used to generate a high-dimensional vector representation of each data item, thereby creating an efficient vector database. The semantic vector generated by the large language model can accurately express the semantic information related to the fault.

[0013] (4) Storage and indexing of vector data: Use the AnythingLLM platform (MintplexLabs.AnythingLLM:A platform for running and integrating local language models with privacy-first features. Available: https: / / anythingllm.com, 2024.) to store and index the generated vectorized data to support subsequent fast similarity retrieval, improve retrieval efficiency and save storage space. Specifically include:

[0014] First, the generated vector is optimized for dimensionality reduction. Let the vector in the database be represented by X∈R n×d , where n is the number of data and d is the vector dimension. Set the mean of each dimension to zero and center X to obtain:

[0015] X C =X-μ,

[0016] in, x i is the mean vector.

[0017] Next, calculate the covariance matrix of the centered data:

[0018]

[0019] It describes the changing relationship of data in each dimension.

[0020] Then, the covariance matrix C is decomposed into eigenvalues ​​to obtain:

[0021] C=UΛU T ,

[0022] Among them, U∈R d×d is the eigenvector matrix, Λ=diag(λ1,λ2,…,λ d ) is a diagonal matrix of eigenvalues, and the eigenvalues ​​are arranged from large to small.

[0023] Then, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected to construct the dimension reduction transformation matrix: Uk = [u1,u2,…,u k ]∈R d×k . The original centralized data X C Projected into k-dimensional space, we get the vector representation after dimensionality reduction:

[0024] Z=X c U k ∈R n×k .

[0025] Finally, the reduced-dimensional vector Z is stored as a new vector database for subsequent retrieval enhancement generation tasks. This method significantly reduces storage space and computational complexity by retaining the main information, improving the efficiency and performance of the model.

[0026] (5) Perform load balancing to ensure that the system can efficiently process large amounts of data. Specifically, this includes:

[0027] (a) Request layer load balancing: balance user requests and distribute them to the entry server;

[0028] (b) Retrieval layer load balancing: distributing retrieval tasks among nodes of the distributed vector database;

[0029] (c) Generation layer load balancing: Distribute computing tasks among generation modules.

[0030] (ii) The large language model training module trains the large language model to enhance its analysis and reasoning capabilities for data in the field of steel production equipment. In this process, low-rank adaptation (LoRA, excerpted from "Hu JE, Shen YL, Wallis P, et al. LoRA: Low-Rank Adaptation of Large Language Models[C]. Proceedings of the International Conference on Learning Representations(ICLR), 2021") technology is used to achieve efficient fine-tuning of data in the field of steel production equipment, ensuring that the large model retains general capabilities while focusing more on the specific needs of steel equipment fault diagnosis. The specific process is as follows:

[0031] During the training process, appropriate training parameters are set to ensure that the model can converge effectively; the training parameters include key parameters such as learning rate, batch size, and training rounds, and the learning rate scheduler is used in combination to adapt to the gradual convergence of the model to ensure stable model performance during training. The training tasks mainly cover the analysis of equipment failure causes and solution suggestions. Through multiple rounds of training, the domain adaptability of the model is gradually optimized so that it can provide structured and logically clear analysis and suggestions when answering user questions.

[0032] Furthermore, the goal of the RAG model is to jointly optimize the retrieval and generation parts so that the generated text is as close to the target text as possible. Assuming that the input of the model is the query q, the goal is to generate the correct answer y. The output of the model is the generated answer And some candidate documents D = {d1, d2, ..., d k}, these documents are used to enhance the generated context. During training, the loss function needs to consider both the retrieval effect and the generation effect. Therefore, the loss function of RAG is:

[0033]

[0034] L retrieval (q,D) is the loss of the retrieval part, which is optimized by maximizing the score of the retrieved relevant documents:

[0035] L retrieval (q,D)=-logP(D * |q),

[0036] Among them, D * is the correct document most relevant to query q, P(D * |q) is the probability given by the model for the document, i.e., cosine similarity. is the loss of the generation part, based on the cross entropy loss function:

[0037]

[0038] Here, y t is the tth word in the target answer, is the conditional probability of generating the target answer y given the query q and the retrieved documents D.

[0039] In addition, to ensure that the model output meets the needs of the domain, specific templates and instruction designs are adopted during the training process. During the model fine-tuning process, the "adoption of specific templates and instruction designs" is achieved by introducing structured domain-specific data formats and clear task instructions in the training data to ensure that the model can accurately identify equipment failures and provide effective solutions. In the training preparation stage, standardized templates are first designed for the input and output of the model. For example, for the fault diagnosis of steel equipment, the input template may include "fault phenomenon description" and "faulty component location", and the output template may include "fault cause analysis", "recommended solutions" and "operation steps". Through this structured template, the input and output information is ensured to be clear, which facilitates the model to learn how to extract key content and generate targeted answers.

[0040] After training is completed, the model output content is strictly quality controlled to filter out irrelevant or duplicate content and ensure the integrity and logic of the output answers.

[0041] (3) The retrieval enhancement generation (RAG) module, after completing the model training, uses the retrieval enhancement generation (RAG) module to improve the accuracy of fault diagnosis. The RAG module integrates a vector database and a large language model, combines the mode of generative questioning and retrieval answering, and provides users with accurate fault analysis and operation guidance.

[0042] When a user raises a question about equipment failure, the system converts the fault phenomenon entered by the user into a vector representation and searches the vector database to find historical records and fault solutions that are similar to the question semantics. Through this search, the system can quickly locate the fault cases, solutions, and operation instructions related to the user's question. Based on the retrieval results, the RAG module passes the retrieved results as context information to the large language model, so that the generated answers can refer to this related information, thereby generating more accurate and detailed fault diagnosis and operation suggestions.

[0043] Furthermore, in order to improve the accuracy and structure of the answers, a specific template and instruction design is used in the generation process of the RAG module. This design enables the model answer to present the cause of the equipment failure, the recommended solution and detailed operation steps.

[0044] Regarding specific templates, by introducing domain-specific instruction sets (such as "fault phenomenon description", "fault component location", "repair method guidance", etc.), the model is ensured to include information such as fault causes, operating procedures and preventive measures in its answers. Including:

[0045] (1) Fault cause analysis: Analyze the possible causes of the fault;

[0046] (2) Operational instructions: Provide specific operating steps for maintenance and resumption of production:

[0047] Regarding instruction design, in order to facilitate program data processing, the input and output are uniformly in json format. In order to respond quickly, the present invention adopts a streaming output interface: that is, each time the model generates a word, it immediately outputs a json containing the generated content. JSON transmits two parts of data, the first part is the content of the question, and the second part is the specified LLM answer mode. LLM's answer mode is divided into two types: query and chat: the query mode is concise and accurate, providing direct fault diagnosis and solutions; the chat mode is more flexible, and can gradually analyze the cause of equipment failure through multiple rounds of interaction.

[0048] The present invention has at least the following beneficial effects:

[0049] By combining the large language model with the retrieval-augmented generation (RAG) technology, domain-related knowledge can be retrieved before generating answers, thereby effectively combining historical fault records and operating manuals and other information to achieve high-precision fault diagnosis, improve the accuracy and completeness of diagnosis, and avoid misjudgments caused by knowledge blind spots in traditional methods. In addition, through the data processing module, the system can quickly structure the sampled data and store it in the vector database, so that when the equipment fails, it can quickly complete retrieval and diagnosis generation, greatly shorten the analysis time, provide timely technical support for the production process, and reduce equipment downtime. By using LoRA technology to train the large language model, the model's understanding and reasoning ability in the field of steel production equipment is further enhanced, ensuring that the model can provide accurate analysis and operation suggestions for complex faults, and improving the professionalism and adaptability of the model. The system's local deployment environment ensures data privacy and security. Combined with the retrieval-augmented generation method, there is no need for external network access, which avoids the leakage of equipment failure data and maintenance records and meets the high standards for data privacy in steel production. Since the system adopts a modular design, each module is independent and easy to expand, and data processing, fine-tuning models and retrieval modules can be flexibly updated to ensure the system's rapid adaptability when facing new fault types and data updates, while facilitating maintenance and continuous optimization, thereby enhancing the system's practicality and scalability. The system runs entirely on local or private servers without the need for networking, ensuring that all data processing, model calculation and generation processes are completed in a controlled environment, fully protecting the privacy of user data.

[0050] Other advantages, objectives and features of the present invention will be reflected in part from the following description, and in part will be understood by those skilled in the art through study and practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0052] The intelligent steel production equipment fault diagnosis system based on a large language model proposed in the present invention includes a data processing module, a large language model training module, and a use module based on retrieval enhancement generation, which combines multiple data sources to realize efficient fault diagnosis of steel production equipment. The system first converts the diversified data sources of steel production equipment into structured data through a data processing module suitable for retrieval enhancement generation, which is convenient for model use. The data processing module first pre-processes the collected original data such as equipment fault reports, operation manuals, maintenance records, etc., specifically including steps such as data cleaning, format normalization and field labeling, so as to facilitate subsequent vectorization processing. In the data cleaning process, irrelevant information such as timestamps, numbers and task-irrelevant descriptions are removed, which further improves the efficiency of the model during training and retrieval. Subsequently, after format normalization processing, the cleaned data is converted into a unified format such as JSON or CSV to ensure data format consistency and facilitate vectorization processing. After processing, the data fields are labeled or classified, such as "fault type" or "operation steps", which is convenient for extracting key features of the data during vectorization.

[0053] The structured data is embedded in the model to generate a high-dimensional semantic vector representation for each piece of data, and an efficient vector database is constructed. Large language models such as Qwen-2.5-32b are used to generate semantic vectors to effectively capture the expression of fault-related information. Then, the AnythingLLM platform is used to store and index the generated vector data, which facilitates subsequent rapid retrieval. In order to improve the retrieval speed and save storage space, the system reduces the dimension of the generated vectors and configures the database for load balancing to ensure the efficiency and stability of the system when processing large-scale data.

[0054] In order to improve the understanding and reasoning ability of the large language model in the field of steel production, the present invention fine-tunes the model for domain adaptability through large language model training. The fine-tuning process uses LoRA (low-rank adaptation) technology to focus on the fault diagnosis needs of steel production equipment while maintaining the general capabilities of the model. The training data set contains equipment failure reports, operating manuals, maintenance records, and typical failure history cases. The data preprocessing process eliminates non-critical information and only retains key information such as fault type, cause, operating steps, and preventive measures, thereby enhancing the accuracy and professionalism of the model's answers. During fine-tuning, the learning rate, batch size, and training rounds are reasonably set, and the training parameters are dynamically adjusted in combination with the learning rate scheduler to ensure that the model has stable performance during the convergence process. During the training process, emphasis is placed on equipment failure cause analysis and solution recommendation to ensure that the model can generate logically clear and structured fault diagnosis and operational recommendations.

[0055] In addition, to make the model structured and logically clear, domain-specific instructions and template designs were added to guide the model to present the cause of the fault, recommended solutions, and detailed operation steps in its answers. By using these domain-specific instruction sets in each training, the model can accurately identify and parse different fault information, thereby providing practical guidance content. After fine-tuning, the system strictly controls the quality of the model output, filters irrelevant or repeated answers, and ensures the integrity and logic of the diagnostic output.

[0056] Finally, in the answer module based on retrieval enhancement generation, the accuracy of fault diagnosis is further improved by combining the historical data retrieved from the vector database with the model-generated answer. When the user enters a question, the system first converts it into a vector representation and retrieves historical fault records and solutions with similar semantics to the question in the vector database, so as to quickly locate the cases and operation instructions related to the user's question. The retrieved results are passed to the large language model as context information, and these related information are referenced when generating the final answer, making the generated answer more accurate and complete. To ensure the accuracy and structure of the answer, specific instructions and templates are used in the RAG module to ensure that the generated answer contains necessary information such as the cause of the fault, operating steps and preventive measures, so as to avoid redundant content affecting the final answer output quality. Through refined vectorization technology and secondary screening of the large language model, the answers generated by the system meet the complex fault diagnosis requirements of steel production equipment in terms of accuracy and real-time performance, and through the cache and automatic update mechanism, the system can continuously use the latest fault records and diagnostic information, thereby continuously optimizing the answer quality in a dynamic production environment.

[0057] It can be seen from the data that the training module proposed in the present invention and the usage module based on retrieval enhancement generation have a significant improvement in prediction accuracy as a whole. Specifically, when the Qwen2.5-32b model is used without fine-tuning and RAG technology is not used, the accuracy of cause analysis is only 61.8%, and the accuracy of the operation guidance part is 14.7%; using the Qwen2.5-32b model after fine-tuning and using RAG technology for experiments, the accuracy of the fault cause analysis task is 86.7%, and the accuracy of the technical guidance task is 81.0%. The average response time for each conversation is increased to 2.24s, which shows that the method proposed in this article is accurate and efficient in the task of fault diagnosis of steel production equipment.

Claims

1. A steel production equipment fault diagnosis system based on a large language model, characterized in that: It includes a data processing module suitable for retrieval enhancement generation, a large language model training module, and a large language model usage module after training, namely the retrieval enhancement generation (RAG) module; through the processing of domain data and model training, an intelligent diagnosis platform dedicated to steel production equipment is constructed; among them, the data processing module is responsible for feature extraction of sampled data and construction of vector database to ensure that the model can perform efficient semantic matching; the training module makes domain adaptability adjustments to the large language model to enhance its reasoning ability in specific fields; the retrieval enhancement generation module realizes highly targeted and accurate fault analysis and technical guidance by integrating vector database and generative answers; specifically: (1) The data processing module specifically includes: (1) Collecting raw data: Collecting raw data from multiple data sources such as fault reports, equipment operation manuals, and maintenance records of steel production equipment; (2) Data preprocessing, including data cleaning, format normalization, and field labeling; (3) Data vectorization: After data preprocessing, a large language model is used to generate a high-dimensional vector representation of each piece of data, thereby creating an efficient vector database. The semantic vector generated by the large language model can accurately express the semantic information related to the fault. (4) Storage and indexing of vector data: Use the AnythingLLM platform to store and index the generated vectorized data to support subsequent fast similarity retrieval, improve retrieval efficiency and save storage space; (5) Perform load balancing to ensure that the system can efficiently process large amounts of data; (ii) the large language model training module, which trains the large language model to enhance its analysis and reasoning capabilities for data in the field of steel production equipment; in this process, efficient fine-tuning of data in the field of steel production equipment is achieved specifically through low-rank adaptive technology, ensuring that the large model retains its general capabilities while focusing more on the specific needs of steel equipment fault diagnosis; During the training process, appropriate training parameters are set to ensure that the model can converge effectively; the training parameters include key parameters such as learning rate, batch size, and training rounds, and the learning rate scheduler is used in combination to adapt to the gradual convergence of the model to ensure stable model performance during the training process; the training tasks mainly cover equipment failure cause analysis and solution suggestions, and the domain adaptability of the model is gradually optimized through multiple rounds of training, so that it can provide structured and logically clear analysis and suggestions when answering user questions; (3) After completing the model training, the retrieval enhancement generation (RAG) module is used to improve the accuracy of fault diagnosis; the RAG module integrates a vector database and a large language model, combines the mode of generative questioning and retrieval answering, and provides users with accurate fault analysis and operation guidance; specifically: When a user raises a question about equipment failure, the system converts the fault phenomenon input by the user into a vector representation and searches the vector database to find historical records and fault solutions that are similar to the question semantics; through the search, it can quickly locate the fault cases, solutions and operation instructions related to the user's question; based on the retrieval results, the RAG module passes the retrieved results as context information to the large language model, so that the generated answers can refer to this related information, thereby generating more accurate and detailed fault diagnosis and operation suggestions.

2. The steel production equipment fault diagnosis system according to claim 1, characterized in that: In the data processing module: The data preprocessing includes: Data cleaning, including removing irrelevant information from the original data, including numbers, timestamps, and other irrelevant descriptions, to improve the training and retrieval efficiency of the model; Format normalization: converting the cleaned data into a unified format to ensure consistency in further processing; this process also removes redundant spaces, symbols, and noise data, retaining only key fields that are useful for fault analysis; Field annotation, that is, adding labels or classification identifiers to each data field in a standardized format so as to extract data features during vectorization; The storage and index construction of the vector data specifically include: First, the generated vector is optimized for dimensionality reduction. Let the vector in the database be represented by X∈R n×d , where n is the number of data and d is the vector dimension; return the mean of each dimension to zero and center X to obtain: X C =X-μ, in, x i is the mean vector; Next, calculate the covariance matrix of the centered data: It describes the changing relationship of data in each dimension; Then, the covariance matrix C is decomposed into eigenvalues ​​to obtain: C=UΛU T , Among them, U∈R d×d is the eigenvector matrix, Λ=diag(λ1,λ2,…,λ d ) is a diagonal matrix composed of eigenvalues, and the eigenvalues ​​are arranged from large to small; Then, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected to construct the dimension reduction transformation matrix: Uk = [u1,u2,…,u k ]∈R d×k ; The original centralized data X C Projected into k-dimensional space, we get the vector representation after dimensionality reduction: Z=X c U k ∈R n×k ; Finally, the dimension-reduced vector Z is stored as a new vector database for subsequent retrieval enhancement generation tasks; The load balancing specifically includes: (a) Request layer load balancing: balance user requests and distribute them to the entry server; (b) Retrieval layer load balancing: distributing retrieval tasks among nodes of the distributed vector database; (c) Generation layer load balancing: Distribute computing tasks among generation modules.

3. The steel production equipment fault diagnosis system according to claim 2, characterized in that: In the large language model training module, for the RAG model, the goal is to jointly optimize the retrieval and generation parts so that the generated text is as close to the target text as possible; assuming that the input of the model is the query q, the goal is to generate the correct answer y, and the output of the model is the generated answer And retrieve some candidate documents D = {d1, d2, ..., d k }, these documents are used to enhance the generated context; during training, the loss function considers both the retrieval effect and the generation effect; therefore, the loss function of RAG is: L retrieval (q,D) is the loss of the retrieval part, which is optimized by maximizing the score of the retrieved relevant documents: L retrieval (q,D)=-logP(D * |q), Among them, D * is the correct document most relevant to query q, P(D * |q) is the probability given by the model for the document, i.e., cosine similarity; is the loss of the generation part, based on the cross entropy loss function: Here, y t is the tth word in the target answer, is the conditional probability of generating the target answer y given the query q and the retrieved documents D; In addition, to ensure that the model output meets the needs of the domain, specific templates and instruction designs are used in the training process; in the process of model fine-tuning, the "specific templates and instruction designs" are implemented by introducing structured domain-specific data formats and clear task instructions in the training data to ensure that the model can accurately identify equipment failures and provide effective solutions; in the training preparation stage, standardized templates are first designed for the input and output of the model; for example, for the fault diagnosis of steel equipment, the input template includes "fault phenomenon description" and "fault component location", and the output template includes "fault cause analysis", "recommended solutions" and "operation steps"; through this structured template, the input and output information is ensured to be clear, which facilitates the model to learn how to extract key content and generate targeted answers; After training is completed, the model output content is strictly quality controlled to filter out irrelevant or duplicate content and ensure the integrity and logic of the output answers.

4. The steel production equipment fault diagnosis system according to claim 3, characterized in that: In the retrieval enhancement generation (RAG) module, in order to improve the accuracy and structure of the answers, a specific template and instruction design are used in the generation process of the RAG module so that the model answer can present the cause of the equipment failure, the recommended solution and the detailed operation steps; The specific template introduces a domain-specific instruction set, including "fault phenomenon description", "fault component location", and "repair method guidance", to ensure that the model includes fault cause, operation steps and preventive measures information in the answer; specifically, it includes: (1) Fault cause analysis: Analyze the possible causes of the fault; (2) Operational instructions: provide specific operating steps for maintenance and resumption of production; In order to facilitate the program to process data, the instructions use a unified json format for input and output; in order to respond quickly, a streaming output interface is used: that is, each time the model generates a word, it immediately outputs a json containing the generated content; json transmits two parts of data, the first part is the content of the question, and the second part is the specified LLM answer mode; LLM's answer mode is divided into query and chat: the query mode is concise and accurate, providing direct fault diagnosis and solutions; the chat mode is more flexible, and can gradually analyze the cause of the equipment failure through multiple rounds of interaction.

Images