A steel production equipment fault diagnosis system based on a large language model

Abstract

The application belongs to the technical field of steel production equipment fault diagnosis, and particularly relates to a steel production equipment fault diagnosis system based on a large language model.The system comprises a data processing module, a large language model training module and a retrieval enhancement generation module; through processing of field data and model training, an intelligent diagnosis platform special for steel production equipment is constructed; the data processing module is responsible for feature extraction on sampling data and construction of a vector database, so that the model can perform efficient semantic matching; the training module performs field adaptability adjustment on the large language model, so as to improve the reasoning capability of the large language model in a specific field; the retrieval enhancement generation module realizes fault analysis and technical guidance with strong pertinence and high accuracy through integration of the vector database and a generative answer; the system runs on a local or private server, so that all data processing, model calculation and generation processes are completed in a controlled environment, and the privacy of user data is fully protected.

Description

Technical Field

[0001] This invention belongs to the field of fault diagnosis technology for steel production equipment, and specifically relates to a fault diagnosis system for steel production equipment based on a large language model. Background Technology

[0002] In recent years, with the rapid development of sensor technology and data acquisition methods, fault diagnosis of steel production equipment has increasingly relied on large amounts of real-time data. However, extracting effective information from massive amounts of sensor data and making accurate fault predictions remains a technical challenge. At the same time, fault diagnosis is not merely a data-driven problem; it also involves understanding and processing various forms of information generated during equipment operation, including logs, error messages, and maintenance records. Most existing systems lack the ability to effectively process this unstructured text data.

[0003] Natural Language Processing (NLP), as an important branch of artificial intelligence, has been widely applied to the understanding and processing of various types of text data, including speech recognition, sentiment analysis, and machine translation. In fault diagnosis, large language models can be used to parse textual information such as equipment fault logs, maintenance records, and user feedback, thereby helping the system better understand the type, location, and severity of equipment faults. However, current fault diagnosis systems based on large language models still face several challenges: First, how to handle the complex semantics and contextual information contained in equipment fault logs; second, how to automatically extract valuable diagnostic information from a large amount of equipment maintenance and operation records; and finally, how to effectively integrate this textual data from different sources with the equipment's operational data to improve the accuracy and timeliness of diagnosis. Summary of the Invention

[0004] The purpose of this invention is to provide a fault diagnosis system for steel production equipment based on a large language model that is highly accurate and timely.

[0005] The steel production equipment fault diagnosis system based on a large language model proposed in this invention mainly includes a data processing module suitable for retrieval enhancement generation, a large language model training module, and a module for using the trained large language model, namely the retrieval enhancement generation (RAG) module. This system constructs an intelligent diagnostic platform specifically for steel production equipment through domain data processing and model training. The data processing module is responsible for extracting features from sampled data and constructing a vector database to ensure the model can perform efficient semantic matching. The training module performs domain-adaptive adjustments to the large language model, improving its reasoning ability in specific domains. The retrieval enhancement generation module, through integrating the vector database and generative responses, achieves targeted and highly accurate fault analysis and technical guidance. Specifically:

[0006] (a) The data processing module specifically includes:

[0007] (1) Collect raw data; First, collect raw data from various data sources such as fault reports of steel production equipment, equipment operation manuals, and maintenance records.

[0008] (2) Data preprocessing; To ensure the data can be used for vectorization, multiple preprocessing steps are required; these include data cleaning, format normalization, and field labeling. Among these:

[0009] Data cleaning involves removing irrelevant information from the original data, such as serial numbers, timestamps, and other irrelevant descriptions, to improve the training and retrieval efficiency of the model.

[0010] Format standardization involves converting the cleaned data into a uniform format, such as JSON or CSV, to ensure consistency during further processing. This process also removes redundant spaces, symbols, and noisy data, retaining only key fields useful for fault analysis.

[0011] Field labeling involves adding labels or category identifiers to each standardized data field, such as "fault type" or "operation steps," to extract data features during vectorization.

[0012] (3) Data vectorization: After data preprocessing, a large language model (e.g., Qwen-2.5-32b) is used to generate a high-dimensional vector representation of each data point, thereby creating an efficient vector database. The semantic vectors generated by the large language model can accurately express fault-related semantic information.

[0013] (4) Storage and indexing of vector data; The AnythingLLM platform (MintplexLabs. AnythingLLM: A platform for running and integrating local language models with privacy-first features. Available: https: / / anythingllm.com, 2024.) is used to store and index the generated vectorized data to support subsequent fast similarity retrieval, and to improve retrieval efficiency and save storage space. Specifically, this includes:

[0014] First, the generated vectors are optimized for dimensionality reduction. Let the vector representation in the database be X∈R. n×d Where n is the number of data entries and d is the vector dimension. By setting the mean of each dimension to zero and centering X, we obtain:

[0015] X C =X-μ,

[0016] in, x i It is the mean vector.

[0017] Next, calculate the covariance matrix of the centered data:

[0018]

[0019] It describes the relationship of data changes in each dimension.

[0020] Then, eigenvalue decomposition is performed on the covariance matrix C to obtain:

[0021] C=UΛU T ,

[0022] Where, U∈R d×d It is the eigenvector matrix, Λ=diag(λ1,λ2,…,λ d ) is a diagonal matrix composed of eigenvalues, and the eigenvalues ​​are arranged in descending order.

[0023] Subsequently, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected to construct the dimension reduction transformation matrix: Uk = [u1, u2, ..., u k ]∈R d×k The original centralized data X C Projecting onto a k-dimensional space yields the reduced-dimensional vector representation:

[0024] Z = X c U k ∈R n×k .

[0025] Finally, the dimensionality-reduced vector Z is stored as a new vector database for subsequent retrieval-enhanced generation tasks. This method significantly reduces storage space and computational complexity by preserving key information, thereby improving the model's efficiency and performance.

[0026] (5) Implement load balancing to ensure the system can efficiently process large-scale data. Specifically, this includes:

[0027] (a) Request layer load balancing: Balances user requests and distributes them to the entry server;

[0028] (b) Load balancing at the retrieval layer: Distributing retrieval tasks among nodes in the distributed vector database;

[0029] (c) Generation layer load balancing: Distribute computing tasks across the generation module.

[0030] (II) The large language model training module enhances the large language model's ability to analyze and reason about data from the steel production equipment domain by training it. Specifically, this is achieved through low-rank adaptive modeling (LoRA, excerpted from "Hu JE, Shen YL, Wallis P, et al. LoRA: Low-Rank Adaptation of Large Language Models[C]. Proceedings of the International Conference on Learning Representations(ICLR),2021") technology, enabling efficient fine-tuning of data from the steel production equipment domain. This ensures that the large model retains its general capabilities while focusing more on the specific needs of steel equipment fault diagnosis. The specific process is as follows:

[0031] During training, appropriate training parameters are set to ensure effective model convergence. These parameters include key parameters such as learning rate, batch size, and training epochs. A learning rate scheduler is used to adapt to the gradual convergence of the model, ensuring stable model performance during training. The training tasks mainly cover equipment failure cause analysis and solution suggestions. Through multiple rounds of training, the domain adaptability of the model is gradually optimized, enabling it to provide structured and logically clear analysis and suggestions when answering user questions.

[0032] Furthermore, the goal of the RAG model is to jointly optimize the retrieval and generation parts, making the generated text as close as possible to the target text. Assume the model's input is a query q, and the goal is to generate the correct answer y. The model's output is the generated answer. Furthermore, some candidate documents D = {d1, d2, ..., d...} were retrieved from the knowledge base. k These documents are used to enhance the context of the generation. During training, the loss function needs to consider both the retrieval and generation effects. Therefore, the loss function for RAG is:

[0033]

[0034] L retrieval (q,D) is the loss of the retrieval part, which is optimized by maximizing the score of the relevant documents retrieved:

[0035] L retrieval (q,D)=-logP(D * |q),

[0036] Among them, D * The correct document most relevant to query q, P(D) * |q) is the probability that the model gives of the document, i.e., cosine similarity. The loss is for the generated part, based on the cross-entropy loss function:

[0037]

[0038] Here, y t It is the t-th word in the target answer. It is the conditional probability of generating the target answer y given the query q and the retrieved document D.

[0039] Furthermore, to ensure the model output meets domain requirements, specific templates and instructions are designed during training. During model fine-tuning, this "specific template and instruction design" is achieved by introducing structured domain-specific data formats and explicit task instructions into the training data, ensuring the model can accurately identify equipment faults and provide effective solutions. In the training preparation phase, standardized templates are first designed for the model's input and output. For example, for fault diagnosis of steel equipment, the input template may include "fault phenomenon description" and "fault component location," while the output template may include "fault cause analysis," "recommended solutions," and "operation steps." This structured template ensures clear input and output information, facilitating the model's learning of how to extract key content and generate targeted answers.

[0040] After training, the model output is subjected to strict quality control to filter out irrelevant or repetitive content, ensuring the completeness and logic of the output answer.

[0041] (3) The Retrieval Enhancement Generation (RAG) module improves the accuracy of fault diagnosis after model training is completed. This RAG module integrates a vector database and a large language model, combining generative questioning and retrieval-based answering to provide users with accurate fault analysis and operational guidance.

[0042] When a user reports a device malfunction, the system converts the user's input malfunction symptom into a vector representation and searches the vector database to find historical records and solutions with similar semantics to the problem. Through this retrieval, the system can quickly locate malfunction cases, solutions, and operational guidance related to the user's problem. Based on the search results, the RAG module passes the retrieved results as contextual information to the large language model, allowing the generated answer to reference this related information, thereby producing more accurate and detailed fault diagnosis and operational suggestions.

[0043] Furthermore, to improve the accuracy and structure of the responses, specific templates and instructions are designed during the generation of the RAG module. This design enables the model's responses to present the causes of equipment malfunctions, recommended solutions, and detailed operating procedures.

[0044] Regarding specific templates, by introducing domain-specific instruction sets (such as "fault phenomenon description," "fault component location," and "repair method guidance"), the model ensures that its responses include information such as the cause of the fault, operating procedures, and preventative measures. This includes:

[0045] (1) Fault cause analysis: Analyze the possible causes of this fault;

[0046] (2) Operational instructions: Provide specific operational steps for maintenance and production restoration:

[0047] Regarding instruction design, to facilitate data processing, input and output are uniformly formatted as JSON. For rapid response, this invention employs a streaming output interface: each time the model generates a word, it immediately outputs a JSON file containing that generated content. The JSON transmits two parts of data: the first part is the question content, and the second part specifies the LLM's response mode. The LLM's response modes are divided into query and chat: the query mode is concise and precise, providing direct fault diagnosis and solutions; the chat mode is more flexible, allowing for gradual analysis of the cause of equipment failure through multiple rounds of interaction.

[0048] The present invention has at least the following beneficial effects:

[0049] By combining a large language model with Retrieval Augmentation (RAG) technology, domain-related knowledge can be retrieved before generating answers. This effectively integrates information such as historical fault records and operation manuals to achieve high-precision fault diagnosis, improving the accuracy and completeness of diagnosis and avoiding misjudgments caused by knowledge blind spots in traditional methods. Furthermore, through the data processing module, the system can quickly structure and store sampled data in a vector database, enabling rapid retrieval and diagnostic generation when equipment malfunctions, significantly shortening analysis time, providing timely technical support for the production process, and reducing equipment downtime. By using LoRA technology to train the large language model, the model's understanding and reasoning capabilities in the steel production equipment domain are further enhanced, ensuring that the model can provide accurate analysis and operational suggestions for complex faults, improving the model's professionalism and adaptability. The system's local deployment environment ensures data privacy and security. Combined with the retrieval augmentation generation method, no external network access is required, preventing the leakage of equipment fault data and maintenance records, meeting the high standards of data privacy required in steel production. Because the system adopts a modular design, each module is independent and easily expandable, allowing for flexible updates to data processing, model fine-tuning, and retrieval modules. This ensures the system's rapid adaptability to new fault types and data updates, while also facilitating maintenance and continuous optimization, thereby enhancing the system's practicality and scalability. The system operates entirely locally or on a private server, without requiring an internet connection, ensuring that all data processing, model calculations, and generation processes are completed in a controlled environment, fully protecting user data privacy.

[0050] Other advantages, objectives and features of the present invention will be apparent in part from the following description, and in part from the understanding of those skilled in the art through study and practice of the invention. Attached Figure Description

[0051] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0052] This invention proposes an intelligent fault diagnosis system for steel production equipment based on a large language model. It includes a data processing module, a large language model training module, and a retrieval-enhanced generation module, combining multiple data sources to achieve efficient fault diagnosis of steel production equipment. The system first transforms diverse data sources from steel production equipment into structured data through a data processing module suitable for retrieval-enhanced generation, facilitating model use. The data processing module preprocesses raw data such as equipment fault reports, operation manuals, and maintenance records, including data cleaning, format normalization, and field labeling, to facilitate subsequent vectorization. During data cleaning, irrelevant information, such as timestamps, serial numbers, and task-irrelevant descriptions, is removed, further improving the model's efficiency during training and retrieval. Subsequently, format normalization converts the cleaned data into a unified format such as JSON or CSV, ensuring data consistency and facilitating vectorization. After processing, data fields are labeled or categorized, such as "fault type" or "operation steps," to facilitate the extraction of key features during vectorization.

[0053] The structured data is used to generate high-dimensional semantic vector representations for each data point through an embedding model, and an efficient vector database is constructed. Large language models such as Qwen-2.5-32b are employed to generate semantic vectors, effectively capturing fault-related information. Next, the generated vector data is stored and indexed using the AnythingLLM platform, facilitating subsequent rapid retrieval. To improve retrieval speed and save storage space, the system performs dimensionality reduction on the generated vectors and implements load balancing for the database, ensuring system efficiency and stability when processing large-scale data.

[0054] To improve the understanding and reasoning capabilities of a large language model in the steel production field, this invention employs domain-adaptive fine-tuning during large language model training. The fine-tuning process utilizes LoRA (Low-Rank Adaptive) technology to focus on the fault diagnosis needs of steel production equipment while maintaining the model's general capabilities. The training dataset includes equipment fault reports, operation manuals, maintenance records, and typical fault historical cases. Data preprocessing removes non-critical information, retaining only key information such as fault type, cause, operating steps, and preventative measures, thereby enhancing the accuracy and professionalism of the model's responses. During fine-tuning, the learning rate, batch size, and training epochs are appropriately set, and training parameters are dynamically adjusted using a learning rate scheduler to ensure stable model performance during convergence. The training process emphasizes equipment fault cause analysis and solution recommendations to ensure the model can generate logically clear and structured fault diagnoses and operational suggestions.

[0055] Furthermore, to ensure the model's structured and logically clear operation, domain-specific instructions and templates were incorporated to guide the model in presenting the cause of the fault, recommended solutions, and detailed operational steps in its responses. By using these domain-specific instruction sets in each training session, the model accurately identifies and parses different fault information, thereby providing highly practical guidance. After fine-tuning, the system performs rigorous quality control on the model's output, filtering out irrelevant or duplicate responses to ensure the completeness and logical consistency of the diagnostic output.

[0056] Finally, in the response module based on retrieval enhancement, the accuracy of fault diagnosis is further improved by combining historical data retrieved from the vector database with model-generated responses. When a user inputs a question, the system first converts it into a vector representation and then retrieves historical fault records and solutions with semantically similar questions from the vector database, thereby quickly locating cases and operational guidance related to the user's question. The retrieved results are passed as contextual information to the large language model, which references this related information when generating the final response, making the generated response more accurate and complete. To ensure the accuracy and structure of the response, specific instructions and templates are used in the RAG module to ensure that the generated response includes necessary information such as fault causes, operational steps, and preventive measures, avoiding redundant content that could affect the quality of the final response output. Through refined vectorization technology and secondary filtering by the large language model, the system-generated responses meet the complex fault diagnosis needs of steel production equipment in terms of accuracy and real-time performance. Furthermore, caching and automatic update mechanisms ensure that the system can continuously utilize the latest fault records and diagnostic information, thereby continuously optimizing response quality in a dynamic production environment.

[0057] As the data shows, the training module and the usage module based on retrieval enhancement proposed in this invention significantly improve the overall prediction accuracy. Specifically, when using the Qwen2.5-32b model without fine-tuning and without RAG technology, the accuracy rate for cause analysis is only 61.8%, and the accuracy rate for operation guidance is 14.7%. After fine-tuning the Qwen2.5-32b model and using RAG technology, the accuracy rate reaches 86.7% for fault cause analysis and 81.0% for technical guidance, with the average response time per dialogue improved to 2.24 seconds. This demonstrates that the proposed method is accurate and efficient in fault diagnosis of steel production equipment.

Claims

1. A fault diagnosis system for steel production equipment based on a large language model, characterized in that, This includes a data processing module for retrieval enhancement generation, a large language model training module, and a module for using the trained large language model, namely the retrieval enhancement generation (RAG) module. Through processing domain data and training the model, a smart diagnostic platform specifically for steel production equipment is constructed. The data processing module is responsible for extracting features from sampled data and constructing a vector database to ensure the model can perform efficient semantic matching. The training module performs domain-adaptive adjustments to the large language model, improving its reasoning ability in specific domains. The retrieval enhancement generation module, through integrating the vector database and generating responses, achieves targeted and highly accurate fault analysis and technical guidance. Specifically: (a) The data processing module specifically includes: (1) Collect raw data; collect raw data from various data sources such as fault reports of steel production equipment, equipment operation manuals, and maintenance records; (2) Data preprocessing; including data cleaning, format normalization, and field labeling; (3) Data vectorization; After data preprocessing, a high-dimensional vector representation of each data item is generated using a large language model, thereby creating an efficient vector database; The semantic vectors generated by the large language model can accurately express fault-related semantic information; (4) Storage and indexing of vector data; The AnythingLLM platform is used to store and index the generated vectorized data to support subsequent fast similarity retrieval and to improve retrieval efficiency and save storage space. (5) Perform load balancing to ensure that the system can efficiently process large-scale data; (ii) The large language model training module enhances the ability of the large language model to analyze and reason about data in the field of steel production equipment by training the large language model. In this process, low-rank adaptive technology is used to achieve efficient fine-tuning of data in the field of steel production equipment, ensuring that the large model retains its general capabilities while focusing more on the specific needs of steel equipment fault diagnosis. During training, appropriate training parameters are set to ensure that the model can converge effectively. These training parameters include key parameters such as learning rate, batch size, and training epochs. A learning rate scheduler is used in conjunction with the model to adapt to its gradual convergence and ensure stable model performance during training. The training tasks mainly cover the analysis of equipment failure causes and the suggestion of solutions. Through multiple rounds of training, the domain adaptability of the model is gradually optimized so that it can provide structured and logically clear analysis and suggestions when answering user questions. In the large language model training module, for the RAG model, the goal is to jointly optimize the retrieval and generation parts, making the generated text as close as possible to the target text. Assuming the model input is a query q, the goal is to generate the correct answer y, and the model output is the generated answer. Furthermore, some candidate documents D = {d1, d2, ..., d...} were retrieved from the knowledge base. k These documents are used to enhance the context of generation; during training, the loss function considers both the retrieval and generation effects; therefore, the loss function for RAG is: L retrieval (q,D) is the loss of the retrieval part, which is optimized by maximizing the score of the relevant documents retrieved: L retrieval (q,D)=-logP(D * ∣q), Among them, D * The correct document most relevant to query q, P(D) * |q) is the probability that the model gives of the document, i.e., cosine similarity; The loss is for the generated part, based on the cross-entropy loss function: Here, y t It is the t-th word in the target answer. It is the conditional probability of generating the target answer y given the query q and the retrieved document D; Furthermore, to ensure that the model output meets domain requirements, specific templates and instructions are designed during training. During model fine-tuning, this "specific template and instruction design" is achieved by introducing structured domain-specific data formats and explicit task instructions into the training data, ensuring the model can accurately identify equipment faults and provide effective solutions. In the training preparation phase, standardized templates are first designed for the model's input and output. For example, for fault diagnosis of steel equipment, the input template includes "fault phenomenon description" and "fault component location," while the output template includes "fault cause analysis," "recommended solutions," and "operation steps." This structured template ensures clear input and output information, facilitating the model's learning of how to extract key content and generate targeted answers. After training, the model output is subjected to strict quality control to filter out irrelevant or repetitive content, ensuring the completeness and logic of the output answer; (3) The retrieval enhancement generation RAG module, after model training is completed, improves the accuracy of fault diagnosis; this RAG module integrates a vector database and a large language model, combining generative questioning and retrieval-based answering modes to provide users with accurate fault analysis and operational guidance; specifically: When a user raises a device malfunction issue, the system converts the user's input malfunction symptom into a vector representation and searches the vector database to find historical records and solutions with similar semantics to the problem. Through this search, the system can quickly locate malfunction cases, solutions, and operational guidance related to the user's problem. Based on the search results, the RAG module passes the retrieved results as contextual information to the large language model, enabling the generated answer to refer to this related information, thereby generating more accurate and detailed fault diagnosis and operational suggestions.

2. The steel production equipment fault diagnosis system according to claim 1, characterized in that, In the data processing module: The data preprocessing includes: Data cleaning includes removing irrelevant information from the raw data, including serial numbers, timestamps, and other irrelevant descriptions, in order to improve the training and retrieval efficiency of the model. Format standardization converts the cleaned data into a uniform format to ensure consistency in further processing; this process also removes redundant spaces, symbols, and noisy data, retaining only key fields useful for fault analysis. Field labeling involves adding labels or category markers to each standardized data field to facilitate the extraction of data features during vectorization. The storage and indexing of the vector data specifically includes: First, the generated vectors are optimized for dimensionality reduction. Let the vector representation in the database be X∈R. n×d Where n is the number of data entries and d is the vector dimension; by setting the mean of each dimension to zero and centering X, we obtain: X C =X-μ, in, x i It is the mean vector; Next, calculate the covariance matrix of the centered data: It describes the relationship of data changes in each dimension; Then, eigenvalue decomposition is performed on the covariance matrix C to obtain: C=UΛU T , Where, U∈R d×d It is the eigenvector matrix, Λ=diag(λ1,λ2,…,λ d ) is a diagonal matrix composed of eigenvalues, and the eigenvalues ​​are arranged in descending order; Subsequently, the eigenvectors corresponding to the first k largest eigenvalues ​​are selected to construct the dimension reduction transformation matrix: Uk = [u1, u2, ..., u k ]∈R d×k ;Transfer the original centralized data X C Projecting onto a k-dimensional space yields the reduced-dimensional vector representation: Z=X c U k ∈R n×k ; Finally, the dimensionality-reduced vector Z is stored as a new vector database for subsequent retrieval enhancement generation tasks; The load balancing process specifically includes: (a) Request layer load balancing: Balances user requests and distributes them to the entry server; (b) Load balancing at the retrieval layer: Distributing retrieval tasks among nodes in the distributed vector database; (c) Generation layer load balancing: Distribute computing tasks across the generation module.

3. The steel production equipment fault diagnosis system according to claim 1, characterized in that, In the aforementioned search enhancement generation RAG module, in order to improve the accuracy and structure of the answers, a specific template and instruction design are adopted during the generation process of the RAG module, so that the model answers can present the cause of the equipment failure, the recommended solution, and detailed operation steps. The specific template, by introducing a domain-specific instruction set, including "fault phenomenon description," "fault component location," and "repair method guidance," ensures that the model's response includes information on the fault cause, operating steps, and preventative measures; specifically including: (1) Fault cause analysis: Analyze the possible causes of this fault; (2) Operational instructions: Provide specific operational steps for maintenance and production resumption; To facilitate data processing, the instructions use JSON format for both input and output. For rapid response, a streaming output interface is used: each time the model generates a word, it immediately outputs a JSON file containing that word. The JSON file transmits two parts: the first part is the question content, and the second part specifies the LLM's response mode. The LLM's response modes are query and chat: the query mode is concise and precise, providing direct fault diagnosis and solutions; the chat mode is more flexible, allowing for gradual analysis of the cause of equipment failure through multiple rounds of interaction.

Images