Intelligent operation and maintenance system for phase modifier based on mT5 model optimization

Abstract

The phase modifier intelligent operation and maintenance system based on mT5 model optimization, in order to solve the current phase modifier intelligent operation and maintenance method knowledge distribution fragmentation, fault judgment lag and intelligent question and answer accuracy rate is low, when the sudden failure occurs, the staff cannot judge the problem in time, lead to low diagnosis efficiency, and the operation and maintenance suggestion generated often appears technical deviation or practicality insufficient problem, the present application based on retrieval enhancement generates RAG model and large language model LLM, using phase modifier database develops a phase modifier intelligent operation and maintenance system for phase modifier, the phase modifier intelligent system includes RAG model and user interaction layer, the embedding model of RAG model is fine tuned to the specific knowledge of phase modifier field by using mT5 model, and the fine tuning ensures that the phase modifier intelligent operation and maintenance system can more accurately identify and respond to the query of the phase modifier related knowledge, the present application belongs to the field of artificial intelligence technology.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence, in particular to a phase modifier knowledge intelligent operation and maintenance system based on a retrieval enhancement generation technology and an mT5 model optimized embedding model. BACKGROUND

[0002] With the rapid development of new power systems, as the core equipment for ensuring voltage stability and reactive power compensation of power grids, the operation and maintenance complexity and professional requirements of phase modifiers are increasingly improved. However, the current intelligent operation and maintenance field of phase modifiers faces the following key challenges:

[0003] 1. Inefficient knowledge management: Phase modifier technology involves multi-disciplinary knowledge such as electromagnetic design, cooling system, vibration monitoring, etc., and industry standards, fault cases, operation procedures, etc. are scattered in technical manuals, expert experience and historical work orders, which have serious fragmentation problems, making it difficult for field personnel to obtain accurate information in time.

[0004] 2. Response lag: Traditional operation and maintenance relies on manual retrieval of documents or expert intervention. When facing sudden failures such as rotor overheating and insulation deterioration, the diagnosis efficiency is low, which may delay the processing time and even cause cascading power grid risks.

[0005] 3. Lack of intelligence: Existing general large models (such as GPT, BERT) lack deep understanding of professional terms and fault logic in the field of phase modifiers, and the generated operation and maintenance suggestions often have technical deviations or insufficient practicality, making it difficult to meet the high reliability requirements.

[0006] Therefore, there is an urgent need for a vertical field knowledge management method for phase modifier intelligent operation and maintenance. SUMMARY

[0007] The application is to solve the problems of current phase modifier intelligent operation and maintenance method, such as knowledge distribution fragmentation, fault judgment lag and low accuracy of intelligent question and answer. When a sudden failure occurs, the staff cannot judge the problem in time, resulting in low diagnosis efficiency, and the generated operation and maintenance suggestions often have technical deviations or insufficient practicality. Therefore, a phase modifier intelligent operation and maintenance system based on an mT5 model optimization is proposed.

[0008] The technical scheme adopted by the application is:

[0009] It includes the following steps:

[0010] The phase modifier intelligent operation and maintenance system includes an RAG model and a user interaction layer, and the RAG model includes a data preprocessing module, an embedding model, a vector database and a large language model.

[0011] The data preprocessing module is used for preprocessing the text problem input by the user interaction layer, and sending the preprocessed text problem to the embedding model.

[0012] The embedding model is used to convert the preprocessed text question into vector data, and send the vector data to the vector database;

[0013] The vector database is used to store the vector data sent by the embedding model, and design an index structure to select vector information related to the vector data, and send the vector information to the large language model;

[0014] The large language model generates a question and answer result according to the vector information sent by the vector database and the text question input by the user interaction layer, and sends the question and answer result to the user interaction layer;

[0015] The user interaction layer is used to send the user's text question to the data preprocessing module and the large language model, and receive and display the question and answer result sent by the large language model;

[0016] Based on the mT5 model, the intelligent operation and maintenance system of the phase modifier is optimized, and the specific process is as follows:

[0017] S1, obtain the text knowledge related to the phase modifier, process the text knowledge, and take the processed text knowledge as the phase modifier database;

[0018] S2, use the large language model GPT3.5 to process the text in the phase modifier database, and generate a corresponding question and answer pair through the context of the text;

[0019] S2, based on the question and answer pair generated in S2, train the mT5 model, input the question, and output the corresponding answer to obtain the QA-mT5 model;

[0020] S3, input the context of the text in the phase modifier database into the QA-mT5 model, and the QA-mT5 model automatically generates a question and answer pair;

[0021] S4, based on the question and answer pair generated in S3, train the embedding model, input the question, and output the vector data of the question, store the vector data of the question into the vector database, and obtain the trained embedding model and the complete vector database;

[0022] S5, the user inputs a phase modifier problem to be queried in the user interaction layer, the phase modifier problem input data preprocessing module, the data preprocessing module outputs the preprocessed phase modifier problem, the preprocessed phase modifier problem inputs the trained embedding model, the trained embedding model outputs the vector data corresponding to the preprocessed phase modifier problem, the vector data inputs the storage complete vector database, the storage complete vector database outputs the vector information related to the vector data, the vector information related to the vector data and the phase modifier problem input the large language model, the large language model outputs the answer corresponding to the phase modifier problem, and displays in the user interaction layer.

[0023] Further, the preprocessing method of the data preprocessing module includes text cleaning, segmentation and blocking in sequence.

[0024] Further, the vector database updates the knowledge base through document uploading, URL importing, ShowDoc and custom content.

[0025] Further, the user interaction layer is a Web interface, an APP, and a WeChat mini-program.

[0026] Further, in S1, the text knowledge related to the phase modifier is acquired, and the processed text knowledge is taken as the phase modifier database, and the specific process is as follows:

[0027] The text knowledge related to the phase modifier is acquired from training manuals, operation manuals, technical documents, related literature, guidance documents and related cases, the images, tables, directories, quotations and unnecessary technical contents contained in the text knowledge are deleted, the deleted text knowledge is obtained, and the deleted text knowledge is taken as the phase modifier database.

[0028] Further, the large language model LLM is the GPT-4o model of OpenAI.

[0029] Further, the phase modifier intelligent operation and maintenance system further comprises a LangChain, and the LangChain connects the RAG model and the user interaction layer together.

[0030] The beneficial effects of the application are:

[0031] The application is based on a retrieval-augmented generation (RAG) model and a large language model (LLM), and a scram smart operation and maintenance system (SCRAM-FT) for scrams is developed using a scram database that integrates all knowledge and data about scrams in practical applications, solving the problem of fragmented knowledge distribution in scram intelligent operation and maintenance methods. To further enhance the relevance of the retrieved content to user queries, the mT5 model is used to fine-tune the embedding model of the RAG model with scram domain knowledge. Through fine-tuning, the mT5 model can switch from general text generation capabilities to specific domain tasks, outputting more demand-oriented text. This fine-tuning ensures that the SCRAM-FT can more accurately identify and respond to queries related to scram knowledge, solving the difficulty of staff being unable to timely determine the problem when a sudden failure occurs. The application significantly improves the performance and user experience of the RAG model, as well as the diagnosis efficiency and accuracy of operation and maintenance recommendations when a scram failure occurs. Even with only a small amount of scram domain data, the RAG model can significantly improve its performance on specific tasks without the need for retraining from scratch.

[0032] The application combines the high-quality scram question and answer dataset constructed with the embedding model trained based on the mT5 model, enabling the RAG model to better understand and generate relevant answers in a specific domain, and more efficiently retrieve relevant knowledge fragments from enterprise documents, operation manuals, training materials, and other resources to generate accurate and contextually relevant answers in real time. In this way, the SCRAM-FT not only provides professional technical support for users, but also adapts to complex and variable problem scenarios in scrams. The introduction of RAG technology enables the SCRAM-FT to access and utilize the latest relevant knowledge base in real time when facing different industrial applications, without relying on static training data, thereby significantly improving the accuracy of problem solving. The SCRAM-FT demonstrates the powerful ability of large language models and retrieval technology to handle specific industry problems, providing a new approach to intelligent operation and maintenance of traditional industrial equipment.

[0033] The application focuses on the application of retrieval-augmented generation in the industrial field, aiming to provide training materials such as operation manuals and step-by-step guides for new employees or non-experts, helping them adapt to the work environment more quickly, and also serving as a technical support tool to answer questions encountered by employees when using specific machines or systems, as well as answering queries related to the reasons for abnormal operation of employee equipment. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a working principle diagram of the scram intelligent operation and maintenance system;

[0035] Figure 2 is a flowchart of obtaining embedding model fine-tuning dataset based on the mT5 model;

[0036] Figure 3 is an mT5-Transformer hybrid time series modeling architecture diagram;

[0037] Figure 4 is a text embedding schematic diagram;

[0038] Figure 5 is a SCRAM-FT user interface schematic diagram; DETAILED DESCRIPTION

[0039] Detailed implementation one: combined with Figures 1-5 To illustrate this embodiment, the intelligent operation and maintenance system for a phase modifier based on an mT5 model optimization includes an RAG model and a user interaction layer, and the RAG model includes a data preprocessing module, an embedding model, a vector database, and a large language model. The large language model LLM is the GPT-4o model of OpenAI.

[0040] The data preprocessing module is used to preprocess the text question input by the user interaction layer, and send the preprocessed text question to the embedding model. The preprocessing method of the data preprocessing module includes text cleaning, segmentation, and blocking in sequence.

[0041] The embedding model is used to convert the preprocessed text question into vector data, and send the vector data to the vector database.

[0042] The vector database itself is used to store all questions and corresponding reply data vectors related to the phase modifier, that is, the vector database is used to store the vector data sent by the embedding model, and an index structure is designed to select vector information related to the vector data in the vector database, and then send the vector information to the large language model. The vector database updates the knowledge base through document uploading, URl importing, ShowDoc, and custom content.

[0043] The large language model generates a question and answer result according to the vector information sent by the vector database and the text question input by the user interaction layer, and sends the question and answer result to the user interaction layer.

[0044] RAG is a method that combines traditional neural network language models and information retrieval techniques to handle natural language processing tasks that require extensive background knowledge. The retrieval method refers to vector retrieval, and RAG uses the synergy between text embedding models and generation models to combine Seq2Seq models with parameterized memory and external knowledge bases with non-parametric memory. The embedding model is used to vectorize the input question, and then the vector database is combined with an efficient and accurate approximate nearest neighbor search algorithm to search for relevant retrieval results and user queries together to input into a large language model to obtain more accurate text generation results. Retrieval-enhanced generation technology combines text embedding models and generation models to achieve organic integration of semantic matching retrieval and natural language generation, improving the explainability and credibility of the question and answer system.

[0045] This technology presents two major advantages: first, knowledge does not need to be implicitly embedded in model parameters, but is explicitly introduced in a plug-and-play manner, with good scalability. Second, compared to generating text from scratch, retrieval-enhanced text generation uses retrieved documents as references for easy traceability. This invention demonstrates that combining retrieval mechanisms can significantly improve the ability of RAG models to handle complex problems, and has a significant impact on the field of natural language processing.

[0046] The user interaction layer is used to send the user's text question to the data preprocessing module and the large language model, and to receive and display the question and answer results sent by the large language model. The user interaction layer is a Web interface, APP, and WeChat mini-program.

[0047] Figure 1 The entire process of the phase modulation machine intelligent operation and maintenance system (SCRAM-FT) from inputting a question to generating an answer is covered. The user inputs the problem encountered in text form on the client side. Before the input text is encoded, it will go through preprocessing, word embedding, etc. to generate a high-dimensional vector containing semantic and contextual information, which converts the user's query into a format that the RAG model can understand and utilize, and matches it with the vector database. By adding relevant data retrieved in the context, the user input is enhanced, and this step uses prompt engineering techniques to effectively communicate with the LLM, and enhanced prompts allow the large language model to generate accurate answers for user queries.

[0048] The mT5 model of the present application optimizes the intelligent system of the phase modulator. First, a data set is constructed. The training manual, operation manual, technical documents, related literature, guidance documents, and related cases of a certain company are combined as a data set, that is, the text knowledge related to the phase modulator included in the data set is existing, but at this time the data set contains a large amount of information unrelated to the phase modulator, such as image data, directory content, references, and materials not directly related to the topic. These unrelated information may interfere with the subsequent training process and affect the accuracy and effectiveness of the RAG model, therefore, the data set at this time needs to be comprehensively preprocessed to retain the text data and text knowledge related to the phase modulator. The first step of preprocessing is to filter out the knowledge information related to the phase modulator from the original data, and remove the images, tables, directories, citations and unrelated technical content contained therein. After these steps of processing, the final clean data set will only contain useful information such as professional terms, technical specifications, production processes, and equipment operation manuals closely related to the phase modulator, ensuring the quality and effectiveness of the phase modulator data set. The processed data set is named as the phase modulator database, which is the core training data source of the present application, and is further used for fine-tuning and retrieval enhancement generation of the RAG model.

[0049] Secondly, the present application proposes an automatic method for constructing the question and answer data set of the embedding model of RAG to overcome the efficiency bottleneck of traditional manual work. As shown in Figure 2 First, the text in the phase modulator database is processed by using the large language model GPT3.5, and the corresponding question and answer pairs are generated based on the context of the text. Based on the question and answer pairs, the mT5 model is fine-tuned and trained, the question is input, and the corresponding answer of the question is output, obtaining the QA-mT5 model, which is the trained mT5 model. The QA-mT5 model can skillfully process the related text in the phase modulator field. The context of the text in the phase modulator database is input into the QA-mT5 model, and the QA-mT5 model can automatically generate question and answer pairs. Based on the question and answer pairs, the embedding model in the RAG process is further fine-tuned and trained, the question is input, and the vector data of the question is output, and the vector data of the question is stored in the vector database, thereby obtaining the fine-tuned embedding model and the complete vector database. In application, the user inputs the phase modulator question to be queried in the user interaction layer, the phase modulator question is sequentially input into the data preprocessing module, the trained embedding model, the complete vector database and the large language model, and the corresponding answer of the phase modulator question is output and displayed in the user interaction layer.

[0050] mT5 (Multilingual T5) is a multilingual extension of T5 (Text-To-Text Transfer Transformer). T5 adopts a unified "text-to-text" framework, treating all tasks (such as classification, translation, generation, etc.) as text generation tasks. The architecture is as follows: Figure 3 The architecture of mT5 is based on Transformer, mainly including the following components:

[0051] a. Transformer Encoder-Decoder Architecture

[0052] mT5 adopts the standard Transformer encoder-decoder architecture:

[0053] Encoder: Encodes the input text into a hidden representation.

[0054] Decoder: Generates the target text step by step based on the output of the encoder and the previous generation results.

[0055] b. Multi-Head Self-Attention Mechanism

[0056] The core of mT5 is the multi-head self-attention mechanism (Multi-Head Self-Attention), whose formula is as follows:

[0057]

[0058] where, are the query (Query), key (Key), and value (Value) matrices, is the dimension of the key vector.

[0059] Multi-head attention projects the input into multiple subspaces and concatenates the attention results:

[0060]

[0061] where, , is the projection matrix, is the output projection matrix.

[0062] c. Position Encoding

[0063] Since Transformer does not have explicit sequence order information, mT5 uses position encoding (PositionalEncoding) to add position information to the input sequence:

[0064]

[0065] wherein, is the position index, i is the dimension index, is the model dimension.

[0066] (1) mT5 fine-tuning principle

[0067] Fine-tuning refers to further training a pre-trained model on specific task data to adapt it to a specific task. The present application adjusts the embedding model parameters by adjusting the camera-related field data, so that it performs better on specific tasks. The pre-training goal of mT5 is Masked Language Modeling (MLM) and Denoising Autoencoding. In the fine-tuning stage, mT5 regards the task as a "text-to-text" conversion problem, and the goal of fine-tuning is to minimize the task-related loss function.

[0068] (2) Loss function

[0069] The loss function of mT5 fine-tuning usually adopts Cross-Entropy Loss, which is used to measure the difference between the target text generated by the model and the real text. For each time step t, the probability distribution generated by the model is , the real label is , and the cross-entropy loss is:

[0070]

[0071] where T is the length of the target sequence.

[0072] For batch data, the overall loss is the average of all sample losses:

[0073]

[0074] where N is the batch size.

[0075] (3) Optimization method

[0076] MT5 fine-tuning usually uses gradient descent-based optimization methods, such as the AdamW optimizer. AdamW is an improved version of the Adam optimizer, which adds weight decay to prevent overfitting. Its update rule is as follows:

[0077]

[0078] where, is the gradient, and are the first and second moments of the gradient, is the learning rate, is a weight decay coefficient.

[0079] The present application connects the RAG model and the user interaction layer together using LangChain. LangChain is an open-source LLM-based application development framework that provides a series of tools and interfaces to make it easy for developers to build and deploy LLM-based applications. LangChain is built around the core concept of "linking" different components together.

[0080] To demonstrate the functions of the phase modifier intelligent operation and maintenance system, the present application uses Streamlit to develop a web application connected to the phase modifier intelligent operation and maintenance system. The interface of the application is as shown in Figure 5 There is a question dialog box below the main interface of the application, where users can input questions related to the phase modifier for inquiry. After receiving the inquiry, the application sends the inquiry question to the phase modifier intelligent operation and maintenance system, which outputs the corresponding answer after processing.

[0081] To comprehensively evaluate the performance of the phase modifier intelligent operation and maintenance system, the research team carefully assembled a diverse evaluation team consisting of 5 researchers with professional backgrounds, 3 senior experts with deep industry experience, and 10 frontline workers from related enterprises. This composition of evaluators ensures both the professionalism and authority of the evaluation, as well as the representativeness of the evaluation results in actual application scenarios.

[0082] The evaluation was conducted using a unified test set. Compared with the other three models, SCRAM-FT demonstrated overall performance advantages, leading in all evaluation indicators. In particular, in terms of correctness and relevance, two core indicators, SCRAM-FT achieved high scores of 8.55 and 8.60 (out of 10), significantly outperforming other models. This result indicates that SCRAM-FT not only accurately understands user query intent but also generates high-quality answers highly relevant to the question, fully demonstrating its outstanding ability in semantic understanding and content generation. SCRAM-FT's performance in the five categories is particularly outstanding, especially in the "device structure" and "safety specifications" fields, which are more professional, achieving excellent scores of 9.04 and 8.88, respectively. This data shows that SCRAM-FT has significant advantages in specific fields: on the one hand, it can accurately analyze the structural characteristics of various devices, providing professional and reliable technical information to users; on the other hand, in key issues related to operation safety, it can provide guidance in line with industry standards, effectively ensuring the accuracy and safety of users' information access.

[0083] The above calculation examples of the present application are only used to illustrate the calculation model and calculation process of the present application, and are not used to limit the embodiments of the present application. Based on the above description, other different forms of changes or variations can be made by those skilled in the art, and all the embodiments cannot be exhausted here. Any obvious changes or variations derived from the technical solutions of the present application are still within the protection scope of the present application.

Claims

1. A smart operation and maintenance system for synchronous condensers based on mT5 model optimization, characterized in that: The intelligent operation and maintenance system for the synchronous condenser includes a RAG model and a user interaction layer. The RAG model includes a data preprocessing module, an embedding model, a vector database, and a large language model. The data preprocessing module is used to preprocess the text questions input by the user interaction layer and send the preprocessed text questions to the embedding model. The embedding model is used to convert the preprocessed text question into vector data and send the vector data to the vector database; The vector database is used to store the vector data sent by the embedding model, and an index structure is designed to select vector information related to the vector data in the vector database and send the vector information to the large language model. The large language model generates question-and-answer results based on the vector information sent from the vector database and the text questions input by the user interaction layer, and sends the question-and-answer results to the user interaction layer; The user interaction layer is used to send the user's text questions to the data preprocessing module and the large language model, and to receive and display the question-and-answer results sent by the large language model; The intelligent operation and maintenance system for synchronous condensers is optimized based on the mT5 model. The specific process is as follows: S1. Obtain textual knowledge related to the camera condenser, process the textual knowledge, and use the processed textual knowledge as a camera condenser database. S2. Use the Large Language Model GPT3.5 to process the text in the camera condenser database and generate corresponding question-answer pairs based on the context of the text; S2. Based on the question-answer pairs generated by S2, train the mT5 model. Input the question and output the corresponding answer to obtain the QA-mT5 model. S3. Input the context of the text in the camera module database into the QA-mT5 model, and the QA-mT5 model automatically generates question-answer pairs; S4. Based on the question-answer pairs generated in S3, train the embedding model by inputting a question and outputting the vector data of the question. Store the vector data of the question in the vector database to obtain the trained embedding model and the fully stored vector database. S5. The user inputs a camera adjustment question to be queried in the user interaction layer. The camera adjustment question is input to the data preprocessing module, which outputs the preprocessed camera adjustment question. The preprocessed camera adjustment question is input to the trained embedding model, which outputs the vector data corresponding to the preprocessed camera adjustment question. The vector data is input to the complete vector database, which outputs vector information related to the vector data. The vector information related to the vector data and the camera adjustment question are input to the large language model, which outputs the answer to the camera adjustment question and displays it in the user interaction layer.

2. The intelligent operation and maintenance system for synchronous condensers based on mT5 model optimization according to claim 1, characterized in that: The preprocessing methods of the data preprocessing module include text cleaning, segmentation, and block division in sequence.

3. The intelligent operation and maintenance system for synchronous condensers based on mT5 model optimization according to claim 1, characterized in that: The vector database updates its knowledge base through document uploads, URL imports, ShowDoc, and custom content.

4. The intelligent operation and maintenance system for synchronous condensers based on mT5 model optimization according to claim 1, characterized in that: The user interaction layer includes web interfaces, apps, and WeChat mini programs.

5. The intelligent operation and maintenance system for synchronous condensers based on mT5 model optimization according to claim 1, characterized in that: In step S1, textual knowledge related to the camera condenser is acquired, the textual knowledge is processed, and the processed textual knowledge is used as the camera condenser database. The specific process is as follows: Textual knowledge related to camera condensers is obtained from training manuals, operation manuals, technical documents, relevant literature, guidance documents, and relevant cases. Images, tables, tables of contents, citations, and unnecessary technical content contained in the textual knowledge are deleted to obtain the deleted textual knowledge, which is then used as the camera condenser database.

6. The intelligent operation and maintenance system for synchronous condensers based on mT5 model optimization according to claim 1, characterized in that: The large language model LLM is OpenAI's GPT-4o model.

7. The intelligent operation and maintenance system for synchronous condensers based on mT5 model optimization according to claim 1, characterized in that: The intelligent operation and maintenance system for synchronous condensers also includes LangChain, which connects the RAG model and the user interaction layer.

Images