Intelligent operation and maintenance system of phase modifier based on mT5 model optimization

Abstract

The invention discloses an intelligent operation and maintenance system for a phase modifier based on mT5 model optimization, and aims to solve the problems of low diagnosis efficiency, high accuracy and the like caused by the fact that a current intelligent operation and maintenance method for the phase modifier is fragmented in knowledge distribution, hysteretic in fault judgment and low in intelligent question and answer accuracy, and a worker cannot judge the problem in time when a sudden fault occurs. And the generated operation and maintenance suggestions often have technical deviation or insufficient practicability. An RAG model and a large language model LLM are generated based on retrieval enhancement, a phase modifier intelligent operation and maintenance system for a phase modifier is developed by utilizing a phase modifier database, the phase modifier intelligent system comprises the RAG model and a user interaction layer, and specific fine adjustment of phase modifier domain knowledge is performed on an embedded model of the RAG model by utilizing an mT5 model. The fine tuning ensures that the intelligent operation and maintenance system of the phase modifier can more accurately identify and respond to query of related knowledge of the phase modifier. The invention belongs to the technical field of artificial intelligence.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a phase shifter knowledge intelligent operation and maintenance system based on retrieval enhancement generation technology and MT5 model optimization embedding model. Background Art

[0002] With the rapid development of new power systems, the complexity and professional requirements for the operation and maintenance of phase regulators, as core equipment for ensuring grid voltage stability and reactive power compensation, are increasing. However, the current intelligent operation and maintenance of phase regulators faces the following key challenges: 1. Inefficient knowledge management: Phase-shifting technology involves multidisciplinary knowledge, including electromagnetic design, cooling systems, and vibration monitoring. Furthermore, industry standards, failure cases, and operation and maintenance procedures are scattered across technical manuals, expert experience, and historical work orders. This creates a serious fragmentation problem, making it difficult for on-site personnel to obtain accurate information in a timely manner.

[0003] 2. Response lag: Traditional O&M relies on manual document retrieval or expert intervention. When faced with sudden faults (such as rotor overheating and insulation degradation), diagnostic efficiency is low, which may delay treatment and even trigger chain reactions to power grid risks.

[0004] 3. Insufficient intelligence: Existing general-purpose large models (such as GPT and BERT) lack a deep understanding of the professional terminology and fault logic in the field of camera adjustment. The generated operation and maintenance suggestions often have technical deviations or lack practicality, making it difficult to meet high reliability requirements.

[0005] Therefore, there is an urgent need for a vertical domain knowledge management method for intelligent operation and maintenance of camera modulators. Summary of the Invention

[0006] In order to solve the problems of fragmented knowledge distribution, delayed fault diagnosis and low accuracy of intelligent question-answering in the current intelligent operation and maintenance methods for phase shifters, the present invention proposes an intelligent operation and maintenance system for phase shifters based on mT5 model optimization. When sudden faults occur, staff are unable to promptly determine the problem, resulting in low diagnostic efficiency and the generated operation and maintenance suggestions often have technical deviations or lack of practicality.

[0007] The technical solution adopted by the present invention is: It includes the following steps: The intelligent operation and maintenance system for the condenser includes a RAG model and a user interaction layer. The RAG model includes a data preprocessing module, an embedding model, a vector database and a large language model. The data preprocessing module is used to preprocess the text questions input by the user interaction layer and send the preprocessed text questions to the embedding model; The embedding model is used to convert the preprocessed text questions into vector data and send the vector data to a vector database; The vector database is used to store the vector data sent by the embedding model, and to design an index structure to select vector information related to the vector data, and to send the vector information to the large language model; The large language model generates a question-answering result based on the vector information sent by the vector database and the text question input by the user interaction layer, and sends the question-answering result to the user interaction layer; The user interaction layer is used to send the user's text questions to the data preprocessing module and the large language model, and receive and display the question and answer results sent by the large language model; The intelligent operation and maintenance system for condenser cameras is optimized based on the mT5 model. The specific process is as follows: S1. Acquire textual knowledge related to the phase shifter, process the textual knowledge, and use the processed textual knowledge as a phase shifter database; S2. Process the text in the camera adjustment database using the large language model GPT3.5, and generate corresponding question-answer pairs based on the context of the text; S2, based on the question-answer pairs generated by S2, train the mT5 model, input the question, output the corresponding answer, and obtain the QA-mT5 model; S3. Input the context of the text in the camera database into the QA-mT5 model, and the QA-mT5 model automatically generates question-answer pairs; S4. Based on the question-answer pairs generated in S3, train the embedding model, input the question, output the vector data of the question, and store the vector data of the question in a vector database to obtain a trained embedding model and a complete vector database. S5. The user inputs the phase adjustment problem to be queried at the user interaction layer, the phase adjustment problem is input into the data preprocessing module, the data preprocessing module outputs the preprocessed phase adjustment problem, the preprocessed phase adjustment problem is input into the trained embedding model, the trained embedding model outputs vector data corresponding to the preprocessed phase adjustment problem, the vector data is input into a complete vector database, the complete vector database outputs vector information related to the vector data, the vector information related to the vector data and the phase adjustment problem are input into the large language model, the large language model outputs the answer corresponding to the phase adjustment problem, and displays it at the user interaction layer.

[0008] Furthermore, the preprocessing method of the data preprocessing module includes text cleaning, segmentation and blocking in sequence.

[0009] Furthermore, the vector database is updated through document upload, URl import, ShowDoc and custom content.

[0010] Furthermore, the user interaction layer is a Web interface, an APP, and a WeChat applet.

[0011] Furthermore, in S1, text knowledge related to the phase shifter is obtained, the text knowledge is processed, and the processed text knowledge is used as the phase shifter database. The specific process is as follows: The text knowledge related to the camera tuner is obtained from training manuals, operating manuals, technical documents, relevant literature, guidance documents and related cases. The images, tables, directories, citations and non-essential technical content contained in the text knowledge are deleted to obtain the deleted text knowledge, which is used as the camera tuner database.

[0012] Furthermore, the large language model LLM is OpenAI's GPT-4o model.

[0013] Furthermore, the camera-modulating intelligent operation and maintenance system also includes LangChain, which connects the RAG model and the user interaction layer.

[0014] The beneficial effects of the present invention are: Based on the retrieval-enhanced generation (RAG) model and the large language model (LLM), this paper develops a SCRAM-FT intelligent operation and maintenance system for tunable ...

[0015] The present invention combines the constructed high-quality camera-tuning question-answering dataset with an embedding model trained based on the mT5 model, so that the RAG model can better understand and generate relevant answers in a specific field, more efficiently retrieve relevant knowledge fragments from corporate documents, operating manuals, training materials and other resources, and generate accurate, context-relevant answers in real time. In this way, SCRAM-FT can not only provide users with professional technical support, but also adapt to the complex and changing problem scenarios in camera-tuning. The introduction of RAG technology enables the camera-tuning intelligent operation and maintenance system to access and utilize the latest relevant knowledge base in real time when facing different industrial applications, without relying on static training data, thereby greatly improving the accuracy of question answering. The camera-tuning intelligent operation and maintenance system demonstrates a powerful ability to handle specific industry problems through the combination of large language models and retrieval technology, providing new ideas for the intelligent operation and maintenance of traditional industrial equipment.

[0016] This invention focuses on the application of retrieval-enhanced generation in the industrial field, aiming to provide new employees or non-experts with training materials, such as operating manuals and step-by-step guidance, to help them adapt to the work environment more quickly. It can also serve as a technical support tool to answer questions encountered by employees when using specific machines or systems, as well as answer employees' inquiries related to the reasons for abnormal equipment operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is the working principle diagram of the intelligent operation and maintenance system of the condenser; Figure 2 This is a flowchart for obtaining the embedding model fine-tuning dataset based on the mT5 model; Figure 3 This is the mT5-Transformer hybrid timing modeling architecture diagram; Figure 4 It is a diagram of text embedding; Figure 5 This is a schematic diagram of the SCRAM-FT user interface; DETAILED DESCRIPTION

[0018] Specific implementation method 1: Combination Figure 1-Figure 5 This embodiment describes an intelligent operation and maintenance system for a phase-shifting camera based on mT5 model optimization. The system includes a RAG model and a user interaction layer. The RAG model includes a data preprocessing module, an embedding model, a vector database, and a large language model (LLM). The large language model (LLM) is OpenAI's GPT-4o model.

[0019] The data preprocessing module is used to preprocess the text questions input by the user interaction layer and send the preprocessed text questions to the embedding model. The preprocessing method of the data preprocessing module includes text cleaning, segmentation and blocking in sequence.

[0020] The embedding model is used to convert the preprocessed text questions into vector data and send the vector data to a vector database.

[0021] The vector database itself is used to store all questions related to the camera and the corresponding answer data vectors. Specifically, the vector database is used to store the vector data sent by the embedding model. An index structure is designed to select vector information related to this vector data from the vector database and then send this vector information to the large language model. The vector database is updated through document uploads, URL imports, ShowDoc, and custom content.

[0022] The large language model generates a question-and-answer result based on the vector information sent by the vector database and the text question input by the user interaction layer, and sends the question-and-answer result to the user interaction layer.

[0023] RAG is a method that combines traditional neural network language models and information retrieval technology to handle natural language processing tasks that require extensive background knowledge. The retrieval method specifically targets vector retrieval. RAG leverages the synergy between text embedding models and generative models, combining the implicit knowledge Seq2Seq model with parametric memory with an external knowledge base with non-parametric memory. The embedding model is used to vectorize the input question. The search is then combined with the vector database and an efficient and accurate approximate nearest neighbor search algorithm. The relevant search results and user questions are input into the large language model to obtain more accurate text generation results. Retrieval-enhanced generation technology achieves an organic fusion of semantic matching retrieval and natural language generation by combining text embedding models with generative models, improving the interpretability and credibility of the question-answering system.

[0024] This technology offers two significant advantages: First, knowledge does not need to be implicitly embedded in model parameters; instead, it is explicitly introduced in a plug-and-play manner, which provides good scalability. Second, compared to generating text from scratch, retrieval-enhanced text generation uses retrieved documents as a reference, facilitating source verification. This invention demonstrates that the ability of RAG models to handle complex problems can be significantly improved by incorporating retrieval mechanisms, which has had a significant impact in the field of natural language processing.

[0025] The user interaction layer is used to send the user's text questions to the data preprocessing module and the large language model, and receive and display the question and answer results sent by the large language model. The user interaction layer is a web interface, APP, and WeChat applet.

[0026] Figure 1This article covers the entire process of the SCRAM-FT intelligent operation and maintenance system, from question input to answer generation. Users enter their questions in text form on the client. Before being encoded, the input text undergoes preprocessing and word embedding to generate high-dimensional vectors containing semantic and contextual information. This transforms the user's question into a format that the RAG model can understand and utilize, and then matches it with the vector database. User input is augmented by adding relevant retrieved data to the context. This step uses hint engineering techniques to effectively communicate with the LLM. This enhanced hint allows the large language model to generate accurate answers to user queries.

[0027] When optimizing the tunable phaser intelligent system, the mT5 model of the present invention first constructs a dataset. A company's training manuals, operating manuals, technical documents, related literature, guidance documents, and related case studies are combined into a dataset. While the dataset contains existing textual knowledge related to tunable phasers, it also contains a significant amount of information unrelated to tunable phasers, such as image data, directory content, references, and other materials not directly related to the topic. This irrelevant information can interfere with subsequent training and affect the accuracy and effectiveness of the RAG model. Therefore, the dataset requires comprehensive preprocessing to preserve the relevant textual data and knowledge related to tunable phasers. The first step in preprocessing is to filter out tunable phaser-related knowledge from the raw data, removing any images, tables, directories, citations, and irrelevant technical content. After these steps, the resulting clean dataset contains only useful information closely related to tunable phasers, such as technical terms, technical specifications, production processes, and equipment operating manuals, ensuring the quality and validity of the tunable phaser dataset. The present invention names this processed dataset the tunable phaser database. The tunable phaser database serves as the core training data source for the present invention and is further used for fine-tuning the RAG model and generating search-enhanced data.

[0028] Secondly, this paper proposes an automated method to construct a question-answering dataset of the RAG embedding model to overcome the efficiency bottleneck of traditional manual methods. Figure 2As shown, the large language model GPT3.5 is first used to process the text in the phase adjustment database, generating corresponding question-answer pairs based on the context of the text. Based on these question-answer pairs, the mT5 model is fine-tuned. A question is input and the corresponding answer is output, resulting in a QA-mT5 model. The QA-mT5 model is a trained mT5 model that is proficient in processing text related to the phase adjustment field. The context of the text in the phase adjustment database is input into the QA-mT5 model, which can automatically generate question-answer pairs. Based on these question-answer pairs, the embedding model in the RAG process is further fine-tuned. A question is input and the vector data of the question is output. The vector data of the question is stored in the vector database, resulting in a fine-tuned embedding model and a fully stored vector database. In application, a user enters a phase adjustment question to be queried in the user interaction layer. The phase adjustment question is sequentially processed through the data preprocessing module, the trained embedding model, the fully stored vector database, and the large language model. The answer to the phase adjustment question is output and displayed in the user interaction layer.

[0029] mT5 (MultilingualT5) is a multilingual extension based on T5 (Text-To-Text Transfer Transformer). T5 adopts a unified "text-to-text" framework, treating all tasks (such as classification, translation, generation, etc.) as text generation tasks. The architecture is shown in Figure 3 The architecture of mT5 is based on Transformer and mainly includes the following components: a. TransformerEncoder-Decoder architecture mT5 uses the standard Transformer encoder-decoder architecture: Encoder: Encodes the input text into a hidden representation.

[0030] Decoder: Generates the target text step by step based on the encoder output and previous generation results.

[0031] b. Multi-head self-attention mechanism The core of mT5 is the Multi-Head Self-Attention mechanism, whose formula is as follows: in, They are query, key and value matrices respectively. is the dimension of the key vector.

[0032] Multi-head attention projects the input into multiple subspaces, calculates the attention separately and concatenates the results: in, , is the projection matrix, is the output projection matrix.

[0033] c. Positional encoding Since Transformer has no explicit sequence order information, mT5 uses positional encoding to add position information to the input sequence: in, is the position index, i is the dimension index, is the model dimension.

[0034] (1) MT5 fine-tuning principle Fine-tuning involves further training a pre-trained model using task-specific data to adapt it to a specific task. This method adjusts the embedding model parameters by using data from relevant fields to optimize performance on specific tasks. mT5's pre-training targets masked language modeling (MLM) and denoising autoencoding. During the fine-tuning phase, mT5 treats the task as a text-to-text translation problem, with the goal of minimizing the task-specific loss function.

[0035] (2) Loss function The loss function of mT5 fine-tuning usually adopts cross-entropy loss (Cross-EntropyLoss), which is used to measure the difference between the target text generated by the model and the real text. For each time step t, the probability distribution generated by the model is , the true label is , then the cross entropy loss is: Where T is the length of the target sequence.

[0036] For batch data, the overall loss is the average of all sample losses: Where N is the batch size.

[0037] (3) Optimization method MT5 fine-tuning typically uses gradient descent-based optimization methods, such as the AdamW optimizer. AdamW is an improved version of the Adam optimizer that incorporates weight decay to prevent overfitting. Its update rule is as follows: in, is the gradient, and are the first and second order momentum of the gradient, is the learning rate, is the weight decay coefficient.

[0038] This invention utilizes LangChain to connect the RAG model and the user interaction layer. LangChain is an open-source upper-layer application development framework based on LLM. It provides a series of tools and interfaces that allow developers to easily build and deploy LLM-based applications. LangChain is built around the core concept of "chaining" different components together.

[0039] In order to demonstrate the functions of the phase shifter intelligent operation and maintenance system, the present invention uses Streamlit to develop a network application, which is connected to the phase shifter intelligent operation and maintenance system. The interface of the application is as follows: Figure 5 Below the main interface of the application, there is a question dialog box where users can enter questions related to the tunable phase shifter. After receiving the query, the application will send the query to the tunable phase shifter intelligent operation and maintenance system, which will process and output the corresponding answer.

[0040] To comprehensively evaluate the performance of the condenser intelligent operation and maintenance system, the research team carefully assembled a diverse evaluation team consisting of five researchers with professional backgrounds, three experienced industry experts, and ten frontline workers from related companies. This composition of evaluators ensured both professionalism and authority in the evaluation and representativeness of the results in real-world application scenarios.

[0041] The evaluation was conducted using a unified test set. Compared to the other three models, SCRAM-FT demonstrated comprehensive performance advantages, leading in all evaluation metrics. In particular, in the core metrics of accuracy and relevance, SCRAM-FT achieved high scores of 8.55 and 8.60 (out of 10), respectively, significantly outperforming the other models. This result demonstrates that SCRAM-FT not only accurately understands user query intent but also generates high-quality, highly relevant answers, fully demonstrating its exceptional capabilities in semantic understanding and content generation. SCRAM-FT also performed particularly well in the five question categories, particularly in the highly specialized areas of "device structure" and "safety specifications," achieving excellent scores of 9.04 and 8.88, respectively. This data demonstrates SCRAM-FT's significant advantages in specific areas: it accurately analyzes the structural characteristics of various types of equipment, providing users with professional and reliable technical information; and it provides guidance that complies with industry standards for critical operational safety issues, effectively ensuring the accuracy and security of the information users receive.

[0042] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. The intelligent operation and maintenance system for condenser phases based on the mT5 model optimization is characterized by: The intelligent operation and maintenance system for the condenser includes a RAG model and a user interaction layer. The RAG model includes a data preprocessing module, an embedding model, a vector database and a large language model. The data preprocessing module is used to preprocess the text questions input by the user interaction layer and send the preprocessed text questions to the embedding model; The embedding model is used to convert the preprocessed text questions into vector data and send the vector data to a vector database; The vector database is used to store the vector data sent by the embedding model, and an index structure is designed to select vector information related to the vector data in the vector database, and send the vector information to the large language model; The large language model generates a question-answering result based on the vector information sent by the vector database and the text question input by the user interaction layer, and sends the question-answering result to the user interaction layer; The user interaction layer is used to send the user's text questions to the data preprocessing module and the large language model, and receive and display the question and answer results sent by the large language model; The intelligent operation and maintenance system for condenser cameras is optimized based on the mT5 model. The specific process is as follows: S1. Acquire textual knowledge related to the phase shifter, process the textual knowledge, and use the processed textual knowledge as a phase shifter database; S2. Process the text in the camera adjustment database using the large language model GPT3.5, and generate corresponding question-answer pairs based on the context of the text; S2, based on the question-answer pairs generated by S2, train the mT5 model, input the question, output the corresponding answer, and obtain the QA-mT5 model; S3. Input the context of the text in the camera database into the QA-mT5 model, and the QA-mT5 model automatically generates question-answer pairs; S4. Based on the question-answer pairs generated in S3, train the embedding model, input the question, output the vector data of the question, and store the vector data of the question in a vector database to obtain a trained embedding model and a complete vector database. S5. The user inputs the phase adjustment problem to be queried at the user interaction layer, the phase adjustment problem is input into the data preprocessing module, the data preprocessing module outputs the preprocessed phase adjustment problem, the preprocessed phase adjustment problem is input into the trained embedding model, the trained embedding model outputs vector data corresponding to the preprocessed phase adjustment problem, the vector data is input into a complete vector database, the complete vector database outputs vector information related to the vector data, the vector information related to the vector data and the phase adjustment problem are input into the large language model, the large language model outputs the answer corresponding to the phase adjustment problem, and displays it at the user interaction layer.

2. The intelligent operation and maintenance system for phase modulators based on mT5 model optimization according to claim 1 is characterized by: The preprocessing method of the data preprocessing module includes text cleaning, segmentation and blocking in sequence.

3. The intelligent operation and maintenance system for phase modulators based on mT5 model optimization according to claim 1 is characterized by: The vector database is updated through document upload, URl import, ShowDoc and custom content.

4. The intelligent operation and maintenance system for phase shifters based on mT5 model optimization according to claim 1 is characterized by: The user interaction layer includes the Web interface, APP, and WeChat applet.

5. The intelligent operation and maintenance system for phase modulators based on mT5 model optimization according to claim 1 is characterized by: In S1, text knowledge related to the phase shifter is obtained, the text knowledge is processed, and the processed text knowledge is used as the phase shifter database. The specific process is as follows: The text knowledge related to the camera tuner is obtained from training manuals, operating manuals, technical documents, relevant literature, guidance documents and related cases. The images, tables, directories, citations and non-essential technical content contained in the text knowledge are deleted to obtain the deleted text knowledge, which is used as the camera tuner database.

6. The intelligent operation and maintenance system for phase shifters based on mT5 model optimization according to claim 1 is characterized by: The large language model LLM is OpenAI's GPT-4o model.

7. The intelligent operation and maintenance system for phase condensers based on mT5 model optimization according to claim 1 is characterized by: The phase regulator intelligent operation and maintenance system also includes LangChain, which connects the RAG model and the user interaction layer.

Images