Power transformation professional information intelligent interaction method

Abstract

The invention relates to the technical field of electric power interaction, in particular to a power transformation professional information intelligent interaction method, which comprises the following steps: S1, receiving problem information input by a user; s2, the type of the input information is judged, and corresponding recognition processing is carried out; s3, performing problem understanding on the information identified in the S2; s4, performing corresponding retrieval based on the question understanding result in S3, and generating a reply result; s5, interaction is carried out according to the reply result of S4, two-way communication of natural languages is achieved, the convenience and efficiency of man-machine interaction are greatly improved, a safer and more efficient working environment is provided for power transformation professionals, meanwhile, user experience is enriched, and the intelligent level of the system is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of electric power interaction technology, and in particular to a method for intelligent interaction of substation professional information. Background Art

[0002] In recent years, State Grid has been committed to promoting the digital and intelligent transformation of the power grid. In the field of substation expertise, the latest RAG enhanced retrieval system and image recognition technologies can effectively improve the professional skills and work efficiency of substation professionals.

[0003] The existing technical solutions mainly have the following shortcomings:

[0004] 1. Unable to recognize image information. In substation operation and maintenance scenarios, visual information is crucial, such as identifying switch positions, instrument readings, equipment damage or abnormal indicator light status, etc., such as the intelligent interactive method for power dispatching based on programmable telephone disclosed in publication number: CN113807642A.

[0005] 2. It cannot support voice input and output. Previous technologies can only support text input and output. The speed of text input is usually much lower than that of voice communication. This means that when dealing with emergency or rapidly changing situations, the system cannot respond to the needs of substation professionals in a timely manner. In addition, the interactive method of text input and output lacks emotional and personalized elements, and cannot meet the needs of modern users for more humanized smart devices and closer to daily communication.

[0006] 3. It is difficult to integrate multiple data sources. Different data sources may use different data formats, such as CSV, JSON, XML, relational database and other formats. This requires the information database to be able to recognize and convert these formats during the integration process, which increases the complexity and cost of data processing.

[0007] 4. Low search efficiency and accuracy. Many traditional information databases rely on relatively simple index structures, such as keyword-based indexes. This indexing method is inefficient when processing large-scale data sets and it is difficult to quickly locate relevant information. The lack of advanced semantic understanding and relevance ranking mechanisms results in low relevance and accuracy of search results.

[0008] 5. The maintenance cost of the information database is high. With the increase and change of data sources, maintaining the data integration pipeline becomes an ongoing and arduous task. Any change in the data source may affect the normal operation of the entire database, requiring frequent manual adjustments and testing. Summary of the invention

[0009] The technical problem to be solved by the present invention is: to overcome the shortcomings of the prior art, provide a method for intelligent interaction of substation professional information, effectively integrate the scattered knowledge in the field of substation operation and maintenance, realize fast and accurate knowledge retrieval and application, improve the efficiency of substation operation and maintenance and the quality of decision-making, and combine the use of graphic recognition technology and whisper speech recognition model technology to realize more intelligent multi-modal professional information intelligent interaction.

[0010] The technical solution adopted by the present invention to solve the technical problem is: the intelligent interaction method of substation professional information includes the following steps:

[0011] S1: accept question information input by the user;

[0012] S2: Determine the type of input information and perform corresponding identification processing;

[0013] S3: Problem understanding of the information identified in S2;

[0014] S4: Based on the question understanding results in S3, perform corresponding searches and generate answer results;

[0015] S5: interact according to the response result of S4.

[0016] The information types input by the user in S2 include voice, text and image.

[0017] If it is determined in S2 that the input information type is text, then the process proceeds to S3; otherwise, the voice or image is first recognized and then the process proceeds to S3.

[0018] If the input information type is determined to be voice in S2, the Whisper model is used to convert the voice audio into text;

[0019] In the above process, the input audio is first converted into sound feature representation, then sent to the encoder to extract features, and the decoder generates the corresponding text.

[0020] If it is determined in S2 that the input information type is an image, the image recognition OCR model is used to perform scene text recognition in the image.

[0021] The S3 uses natural language understanding technology to analyze the text and extract the key information and intent of the question.

[0022] The S4 includes the following sub-steps:

[0023] S4-1: Retrieve relevant information based on the key information and intent of the extraction question;

[0024] S4-2: Fill in the text or answer questions using the information retrieved in S4-1;

[0025] S4-3: Generate the final text content based on S4-2.

[0026] The S4-2 includes the following sub-steps:

[0027] S4-2-1: Based on S4-1 retrieval, determine relevant documents and extract paragraphs or sentences related to the question from the relevant documents as evidence; specifically, S4-2-2: use the conditional generation model to generate answers based on the evidence paragraphs;

[0028] S4-2-3: Generate interaction results based on S4-2-2.

[0029] In S4-3, the interaction result generated in S4-2-3 is optimized to generate the final text content.

[0030] The optimization process includes grammar correction and format adjustment.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] This application introduces speech recognition and synthesis technology to achieve two-way communication in natural language, which not only greatly improves the convenience and efficiency of human-computer interaction, but also provides a safer and more efficient working environment for substation professionals, while also enriching the user experience and enhancing the intelligence level of the system. Through voice input and output, substation professionals can communicate with the system in a more natural way, reduce manual operations, and improve emergency response capabilities and job satisfaction;

[0033] At the same time, the intelligent interaction method for substation professional information enables substation professionals to quickly obtain the required information through intelligent retrieval and auxiliary decision support;

[0034] Combining voice and image recognition, it can handle more complex and diverse tasks, such as equipment maintenance guidance, equipment information query, etc.

[0035] With natural language understanding capabilities, it can generate natural and humanized dialogues, making substation operation and maintenance information interaction closer to real dialogue experience, which not only improves user experience, but also enables more effective information exchange with humans;

[0036] By combining the powerful generation capability and precise information retrieval capability of the RAG enhanced retrieval system, the efficiency and effectiveness of knowledge management and application are improved, and more accurate, timely and secure information interaction services can be provided. At the same time, it has the ability of multi-round dialogue in natural language, voice dialogue, and image recognition to achieve more intelligent interactive services. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The intelligent interaction method of substation professional information includes the following steps:

[0040] S1: accept question information input by the user;

[0041] S2: Determine the type of information input and perform corresponding recognition processing; the information type input by the user in S2 includes voice, text and image. If the input information type is determined to be text in S2, enter S3; otherwise, first recognize the voice or image and then enter S3.

[0042] If the input information type is determined to be speech in S2, the present application converts the speech into text through automatic speech recognition (ASR); uses the Whisper model, which is an automatic speech recognition model based on the Transformer architecture. It uses an encoder-decoder structure to convert speech audio into text, and processes multiple languages ​​and accents by pre-training on large-scale multilingual datasets, and can be fine-tuned through labeled data to adapt to specific task requirements;

[0043] In the above process, the input audio is first converted into a sound feature representation such as a spectrogram, then sent to the encoder to extract features, and the decoder generates the corresponding text. The entire process uses an appropriate loss function to optimize the model parameters through the back propagation algorithm, gradually improving the accuracy and robustness of the audio content transcription in different scenarios. Through this model solution, speech can be converted into text efficiently and accurately.

[0044] This application takes into account that the operation and maintenance team may come from different regions, and is set up to recognize and respond to multiple dialects to ensure barrier-free communication.

[0045] This application uses Whisper model technology to more accurately recognize the voice commands of substation professionals, and maintain a high recognition rate even in noisy environments. It can understand complex and ambiguous verbal commands, improve the accuracy of understanding the intentions of substation professionals, and make more precise responses.

[0046] If the input information type is judged as an image in S2, the image recognition OCR model (Minicpm-V) is used to recognize scene text in the image. This model belongs to the product of Mianbi Intelligent Company, and Minicpm-V is the lightest model in the MiniCPM-V series. The total parameter volume is 2B, and the multimodal comprehensive performance exceeds models with larger parameter scales such as Yi-VL34B, CogVLM-Chat17B, and Qwen-VL-Chat10B. It can accept 1.8 million pixel images of arbitrary aspect ratio input, and achieves scene text recognition capabilities similar to GeminiPro and a low hallucination rate that matches GPT-4V.

[0047] This application uses image recognition technology to detect photographed power equipment, including primary and secondary equipment, and gives corresponding descriptions and related information.

[0048] S3: Understand the problem based on the information identified in S2; in S3, Natural Language Understanding (NLU) technology is used to analyze the text to extract the key information and intent of the problem.

[0049] NLU is a subfield in the field of Natural Language Processing (NLP) that focuses on understanding and interpreting the meaning and intent of human language. It is an important component of NLP. NLU focuses on semantic understanding of natural language input and converting text or speech into structured representations that computers can understand and process.

[0050] The main task of NLU is to convert human language into a form that computers can understand and process. It involves semantic parsing, intent classification, entity recognition, sentiment analysis, and other processing of language input. Specifically, NLU can help computers identify entities in sentences (such as names of people, places, dates, etc.), understand the intent of sentences (such as asking questions, seeking help, expressing needs, etc.), and analyze the emotions and sentiments in sentences. Through these analyses and understandings, computers can better interact and communicate with humans.

[0051] Natural language interaction reduces the requirement for specialized skills, improves user-friendliness, and reduces the time and cost of training new employees.

[0052] S4: Based on the question understanding results in S3, perform corresponding searches and generate answer results;

[0053] The S4 includes the following sub-steps:

[0054] S4-1: Retrieve relevant information based on the key information and intent of the extraction question;

[0055] S4-2: Fill in the text or answer questions using the information retrieved in S4-1;

[0056] S4-3: Generate the final text content based on S4-2.

[0057] Specifically, the present application retrieves relevant content in an information database based on an LLM (Large Language Model) according to the key information of the question. The working principle of RAG (Retrieval Augmented Generation) is to improve the quality and accuracy of prediction by retrieving relevant information from a large-scale document collection and then using this information to guide the generation of text. Specifically, RAG works through three key parts: retrieval, utilization, and generation. In the retrieval stage, the system retrieves relevant information from the document collection; in the utilization stage, the system uses the retrieved information to fill in the text or answer questions; finally, in the generation stage, the system generates the final text content based on the retrieved knowledge.

[0058] Before generating a response, RAG retrieves relevant information from the local information database based on the query. The generated content is not only based on the generalized knowledge within the model, but also incorporates specific and time-sensitive external data, thereby improving the accuracy and relevance of the answer.

[0059] The RAG mechanism significantly improves retrieval accuracy and speed, quickly locates key information, and reduces manual screening time.

[0060] The S4-2 includes the following sub-steps:

[0061] S4-2-1: Retrieve and determine relevant documents based on S4-1, and extract the paragraphs or sentences most relevant to the question from the relevant documents as evidence; specifically, in this application, first determine a question or statement (query) and a document collection based on S3, and divide each document into several paragraphs; then, use the pre-trained language model LLM to encode each paragraph and query, and generate corresponding vectors for representation. The language model LLM has learned rich language features through a large-scale corpus; then, calculate the similarity score between the query vector and each paragraph vector to quantify the degree of correlation between them; finally, sort the paragraphs according to the score, and select the paragraph or paragraphs with the highest score as the final evidence paragraph output. The whole process can also be combined with the attention mechanism to more accurately locate the key information in the paragraph to ensure that the selected paragraph can indeed effectively support or refute the original query.

[0062] S4-2-2: Use conditional generation models (such as Seq2Seq, Transformer, etc.) to generate answers based on the evidence paragraph;

[0063] Specifically, the present application aims to generate specific types of outputs, such as text, images, etc., based on given conditions or contextual information. The core principle is to use input conditions to guide the generation process to ensure that the output content is closely related to the provided conditions. The present application first collects and processes a training data set containing paired condition-target samples, and then selects the corresponding neural network architecture as the basis, such as using an encoder-decoder structure to convert the conditional information into a fixed-length vector representation in the encoding stage, and then gradually generates the final output based on this vector in the decoding stage. For some complex tasks, the model may also be pre-trained on a large-scale general corpus to learn basic language or visual features, and then fine-tuned for condition-target paired data in specific application scenarios, so that the model can better capture the correlation between conditions and outputs, and define appropriate loss functions to measure the difference between the generated results and the true target, and adjust the model parameters through gradient descent or other optimization algorithms to minimize the loss value. After completing the training, when a new conditional input is given in actual application, the model will generate the corresponding output content based on the learned knowledge, and in order to increase diversity and control the generation quality, sampling strategies can also be introduced or beam search algorithms can be used to explore different generation paths.

[0064] S4-2-3: Generate interaction results based on S4-2-2.

[0065] In S4-3, the interaction result generated in S4-2-3 is optimized to generate the final text content. The optimization process includes grammar correction, format adjustment, etc.

[0066] S5: Interact according to the response result of S4. If you need to reply to the user in voice form, you can use text-to-speech (TTS) technology to convert the text interaction results into voice output.

[0067] This application can also understand complex operation and maintenance scenarios based on conversation history and provide accurate context-based feedback, such as automatically identifying and tracking the progress of reports in continuous conversations.

[0068] This application stores data locally, avoiding the risk of privacy leakage caused by uploading data to the cloud. It complies with data protection regulations such as GDPR and is particularly suitable for professionals who handle sensitive information.

[0069] This interactive method can understand the verbal instructions of substation professionals through advanced speech recognition technology, and respond clearly through speech synthesis technology, achieving natural and smooth human-computer dialogue. This allows substation professionals to complete tasks such as equipment monitoring, status query and remote control without manual operation, which is particularly practical in substation equipment operation and emergency scenarios.

[0070] In addition to plain text data, this interactive method can also understand structured data such as tables, lists, and charts through appropriate preprocessing and embedding techniques, and extract information from text in images (OCR). This fusion processing capability enables operation and maintenance professional information to not only store and retrieve traditional text, but also utilize other forms of data resources to enrich its content and application scope.

[0071] The above descriptions are only optional embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A method for intelligent interaction of substation professional information, characterized in that: The steps include: S1: accept question information input by the user; S2: Determine the type of input information and perform corresponding identification processing; S3: Problem understanding of the information identified in S2; S4: Based on the question understanding results in S3, perform corresponding searches and generate answer results; S5: interact according to the response result of S4.

2. The intelligent interaction method for substation professional information according to claim 1 is characterized in that: The information types input by the user in S2 include voice, text and image.

3. The intelligent interaction method for substation professional information according to claim 2 is characterized in that: If it is determined in S2 that the input information type is text, then the process proceeds to S3; otherwise, the voice or image is first recognized and then the process proceeds to S3.

4. The intelligent interaction method for substation professional information according to claim 3 is characterized in that: If it is determined in S2 that the input information type is voice, the Whisper model is used to convert the voice audio into text.

5. The intelligent interaction method for substation professional information according to claim 3 is characterized in that: If it is determined in S2 that the input information type is an image, the image recognition OCR model is used to perform scene text recognition in the image.

6. The intelligent interaction method for substation professional information according to claim 1 is characterized in that: The S3 uses natural language understanding technology to analyze the text and extract the key information and intent of the question.

7. The intelligent interaction method for substation professional information according to claim 6 is characterized in that: The S4 includes the following sub-steps: S4-1: Retrieve relevant information based on the key information and intent of the extraction question; S4-2: Fill in the text or answer questions using the information retrieved in S4-1; S4-3: Generate the final text content based on S4-2.

8. The intelligent interaction method for substation professional information according to claim 7 is characterized in that: The S4-2 includes the following sub-steps: S4-2-1: Determine relevant documents based on S4-1 retrieval, and extract paragraphs or sentences related to the question from the relevant documents as evidence; S4-2-2: Generate answers based on evidence paragraphs using a conditional generative model; S4-2-3: Generate interaction results based on S4-2-2.

9. The intelligent interaction method for substation professional information according to claim 8 is characterized in that: In S4-3, the interaction result generated in S4-2-3 is optimized to generate the final text content.

10. The intelligent interaction method for substation professional information according to claim 9 is characterized in that: The optimization process includes grammar correction and format adjustment.

Images