Power system knowledge retrieval method and platform based on RAG framework

Abstract

The invention provides a power system knowledge retrieval method and platform based on an RAG framework, and relates to the technical field of knowledge retrieval, and the method comprises the steps: carrying out knowledge base construction processing on a power system file based on the RAG framework, and generating a vector database; service problem reconstruction is carried out, and a service adaptation expression is generated; inputting a word embedding model, and converting the word embedding model into a 1024-dimensional vector of the problem; sub-block vector similarity matching is carried out, and K candidate block metadata are screened and recalled; associating the K parent block IDs to which the K parent block IDs belong; after the 1024-dimensional vectors of the K parent blocks are called, M candidate block metadata are screened and output; and returning complete chapter content and traceability information. The technical problem that a traditional knowledge base is difficult to meet the efficient retrieval requirement of the commemorative inspection business of a power system due to the fact that the representation capability of vectors on business information is insufficient because the vectorization in the prior art mostly adopts general dimensions and is not combined with the text complexity of commemorative inspection files to optimize the vector dimensions is solved.

Description

Technical Field

[0001] This invention relates to the field of knowledge retrieval technology, specifically to a power system knowledge retrieval method and platform based on the RAG framework. Background Technology

[0002] With the deepening application of artificial intelligence in the power industry, the demand for "rapidly locating regulations and accurately matching business issues" in disciplinary inspection operations is becoming increasingly urgent. Current RAG-based knowledge base construction methods suffer from three major shortcomings: First, document processing lacks specificity, failing to structurally decompose disciplinary inspection documents according to their chapter characteristics, and information is easily lost after table format conversion; second, the block segmentation method is rigid, often using fixed-length divisions, which easily severs the semantic logic of disciplinary inspection clauses and fails to establish parent-child block relationships, resulting in missing context during retrieval; third, user questions have poor adaptability to business needs, failing to optimize question descriptions for power disciplinary inspection terminology, and retrieval remains only at the single block level, resulting in insufficient accuracy. Furthermore, existing vectorization methods often use general dimensions, such as 768 dimensions, without optimizing vector dimensions to suit the textual complexity of disciplinary inspection documents, leading to insufficient representation of business information by the vectors. These problems make traditional knowledge bases unable to meet the efficient retrieval needs of power system disciplinary inspection operations. Summary of the Invention

[0003] This application provides a power system knowledge retrieval method and platform based on the RAG framework, aiming to solve the technical problem that existing technologies often use general dimensions for vectorization, without optimizing the vector dimensions based on the text complexity of disciplinary inspection documents. This results in insufficient vector representation capabilities for business information, making it difficult for traditional knowledge bases to meet the efficient retrieval needs of power system disciplinary inspection business.

[0004] The first aspect disclosed in this application provides a power system knowledge retrieval method based on the RAG framework. The method includes: constructing a knowledge base for power system documents using the RAG framework to generate a vector database; receiving user input questions and reconstructing the business questions to generate business-adaptive expressions; embedding the business-adaptive expressions into an input word embedding model to convert them into a 1024-dimensional question vector; inputting the 1024-dimensional question vector into the vector database and performing sub-block vector similarity matching to filter and recall K candidate block metadata; associating the K candidate block metadata with K parent-child relationships based on the K parent block metadata with K parent block IDs; calling the K parent block 1024-dimensional vectors based on the K parent block IDs and then filtering and outputting M candidate block metadata from the K candidate block metadata by comparing the question 1024-dimensional vectors; and returning complete chapter content and source information based on the M candidate block metadata.

[0005] The second aspect disclosed in this application provides a power system knowledge retrieval platform based on the RAG framework. This platform is used in the aforementioned power system knowledge retrieval method based on the RAG framework. The platform includes: a knowledge base construction and processing module for constructing a knowledge base from power system documents using the RAG framework, generating a vector database; a business question reconstruction module for receiving user input questions, reconstructing business questions, and generating business-adaptive expressions; a vector conversion module for embedding the input words of the business-adaptive expressions into a model, converting them into a 1024-dimensional question vector; and a similarity matching module for matching the similarity of the input words. The question's 1024-dimensional vector is input into the vector database, and sub-block vector similarity matching is performed to filter and recall K candidate block metadata. The ID association module is used to associate the K parent block IDs with the K parent-child relationships of the K candidate block metadata. The metadata filtering module is used to call the K parent block 1024-dimensional vectors according to the K parent block IDs, and then filter and output M candidate block metadata from the K candidate block metadata by comparing the question's 1024-dimensional vector. The source information return module is used to return the complete chapter content and source information according to the M candidate block metadata.

[0006] One or more technical solutions provided in this application have at least the following beneficial effects: By constructing a knowledge base for power system documents based on the RAG framework, traditional documents can be transformed into a structured knowledge base, providing accurate data support for the retrieval system. After reconstructing user input questions, the generation of business-adapted expressions ensures consistency between the question descriptions and the standards of power system discipline inspection, improving retrieval accuracy. Embedding the input words of the business-adapted expressions into a model and converting them into 1024-dimensional vectors allows for the extraction of semantic features of the questions in high-dimensional vector form. This not only enhances the richness of the question representation but also improves the accuracy of subsequent similarity calculations. Using a vector database for sub-block vector similarity matching enables rapid matching of the question vector with the vectors of each sub-block in the document. This process allows the system to recall the most relevant sub-blocks from a large number of documents, significantly reducing the search scope and improving efficiency. This improves retrieval efficiency. By associating the metadata of K candidate blocks with their parent-child relationships, not only are relevant sub-blocks returned, but the retrieved information also has a complete context. Establishing parent-child block relationships enhances semantic coherence, ensuring that the returned content reflects the hierarchical structure of the document and avoiding contextual gaps that may result from returning only fragmented information. After obtaining candidate sub-blocks, similarity calculation is performed by comparing the 1024-dimensional vector of the parent block with the question vector, achieving accurate matching from sub-blocks to parent blocks. This process further optimizes the retrieval results, ensuring that the returned information not only semantically matches the user's question but also provides a more comprehensive and detailed answer. When finally outputting the retrieval results, not only is the most relevant parent block content returned, but source information is also provided to help users trace the source of information, enhancing the transparency and credibility of the retrieval.

[0007] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0008] Figure 1 A schematic diagram of the power system knowledge retrieval method based on the RAG framework provided in this application embodiment.

[0009] Figure 2 A schematic diagram of the structure of a power system knowledge retrieval platform based on the RAG framework provided in this application embodiment.

[0010] Figure labeling: Knowledge base construction and processing module 10, business problem reconstruction module 20, vector conversion module 30, similarity matching module 40, ID association module 50, metadata filtering module 60, and traceability information return module 70. Detailed Implementation

[0011] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0012] Example 1, as Figure 1 As shown in the figure, this application provides a power system knowledge retrieval method based on the RAG framework, the method including: Based on the RAG framework, a knowledge base is built from power system documents to generate a vector database.

[0013] This document describes how to structure power system documents (such as .docx format disciplinary regulations) using Python scripts. Content is extracted according to chapter structure, such as "Chapter 1 General Provisions" or "Chapter 2 Violation Identification," and then converted into key-value pair structured data. Tabular data (such as violation handling flowcharts) is converted to Markdown format using the python-docx and markdown libraries to ensure the preservation of table row and column logic and data integrity, thus avoiding information loss during vectorization.

[0014] Based on structured data, a dynamic chunking method is used to divide the text into parent-child level chunks. Parent chunks are chapters or large paragraphs of the document, while child chunks are smaller text units further subdivided from the parent chunks. During chunking, care is taken to avoid semantic breaks and ensure that sentences are not truncated during segmentation. A preset threshold length, such as 1000 tokens, is used for chunking, and a certain overlap is ensured between each child chunk, such as 100 tokens before and after, to ensure semantic coherence.

[0015] A pre-trained word embedding model, such as BERT or Sentence-BERT, is used to vectorize the text in parent-child chunks, generating 1024-dimensional vectors. Each chunk's text is transformed into a high-dimensional vector representing its semantic information. The vectorized parent-child chunk vectors are associated one-to-one with chunk information (such as chapter relationships, chunk IDs, token counts, etc.) and stored in a vector database (such as Milvus or FAISS) for easy retrieval later.

[0016] After receiving user input questions, the business problem is restructured to generate a business adaptation statement.

[0017] User-input business questions, such as "How to handle irregular expense reimbursement?", are often imprecise or contain vague expressions. To address this issue, a large language model (such as Llama 2 with fine-tuning) is used to reconstruct user questions. The model is fine-tuned using a corpus of power system discipline inspection business data, enabling it to understand industry-specific terminology and business logic. The reconstructed question expression accurately reflects the user's intent while ensuring compliance with industry standards. For example, "irregular expense reimbursement" is replaced with the more standardized "standards and procedures for identifying and handling irregular expense reimbursement in the regulations and systems of power system discipline inspection." By structuring and standardizing the question expression, it becomes adaptable to subsequent retrieval processes.

[0018] The business adaptation expression input word embedding model is transformed into a 1024-dimensional vector of questions.

[0019] The reconstructed and standardized business-adapted expressions are input into a pre-trained word embedding model, such as Sentence-BERT, which transforms them into a 1024-dimensional vector of questions.

[0020] The 1024-dimensional vector of the problem is input into the vector database, and sub-block vector similarity matching is performed to filter and recall K candidate block metadata.

[0021] The 1024-dimensional vector of the question, obtained after word embedding, is matched with the sub-block vectors in the vector database for similarity. This is achieved by calculating the cosine similarity between the question vector and all sub-block vectors. Based on the similarity score, the Top K sub-blocks are retrieved, representing the metadata of the K most relevant sub-blocks. Each sub-block contains complete text information, context, and the ID of its parent block. The retrieved candidate sub-blocks are then processed using a Rerank algorithm (such as the Cross-Encoder model) to reorder them. By comparing the semantic similarity between the question and the sub-blocks, the Top K most relevant sub-blocks are selected. After obtaining the Top K sub-blocks, the content of the parent blocks is further retrieved based on the relationship between the sub-blocks and their parent blocks. By comparing the question vector and the parent block vector, the parent block content that best matches the user's query is selected. Finally, after the retrieval and sorting of parent and child blocks, the Top K parent block contents containing the complete chapter context are returned, ensuring the completeness and contextual coherence of the results.

[0022] Based on the K parent-child relationships of the K candidate block metadata, associate the K parent block IDs.

[0023] For the obtained TopK candidate sub-blocks, each of these candidate sub-blocks has a corresponding parent block ID. Here, the parent-child relationship means that each sub-block belongs to a larger parent block, such as a chapter or paragraph, and the parent block contains more contextual information. The parent block IDs to which these candidate sub-blocks belong are extracted and associated. This association process finds the parent block corresponding to each sub-block and saves these IDs as the basis for further retrieval and merging. The associated parent block IDs serve as the basis for subsequent steps, ensuring that the search results not only include sub-blocks but also accurately retrieve the parent blocks, thereby improving the contextual completeness of the search results.

[0024] After calling the K parent block 1024-dimensional vectors based on the K parent block IDs, M candidate block metadata are output by comparing the K candidate block metadata with the question 1024-dimensional vectors.

[0025] Based on the obtained K parent block IDs, the corresponding parent block vectors are retrieved from the vector database. Each parent block has a 1024-dimensional vector representing its semantic information. The 1024-dimensional vector of the question is compared with the K parent block vectors. This is achieved by calculating the similarity (e.g., cosine similarity) between the parent block vectors and the question vector, with the aim of finding the parent blocks most relevant to the user's question. After vector matching, the M most relevant parent blocks are selected. The content of these parent blocks best matches the semantics of the user's question, and therefore, these parent blocks are considered the optimal candidate content.

[0026] Based on the M candidate block metadata, return the complete chapter content and source information.

[0027] The selected M candidate parent chunks contain their IDs and related information. Based on these parent chunk IDs, complete chapter or paragraph content is retrieved and returned. This provides more comprehensive contextual information, helping users better understand the search results. Each parent chunk includes not only text content but also source information, such as the source file name, chapter number, page number, chunk ID, and token count. This helps users trace back to the original document and further verify and understand the source of the search results. Finally, the complete chapter or paragraph content of the Top M parent chunks is returned, along with the source information. This ensures that users not only receive accurate answers but also understand the background and context of the answer's source, guaranteeing the accuracy and reliability of the response.

[0028] Furthermore, the method involves constructing a knowledge base for power system documents based on the RAG framework to generate a vector database, and includes: The power system files undergo multi-granularity structuring to generate structured data files. Based on semantic awareness, the structured data files are dynamically hierarchically segmented to obtain a parent-child two-level segmented metadata dataset. This parent-child two-level segmented metadata dataset is then input into a pre-trained word embedding model for vectorization, outputting a 1024-dimensional vector set for the parent-child two-level segments. After mapping and binding the parent-child two-level segmented metadata dataset and the 1024-dimensional vector set, the parent-child two-level segmented metadata dataset and the 1024-dimensional vector set are stored in association according to parent-child relationships, generating the vector database.

[0029] Documents in the power system (such as disciplinary regulations and rules) first undergo multi-granularity structuring processing. This involves using Python scripts or other tools to break down and organize the original document content into chapters, paragraphs, tables, and other granularities. Specifically, this process extracts the chapter information and forms a structured data format, such as JSON or key-value pairs, ensuring that the content of each chapter is clearly segmented and identified. Tables in the document are converted to Markdown format according to specific processing rules to ensure that table information is not lost in subsequent processing and can be correctly mapped to the structured data. The final output is a structured data file containing detailed content for each chapter and table, which can then be used in subsequent dynamic chunking and vectorization processes.

[0030] Based on structured data files, a semantically aware approach is used for dynamic chunking. This means that chunking relies not only on physical chapter or paragraph structures but also on semantic analysis to ensure the rationality and coherence of the chunking. First, the file is divided into parent chunks (larger text blocks) according to chapter structure. Then, based on a preset threshold (e.g., 1000 tokens), the parent chunks are further split into child chunks (smaller text blocks). During splitting, semantic integrity is preserved; for example, splitting occurs at sentence boundaries. A 10%-20% overlap is maintained between child chunks at the same level to compensate for potential semantic breaks caused by segmentation. Each chunk records its parent-child relationship, chunk ID, file origin, and other information. Finally, after semantically aware chunking, a two-level chunked metadata dataset is obtained. The relationship between parent and child chunks, as well as the semantic information of each chunk, are clearly recorded, forming a complete chunked structure.

[0031] After obtaining the parent-child two-level chunked metadata dataset, these chunks are input into pre-trained word embedding models for vectorization processing. Models include BERT and Sentence-BERT. The text content of each parent and child chunk is transformed into a high-dimensional vector (1024-dimensional). These vectors represent the semantics of the text content and can capture the deeper meaning of the text. The output of the vectorization processing is a 1024-dimensional vector set for each parent and child chunk. These vectors are used for subsequent similarity calculations and retrieval matching.

[0032] The generated parent-child block metadata dataset is bound together with the corresponding 1024-dimensional vector sets of the parent and child two-level blocks. The metadata of each parent block and child block (including ID, parent-child relationship, number of tokens, etc.) is associated with its corresponding vector. This binding is to ensure that the semantic information of the block and its corresponding vector can be matched one-to-one, which facilitates subsequent retrieval and information tracing.

[0033] The bound parent-child chunk metadata and its 1024-dimensional vector set are stored in a vector database, such as Milvus or FAISS. These databases support efficient vector storage and approximate nearest neighbor search, enabling the rapid retrieval of the most relevant chunk content during subsequent searches. The IDs and vectors of the parent and child chunks are stored together based on the parent-child relationship, ensuring appropriate merging and matching based on the parent-child hierarchy during retrieval. Finally, a vector database is generated containing all parent-child chunk metadata and their corresponding vectors, which will support subsequent retrieval and matching.

[0034] Furthermore, each block's metadata is identified by a unique ID, parent-child relationship, file origin, and number of tokens.

[0035] Each chunk (parent or child) is assigned a unique identifier, or unique ID. This unique ID is used to distinguish different chunks during subsequent retrieval and helps to trace its position in the original document. Each chunk includes the ID of its parent chunk (for child chunks) to establish the relationship between parent and child chunks. This parent-child relationship ensures that the hierarchical information of the structured text content is fully preserved, providing semantic coherence of context. Chunk metadata also identifies its source, indicating which file or document the chunk comes from, which helps with source tracing during retrieval. Each chunk's metadata also records the number of tokens in that chunk, i.e., the length of the chunk's text. This is important for subsequent chunk analysis and processing, especially when performing dynamic chunking, where the token count serves as one of the criteria for segmentation thresholds.

[0036] Furthermore, the 1024-dimensional vector of the problem is input into the vector database, and sub-block vector similarity matching is performed to filter and recall K candidate block metadata. The method includes: The 1024-dimensional vector of the question is input into the vector database; the 1024-dimensional vector of the question is used to traverse the sub-block vector sets in the parent and child two-level block 1024-dimensional vector set for similarity matching to obtain the sub-block similarity set; after serializing the sub-block similarity set, the K candidate block metadata corresponding to the TOPK block similarity are retrieved.

[0037] The 1024-dimensional vector of the question represents the semantic information of the question. This 1024-dimensional vector is input into a pre-built vector database for similarity matching and retrieval.

[0038] The vector database stores the vector data of the parent and child blocks. The 1024-dimensional vector of the question is matched with the 1024-dimensional vectors of all child blocks in the database. The similarity calculation uses cosine similarity or other similarity measurement methods. The result of the matching is a set of similarity of child blocks, so as to select the most relevant child blocks.

[0039] The generated sub-block similarity set is serialized, and the Top K candidate sub-blocks with the highest similarity scores are selected. These candidate sub-blocks are the most relevant to the user's question, representing the part that best matches the question's semantics. The K value is a preset number, such as Top 10 or Top 20, indicating that the K most relevant sub-blocks are selected from the candidate blocks. Metadata for the corresponding Top K sub-blocks, such as block ID, parent block ID, and file source, is retrieved as part of the search results. This metadata allows for further processing and display of relevant content.

[0040] Furthermore, after receiving user input questions, the business problem is restructured to generate a business adaptation statement. The method includes: The user input question is semantically segmented to obtain a sequence of semantic units; Figure 1 Consistency is used as a constraint to perform term normalization replacement on the semantic unit sequence to generate a standardized term sequence; the structure of the standardized term sequence is standardized by supplementing with business context, and the business-adaptive expression is output.

[0041] User input questions are typically in natural language form and may contain multiple concepts or topics. To make the questions more structured and easier to process later, semantic segmentation is necessary. Semantic segmentation divides a complete question sentence into multiple semantic units. These units can be words, phrases, or higher-level semantic structures. For example, the user input question "How to handle reimbursement violations" can be divided into semantic units such as "handle" (action), "violation" (behavioral nature), and "reimbursement" (target object). The segmentation of semantic units is based on natural language processing (NLP) techniques, such as syntactic analysis, dependency parsing, or deep learning-based semantic segmentation models.

[0042] After semantic segmentation, a set of semantic units is obtained. To ensure that the terminology of the problem conforms to the business context and can find matching entries in the knowledge base, these semantic units need to be standardized. Figure 1 Consistency is a key constraint; that is, the standardization process must maintain the core intent of the original problem. For example, users may use different words to express the same meaning, such as "illegal expense reimbursement" and "improper expense reimbursement," which are semantically similar. Therefore, they should be replaced with a unified term, such as "illegal expense reimbursement." The standardization process includes replacing synonyms, standardizing terminology (e.g., replacing "illegal" with "illegal behavior"), and transforming colloquial expressions into more formal industry terms, such as changing "check illegal expense reimbursement" to "verify illegal expense reimbursement." After terminology standardization, all semantic units will be unified into a standardized sequence of terms, ensuring that these terms can be accurately interpreted in the business context and matched with entries in the knowledge base.

[0043] After terminology standardization, the question structure also needs to be standardized to ensure that the question's wording conforms to the format and structure requirements of power industry discipline inspection. Supplementing the business context refers to adjusting the structure of the question's wording based on the specific business domain (such as power industry discipline inspection regulations). For example, simple questions can be converted into more standardized query formats, or specific information can be added based on business needs. For instance, "How to handle irregular reimbursement?" can be supplemented to "According to the 'Power Industry Discipline Inspection and Supervision Management Measures,' how to handle irregular reimbursement?". The structural standardization process involves converting free text into a structured query format and supplementing relevant background information (such as specific clauses). Finally, after standardization, a business-adapted expression is obtained. This expression is not only semantically accurate but also conforms to industry standards and can find an exact match in the knowledge base.

[0044] Furthermore, the structured data file is dynamically hierarchically segmented based on semantic awareness to obtain a parent-child two-level segmented metadata dataset. The method includes: Using the original chapter structure of the structured data file as the dividing unit, the structured data file is divided into multiple parent-level block metadata. Using a preset length threshold as the segmentation trigger condition, the multiple parent-level block metadata are traversed to perform sub-block segmentation, resulting in multiple candidate sub-block sequences. After detecting sentence boundary symbols in the multiple candidate sub-block sequences, segmentation position adjustment and correction are performed to obtain multiple corrected sub-block sequences. The multiple corrected sub-block sequences are then used to design overlapping regions between adjacent sub-blocks, resulting in multiple sets of child-level block metadata. The multiple parent-level block metadata constitutes a parent-level block metadata dataset, the multiple sets of child-level block metadata constitute a child-level block metadata dataset, and the parent-level block metadata dataset and the child-level block metadata dataset constitute the parent-child two-level block metadata dataset.

[0045] When processing documents, the document is first divided into blocks based on its chapter structure. Each chapter serves as a parent block unit, containing all its content. This clearly preserves the document's hierarchical structure. For example, if a document contains chapters such as "Chapter 1 General Provisions" and "Chapter 2 Violation Identification," the structured data file will be divided into multiple parent blocks, each representing a chapter and containing all information (text, tables, images, etc.) for that chapter. Each parent block is a larger unit within the document, containing a specific theme or content. In this way, the document's content organization is clearly defined, laying the foundation for subsequent fine-grained segmentation (i.e., sub-blocks).

[0046] A preset threshold length, such as 1000 tokens, is determined as the splitting trigger condition. This threshold determines when the content of a parent block reaches a certain length, at which point it begins to split into sub-blocks. The threshold is set based on the average paragraph length or text complexity of the document. For example, documents related to power industry regulations and rules may have long clauses or definitions, so a larger threshold is set to avoid excessively frequent splitting. During splitting, all parent blocks are traversed, and the length of each parent block determines whether sub-block splitting is necessary. If the content of a parent block exceeds the preset threshold length, sub-block splitting is performed. When the trigger condition is met, the parent block is split into multiple candidate sub-blocks. The lengths of these sub-blocks are within the threshold range, ensuring that the lengths of each sub-block are relatively balanced.

[0047] When performing sub-block segmentation, segmenting solely based on a length threshold may lead to semantic incoherence, such as segmenting in the middle of a sentence. To avoid this, sentence boundary symbol detection is performed to ensure that the segmentation point appears at the sentence boundary (such as a period, semicolon, comma, etc.), rather than in the middle of the sentence. By using sentence boundary detection, the segmentation position is adjusted and corrected. For example, if the segmentation point is in the middle of the sentence, the segmentation position is adjusted to the sentence boundary, ensuring that each sub-block contains a complete sentence and avoiding semantic breaks. The adjusted sub-blocks form a corrected sub-block sequence, where each sub-block is semantically more complete and contextually coherent, facilitating subsequent semantic representation and vectorization.

[0048] To ensure the continuity and integrity of information between sub-chunks, overlapping regions are designed between adjacent sub-chunks. This means that there is some overlap between every two adjacent sub-chunks to prevent information loss. The design of these overlapping regions is proportional; for example, the overlap length is set to 10%-20%. If a sub-chunk contains 1000 tokens, then the subsequent sub-chunk can retain 100-200 tokens from the previous sub-chunk as an overlapping region. The purpose of the overlapping regions is to compensate for any contextual information that may be lost during the chunking process, especially in the segmentation of long documents. The overlapping content helps ensure that the information connection between sub-chunks is not severed. In this way, a certain semantic connection is maintained between sub-chunks, improving the contextual coherence during retrieval. The sub-chunks designed with overlapping regions form multiple sets of sub-level chunk metadata. Each sub-chunk contains not only text content but also the chunk's ID, the length of the overlapping region, and other information, ensuring that subsequent retrieval and tracing operations can proceed normally.

[0049] All parent chunks (e.g., chapters or large paragraphs) are organized into a parent chunk metadata dataset. This dataset contains information about all parent chunks in the document, including the chunk's ID, chapter number, text content, and related metadata. All child chunks (fine-grained units resulting from the splitting of parent chunks) are organized into a child chunk metadata dataset. This dataset contains information about all child chunks, with metadata for each child chunk including its ID, parent chunk ID, text content, token count, and overlapping area information. Finally, the parent and child chunk metadata datasets form a complete two-level parent-child chunk metadata dataset. This dataset provides structured text units for subsequent vectorization, retrieval, and semantic matching, and connects the overall document structure through parent-child relationships. The parent-child chunk metadata dataset ensures that information at different levels of the document can be effectively indexed and retrieved, preserving broad information at the chapter level while refining it to smaller semantic units, supporting more accurate semantic matching and context reconstruction.

[0050] Furthermore, the parent-child two-level block metadata is input into a pre-trained word embedding model for vectorization processing, outputting a 1024-dimensional vector set of parent-child two-level blocks. The method includes: The first linear string of the metadata of the first parent block is input into the word embedding model; the word embedding model performs the following: S1: extracting the first semantic feature via the Transformer encoding layer of the word embedding model; the word embedding model performs the following: S2: the mean pooling layer of the word embedding model generates the first sentence vector based on the first semantic feature; S3: the 1024-dimensional fully connected layer of the word embedding model outputs the 1024-dimensional vector of the first parent block based on the first sentence vector.

[0051] The first linear string of the metadata from the first parent block is input into a pre-trained word embedding model for processing. The first linear string refers to the text data extracted from the parent block. This text data has already undergone the previous structured processing to ensure that the content is clear and structured. Word embedding models, such as BERT and Sentence-BERT, are pre-trained natural language processing models that can transform text into high-dimensional vector representations and capture the semantic features of the text.

[0052] When the input text passes through the word embedding model, the Transformer encoding layer processes the text. The Transformer is a widely used architecture in modern natural language processing (NLP), capable of capturing the relationships and semantics between words in the text through a self-attention mechanism. This layer processes the parent block of input text, extracting the first semantic features containing the global context, i.e., the high-level semantic information of the text.

[0053] After semantic features are extracted in the Transformer encoding layer, mean pooling is used to transform these features into a fixed-length vector. Mean pooling generates a comprehensive text representation vector by averaging the vectors of each word or word segment. This process ensures that the semantic features of the entire parent block are compressed into a single vector, facilitating subsequent processing. The result is the first sentence vector, which represents the comprehensive semantic information of the input text (parent block).

[0054] After mean pooling, the resulting first sentence vector is fed into a 1024-dimensional fully connected layer. This layer is responsible for mapping the text features to a high-dimensional vector space (1024 dimensions). Through the fully connected layer, a 1024-dimensional vector is generated based on the input semantic information. This vector is a deep semantic representation of the parent block text, containing rich contextual information. This 1024-dimensional vector is the first parent block's 1024-dimensional vector, used in subsequent similarity matching, retrieval, and analysis processes.

[0055] The processing flow in steps 131 to S3 applies not only to the parent block but also to the child block. Each child block is processed by the word embedding model to generate its corresponding 1024-dimensional vector, ensuring that the semantic information of the parent and child blocks is represented consistently.

[0056] Example 2, based on the same inventive concept as the power system knowledge retrieval method based on the RAG framework in the previous examples, such as... Figure 2 As shown in the embodiment of this application, a power system knowledge retrieval platform based on the RAG framework is provided, the platform comprising: The system comprises the following modules: a knowledge base construction and processing module 10, which constructs a knowledge base for power system documents based on the RAG framework and generates a vector database; a business problem reconstruction module 20, which receives user input questions, reconstructs business problems, and generates business-adaptive expressions; a vector conversion module 30, which embeds the input words of the business-adaptive expressions into a model and converts them into a 1024-dimensional vector of the question; a similarity matching module 40, which inputs the 1024-dimensional vector of the question into the vector database and performs sub-block vector similarity matching to filter and recall K candidate block metadata; an ID association module 50, which associates the K parent block IDs of the K candidate block metadata with the K parent-child relationships; a metadata filtering module 60, which calls the K parent block 1024-dimensional vectors based on the K parent block IDs and then filters and outputs M candidate block metadata by comparing the question 1024-dimensional vectors; and a traceability information return module 70, which returns complete chapter content and traceability information based on the M candidate block metadata.

[0057] Furthermore, the knowledge base construction and processing module 10 is used to perform the following operation steps: The power system files undergo multi-granularity structuring to generate structured data files. Based on semantic awareness, the structured data files are dynamically hierarchically segmented to obtain a parent-child two-level segmented metadata dataset. This parent-child two-level segmented metadata dataset is then input into a pre-trained word embedding model for vectorization, outputting a 1024-dimensional vector set for the parent-child two-level segments. After mapping and binding the parent-child two-level segmented metadata dataset and the 1024-dimensional vector set, the parent-child two-level segmented metadata dataset and the 1024-dimensional vector set are stored in association according to parent-child relationships, generating the vector database.

[0058] Furthermore, each block's metadata is identified by a unique ID, parent-child relationship, file origin, and number of tokens.

[0059] Furthermore, the similarity matching module 40 is used to perform the following operation steps: The 1024-dimensional vector of the question is input into the vector database; the 1024-dimensional vector of the question is used to traverse the sub-block vector sets in the parent and child two-level block 1024-dimensional vector set for similarity matching to obtain the sub-block similarity set; after serializing the sub-block similarity set, the K candidate block metadata corresponding to the TOPK block similarity are retrieved.

[0060] Furthermore, the business problem reconstruction module 20 is used to perform the following operation steps: The user input question is semantically segmented to obtain a sequence of semantic units; Figure 1 Consistency is used as a constraint to perform term normalization replacement on the semantic unit sequence to generate a standardized term sequence; the structure of the standardized term sequence is standardized by supplementing with business context, and the business-adaptive expression is output.

[0061] Furthermore, the knowledge base construction and processing module 10 is used to perform the following operation steps: Using the original chapter structure of the structured data file as the dividing unit, the structured data file is divided into multiple parent-level block metadata. Using a preset length threshold as the segmentation trigger condition, the multiple parent-level block metadata are traversed to perform sub-block segmentation, resulting in multiple candidate sub-block sequences. After detecting sentence boundary symbols in the multiple candidate sub-block sequences, segmentation position adjustment and correction are performed to obtain multiple corrected sub-block sequences. The multiple corrected sub-block sequences are then used to design overlapping regions between adjacent sub-blocks, resulting in multiple sets of child-level block metadata. The multiple parent-level block metadata constitutes a parent-level block metadata dataset, the multiple sets of child-level block metadata constitute a child-level block metadata dataset, and the parent-level block metadata dataset and the child-level block metadata dataset constitute the parent-child two-level block metadata dataset.

[0062] Furthermore, the knowledge base construction and processing module 10 is used to perform the following operation steps: The first linear string of the metadata of the first parent block is input into the word embedding model; the word embedding model performs the following: S1: extracting the first semantic feature through the Transformer encoding layer of the word embedding model; S2: the mean pooling layer of the word embedding model generates the first sentence vector based on the first semantic feature; S3: the 1024-dimensional fully connected layer of the word embedding model outputs the 1024-dimensional vector of the first parent block based on the first sentence vector.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A power system knowledge retrieval method based on the RAG framework, characterized in that, The method includes: Based on the RAG framework, knowledge base construction and processing of power system documents are performed to generate a vector database; After receiving user input questions, the business problem is restructured to generate a business adaptation statement; The business adaptation expression input word embedding model is transformed into a 1024-dimensional vector of questions. The 1024-dimensional vector of the problem is input into the vector database, and sub-block vector similarity matching is performed to filter and recall K candidate block metadata. Based on the K parent-child relationships of the K candidate block metadata, associate them with the K parent block IDs; After calling the K parent block 1024-dimensional vectors based on the K parent block IDs, M candidate block metadata are output by comparing the question 1024-dimensional vectors with the K candidate block metadata. Based on the M candidate block metadata, return the complete chapter content and source information.

2. The power system knowledge retrieval method based on the RAG framework as described in claim 1, characterized in that, The method for constructing a knowledge base from power system documents using the RAG framework to generate a vector database includes: The power system files undergo multi-granularity structuring processing to generate structured data files; Based on semantic awareness, the structured data file is dynamically divided into hierarchical blocks to obtain a parent-child two-level block meta-dataset; The parent-child two-level block metadata is input into a pre-trained word embedding model for vectorization processing, and the output is a 1024-dimensional vector set of parent-child two-level blocks. After mapping and binding the parent-child two-level block metadata dataset and the parent-child two-level block 1024-dimensional vector set, the parent-child two-level block metadata dataset and the parent-child two-level block 1024-dimensional vector set are associated and stored according to the parent-child relationship to generate the vector database.

3. The power system knowledge retrieval method based on the RAG framework as described in claim 2, characterized in that, Each block's metadata identifier includes a unique ID, parent-child relationship, file origin, and number of tokens.

4. The power system knowledge retrieval method based on the RAG framework as described in claim 3, characterized in that, The method involves inputting the 1024-dimensional vector of the problem into the vector database, performing sub-block vector similarity matching, and filtering and recalling K candidate block metadata. Input the 1024-dimensional vector of the problem into the vector database; The sub-block vector sets in the parent-child two-level block 1024-dimensional vector set are traversed using the 1024-dimensional vector of the problem to perform similarity matching, thereby obtaining the sub-block similarity set; After serializing the sub-block similarity set, the metadata of the K candidate blocks corresponding to the TOPK block similarities is retrieved.

5. The power system knowledge retrieval method based on the RAG framework as described in claim 1, characterized in that, After receiving user input questions, the business problem is restructured to generate a business adaptation statement. The method includes: The user input question is semantically segmented to obtain a sequence of semantic units; With intent consistency as a constraint, the semantic unit sequence is subjected to terminology normalization replacement to generate a standardized terminology sequence; The standardized terminology sequence is structurally standardized by supplementing with business context, and the business-adaptive expression is output.

6. The power system knowledge retrieval method based on the RAG framework as described in claim 2, characterized in that, The structured data file is dynamically hierarchically segmented based on semantic awareness to obtain a parent-child two-level segmented metadata dataset. The method includes: Using the original chapter structure of the structured data file as the dividing unit, the structured data file is divided into multiple parent block metadata. Using a preset length threshold as the segmentation trigger condition, the multiple parent block metadata are traversed to perform sub-block segmentation, resulting in multiple candidate sub-block sequences. After performing sentence boundary symbol detection on the multiple candidate sub-block sequences, the segmentation position adjustment and correction are performed to obtain multiple corrected sub-block sequences; Design the overlapping region of adjacent sub-blocks for the multiple modified sub-block sequences to obtain multiple sets of sub-level block metadata; The multiple parent-level block metadata constitutes the parent-level block metadata dataset, the multiple sets of child-level block metadata constitute the child-level block metadata dataset, and the parent-level block metadata dataset and the child-level block metadata dataset constitute the parent-child two-level block metadata dataset.

7. The power system knowledge retrieval method based on the RAG framework as described in claim 2, characterized in that, The parent-child two-level block metadata is input into a pre-trained word embedding model for vectorization processing, outputting a 1024-dimensional vector set of parent-child two-level blocks. The method includes: Input the first linear string of the first parent block metadata into the word embedding model; Execution in the word embedding model: S1: Extract the first semantic features via the Transformer encoding layer of the word embedding model; S2: The mean pooling layer of the word embedding model generates the first sentence vector based on the first semantic feature; S3: The 1024-dimensional fully connected layer of the word embedding model outputs the first parent block 1024-dimensional vector based on the first sentence vector.

8. A power system knowledge retrieval platform based on the RAG framework, characterized in that: The platform is used to implement the power system knowledge retrieval method based on the RAG framework according to any one of claims 1-7, and the platform includes: The knowledge base construction and processing module is used to construct and process knowledge bases for power system documents based on the RAG framework, generating a vector database. The business problem reconstruction module is used to receive user input problems, reconstruct business problems, and generate business adaptation statements. The vector conversion module is used to convert the business adaptation expression input word embedding model into a 1024-dimensional vector. The similarity matching module is used to input the 1024-dimensional vector of the question into the vector database and perform sub-block vector similarity matching to filter and recall K candidate block metadata. The ID association module is used to associate the K parent block IDs with the K parent block IDs based on the K parent-child relationships of the K candidate block metadata. The metadata filtering module is used to call the K parent block 1024-dimensional vectors based on the K parent block IDs, and then filter and output M candidate block metadata from the K candidate block metadata by comparing the question 1024-dimensional vectors. The traceability information return module is used to return the complete chapter content and traceability information based on the M candidate block metadata.

Images