A document retrieval optimization method and system for backbone optical communication equipment

Abstract

The present invention relates to the technical field of language retrieval enhancement, and discloses a method and system for optimizing document retrieval of backbone optical communication equipment. The method comprises: acquiring named entity types and relationship types based on equipment documents, and constructing a data structure; extracting named entities and relationship words through a large language model based on the data structure to generate at least two triple data; performing deduplication and fusion on all generated triple data to obtain multi-tuple data to form a complete knowledge graph; constructing a text vector database; extracting named entities and keywords in the query command in response to a query command, obtaining command entities and command keywords, and converting the query command into an embedded vector; performing mixed retrieval and similarity matching on the command entity, command keyword and embedded vector, obtaining corresponding matching results, and performing deduplication, fusion and rearrangement to obtain an optimized result, and finally generating a query answer, thereby improving query efficiency and accuracy and reducing maintenance costs.

Description

Technical Field

[0001] The present invention relates to the technical field of language retrieval enhancement, and in particular to a document retrieval optimization method and system for backbone optical communication equipment. Background Art

[0002] The backbone optical communication system is an important part of the power grid communication network system. The reliability and stability of its operation are necessary conditions for the safety and stability of the power communication system. With the continuous improvement of the power transmission and communication capabilities of the power grid communication network, the reliability and stability requirements of communication equipment are also increasing. Therefore, the requirements for rapid retrieval of equipment documents and fault location are becoming increasingly higher.

[0003] However, since the backbone optical communication field is a professional field, there are problems such as small data set size, scarce labeled samples, and poor applicability of general entity libraries. It is difficult for various neural network algorithms to obtain good training results with small samples. In addition, rule-based named entity recognition technology is limited by the large number of fault text rules and the high cost of manual compilation, making it difficult to apply. At present, the equipment entity and relationship extraction methods usually rely on manually defined rules or feature engineering, which is not only inefficient, but also requires a lot of adjustments and redesigns in different fields or language environments. In addition, the manual definition of rules is easily affected by human factors and is difficult to handle complex language phenomena and diverse text formats. At the same time, the backbone optical communication system relies on manual on-site inspection, equipment manual query, fault confirmation, reporting and maintenance for equipment installation, maintenance, and fault handling. Manual query of equipment fault manuals requires high time and labor costs. In addition, due to the lag in manual information update, insufficient content, complex operation, and lack of actual guidance cases, there is a possibility of misjudgment.

[0004] Therefore, there is an urgent need for a method that can quickly retrieve backbone optical communication equipment documents, quickly locate and handle faults occurring during the operation of communication equipment, and reduce maintenance difficulty and labor costs. Summary of the invention

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the present invention provides a backbone optical communication equipment document retrieval optimization method and system to solve the problem that the existing backbone optical communication equipment document retrieval method has low information retrieval efficiency when faced with a large number of complex technical documents, requires a lot of time to search and screen, and has shortcomings such as insufficient accuracy, untimely knowledge updating, and poor interactive experience, resulting in low efficiency and high cost of equipment installation, maintenance, and servicing.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, the present invention provides a method for optimizing document retrieval of backbone optical communication equipment, comprising: obtaining named entity types and relationship types required for a knowledge graph based on equipment documents;

[0009] Build and define data structures based on named entity types and relationship types;

[0010] Based on the data structure, a prompt is set and named entities and relation words in any device document are extracted through a large language model to generate at least two triples of data;

[0011] De-duplicate and merge all generated triplet data to obtain optimized multi-tuple data;

[0012] Storing the multi-tuple data in a graph database to form a complete knowledge graph;

[0013] Build a text vector database for device documentation;

[0014] In response to a user query command, named entities and keywords in the query command are extracted to obtain command entities and command keywords, and the query command is converted into an embedding vector;

[0015] Through the text vector database and knowledge graph, the command entities, command keywords and embedded vectors are hybrid searched and similarity matched to obtain the corresponding matching results respectively;

[0016] De-duplication and fusion of the matching results are performed, and they are rearranged with the query command to obtain an optimized result;

[0017] The optimization result is combined with the prompt word to generate a query answer.

[0018] As a preferred solution of the backbone optical communication equipment document retrieval optimization method described in the present invention, wherein: the triple data includes two named entities and a relational word, and the two named entities are combined and connected by a relational word.

[0019] As a preferred solution of the backbone optical communication equipment document retrieval optimization method described in the present invention, wherein: all generated triplet data are deduplicated and merged to obtain optimized multi-tuple data, including:

[0020] Setting a deduplication rule, wherein for the same named entity extracted from different child device documents, if the different child device documents belong to the same parent document, deduplication is performed;

[0021] The deduplicated results are fused to obtain multi-tuple data, where the multi-tuple data includes at least one triple data.

[0022] As a preferred solution of the backbone optical communication equipment document retrieval optimization method described in the present invention, wherein: all generated triplet data are deduplicated and merged to obtain optimized multi-tuple data, including:

[0023] Eliminate duplicate named entities and relation words through hash functions or similarity-based deduplication algorithms;

[0024] The triplet data from different device document sources are merged by a merging algorithm based on similarity calculation to obtain multi-tuple data, wherein the multi-tuple data includes at least one triplet data.

[0025] As a preferred solution of the document retrieval optimization method for backbone optical communication equipment described in the present invention, wherein: a hybrid retrieval and similarity matching of command entities, command keywords and embedded vectors are performed through a text vector database and a knowledge graph to obtain corresponding matching results respectively, including: matching the embedded vector through the text vector database to obtain at least one text block with semantics similar to the query command, specifically:

[0026] Connect vector data through index structure;

[0027] Define a distance metric to calculate the first similarity between the query command vector and the text vector;

[0028] Based on the first similarity, an approximate algorithm is used to search for vectors similar to the query command vector and sort them;

[0029] At least one text block is obtained based on the first similarity search result within the preset sorting range.

[0030] As a preferred solution of the document retrieval optimization method for backbone optical communication equipment described in the present invention, wherein: hybrid retrieval and similarity matching are performed on command entities, command keywords and embedded vectors through a text vector database and a knowledge graph to obtain corresponding matching results respectively, and further comprising: keyword matching is performed on command keywords through the text vector database and based on a text matching algorithm to obtain at least one text block with semantics similar to the query command, specifically:

[0031] The text is retrieved based on the BM25 algorithm, and the frequency of occurrence of each command keyword in each text and the length of the text are counted through the inverted index;

[0032] Calculate the similarity score between the command keyword and the text based on the frequency of occurrence and the length of the text;

[0033] The text blocks are sorted according to the similarity scores, and the text blocks with high scores are recalled first.

[0034] As a preferred solution of the document retrieval optimization method for backbone optical communication equipment described in the present invention, wherein: hybrid retrieval and similarity matching are performed on command entities, command keywords and embedded vectors through a text vector database and a knowledge graph to obtain corresponding matching results respectively, and further comprising:

[0035] Obtaining relevant subgraphs in the knowledge graph according to the command entity and the corresponding relational words;

[0036] Calculate the second similarity between the obtained subgraphs and the query command, and sort them;

[0037] Based on the second similarity within the preset sorting range, a corresponding knowledge subgraph is obtained.

[0038] In a second aspect, the present invention provides a backbone optical communication equipment document retrieval optimization system, comprising:

[0039] The first acquisition module is used to acquire the named entity types and relationship types required for the knowledge graph based on the device document;

[0040] The data structure building module is used to build and define data structures based on named entity types and relationship types;

[0041] A first extraction module, configured to set a prompt based on the data structure and extract named entities and relation words in any device document through a large language model to generate at least two triples of data;

[0042] The first deduplication and fusion module is used to deduplicate and fuse all generated triplet data to obtain optimized multi-tuple data;

[0043] A graph construction module, used for storing the multi-tuple data into a graph database to form a complete knowledge graph;

[0044] A text vector database construction module, used for constructing a text vector database for device documents;

[0045] The second extraction module is used to extract named entities and keywords in the query command in response to the user query command, obtain command entities and command keywords, and convert the query command into an embedding vector;

[0046] The retrieval and matching module is used to perform hybrid retrieval and similarity matching on command entities, command keywords and embedded vectors through the text vector database and knowledge graph to obtain corresponding matching results respectively;

[0047] A second deduplication and fusion module is used to deduplicate and fuse the matching results, and rearrange them with the query command to obtain an optimized result;

[0048] The reply module is used to generate a query answer by combining the optimization result with the prompt word.

[0049] In a third aspect, the present invention provides a computing device, comprising:

[0050] Memory and processor;

[0051] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the backbone optical communication equipment document retrieval optimization method are implemented.

[0052] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the backbone optical communication equipment document retrieval optimization method.

[0053] Compared with the prior art, the present invention has the following beneficial effects: the present invention can quickly query relevant equipment information and solutions through knowledge graph and retrieval enhancement technology, without flipping through the equipment manual page by page, thus improving query efficiency. It can also extract the most relevant content from massive documents according to user query commands, provide accurate answers, and improve information acquisition efficiency; the constructed knowledge graph can be dynamically updated to ensure that the latest equipment information and troubleshooting methods are reflected in the system in a timely manner to avoid information lag; it can associate actual operation cases with the content of the equipment manual through the knowledge graph, and provide detailed case analysis and solutions; it can gradually guide maintenance personnel to operate through entity language relation words, reducing the difficulty of operation. It can also handle a variety of problems according to user needs; at the same time, it can achieve unified management of all equipment manuals and documents to ensure version consistency and accessibility, and maintenance personnel can obtain the required information anytime and anywhere through mobile devices to improve work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0055] Figure 1 A schematic diagram of the overall process logic of a method for optimizing document retrieval for backbone optical communication equipment according to an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of a process for constructing a knowledge graph based on a large language model in a method for optimizing document retrieval for backbone optical communication equipment according to an embodiment of the present invention;

[0057] Figure 3 A schematic diagram of a vector database construction process in a backbone optical communication equipment document retrieval optimization method according to an embodiment of the present invention;

[0058] Figure 4 The present invention is a schematic diagram of the overall process framework structure in the backbone optical communication equipment document retrieval optimization method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0060] Example 1, reference Figure 1-Figure 4 , as an embodiment of the present invention, provides a backbone optical communication equipment document retrieval optimization method, comprising:

[0061] S1: Obtain the named entity types and relationship types required for the knowledge graph based on device documents;

[0062] Specifically, the definitions of the above named entity types and relationship types can be mainly determined in the following ways:

[0063] ① Discuss with domain experts to clarify the actual process and business requirements of the power communication network fault diagnosis business and determine the goal of ontology construction; ② Investigate the ontology construction work in other related fields, such as power grid fault diagnosis, mechanical fault diagnosis, etc., to determine whether it can be reused in this field; ③ Combine business needs, related domain ontology information and power communication network fault text data characteristics to define the core concepts of fault knowledge and preliminarily form a conceptual system for the power communication network fault field; ④ Based on the definition of professional terms in national standards and industry standards related to the power communication field, a standardized conceptual system for the power communication network fault field is formed; ⑤ Collect evaluations and improvement suggestions on the fault knowledge ontology from experts in the field; ⑥ Return to step ③ to improve and optimize the fault knowledge model; ⑦ After the improvement and optimization, the final ontology conceptual model is obtained, including named entity types and relationship types.

[0064] The resulting ontology conceptual model may include: "fault phenomenon", "fault cause", "equipment components", "alarm name", "tools used", "inspection action", "handling action", "subdivision type" and "fault type", etc.

[0065] The relationship types mainly include "cause of occurrence", "inspection operation", "disposal operation", "involvement", "report", "use", "corresponding operation", "belong to" and "details".

[0066] S2: Build and define data structures based on named entity types and relationship types;

[0067] Specifically, build and define the data structure (Schema), including: defining entity types, relationship types, and corresponding attributes and constraints.

[0068] Exemplarily, for example, the attributes defining the device entity type include model, manufacturer, and the like.

[0069] S3: Based on the data structure, set prompts and extract named entities and relation words in any device document through a large language model to generate at least two triples of data;

[0070] Specifically, named entities include: product, fault category, alarm, fiber jumper, site, board, received optical power, and transmitted optical power;

[0071] Relational words such as: associate, appear in, connect, receive, send.

[0072] Exemplarily, based on the above types, the ternary relationships between nine entity types are defined as follows:

[0073] ① {fault phenomenon, cause of occurrence, fault cause}: Describe the causal relationship between the fault phenomenon and the fault cause.

[0074] ②{fault phenomenon, inspection operation, inspection action}: Describe the relationship between the fault phenomenon and the inspection action necessary to eliminate the fault.

[0075] ③{fault phenomenon, handling operation, handling action}: Describe the relationship between the fault phenomenon and the handling action necessary to eliminate the fault.

[0076] ④{fault phenomenon, involved, equipment components}: Describe the relationship between the fault phenomenon and the equipment components involved when the fault phenomenon occurs.

[0077] ⑤{Equipment component, report, alarm name}: describes the relationship between the equipment component and the alarm reported by the equipment component when a fault occurs.

[0078] ⑥{Fault phenomenon, use, tools used}: Describe the set of tools that may be used to troubleshoot the fault.

[0079] ⑦ {Inspection action, corresponding operation, disposal action}: Describe the corresponding disposal operation to be taken after performing the necessary inspection action.

[0080] ⑧{fault phenomenon, belongs to, specific type}: Describe the relationship between the fault phenomenon and the specific fault type to which it belongs.

[0081] ⑨{fault type, refinement, specific type}: describes the relationship between the fault type and the specific fault types that can be further refined and classified.

[0082] Specifically, the large language model selects a suitable large language model, such as GPT-4 or Llama2 model, based on factors such as model performance, domain adaptability, resource requirements and regulations. At the same time, natural language processing tools such as SpaCy and Stanford NLP can be used for auxiliary extraction.

[0083] In an embodiment of the present application, triple data includes two named entities and a relational word, and the two named entities are combined and connected by the relational word.

[0084] Exemplarily, two named entities are extracted as "network element amplifier board" and "MUT_LOS alarm", and a "report" relationship. The triple data connects the two named entities through the "report" relationship word, expressed as: network element amplifier board-report-MUT_LOS alarm.

[0085] It should be noted that in this step, the introduction of a large language model to extract named entities and relational words can learn rich language representations and deep semantic information, thereby performing well in named entity recognition and relation extraction tasks. Large language models such as BERT capture contextual information through a bidirectional encoder, which can better understand the meaning of words in different contexts, thereby improving the accuracy of entity recognition. GPT uses a generative approach to generate reasonable text fragments, predict and complete sentences given a context, and thus better understand the relationship between entities.

[0086] The application of large language models not only improves the performance of named entity and relationship extraction, but also simplifies the development process, making the construction of knowledge graphs more efficient and feasible. The pre-trained model has strong generalization capabilities through learning from a large amount of data, and can accurately identify entities and relationships in complex texts. Since the model has been pre-trained on large-scale data, developers no longer need to manually define rules or features for each specific task, greatly reducing the workload in the early stages. In addition, the pre-trained model can quickly adapt to different fields and languages, such as professional terms and nouns in backbone optical communication equipment documents, which can significantly improve the scalability and adaptability of the system.

[0087] S4: De-duplicate and merge all generated triplet data to obtain optimized multi-tuple data;

[0088] In the embodiment of the present application, all generated triplet data are deduplicated and merged to obtain optimized multi-tuple data, including:

[0089] Set deduplication rules. For the same named entity extracted from different sub-device documents, if the different sub-device documents belong to the same parent document, deduplication will be performed.

[0090] The results after deduplication are merged to obtain multi-tuple data, where the multi-tuple data includes at least one triple data.

[0091] For example, entities and relations are extracted from two documents through a large language model. It is assumed that the two documents are from the device document of OSN8800.

[0092] In document 1, the two entities "network element amplifier board" and "MUT_LOS alarm" are obtained and connected with the "report" relationship, and the triple "network element amplifier board-report-MUT_LOS alarm" is obtained;

[0093] In document 2, the two entities "network element amplifier board" and "communication failure" are obtained and connected with the relation word "main cause" to obtain the triple "network element amplifier board-main cause-communication failure";

[0094] Through the deduplication rules, it can be found that the entity "Network Element Amplifier Board" appears twice, and both are from the same parent document. Then according to the deduplication rules, it is determined that these entities are the same entity and deduplication is performed;

[0095] The results of the two extractions are combined to obtain the multi-tuple data of "communication failure - main reason - network element amplifier board - report - MUT_LOS alarm".

[0096] In another embodiment of the present application, all generated triplet data are deduplicated and merged to obtain optimized multi-tuple data, including:

[0097] Eliminate duplicate named entities and relation words through hash functions or similarity-based deduplication algorithms;

[0098] Specifically, the similarity-based deduplication algorithm may be Jaccard similarity;

[0099] The triplet data from different device document sources are merged through a merging algorithm based on similarity calculation to obtain multi-tuple data, which includes at least one triplet data.

[0100] S5: Store the multi-tuple data into a graph database to form a complete knowledge graph;

[0101] Specifically, the graph database may preferably be a NEO4J graph database. A batch import tool of NEO4J, such as neo4j-admin import, is used to import multi-tuple data into the graph database, thereby creating an index to improve query performance, and performing database optimization, such as configuring cache and adjusting memory settings.

[0102] S6: Build a text vector database for device documents;

[0103] Specifically, Figure 3 As shown in the figure, the steps for constructing the text vector database include:

[0104] Step 1: Preprocess the device documents

[0105] (1) Data cleaning: remove irrelevant information such as HTML tags, footnotes, and advertisements;

[0106] (2) Formatting: Unify the document format, for example: unify the time format and unit.

[0107] Step 2: Divide the text into chunks

[0108] (1) Chunking strategy: Chunking is performed according to the token capacity of the model while ensuring semantic coherence;

[0109] (2) Chunking tools: Use tools such as NLTK and SpaCy to chunk text.

[0110] Step 3: Text vectorization: Convert text blocks into vectors through an Embedding model such as BERT, RoBERTa, or OpenAI’s text-embedding-3 model.

[0111] Step 4: Store vector database

[0112] (1) Database selection: Choose a vector database such as Faiss, Milvus or Pinecone;

[0113] (2) Index construction: Use index structures such as HNSW and IVF to optimize vector retrieval efficiency.

[0114] S7: In response to a user query command, extract named entities and keywords in the query command to obtain command entities and command keywords, and convert the query command into an embedding vector;

[0115] Specifically, in this step, extraction can be performed by using the same functional tools such as SpaCy's NER; the query is converted into a vector using the same Embedding model as the document.

[0116] S8: Perform hybrid retrieval and similarity matching on command entities, command keywords and embedded vectors through the text vector database and knowledge graph to obtain corresponding matching results respectively;

[0117] In the embodiment of the present application, the command entity, command keyword and embedded vector are subjected to hybrid retrieval and similarity matching through the text vector database and the knowledge graph to obtain corresponding matching results respectively, including: matching the embedded vector through the text vector database to obtain at least one text block with semantics similar to the query command, specifically:

[0118] Connect vector data through index structure;

[0119] Specifically, in order to speed up the retrieval process, index structures such as inverted indexes, tree structures, hash tables, etc. are used to organize vector data.

[0120] Define a distance metric to calculate the first similarity between the query command vector and the text vector;

[0121] Specifically, the distance metric may be, for example, Euclidean distance, cosine similarity, Manhattan distance, or the like.

[0122] Based on the first similarity, an approximate algorithm is used to search for vectors similar to the query command vector and sort them;

[0123] Specifically, since the exact nearest neighbor search is computationally expensive in high-dimensional space, approximate algorithms such as HNSW, Annoy, and FAISS are used to efficiently find vectors similar to the query vector;

[0124] Based on the first similarity search result within the preset sorting range, obtain at least one text block;

[0125] Specifically, top-k vector matching is performed in the text vector database, preferably using KNN search of Faiss.

[0126] Exemplarily, there are 10 original device text data in total, the text data and user commands are vectorized, the cosine similarities between the user commands and the 10 text data are calculated respectively, and the three most similar results are returned.

[0127] In the embodiment of the present application, step S8 further includes: performing keyword matching on the command keywords through a text vector database and based on a text matching algorithm to obtain at least one text block that is semantically similar to the query command, specifically:

[0128] The text is retrieved based on the BM25 algorithm, and the frequency of occurrence of each command keyword in each text and the length of the text are counted through the inverted index;

[0129] Calculate the similarity score between the command keyword and the text based on the frequency of occurrence and the length of the text;

[0130] The text blocks are sorted according to the similarity scores, and the text blocks with high scores are recalled first.

[0131] Specifically, other algorithms such as TF-IDF and cosine similarity can be combined to optimize the recall effect.

[0132] In the embodiment of the present application, step S8 further includes:

[0133] Obtain relevant subgraphs in the knowledge graph according to the command entity and the corresponding relational words;

[0134] Calculate the second similarity between the obtained subgraphs and the query command, and sort them;

[0135] Based on the second similarity within the preset sorting range, a corresponding knowledge subgraph is obtained.

[0136] Specifically, the similarity can be calculated using Graph Embedding technology.

[0137] S9: De-duplicate and merge the matching results, and rearrange them with the query command to obtain an optimized result;

[0138] Specifically, step S9 is:

[0139] Deduplication algorithm: Use algorithms such as MinHash to deduplicate.

[0140] Fusion algorithm: Reciprocal Rank Fusion, Relative-score Fusion or Distance-Based Fusion.

[0141] Reranking model: Use reranking models such as Cohere Reranker and Jina Reranker to more finely rank the matching results, thereby improving the input quality of the generation model.

[0142] According to the deduplication, fusion and re-ranking methods, the text blocks and knowledge sub-graphs obtained by vector retrieval, BM 25 retrieval and knowledge graph retrieval are merged to obtain the optimized results as the overall background knowledge for the next step of prompt word construction.

[0143] For example, for S8-S9:

[0144] Respond to the user query command, the query command is: What is the reason why the downstream site OAU board reports the OUT_LOS alarm?

[0145] Vector search: Convert the user's query into a vector and perform semantic search. The semantics of chunk2 and chunk3 are closest to the user's query.

[0146] BM25 search: Based on the keywords "downstream site", "OAU board", "OUT_LOS alarm", "reason", etc. in the user query, the most relevant text blocks are retrieved as chunk1 and chunk2;

[0147] Knowledge graph retrieval: Similarly, the keywords in the user query are matched with the entities in the knowledge graph, and the matches are "OAU board-reported alarm-OTU_LOS alarm" and "OUT_LOS alarm-main reason-received optical power is too low".

[0148] The results of the three searches were deduplicated, merged and rearranged;

[0149] It is found that the vector retrieves chunks 2 and 3, and BM25 retrieves chunks 1 and 2, so the duplicates are removed and merged into chunks 1, 2, and 3;

[0150] The knowledge graph retrieves "OUT_LOS alarm - main reason - received optical power is too low";

[0151] Use the optimization results after deduplication as background knowledge;

[0152] The background knowledge and query commands are rearranged so that the most relevant content is placed at the top of the prompt word.

[0153] For example: The background knowledge of the prompt words after sorting is as follows:

[0154] "OUT_LOS alarm - main reason - received optical power is too low"

[0155] Chunk3

[0156] Chunk2

[0157] Chunk1.

[0158] It should be noted that the multimodal retrieval in S8-S9 combines vector retrieval, keyword retrieval and knowledge graph retrieval, and can obtain the most relevant information to the user query from multiple dimensions. Vector retrieval represents text as a high-dimensional vector and uses the similarity in the vector space to retrieve relevant content. This method performs well when dealing with semantic similarity and fuzzy queries. Keyword retrieval relies on traditional inverted index technology, which can quickly locate documents containing specific keywords and is suitable for precise matching query requirements. Knowledge graph retrieval uses structured knowledge graphs to provide deeper semantic understanding and reasoning capabilities through the connection of entities and relationships, which is particularly suitable for complex queries and scenarios that require background knowledge. This avoids the current single retrieval method being unable to fully cover the needs of user queries; although the traditional keyword retrieval method is simple and intuitive, it is difficult to handle complex queries or understand semantics.

[0159] After obtaining the search results, the fusion algorithm merges the search results and generates a preliminary comprehensive result set through weight distribution, voting mechanism or other statistical methods. Then, the re-ranking algorithm re-ranks the comprehensive result set to ensure the relevance and accuracy of the final result.

[0160] The above multimodal retrieval method can not only adapt to different types of query requirements, but also significantly improve the overall performance of the question-answering system. For simple keyword queries, the system can quickly return accurate matching results; for semantically complex or ambiguous queries, vector retrieval and knowledge graph retrieval can provide more valuable answers. Through fusion and re-ranking, the system can comprehensively consider multiple factors and provide the most relevant and accurate results, thereby improving user experience and satisfaction.

[0161] S10: The optimization result is combined with the prompt word to generate a query answer.

[0162] Exemplarily, based on the background knowledge obtained above and the preset prompt word template, a complete prompt word is constructed, and the large language model performs question and answer based on the background knowledge, for example:

[0163] Tips:

[0164] Please answer users' questions based only on the background knowledge provided.

[0165] User query command: What is the reason why the downstream site OAU board reports the OUT_LOS alarm?

[0166] Background knowledge: "OUT_LOS alarm - main reason - received optical power is too low", Chunk3, Chunk2, Chunk1;

[0167] LLM's answer: The main reason why the downstream site OAU board reports the OUT_LOS alarm is that the received optical power is too low, which is usually caused by problems with the fiber jumper end face (Chunk3 content). You can check the fiber jumper between the ODF rack and the RPC (Chunk2 content), as well as product X (Chunk1 content).

[0168] It should be noted that the above solution combines knowledge graph with retrieval enhancement generation technology and builds a backbone optical communication system equipment document question-and-answer system. When answering user questions, it can not only use the structured information in the knowledge graph to quickly answer, but also provide smarter, more accurate and more comprehensive information processing and generation capabilities through retrieval enhancement generation (RAG) technology, helping staff to quickly and accurately obtain the required information, improve work efficiency, and reduce the time to find and analyze documents. At the same time, this combination can also continuously learn and update the knowledge base, maintain the efficiency and cutting-edge nature of the system, and adapt to changing technologies and needs.

[0169] The above is a schematic scheme of a backbone optical communication equipment document retrieval optimization method of this embodiment. It should be noted that the technical scheme of the backbone optical communication equipment document retrieval optimization system and the technical scheme of the backbone optical communication equipment document retrieval optimization method described above belong to the same concept. For details not described in detail in the technical scheme of the backbone optical communication equipment document retrieval optimization system in this embodiment, please refer to the description of the technical scheme of the backbone optical communication equipment document retrieval optimization method described above.

[0170] The backbone optical communication equipment document retrieval optimization system in this embodiment includes:

[0171] The first acquisition module is used to acquire the named entity types and relationship types required for the knowledge graph based on the device document;

[0172] The data structure building module is used to build and define data structures based on named entity types and relationship types;

[0173] A first extraction module, configured to extract named entities and relation words in any device document based on the data structure, set prompts, and extract them through a large language model to generate at least two triples of data;

[0174] The first deduplication and fusion module is used to deduplicate and fuse all generated triplet data to obtain optimized multi-tuple data;

[0175] Graph construction module, used to store multi-tuple data into a graph database to form a complete knowledge graph;

[0176] A text vector database construction module, used for constructing a text vector database for device documents;

[0177] The second extraction module is used to extract named entities and keywords in the query command in response to the user query command, obtain command entities and command keywords, and convert the query command into an embedding vector;

[0178] The retrieval and matching module is used to perform hybrid retrieval and similarity matching on command entities, command keywords and embedded vectors through the text vector database and knowledge graph to obtain corresponding matching results respectively;

[0179] The second deduplication and fusion module is used to deduplicate and fuse the matching results, and rearrange them with the query command to obtain an optimized result;

[0180] The reply module is used to generate query answers by combining the optimized results with the prompt words.

[0181] This embodiment further provides a computing device, which is applicable to the case of optimizing document retrieval of backbone optical communication equipment, including:

[0182] Memory and processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the document retrieval optimization method for backbone optical communication equipment proposed in the above embodiment.

[0183] This embodiment further provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for optimizing document retrieval of backbone optical communication equipment proposed in the above embodiment is implemented.

[0184] The storage medium proposed in this embodiment and the method for realizing document retrieval optimization of backbone optical communication equipment proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0185] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FLASH), a hard disk or an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0186] Example 2, referring to Table 1, based on the previous example, this example provides an application comparison case of a document retrieval optimization method for backbone optical communication equipment to illustrate the feasibility and beneficial effects of our solution.

[0187] Based on 10 randomly provided device documents, 10 questions were generated using a large language model, and 10 questions were manually designed, totaling 20 questions per group. Five people were assigned to participate in the experiment.

[0188] The process of traditional method:

[0189] a) Reading questions;

[0190] b) Use the device document reading software to search based on the keywords in the question;

[0191] c) Determine whether the search results are correct one by one. If you think the retrieved documents can answer the question, organize the answer based on the document content and submit it;

[0192] d) The time required for manual record retrieval and question answering.

[0193] The process of the method of the present invention:

[0194] a) Users directly input questions into the question-answering system;

[0195] b) The question-answering system searches and answers according to the design;

[0196] c) During this period, the system automatically counts parameters such as retrieval time and answer time.

[0197] Comparison: The results of the method of the present invention and the traditional method were submitted to experts for manual evaluation. The evaluation criteria included whether the questions and answers were accurate, whether the questions and answers were comprehensive, whether there were inaccuracies in the answers, etc. At the same time, the experimenters needed to evaluate the difficulty and satisfaction of the methods used. The specific comparison results are shown in Table 1.

[0198] Table 1 Comparison of the two solutions

[0199] ,

[0200] It can be seen from Table 1 that the solution of the present invention has greatly improved the accuracy and the time of retrieval and answering.

[0201] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A document retrieval optimization method for backbone optical communication equipment, characterized in that: include: Obtain the named entity types and relationship types required for the knowledge graph based on device documents; Build and define data structures based on named entity types and relationship types; Based on the data structure, a prompt is set and named entities and relation words in any device document are extracted through a large language model to generate at least two triples of data; The triple data includes two named entities and a relation word, and the two named entities are combined and connected by the relation word; All generated triplet data are deduplicated and merged to obtain optimized multi-tuple data, including: Setting a deduplication rule, wherein for the same named entity extracted from different child device documents, if the different child device documents belong to the same parent document, deduplication is performed; The deduplicated results are merged to obtain multi-tuple data, wherein the multi-tuple data includes at least one triple data; Storing the multi-tuple data in a graph database to form a complete knowledge graph; Build a text vector database for device documentation; Specifically, the steps for building a text vector database include: Perform equipment document preprocessing; The preprocessed data is segmented into text blocks using a segmentation tool based on the token capacity of the model while ensuring semantic coherence to obtain text blocks; Convert text blocks into vectors through the Embedding model; Store vector database and build indexes; In response to a user query command, named entities and keywords in the query command are extracted to obtain command entities and command keywords, and the query command is converted into an embedding vector; Through the text vector database and knowledge graph, the command entities, command keywords and embedded vectors are hybrid searched and similarity matched to obtain the corresponding matching results respectively; De-duplication and fusion of the matching results are performed, and they are rearranged with the query command to obtain an optimized result; The optimization result is combined with the prompt word to generate a query answer.

2. The backbone optical communication equipment document retrieval optimization method according to claim 1, characterized in that: All generated triple data are deduplicated and merged to obtain optimized multi-tuple data. Duplicate named entities and relation words can also be eliminated through hash functions or similarity-based deduplication algorithms. The triplet data from different device document sources are merged by a merging algorithm based on similarity calculation to obtain multi-tuple data, wherein the multi-tuple data includes at least one triplet data.

3. The document retrieval optimization method for backbone optical communication equipment according to claim 2, characterized in that: The command entity, command keyword and embedded vector are subjected to mixed retrieval and similarity matching through the text vector database and the knowledge graph to obtain corresponding matching results respectively, including: matching the embedded vector through the text vector database to obtain at least one text block with semantics similar to the query command, specifically: Connect vector data through index structure; Define a distance metric to calculate the first similarity between the query command vector and the text vector; Based on the first similarity, an approximate algorithm is used to search for vectors similar to the query command vector and sort them; At least one text block is obtained based on the first similarity search result within the preset sorting range.

4. The document retrieval optimization method for backbone optical communication equipment according to claim 3, characterized in that: The command entity, the command keyword and the embedded vector are subjected to hybrid retrieval and similarity matching through the text vector database and the knowledge graph to obtain corresponding matching results respectively, and further comprising: performing keyword matching on the command keyword through the text vector database and based on the text matching algorithm to obtain at least one text block with semantics similar to the query command, specifically: The text is retrieved based on the BM25 algorithm, and the frequency of occurrence of each command keyword in each text and the length of the text are counted through the inverted index; Calculate the similarity score between the command keyword and the text based on the frequency of occurrence and the length of the text; The text blocks are sorted according to the similarity scores, and the text blocks with high scores are recalled first.

5. The backbone optical communication equipment document retrieval optimization method according to claim 4, characterized in that: Through the text vector database and knowledge graph, command entities, command keywords and embedded vectors are hybrid searched and similarity matched to obtain corresponding matching results respectively, including: Obtaining relevant subgraphs in the knowledge graph according to the command entity and the corresponding relational words; Calculate the second similarity between the obtained subgraphs and the query command, and sort them; Based on the second similarity within the preset sorting range, a corresponding knowledge subgraph is obtained.

6. A system using the backbone optical communication equipment document retrieval optimization method as claimed in claim 1, characterized in that: include: The first acquisition module is used to acquire the named entity types and relationship types required for the knowledge graph based on the device document; The data structure building module is used to build and define data structures based on named entity types and relationship types; A first extraction module, configured to set a prompt based on the data structure and extract named entities and relation words in any device document through a large language model to generate at least two triples of data; The first deduplication and fusion module is used to deduplicate and fuse all generated triplet data to obtain optimized multi-tuple data; A graph construction module, used for storing the multi-tuple data into a graph database to form a complete knowledge graph; A text vector database construction module, used for constructing a text vector database for device documents; The second extraction module is used to extract named entities and keywords in the query command in response to the user query command, obtain command entities and command keywords, and convert the query command into an embedding vector; The retrieval and matching module is used to perform hybrid retrieval and similarity matching on command entities, command keywords and embedded vectors through the text vector database and knowledge graph to obtain corresponding matching results respectively; A second deduplication and fusion module is used to deduplicate and fuse the matching results, and rearrange them with the query command to obtain an optimized result; The reply module is used to generate a query answer by combining the optimization result with the prompt word.

7. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the backbone optical communication equipment document retrieval optimization method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, can implement the steps of the backbone optical communication equipment document retrieval optimization method described in any one of claims 1 to 5.

Images