Knowledge graph and large language model-based retrieval method and electronic device

Abstract

The embodiment of the application provides a retrieval method based on a knowledge graph and a large language model and electronic equipment, and relates to the technical field of data retrieval. The method comprises the following steps: receiving a technical question of a user, retrieving a target node most relevant to the technical question in a constructed standard knowledge graph; determining a first evidence node and a second evidence node associated with the target node in the standard knowledge graph; the first evidence node and the target node belong to the same standard node, and the second evidence node and the target node belong to different standard nodes; each second evidence node has a cross-standard reference with the target node or the first evidence node; constructing evidence information comprising the first evidence node and the standard node to which the first evidence node belongs, the second evidence node and the standard node to which the second evidence node belongs; inputting the technical question and the evidence information into a large language model for reasoning to obtain a reply result of the technical question output by the large language model.

Description

Technical Field

[0001] This invention relates to the field of data retrieval technology, specifically to a retrieval method and electronic device based on knowledge graphs and large language models. Background Technology

[0002] In scenarios such as security governance, operation and maintenance management, compliance verification, fault diagnosis and technical training, when relevant personnel conduct standard searches around a specific technical issue, they often need to repeatedly consult multiple standard documents, multiple clauses and their cited clauses. The search is time-consuming, the manual understanding cost is high, key evidence is easily missed, and it is difficult to quickly form actionable governance recommendations.

[0003] Standard queries still primarily rely on manual directory searches, keyword retrieval, or full-text semantic matching. Manual searches are inefficient, keyword searches are easily affected by differences in wording, and while general semantic searches can return similar terms, they typically treat standard text as flat text fragments.

[0004] Taking the energy storage system field as an example, in tasks such as cell short-circuit testing, module withstand voltage testing, and decommissioning determination, maintenance personnel, testing personnel, or safety management personnel often do not simply search for a single clause, but rather hope that the system can answer the following questions: Which standard clauses should correspond to a certain type of anomaly; In what order should a certain test be performed; What are the requirements of other clauses or other standards cited in a certain clause; and In the case of multiple clauses jointly applying, what governance recommendations should be given in the end? However, existing search methods usually only return the "most similar" clauses, making it difficult to form a complete and actionable standard interpretation result. Summary of the Invention

[0005] The purpose of this invention is to provide a retrieval method and electronic device based on knowledge graphs and large language models, which can significantly reduce the workload of manually tracing citation relationships one by one, improve the efficiency and accuracy of standard retrieval and technical analysis around specific technical issues, and enhance the interpretability and traceability of the results.

[0006] To achieve the above objectives, this invention provides a retrieval method based on knowledge graphs and large language models, comprising: The system receives technical questions from users and retrieves the target node most relevant to the technical question from the constructed standard knowledge graph. The standard knowledge graph includes: several standard nodes, clause requirement nodes under each standard node, and the reference relationships between the clause requirement nodes. In a standard knowledge graph, a first evidence node and a second evidence node are identified that are associated with the target node; the first evidence node and the target node belong to the same standard node, and the second evidence node and the target node belong to different standard nodes, and each second evidence node has a cross-standard reference to the target node or the first evidence node; Construct evidence information including the first evidence node and the standard node to which the first evidence node belongs, the second evidence node and the standard node to which the second evidence node belongs; The technical problem and the evidence information are input into a large language model for reasoning, and the response to the technical problem is obtained by the output of the large language model. The response includes the technical conclusion, the source of evidence, and the association path.

[0007] The present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the retrieval method based on knowledge graph and large language model as described above.

[0008] The present invention also provides a computer-readable storage medium, which is a non-volatile or non-transient storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the retrieval method based on knowledge graph and large language model as described above.

[0009] In one embodiment, determining the first evidence node and the second evidence node associated with the target node in a standard knowledge graph includes: Based on the first standard node to which the target node belongs, a search step is performed with the target node as the search starting point. The search step is as follows: based on the standard node to which the search starting point belongs, N first evidence nodes associated with the search starting point are searched in the standard knowledge graph, where N≥1. After each execution of the search step, if there is a specified node that is referenced across standards among the N first evidence nodes obtained in this search, then based on the standard node referenced across standards by the specified node, determine M second evidence nodes associated with the specified node in the standard knowledge graph, where M≥1; If the preset stopping condition is not met, the search steps will be executed again, taking the N first evidence nodes and the M second evidence nodes obtained in this search as the search starting points respectively. If the preset stopping condition is met, all the first evidence nodes and the second evidence nodes currently acquired will be retained; The preset stopping conditions include at least one of the following: the depth between the target node and the first evidence node obtained in this search reaches a preset maximum expansion depth; the sum of the number of the determined first evidence node and the second evidence node reaches a preset upper limit; and no new first evidence node or second evidence node is obtained in this search.

[0010] In one embodiment, based on the standard node to which the search starting point belongs, N first evidence nodes associated with the search starting point are searched in the standard knowledge graph, including: If the search starting point is a clause requirement node, then one or more target clause requirement nodes referenced by the search node are obtained by backtracking along the clause requirement dependency edges in the standard knowledge graph; all the target clause requirement nodes constitute the N first evidence nodes.

[0011] In one embodiment, based on the standard nodes referenced across standards by the specified node, M second evidence nodes associated with the specified node are determined within the standard knowledge graph, including: Based on at least a portion of the text content of the target node that is related to the standard node referenced across standards, a local retrieval constraint is generated; Within the standard knowledge graph, determine a set of candidate nodes for the standard nodes referenced across standards by the specified node. The set of candidate nodes includes all clause requirement nodes belonging to the standard nodes referenced across standards by the specified node. The local retrieval constraint is matched with all nodes in the candidate node set, and the top M nodes with the highest matching degree with the local retrieval constraint are selected from the candidate node set as the second evidence nodes associated with the specified node, where M≥1.

[0012] In one embodiment, retrieving the target node most relevant to the technical problem from the constructed standard knowledge graph includes: Based on the similarity between the technical problem and the text content of each of the clause requirement nodes, the clause requirement node most relevant to the technical problem is selected as the target node.

[0013] In one embodiment, based on the similarity between the technical problem and the text content of each of the clause requirement nodes, the clause requirement node most relevant to the technical problem is obtained as the target node, including: Select the text content that has the highest similarity to the technical problem as the target candidate text; If the similarity between the target candidate text and the technical problem is greater than or equal to a preset similarity threshold, then the clause requirement node corresponding to the target candidate text is taken as the target node.

[0014] In one embodiment, the similarity between the technical problem and the text content of the clause requirement node is obtained as follows: Obtain the query vector encoded by the technical problem, and the encoded vector of the text content of the clause requirement node; The cosine similarity between the query vector of the technical problem and the encoded vector of the text content of the clause requirement node is calculated as the similarity between the technical problem and the text content of the clause requirement node.

[0015] In one embodiment, the standard knowledge graph construction process is as follows: Read standard-level metadata and clause requirement data from the full-text data of the standard, and obtain the attribution relationship between standard-level metadata and clause requirements, the subordinate reference relationship between clause requirement data, and the cross-standard reference relationship between clause requirement data and standard-level metadata; In the standard knowledge graph, the standard-level metadata is used as the standard node, and the clause requirement data is used as the clause requirement node. Based on the aforementioned attribution relationship, construct the relationship edges between the standard node and the clause requirement node; Based on the aforementioned subordinate reference relationship, construct requirement dependency edges between the clause requirement nodes under the same standard node; Based on the cross-standard reference relationship, cross-standard reference edges are constructed between the standard node, the clause requirement node, and the cross-standard reference standard node. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the construction process of the standard knowledge graph in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the retrieval method based on knowledge graph and large language model in the first embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the process of obtaining the first evidence node and the second evidence node in the first embodiment of the present invention; Figure 4 This is a schematic diagram of the retrieval process in the first embodiment of the present invention, which addresses the technical problem of "how should the test connection method, initialization conditions, and judgment steps be executed in a certain technical solution to address the problem related to short circuits in battery cells?" Figure 5 This is a schematic diagram of the retrieval process in the first embodiment of the present invention, which addresses the technical problem of "which relevant standards should be prioritized for the withstand voltage test and retirement determination of battery modules and batteries, and how to form a complete technical implementation chain?" Detailed Implementation

[0017] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings to provide a clearer understanding of the purpose, features, and advantages of the present invention. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of the present invention, but are merely illustrative of the essential spirit of the technical solution of the present invention.

[0018] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known apparatuses, structures, and techniques associated with this application may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0019] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.

[0020] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0021] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to include the meaning of “or / and” unless otherwise expressly stated herein.

[0022] In the following description, in order to clearly demonstrate the structure and working method of the present invention, a number of directional terms will be used. However, terms such as "front", "back", "left", "right", "outside", "inside", "outward", "inward", "up", and "down" should be understood as convenient terms and not as limiting terms.

[0023] Existing data retrieval technologies mainly fall into two categories: The first type utilizes large language models to perform structured analysis of technical texts, constructing a correspondence between features and effects, and combining vector retrieval to improve the matching effect of complex queries. This method demonstrates the application value of large language models in the analysis and retrieval of complex technical texts, but its processing objects are mainly standard texts, focusing on feature extraction, effect analysis, and standard evaluation, without modeling the citation dependencies between standard clauses.

[0024] The second type involves setting up multiple intelligent agents, such as search, parsing, judgment, reporting, and scheduling, to collaboratively complete complex retrieval tasks, thereby enhancing the system's understanding, reasoning, and generation capabilities. This method embodies the idea of ​​multi-agent collaborative processing of complex text retrieval tasks, but it is mainly geared towards standard retrieval scenarios and does not establish a dedicated clause-level structure for common issues such as intra-standard and cross-standard references in standard documents.

[0025] In existing methods, standards analysts typically need to first identify several candidate standards through keywords or semantic similarity, and then read through each requirement document, specification, and figure description to determine which requirement defines the target technical feature, which implementation method supports it, and whether it is necessary to consider other cited standard documents for understanding. This process is inefficient for the following reasons.

[0026] It is evident that most existing technologies treat standard documents as independent text units, ignoring the inherent hierarchical dependencies within the standards. In reality, there is a clear logical support relationship between independent clause requirements, dependent clause requirements, and specification embodiments. If only a superficially similar section of text is returned without further tracing its dependent prerequisite clause requirements or implementation methods, the user will still not obtain the complete technical solution.

[0027] Secondly, cross-standard citations are often not explicit citations "precise to specific clause requirements." In many cases, the standard text only mentions another standard, a certain existing technical solution, or a certain existing process, without directly pointing to specific paragraphs or technical features in the other document. Even if existing retrieval systems identify the existence of a citation relationship, it is difficult to automatically complete the chain process of "citation intent analysis—target standard local location—relevant feature retrieval," so users still need to manually track it.

[0028] Furthermore, if multiple candidate standards are directly fed into a large language model, although the model can generate relatively fluent responses, the lengthy and unstructured input often leads to problems such as excessive contextual noise, blurred evidence boundaries, and a lack of clear sources in the responses. For tasks like standards analysis that emphasize evidence and tracing, simply relying on a large model to generate responses freely cannot meet reliability requirements.

[0029] Therefore, while existing technologies offer valuable insights into large language model-assisted retrieval, complex text analysis, and multi-agent collaboration, they have yet to address the challenges of complex clause hierarchies, dense citation relationships, semantic inconsistencies in individual clauses, and difficulties in cross-standard location within standard analysis scenarios. Particularly when performing standard queries related to specific engineering problems, existing methods struggle to automatically recover clause dependency chains and generate governance recommendations with clear evidence. Based on this, this application proposes a retrieval method based on knowledge graphs and large language models to improve the completeness, accuracy, and executability of standard queries and interpretations.

[0030] The first embodiment of this invention relates to a retrieval method based on knowledge graphs and large language models, applicable to electronic devices such as servers, workstations, or cloud computing platforms, as well as common computer devices such as desktop computers and laptops. The method of this embodiment can be used in multiple fields such as energy storage systems, fire safety, industrial control, power equipment, rail transportation, and special equipment.

[0031] Standard knowledge graphs are pre-built in electronic devices, such as Figure 1 As shown, the standard knowledge graph construction process is as follows: First, obtain the full-text data of the standard, such as from a local standard database, enterprise standard knowledge base, public standard data interface, or document storage system. When the obtained raw data is a PDF, scanned copy, or a document with mixed text and images, further page layout analysis, character recognition, numbering recognition, and paragraph segmentation can be performed to obtain standardized standard text objects, forming the full-text data of the standard. This standard text object includes at least: standard number, name, abstract, requirements, specification paragraphs, figure descriptions, referenced standard identifiers, and their locations.

[0032] The acquired full-text standard data has a clear hierarchical structure and citation relationships. A single full-text standard contains standard-level metadata and several clause requirements. There are subordinate citation relationships among the clause requirements, and there may also be background citations, prior art citations, and family citation relationships between different standards.

[0033] Therefore, standard-level metadata and clause requirement data can be read from the full-text data of the standard, and the attribution relationship between standard-level metadata and clause requirements, the subordinate reference relationship between clause requirement data, and the cross-standard reference relationship between clause requirement data and standard-level metadata can be obtained. In the standard knowledge graph, the standard-level metadata is used as the standard node, and the clause requirement data is used as the clause requirement node; that is, all standard-level metadata and all clause requirement data constitute the corresponding nodes in the standard knowledge graph.

[0034] The process of reconstructing the connecting edges between adjacent nodes in the standard knowledge graph is as follows: Based on the aforementioned attribution relationship, construct the relationship edges between the standard node and the clause requirement node; Based on the aforementioned subordinate reference relationship, construct requirement dependency edges between the clause requirement nodes under the same standard node; Based on the cross-standard reference relationship, cross-standard reference edges are constructed between the standard node, the clause requirement node, and the cross-standard reference standard node.

[0035] The standard knowledge graph constructed based on the above process for: ; in, For standard knowledge graphs The set of all nodes in the set, For standard knowledge graphs The set of edges formed by connecting all adjacent nodes in a given region.

[0036] Node set Specifically: ; in, For a standard set of nodes, The set of nodes required by the terms.

[0037] For any standard node in the standard node set Its corresponding standard number is denoted as .

[0038] For any clause requirement node in the set of clause requirement nodes The corresponding clause requirement number is recorded as follows: The corresponding text content is denoted as .

[0039] edge set It includes at least the following types of relationships: ; in: (1) An inclusion edge is used to indicate that a certain clause requires a node to belong to a certain standard node; (2) A clause requires a dependency edge, used to indicate that a subordinate clause requires a reference to a preceding clause. (3) A cross-standard reference edge is used to indicate that a standard node or a clause requirement node references another standard node or a clause requirement node under another standard node.

[0040] More specifically: like This indicates that the terms require nodes Belongs to standard nodes ; like This indicates that the terms require nodes Subordinate or referenced terms require nodes ; like ,in If it is a standard node or a clause requirement node, then it indicates that the node... Another standard node was referenced. .

[0041] For example, Figure 1 In the standard node GB / T XXX, the subordinate clause requirement nodes include: Clause 1.1.1, Clause 1.1.2, Clause 1.1.3, Clause 2.1.1, Clause 2.1.2, Clause 2.2, etc. Clause 2.1.1 is a prerequisite clause requirement node for Clause 1.1.3, and Clause 2.2 is a prerequisite clause requirement node for Clause 2.1.2.

[0042] As described above, a standard knowledge graph containing standard nodes, clause requirement nodes, and their interrelationships is automatically constructed in the standard library, so that the dependency structure within and between standards can be explicitly processed by the machine.

[0043] Furthermore, the nodes of each clause in the standard knowledge graph will be vectorized to support subsequent semantic retrieval.

[0044] For example, text encoding models used for vector transformation. for: ; Then any clause requires the vector representation of node u to be: ; in, This represents the text content corresponding to node u; The vectors of the above nodes (standard nodes and clause requirement nodes) and the graph relationships in the standard knowledge graph are stored together in the vector database and the graph database to form the basis for subsequent retrieval.

[0045] like Figure 2 The diagram shown is a flowchart illustrating the retrieval method based on knowledge graphs and large language models in this embodiment.

[0046] Step 101: Receive the user's technical question and retrieve the target node most relevant to the technical question from the constructed standard knowledge graph; wherein the standard knowledge graph includes: several standard nodes, clause requirement nodes under each standard node, and the reference relationship between the clause requirement nodes.

[0047] Specifically, users can input technical questions through their user terminals. The user terminal will then address this technical issue. Transmitted to electronic devices; technical issues encountered by electronic devices receiving user data. Then, semantic matching is performed to retrieve the target node most relevant to the technical problem from the standard knowledge graph; this target node is the clause requirement node.

[0048] For example, based on the similarity between the technical problem and the text content of each clause requirement node, the clause requirement node most relevant to the technical problem is obtained as the target node; Details: Obtain the query vector encoded from the technical problem, for example, using a text encoding model. Technical issues Encode the query vector of the technical problem to obtain the query vector. .

[0049] The encoding vector of the candidate text is as described above. .

[0050] Using the text content of each clause's requirement node as candidate text, the cosine similarity between the query vector of the technical issue and the encoding vector of the candidate text is calculated as the similarity between the technical issue and the candidate text. ; After obtaining the similarity between the technical problem and each candidate text, the target candidate text with the highest similarity to the technical problem is selected from all candidate texts. The node corresponding to the target candidate text is the most relevant node. ; ; At this point, we can directly use the most relevant node. As the target node; in some embodiments, a similarity threshold can be preset. If the similarity between the target candidate text and the technical problem is greater than or equal to a preset similarity threshold, that is... Then, the most relevant node is the clause requirement node corresponding to the target candidate text. As the target node; If the similarity between the target candidate text and the technical problem is less than a preset similarity threshold, that is... If the search fails, a message indicating "not enough relevant evidence nodes were found" will be returned; at this point, you can choose to switch to a more lenient candidate search mode.

[0051] Step 102: In the standard knowledge graph, determine the first evidence node and the second evidence node associated with the target node; the first evidence node and the target node belong to the same standard node, the second evidence node and the target node belong to different standard nodes, and each second evidence node has a cross-standard reference with the target node or the first evidence node.

[0052] Step 103: Construct evidence information including the first evidence node and the standard node to which the first evidence node belongs, the second evidence node and the standard node to which the second evidence node belongs.

[0053] Specifically, regarding the technical issues raised by users, the most relevant clause requirement nodes have been located semantically as target nodes. Once the technical issues are identified... After identifying the most relevant target node, use that target node as the starting node for graph expansion; then automatically expand the dependency context along the graph relationships indicated by the standard knowledge graph; furthermore, for cross-standard reference scenarios, automatically parse the reference intent of the current node, perform secondary local matching within the referenced standard, and recover the truly relevant target technical features; this can be summarized as recursively executing the following two types of expansion actions: The first type of extension action is a standard internal extension, which adds the dependent clause requirement node, i.e., the first evidence node; The second type of extension action is cross-standard extension, which introduces relevant evidence nodes, namely second evidence nodes, by performing local intent matching on the referenced standard nodes across standards.

[0054] After each execution of the first type of expansion action, a new first evidence node can be introduced, which can trigger a new round of graph expansion, and thus the above process can be executed recursively. This embodiment also includes a preset stop condition to prevent graph expansion from getting out of control.

[0055] For example, please refer to Figure 3 The target node is clause 1.1.3, and the detailed expansion process is as follows: Using the target node as the starting point, a search step is performed, which is as follows: based on the standard node to which the starting point belongs, N first evidence nodes associated with the starting point are searched in the standard knowledge graph, where N≥1.

[0056] Wherein, the search starting point is the clause requirement node, then the target clause requirement node referenced by the search node is obtained by backtracking along the clause requirement dependency edge in the standard knowledge graph; all the target clause requirement nodes constitute the N first evidence nodes.

[0057] After each search step, the search node and its corresponding N first evidence nodes form a continuous technical path within the same standard from the requirements of the preconditions and the requirements of the subordinate conditions, reorganizing the technical features scattered in different conditions; avoiding retrieval for only a single similar fragment, but automatically replying with the skeleton of the standard's internal technical solution related to the user's question.

[0058] After each execution of the search step, if among the N first evidence nodes obtained in this search, there is a specified node that is referenced across standards, then based on the standard node referenced across standards by the specified node, M second evidence nodes associated with the specified node are determined within the standard knowledge graph. Specifically: Based on at least a portion of the text content of the target node that is related to the standard node referenced across standards, local retrieval constraints are generated; that is, semantic features related to the standard node referenced by the specified node are obtained from the text content of the specified node, and these semantic features are used as local retrieval constraints for the standard referenced by the specified node. There is a cross-standard reference edge ,in For the specified node The other standard node referenced is called the referenced standard node. In this case, the referenced standard node is not directly imported. Instead of reading the full text, it first depends on the specified node. In and the referenced standard node Based on the relevant contextual reference semantic features, generate references for the referenced standard nodes. Local search constraints, denoted as This local search constraint It can be generated by a large language model under restricted prompts, which is a response to "specified nodes". The semantic description of "what technical content do you want to borrow from the referenced standard".

[0059] Within the standard knowledge graph, determine a set of candidate nodes for the standard nodes referenced across standards by the specified node. The set of candidate nodes includes all clause requirement nodes belonging to the standard nodes referenced across standards by the specified node. The local retrieval constraint is matched with all nodes in the candidate node set, and the top M nodes with the highest matching degree with the local retrieval constraint are selected from the candidate node set as the M second evidence nodes associated with the specified node, where M≥1.

[0060] For example, the referenced standard node The internal set of candidate nodes is: .

[0061] Then in the referenced standard node The first M nodes obtained after performing local matching on the internal candidate node set are: ; in, This indicates selecting the top scorer. Each node.

[0062] In this way, it is possible to reference standard nodes. The system automatically retrieves the clauses and requirements most likely to be related to the current technical issues, eliminating the need for manual page-by-page browsing.

[0063] exist Figure 3 In the search, the target node is clause 1.1.3. When the target node is the starting point of the search, the starting point of the search is the clause requirement node. By backtracking along the clause requirement dependency edge in the standard knowledge graph, the preceding clause requirement node referenced by the search node is clause 2.1.1; clause 2.1.1 is the first evidence node.

[0064] Clause 2.1.1 contains cross-standard references, with the referenced standard node being GB / T YYYY. The clauses under standard node GB / T YYYY include: Clause 1.1.1, Clause 1.1.2, Clause 4.1, and Clause 4.2; thereby, the second evidence node can be further determined.

[0065] If none of the N first evidence nodes obtained in this search are cross-standard references to the specified node, then directly determine whether the preset stopping condition is met; If the preset stopping condition is not met, the N first evidence nodes and M second evidence nodes obtained in this search will be used as search starting points, and the search steps will be executed again. That is, each first evidence node and each second evidence node will be used as a search starting point, and the above process will be repeated to obtain the first evidence node and the second evidence node under each search starting point. Then, the first evidence node and the second evidence node under each search starting point will be used as search nodes, and the above process will be repeated until the preset stopping condition is met.

[0066] If the preset stopping condition is met, all the first evidence nodes and the second evidence nodes currently acquired will be retained; The preset stopping condition includes at least one of the following: the depth between the target node and the first evidence node obtained in this search reaches a preset maximum expansion depth. The sum of the number of the first and second evidence nodes identified reaches the preset upper limit. No new first or second evidence nodes were found in this search.

[0067] Based on the above process, after the preset stopping condition is met, no new evidence nodes are searched. The target node, all first evidence nodes, and all second evidence nodes can be combined to obtain the evidence node set. Each node in this set of evidence nodes can be called an evidence node.

[0068] Therefore, by employing a combined strategy of citation intent analysis and local matching of the cited standard for cross-standard citation relationships, more suitable cross-standard evidence nodes can be found.

[0069] Subsequently, an evidence package is constructed based on the set of evidence nodes, denoted as evidence information. The evidence information specifically includes: the standards involved, the evidence nodes under each standard, and the reference relationships between the evidence nodes. Each evidence node also records reference relationship information, which indicates whether the evidence node was obtained through "initial hit", "dependent extension", or "cross-standard reference extension".

[0070] Step 104: Input the technical problem and the evidence information into the large language model for reasoning, and obtain the answer to the technical problem output by the large language model. The answer includes the technical conclusion, the source of evidence, and the association path.

[0071] Specifically, the user-inputted technical questions, evidence information, graph relationships between evidence nodes (i.e., connections with subordinate or attributive relationships), and optional domain constraint information (such as energy storage system scenario labels and search target type labels) are input into the Large Language Model (LLM). As a result, the Large Language Model can perform constrained reasoning under constrained conditions and output a response that includes technical conclusions, evidence sources, and related paths. This response can be displayed on the user terminal for the user to view.

[0072] Large language models perform evidence fusion and technical reasoning under constraints, specifically including the following three types of tasks: First, analyze the logical dependencies between the various evidence nodes to obtain the complete clauses addressing the technical issues and restore the complete technical solution; Second, determine which standard features are truly relevant based on user questions, and summarize their technical implementation logic; Third, generate analytical reports, proposed solutions, lists of relevant standards, or technical comparison conclusions with supporting evidence.

[0073] In this way, the large language model does not directly answer based on free memory, but rather it extracts the semantics of the technical question and then performs a fusion analysis based on the evidence information obtained from graph retrieval to arrive at the final answer, thereby improving reliability and interpretability.

[0074] In one example, please refer to Figure 4 Taking a standard analysis task in the field of energy storage systems concerning cell short-circuit testing or protection strategies as an example, the technical question raised by the user is: "For issues related to cell short circuits, how should the test connection method, initialization conditions, and judgment steps be executed in a certain technical solution?"

[0075] First, the node most similar to the technical problem is retrieved from the standard knowledge graph, i.e., the target node. From this, a core clause requirement node is located within the target standard node, namely target clause 6.7.1.4.1. This target clause 6.7.1.4.1 directly describes the main process of short-circuit testing. This target clause 6.7.1.4.1 also relies on the connection circuit limitations and related initialization parameters and execution order descriptions in the preceding clause requirement node to form a semantically consistent complete solution. Therefore, within the target standard to which target clause 6.7.1.4.1 belongs, the solution can be derived by following the internal standards... Laibian automatically backtracks to obtain the preceding clause requirement nodes, including: reference clauses 6.2.4.1.1 and 6.2.2.1, which are both first evidence nodes. There is no cross-standard reference between the target node and the first evidence nodes. Therefore, the target clause 6.7.1.4.1, reference clause 6.2.4.1.1, and reference clause 6.2.2.1 can be directly used as evidence nodes. The evidence nodes and the standard nodes to which the evidence nodes belong constitute evidence information. This evidence information is no longer the text of a single clause requirement node, but a complete evidence package including the target clause and its preceding clauses.

[0076] The technical problem and the evidence information are input into a large language model. Under the constraints of the evidence package, the large language model fuses these elements to generate a summary of steps for an "executable clause" addressing the user's technical problem. This summary includes the test object, connection order, sampling conditions, threshold constraints, and result recording method. Therefore, this embodiment does not return the standard text most similar to the technical problem, but rather automatically recovers the complete technical context that the standard text relies on within the same standard. This integrates technical information originally scattered across multiple clause requirements into a directly readable and usable analysis result.

[0077] In one example, please refer to Figure 5Taking a technical issue concerning the withstand voltage test, anomaly determination, or retirement determination of battery modules in the field of energy storage systems as an example, the technical question raised by the user is: "Regarding the withstand voltage test and retirement determination of battery modules and batteries, which relevant standards should be prioritized for research, and how to form a complete technical implementation chain?"

[0078] First, retrieve the node most similar to the technical problem from the standard knowledge graph, i.e., the target node; then locate the target standard node GB / T 43540 that is most relevant to "pressure withstand test procedure" in the standard knowledge graph, and hit the target clause requirement node describing the test procedure in the target standard, i.e., the target node, which is target clause 6.2.5.

[0079] Within the target standard GB / T 43540 to which target clause 6.2.5 belongs, the preceding clause requirement node is automatically traced back along the internal dependency edge of the standard, which is reference clause 5.2.5, and is the first evidence node.

[0080] Clause 6.2.5 further references the test connection method, high-altitude operating condition compensation steps, or judgment threshold setting method disclosed in another standard, GB / T 36276. This identifies a cross-standard reference in Clause 6.2.5. First, the reference intent of Clause 6.2.5 is analyzed to determine which technical elements from the cited standard need to be borrowed, thus obtaining the local retrieval constraints of Clause 6.2.5 on GB / T 36276, such as: "test connection loop configuration," "test environment condition settings," and "pressure withstand anomaly judgment rules." Then, using these local retrieval constraints, a cross-standard extension is performed within the cited standard GB / T 36276. The top M nodes with the highest degree of matching with these local retrieval constraints are used as the second evidence nodes associated with Clause 6.2.5. For example, if M=2, the two determined second evidence nodes include: reference clause 6.7.3.4.1 and reference clause 6.7.1.6.1. Within the standard GB / T 36276 to which reference clause 6.7.3.4.1 belongs, the preceding clause requirement node is automatically traced back along the internal dependency edge of the standard, which is reference clause 6.2.4.1.2, and is the first evidence node.

[0081] Within the standard GB / T 36276 to which reference clause 6.7.1.6.1 belongs, the preceding clause requirement node is automatically traced back along the internal dependency edge of the standard, which is also reference clause 6.2.4.1.2, and has been retained as the first evidence node.

[0082] Within the standard GB / T 36276 to which reference clause 6.2.4.1.2 belongs, the preceding clause requirement node is automatically traced back along the internal dependency edge of the standard, which is reference clause 6.2.2.2, and is the first evidence node.

[0083] After the expansion is completed, the target clause 6.2.5, reference clause 5.2.5, reference clause 6.7.3.4.1, reference clause 6.7.1.6.1, reference clause 6.2.4.1.2, and reference clause 6.2.2.2 can be directly used as evidence nodes. The evidence nodes and the standard nodes to which the evidence nodes belong constitute evidence information. This evidence information is no longer the text of a single clause requiring a node, but a complete evidence package including the target clause and its preceding clauses. The technical issues and the evidence information are input into the large language model. The large language model integrates the core process features in the target standard with the supplementary features in the cited standard to generate a complete analysis result (i.e., the executable clause shown in the figure) around the user's problem, including: candidate related standards, the technical functions undertaken by each standard, the source of evidence corresponding to the key technical features, and the integrated implementation chain.

[0084] In this embodiment, the standard document is first structured and parsed to construct a standard knowledge graph consisting of standard nodes, clause nodes, and inter-clause citation relationships. Semantic retrieval is then performed based on user questions or anomalies to locate relevant clauses. The relevant clauses are then expanded along the internal citation relationships within the same standard and the cross-standard citation relationships. Finally, a large language model is combined to generate governance recommendations, compliance judgment results, or execution steps that are supported by evidence.

[0085] In other words, the process begins by extracting standard numbers, clause numbers, clause texts, and citation relationships from standard documents. This explicit modeling of the structural relationships within the standard documents constructs a standard knowledge graph. Then, a graph-guided standard retrieval is performed. Based on semantic matching, the cited clauses and standards are recursively expanded during the retrieval phase using this structural relationship for contextual expansion. Finally, the expanded evidence package is input into a large language model, thus achieving a "structured constraint first, then language model reasoning" approach. This allows the large language model to perform reasoning on the constrained evidence package, merging previously scattered, cross-referenced, and inconsistent standard clauses into actionable recommendations, compliance judgments, or supporting evidence with clear sources.

[0086] Therefore, it is possible to perform structured modeling of standard texts and, by combining the citation relationships between standards and clauses, quickly locate the core clauses and supporting clauses related to the issue, reducing the workload of manually reviewing each clause; it can significantly reduce the workload of manually tracing citation relationships in each document, improve the efficiency and accuracy of standard retrieval and technical analysis around specific technical issues, and enhance the interpretability and traceability of the results.

[0087] Meanwhile, this invention combines a large language model to fuse and analyze multiple pieces of evidence retrieved from the search, transforming previously scattered and cross-referenced standard content into more complete, clear, and well-supported implementation or governance recommendations. This method helps improve the completeness, accuracy, and enforceability of standard queries, enabling operations, testing, and management personnel to quickly obtain standard evidence corresponding to actual problems and conduct problem analysis, compliance assessment, and rectification and governance work accordingly.

[0088] The second embodiment of the present invention relates to an electronic device, which may be, for example, a server, a workstation or a cloud computing platform, or a common computer device, such as a desktop computer or a laptop computer.

[0089] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the knowledge graph and large language model-based retrieval method of the first embodiment.

[0090] It should be noted that the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Similarly, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0091] Since the first embodiment corresponds to this embodiment, this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment remain valid in this embodiment, and the technical effects achievable in the first embodiment can also be achieved in this embodiment. To reduce repetition, they will not be repeated here. Correspondingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0092] The third embodiment of the present invention relates to a computer-readable storage medium, which is a non-volatile or non-transient storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the retrieval method based on knowledge graph and large language model in the first embodiment.

[0093] The preferred embodiments of the present invention have been described in detail above, but it should be understood that, if necessary, aspects of the embodiments can be modified to utilize aspects, features, and concepts from various patents, applications, and publications to provide other embodiments.

[0094] In light of the detailed description above, these and other changes can be made to the embodiments. Generally, the terminology used in the claims should not be considered limited to the specific embodiments disclosed in the specification and claims, but should be understood to include all possible embodiments together with the full scope of equivalents enjoyed by these claims.

Claims

1. A retrieval method based on knowledge graphs and large language models, characterized in that, include: The system receives technical questions from users and retrieves the target node most relevant to the technical question from the constructed standard knowledge graph. The standard knowledge graph includes: several standard nodes, clause requirement nodes under each standard node, and the reference relationships between the clause requirement nodes; In a standard knowledge graph, a first evidence node and a second evidence node are identified that are associated with the target node; the first evidence node and the target node belong to the same standard node, and the second evidence node and the target node belong to different standard nodes, and each second evidence node has a cross-standard reference to the target node or the first evidence node; Construct evidence information including the first evidence node and the standard node to which the first evidence node belongs, the second evidence node and the standard node to which the second evidence node belongs; The technical problem and the evidence information are input into a large language model for reasoning, and the response to the technical problem is obtained by the output of the large language model. The response includes the technical conclusion, the source of evidence, and the association path.

2. The retrieval method based on knowledge graph and large language model according to claim 1, characterized in that, Identifying the first evidence node and the second evidence node associated with the target node in the standard knowledge graph includes: Based on the first standard node to which the target node belongs, a search step is performed with the target node as the search starting point. The search step is as follows: based on the standard node to which the search starting point belongs, N first evidence nodes associated with the search starting point are searched in the standard knowledge graph, where N≥1. After each execution of the search step, if there is a specified node that is referenced across standards among the N first evidence nodes obtained in this search, then based on the standard node referenced across standards by the specified node, determine M second evidence nodes associated with the specified node in the standard knowledge graph, where M≥1; If the preset stopping condition is not met, the search steps will be executed again, taking the N first evidence nodes and the M second evidence nodes obtained in this search as the search starting points respectively. If the preset stopping condition is met, all the first evidence nodes and the second evidence nodes currently acquired will be retained; The preset stopping conditions include at least one of the following: the depth between the target node and the first evidence node obtained in this search reaches a preset maximum expansion depth; the sum of the number of the determined first evidence node and the second evidence node reaches a preset upper limit; and no new first evidence node or second evidence node is obtained in this search.

3. The retrieval method based on knowledge graphs and large language models according to claim 2, characterized in that, Based on the standard node to which the search starting point belongs, N first evidence nodes associated with the search starting point are obtained in the standard knowledge graph, including: If the search starting point is a clause requirement node, then one or more target clause requirement nodes referenced by the search node are obtained by backtracking along the clause requirement dependency edges in the standard knowledge graph; all the target clause requirement nodes constitute the N first evidence nodes.

4. The retrieval method based on knowledge graph and large language model according to claim 2, characterized in that, Based on the standard nodes referenced across standards by the specified node, M second evidence nodes associated with the specified node are determined within the standard knowledge graph, including: Based on at least a portion of the text content of the target node that is related to the standard node referenced across standards, a local retrieval constraint is generated; Within the standard knowledge graph, determine a set of candidate nodes for the standard nodes referenced across standards by the specified node. The set of candidate nodes includes all clause requirement nodes under the standard nodes referenced across standards by the specified node. The local retrieval constraint is matched with all nodes in the candidate node set, and the top M nodes with the highest matching degree with the local retrieval constraint are selected from the candidate node set as the second evidence nodes associated with the specified node, where M≥1.

5. The retrieval method based on knowledge graph and large language model according to claim 1, characterized in that, The target nodes most relevant to the technical problem are retrieved from the constructed standard knowledge graph, including: Based on the similarity between the technical problem and the text content of each of the clause requirement nodes, the clause requirement node most relevant to the technical problem is selected as the target node.

6. The retrieval method based on knowledge graph and large language model according to claim 5, characterized in that, Based on the similarity between the text content of the technical problem and each of the clause requirement nodes, the clause requirement node most relevant to the technical problem is selected as the target node, including: Select the text content that has the highest similarity to the technical problem as the target candidate text; If the similarity between the target candidate text and the technical problem is greater than or equal to a preset similarity threshold, then the clause requirement node corresponding to the target candidate text is taken as the target node.

7. The retrieval method based on knowledge graph and large language model according to claim 5, characterized in that, The method for obtaining the similarity between the technical problem and the text content of the clause requirement node is as follows: Obtain the query vector encoded by the technical problem, and the encoded vector of the text content of the clause requirement node; The cosine similarity between the query vector of the technical problem and the encoded vector of the text content of the clause requirement node is calculated as the similarity between the technical problem and the text content of the clause requirement node.

8. The retrieval method based on knowledge graph and large language model according to claim 1, characterized in that, The standard knowledge graph construction process is as follows: Read standard-level metadata and clause requirement data from the full-text data of the standard, and obtain the attribution relationship between standard-level metadata and clause requirements, the subordinate reference relationship between clause requirement data, and the cross-standard reference relationship between clause requirement data and standard-level metadata; In the standard knowledge graph, the standard-level metadata is used as the standard node, and the clause requirement data is used as the clause requirement node. Based on the aforementioned attribution relationship, construct the relationship edges between the standard node and the clause requirement node; Based on the aforementioned subordinate reference relationship, construct requirement dependency edges between the clause requirement nodes under the same standard node; Based on the cross-standard reference relationship, cross-standard reference edges are constructed between the standard node, the clause requirement node, and the cross-standard reference standard node.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the retrieval method based on knowledge graph and large language model as described in any one of claims 1 to 8.

10. A computer-readable storage medium, wherein the computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer program is stored thereon, characterized in that, The computer program is executed by the processor to perform the retrieval method based on knowledge graphs and large language models as described in any one of claims 1 to 8.

Images