Information retrieval reasoning optimization method and system based on knowledge graph and feedback enhancement

Abstract

The invention discloses an information retrieval reasoning optimization method and system based on a knowledge graph and feedback enhancement, and the method comprises the steps: converting natural language data in a text material into a directed graph, and constructing and dynamically updating the knowledge graph containing rules and historical cases; extracting features based on historical cases, constructing a sparse matrix and a feedback database, and recording a prediction success rate; acquiring and preprocessing questions of a user, and performing semantic matching with the map nodes to generate a candidate path set; establishing a path quality scoring device, and comprehensively evaluating the path confidence, the rule matching degree, the case score item and the path length; introducing an exponential decay feedback mechanism, dynamically updating a case prediction success rate and calculating a candidate path preferential selection score; and selecting a high-score path as a background prompt, converting the high-score path into a text, and inputting the text into the large model to generate an answer. According to the method, large model illusion and Token consumption can be reduced, retraining is not needed, and dynamic updating of rules is supported.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to an information retrieval reasoning optimization method and system based on knowledge graphs and feedback enhancement. Background Technology

[0002] With the widespread application of Large Language Models (LLMs) in service consulting, business support, and other fields, their ability to handle complex logical reasoning and rule understanding has attracted considerable attention. However, in practical applications, large models still face the following key challenges:

[0003] 1. The output of an LLM is highly dependent on the accuracy of its input; however, in reality, conversations between people are insufficient to provide the LLM with enough clues.

[0004] 2. Existing methods based on static knowledge bases or retrieval-enhanced generation (RAG) are difficult to adapt to the real-time updates of similar policies and regulations. Existing systems cannot effectively identify the numerous flexible processing mechanisms that exist in reality, and it is difficult to make reasonable use of such information.

[0005] 3. Traditional RAG (augmented search) schemes require segmenting and vectorizing a large number of paragraphs, rely on text similarity, and cannot perceive changes in rule structure, resulting in weak semantic relevance and easy loss of inherent relationship information between different texts and paragraphs.

[0006] 4. Under the current model, the illusion of LLM is still unavoidable. In some service-oriented consulting scenarios, providing incorrect guidance can lead to serious consequences.

[0007] To address the aforementioned issues, while existing research has attempted to optimize paths using methods such as graph neural networks and reinforcement learning, these methods rely on large-scale training data, are complex to deploy, and struggle to meet the application requirements of rapid iteration and low resource constraints. Furthermore, upgrading the network lacks complete interpretability and cannot meet compliance requirements in specific scenarios. Summary of the Invention

[0008] Objective: This invention provides an information retrieval reasoning optimization method and system based on knowledge graphs and feedback enhancement. It aims to address the problems of existing technologies, such as strong reliance on input prompts, slow and inflexible response to dynamic rules, weak semantic relevance, lack of interpretability and illusion problems, and the difficulty of graph neural networks and reinforcement learning methods in meeting the requirements of rapid iteration and low resource requirements. The invention addresses these issues by structurally storing background content text in natural language form. When a question is posed, multiple candidate reference reasoning paths are generated. Simultaneously, the optimal path is selected based on a multi-fitness evaluation function and a comprehensive score of historical feedback. This allows the model to more accurately understand the associated information in the text (such as repair steps and fault associations in a fault repair scenario). The final output is a highly accurate natural language suggestion that fits the actual scenario. This achieves reasoning optimization without deep training, supports dynamic rule updates, provides interpretability of LLM output content, possesses adaptive capabilities, and significantly reduces the input length of LLM.

[0009] Technical solution: This invention provides an information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement, comprising:

[0010] The process involves acquiring text materials, converting the natural language data within the text materials into directed graphs, constructing a knowledge graph based on the directed graphs, and dynamically updating the knowledge graph; the knowledge graph includes rule-oriented directed graphs and historical case-oriented directed graphs.

[0011] Based on the directed graph of historical cases in the knowledge graph, the features of historical cases are extracted and a sparse matrix of cases is constructed; a feedback database is constructed based on the sparse matrix of cases to record the prediction success rate of the features of historical cases.

[0012] The system acquires and preprocesses user query information, performs semantic matching queries between the preprocessed user query information and nodes in the directed graph, and filters out nodes with higher matching thresholds to form a candidate node set. A path search algorithm is used to obtain the paths from the starting node to the ending node in the directed graph from the candidate node set as candidate paths, and all candidate paths are combined into a candidate path set.

[0013] A path quality scorer is established. The candidate path set is input into the path quality scorer, and the comprehensive quality score of each candidate path is output. The path quality scorer comprehensively evaluates the path confidence, rule matching degree, case bonus items, and path penalty length.

[0014] Establish a feedback-enhanced selection mechanism based on exponential decay, dynamically update the prediction success rate of historical case features using an exponentially weighted moving average method, and calculate the optimal selection score of candidate paths based on the prediction success rate and path quality scorer.

[0015] The candidate paths in the candidate path set are ranked by their selection scores, and the top-ranked candidate paths are selected as background prompts. The background prompts are then processed and converted into text information, which is input into the large model to generate answers to user questions.

[0016] Furthermore, the step of converting natural language data in textual materials into directed graphs, constructing a knowledge graph based on the directed graphs, and dynamically updating it includes:

[0017] The text materials are classified according to domain relationships and relevance, and multiple tree-like domain classification diagrams are constructed.

[0018] Extract the target object entities, target object association conditions or rules, and final conclusions from each tree-like domain classification graph of the text material. The target object entities include rule entities and case entities. Use the target object entities, target object association conditions or rules, and final conclusions as nodes of a directed graph. Construct edges of the directed graph based on the relationships between each node to form a directed graph. The directed graph starting with rule entities is called a rule-oriented graph, and the directed graph starting with case entities is called a historical case-oriented graph. Construct all the directed graphs into a knowledge graph. For updated text materials, the knowledge graph is updated synchronously and dynamically.

[0019] Furthermore, the step of extracting historical case features from a directed graph of historical cases in a knowledge graph and constructing a sparse case matrix includes:

[0020] The historical case features are extracted, classified, and combined according to the set features from the directed graph of historical cases. All historical case features are used as rows of the case sparse matrix, and the numbers of each starting point in the directed graph of historical cases are used as columns of the case sparse matrix. For each directed graph of historical cases, it is analyzed whether there are historical case features. If there are, the corresponding row and column position of the case sparse matrix is ​​1, otherwise it is 0.

[0021] A feedback database is constructed based on the rows of the sparse case matrix to record the prediction success rate of historical case features.

[0022] Furthermore, the path quality scorer includes:

[0023] Establish a path quality scorer Each candidate path p in the candidate path set is input into the path quality scorer to calculate the overall quality score of candidate path p.

[0024] The formula for candidate path p is:

[0025] ;

[0026] in, Represents the starting node. Represents the i-th node. represents the edge from node i to node i+1; k is the number of nodes in the candidate path p;

[0027] Path quality scorer The formula is:

[0028] ;

[0029] in, The formula for path confidence is:

[0030] ;

[0031] in, For the edge The weights;

[0032] The formula for rule matching degree is:

[0033] ;

[0034] Wherein, the denominator represents the total number of all rule nodes in the candidate path, and the numerator represents the total number of rule nodes in the candidate path that match the user's query information;

[0035] Bonus points for case studies, the formula is:

[0036] ;

[0037] in, Condition coefficients, The formula for the match degree of the case is:

[0038] ;

[0039] in, This indicates the number of parameters to be evaluated for the path, including the timeliness, specificity, and frequency of use of the case. Let be the coefficient of the i-th parameter. This represents the value of the i-th parameter;

[0040] The path penalty length is the number of nodes in the candidate path p. They correspond to as The weighting coefficients.

[0041] Furthermore, the prediction success rate of dynamically updating historical case features using the exponentially weighted moving average method is calculated using the following formula:

[0042] ;

[0043] in, The prediction success rate is dynamically updated. The prediction success rate before dynamic updates; The adoption results of users' answers provided by the large model; This represents the attenuation rate.

[0044] Furthermore, the selection of multiple candidate paths based on prediction success rate and path quality scorer is performed using the following formula:

[0045] ;

[0046] in, A score is selected to optimize the candidate paths. This represents the overall quality score of the candidate paths. For feedback adjustment coefficient, As the feature influence weight, The current prediction success rate is a feature. It is a collection of text information in the text materials corresponding to each node in the candidate path.

[0047] Furthermore, the process of converting the background prompts into text information includes: based on a knowledge graph, performing structured semantic merging on the text information of the text materials corresponding to the nodes in each candidate path of the background prompts; and removing the text information of nodes with a repetition degree higher than the repetition threshold according to the degree of repetition between nodes, thereby obtaining the merged and deduplicated text information.

[0048] This invention also provides an information retrieval reasoning optimization system based on knowledge graphs and feedback enhancement, comprising:

[0049] The knowledge graph construction module is used to acquire text materials, convert the natural language data in the text materials into directed graphs, construct knowledge graphs based on the directed graphs, and dynamically update them; the knowledge graphs include rule-oriented directed graphs and historical case-oriented directed graphs;

[0050] The matrix and database building module is used to extract features of historical cases based on the directed graph of historical cases in the knowledge graph, construct a sparse matrix of cases, and build a feedback database based on the sparse matrix of cases to record the prediction success rate of the features of historical cases.

[0051] The candidate node module is used to acquire and preprocess user query information, perform semantic matching queries between the preprocessed user query information and nodes in the directed graph, and filter out nodes with higher matching thresholds to form a candidate node set; a path search algorithm is used to obtain the path from the starting node to the ending node in the directed graph from the candidate node set as candidate paths, and all candidate paths are combined into a candidate path set.

[0052] The comprehensive scoring module is used to establish a path quality scorer. It inputs the candidate path set into the path quality scorer and outputs the comprehensive quality score of each candidate path. The path quality scorer comprehensively evaluates path confidence, rule matching degree, case bonus items, and path penalty length.

[0053] The feedback enhancement selection module is used to establish a feedback enhancement selection mechanism based on exponential decay. It dynamically updates the prediction success rate of historical case features using an exponentially weighted moving average method, and calculates the optimal selection score of candidate paths based on the prediction success rate and the path quality scorer.

[0054] The large model output module is used to sort the candidate paths in the candidate path set by their selection scores, select the top-ranked candidate paths as background prompts, process the background prompts into text information, and input them into the large model to generate answers to user questions.

[0055] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0056] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method.

[0057] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0058] 1. Through precise information retrieval, reference prompts can be quickly and accurately input into the LLM model, greatly reducing the illusion problem caused by the model's free play and minimizing various losses caused by model error prompts and suggestions.

[0059] 2. By generating, evaluating, filtering, and semantically compressing knowledge graph paths, we can reduce the cost issues caused by excessive token consumption due to a large amount of reference information in the actual production environment and improve operational efficiency.

[0060] 3. By combining historical case data for data augmentation, the model can self-enhance without needing to retrain and adjust it, enabling it to absorb experience and thus improve the accuracy of the answers.

[0061] 4. This invention plays a positive role in specific application scenarios, such as AI assistant consultation scenarios for on-site maintenance reference assistance, operation and maintenance alarms and their feedback suggestions, and consultation suggestions. Attached Figure Description

[0062] Figure 1 This is a flowchart of the method of the present invention.

[0063] Figure 2 This is a flowchart illustrating the knowledge graph construction process of this invention.

[0064] Figure 3 This is a diagram illustrating how the present invention transforms natural language into structured data.

[0065] Figure 4 This is a schematic diagram of a sparse matrix, a special case of the present invention.

[0066] Figure 5 This is a flowchart illustrating the feedback enhancement process of the present invention. Detailed Implementation

[0067] like Figure 1 As shown, the information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement described in this invention includes:

[0068] Obtain user question information;

[0069] The process involves acquiring text materials, converting the natural language data in the text materials into directed graphs, constructing a knowledge graph based on the directed graphs, and dynamically updating it. The knowledge graph includes a rule-oriented directed graph and a historical case directed graph. Based on the historical case directed graph in the knowledge graph, features of historical cases are extracted, and a case sparse matrix is ​​constructed. A feedback database is constructed based on the case sparse matrix to record the prediction success rate of historical case features.

[0070] The user query information is preprocessed, and the preprocessed user query information is semantically matched with the nodes in the directed graph for preliminary screening. Nodes with a matching threshold are selected to form a candidate node set. A path search algorithm is used to obtain the path from the starting node to the ending node in the directed graph from the candidate node set as candidate paths, and all candidate paths are combined into a candidate path set.

[0071] A path quality scorer is established. The candidate path set is input into the path quality scorer, and the comprehensive quality score of each candidate path is output. The path quality scorer comprehensively evaluates the path confidence, rule matching degree, case bonus items, and path penalty length.

[0072] Establish a feedback-enhanced selection mechanism based on exponential decay, dynamically update the prediction success rate of historical case features using an exponentially weighted moving average method, and calculate the optimal selection score of candidate paths based on the prediction success rate and path quality scorer.

[0073] The candidate paths in the candidate path set are sorted by their selection scores, and the top-ranked candidate paths are selected as background prompts. The background prompts are then processed and converted into text information, which is input into the large model to generate answers to user questions.

[0074] The system records user adoption of answers generated by the large model in the feedback database and updates the prediction success rate of historical case features in the answers.

[0075] Specifically, first, obtain user question information;

[0076] Next, transforming various background text information and natural language information into a structured, computable knowledge network is the foundation for improving the system's reasoning ability.

[0077] like Figure 2 As shown, this invention, taking into account the characteristics of background knowledge text content in real-world scenarios, processes text materials composed of various background text information and natural language information as follows:

[0078] 1. Classify text materials according to different domain relationships and relevance, construct multiple tree-like domain classification diagrams, clearly separate the relationships between different domains, and narrow down the scope of useless searches.

[0079] 2. Extract the target object entities, associated conditions or rules, and final conclusions of the text material from each tree-like domain classification graph. The target object entities include rule entities and case entities. The target object entities, associated conditions or rules, and final conclusions are used as nodes in a directed graph (target object entities as starting nodes, target nodes, or phenomenon nodes, etc.; conditions and rules as condition nodes or rule nodes, etc.; and final conclusions as conclusion nodes or end nodes, etc.). Based on the relationships between each node, edges are constructed from four perspectives: logical rules, dependent conditions, process order, and whether it is an experience case, forming a directed graph. The directed graph starting with rule entities is a rule-oriented graph, and the directed graph starting with case entities is a historical case-oriented graph. All directed graphs are used to construct a computable, reasonable, updatable, and structured knowledge graph. For updated text materials, the knowledge graph is dynamically updated synchronously.

[0080] For example, suppose the text material is an electronic device repair manual. The repair reference manual contains the following: If the screen flickers noticeably, first check if the connection harness is loose. If it is loose, directly tighten the harness. Next, use a multimeter to check if the voltage at the pins of the display driver chip is normal. If it is not normal, check if the bypass capacitor of the chip is damaged. Otherwise, check if the output of the corresponding chip is normal.

[0081] Therefore, screen flickering is taken as the starting node, loose wiring harness, normal voltage at chip pins, and damaged bypass capacitors are taken as conditional nodes, and re-secured wiring harness, replaced chip, and replaced capacitor are taken as conclusion nodes. Based on this, the natural language of the text material is transformed into structured data, and the edges corresponding to the graph are constructed from four perspectives: logical rules, dependencies, process sequence, and whether it is an empirical case, thus constructing a directed graph, such as... Figure 3 As shown. Similarly, if the text material contains content about detecting chip pin operations when the screen is black, the pin detection part can be directly reused to complete the structure of the entire knowledge graph, such as... Figure 3 The system detects the waveform of the driver chip to determine if it needs replacement. If a power-on failure is described in the text, the system checks the transformer and replaces any damaged windings. A directed graph is then constructed and updated in the knowledge graph. The set of all nodes traversed from the abnormal state to the final operation is called a path.

[0082] This invention provides a means for rule change awareness for knowledge graphs, and constructs an access layer that supports manual or external system information access. During access, a lightweight NLP module is used or node information is extracted according to preset fixed rules.

[0083] Taking the equipment maintenance manual as an example, when there is new information such as: when the equipment power supply is making a whistling sound, check whether the corresponding varnished power inductor has any damage to the external paint surface. If it is damaged, the paint surface can be repaired. If the situation is serious, the inductor can be replaced.

[0084] Extract the device as the target node, the howling as the phenomenon node (the starting point node for special cases), the external paint surface as the condition node, and repair or replace the paint surface as the conclusion node; merge the new node and edges into the directed graph corresponding to the existing knowledge graph, that is, update the knowledge graph. The merging refers to adding the new node and edges into the knowledge graph, comparing the degree of repetition between nodes (such as similarity), and removing nodes or edges with a repetition degree higher than the repetition threshold.

[0085] In real-world scenarios, building a knowledge graph solely based on specific regulations or rules is insufficient, especially in special situations. The lack of real-world context and reliance on textual content for judgment can lead to unsatisfactory results. Therefore, it's necessary to incorporate historical operational cases, particularly special cases, along with their corresponding operational records, into the knowledge graph. During the import of these cases into the knowledge graph, a directed graph unique to each case is constructed using the method described above. The corresponding knowledge network graph is then located, and a virtual link is established to attach the case's directed graph as a subgraph to that knowledge graph. This ensures that the import of historical case data is completed without interfering with the original text.

[0086] Since historical records and cases are mostly natural language text, this invention extracts and classifies the textual information in historical records and cases according to predefined historical case features. Whether a case possesses a feature is represented by 1 or 0 corresponding to that historical case feature, thus forming a feature vector for that case. Based on the feature vectors of multiple cases, a sparse matrix is ​​constructed to express the relationship between each historical case and its features. The rows of the matrix represent predefined historical case features, the columns represent the ID of the historical case in the knowledge graph, and the element value is 0 or 1, indicating whether the historical case possesses that historical case feature (0 indicates no feature, 1 indicates feature).

[0087] like Figure 4 As shown, the rows of the sparse matrix include historical case features such as "loose terminal J12", "damaged bypass capacitor C24", "unstable voltage of display driver chip U4", "broken transistor Q18", and "broken freewheeling diode D7". The px324 black screen-0223 case includes historical case features such as "loose terminal J12" and "broken transistor Q18", with the first 5 bits of the feature vector being [1,0,0,1,0]. The px391 screen flickering-0407 case includes historical case features such as "damaged bypass capacitor C24" and "unstable voltage of display driver chip U4", with the first 5 bits of the feature vector being [0,1,1,0,0]. The px453 unable to power on-0731 case includes historical case features such as "broken transistor Q18" and "broken freewheeling diode D7", with the first 5 bits of the feature vector being [0,0,0,1,1]. A complete case can be clearly represented in the form of entities and feature vectors.

[0088] This invention is applicable to multi-domain knowledge management and problem-solving scenarios (such as industrial equipment maintenance, engineering safety management, and other fields that require the integration of professional documents and cases). By constructing a multi-dimensional associated knowledge graph of target object-problem description-solution and a retrieval and reasoning method, the system can automatically integrate fragmented information such as domain manuals, historical business records, and professional experience summaries to form a structured knowledge network. At the same time, it extracts corresponding features from case records in actual scenarios to construct a sparse matrix.

[0089] In addition to the sparse matrix, this invention maintains a case feedback database to record the adoption results of users' responses to the large model (hereinafter referred to as prediction suggestions) and the historical prediction success rate of features when calculating the prediction suggestions.

[0090] This invention addresses the challenges of traditional knowledge bases in responding to rule changes and handling special cases in real-world scenarios through dynamic updates and path modeling. It also provides accurate and compliant semantic support for subsequent path generation and optimization. For documents such as safety regulations and maintenance manuals, the invention unifies and abstracts the natural language text content, defining nodes from four perspectives: the target object, the corresponding characteristic phenomena, the relationship between the object and the phenomena, and the corresponding cases and final conclusions. This forms a directed heterogeneous graph.

[0091] The user query information is preprocessed, including extracting key entities such as user information, operation goals, and background conditions from the user query information and the user's role. The preprocessed user query information is then semantically matched with nodes in the directed graph, and nodes with matching thresholds are selected to form a candidate node set. A path search algorithm (BFS or Dijkstra) is used to obtain the paths from the starting node to the ending node in the directed graph from the candidate node set as candidate paths, and all candidate paths are combined into a candidate path set. When generating paths, samples are taken from original documents and actual cases according to a preset ratio coefficient.

[0092] When faced with a question, diverse candidate references are generated according to corresponding rules, covering standard processes, exception handling, and historical success cases to avoid reasoning blind spots caused by a single path. By using a rule-driven rather than randomly generated approach, it ensures that all paths conform to business logic, solving the problems of incomplete path coverage and insufficient flexibility in traditional methods.

[0093] The candidate path set still cannot achieve precise matching, and contains much redundant and invalid information, such as paths with consistent target nodes that generally meet the correlation criteria but deviate significantly at several key condition nodes. Therefore, a multi-objective weighted function is designed to comprehensively evaluate each path. This invention uses an evaluation function to screen candidate paths, eliminating inaccurate and redundant information and refining the information range. Compared to the method relying on LLM's free judgment, this module provides an interpretable, configurable, and training-free scoring mechanism, improving the accuracy and controllability of inference. The path quality scorer includes:

[0094] Establish a path quality scorer Each candidate path p in the candidate path set is input into the path quality scorer to calculate the overall quality score of candidate path p.

[0095] The formula for candidate path p is:

[0096] ;

[0097] in, Represents the starting node. Represents the i-th node. represents the edge from node i to node i+1; k is the number of nodes in the candidate path p;

[0098] Path quality scorer The formula is:

[0099] ;

[0100] in, Path confidence measures the reliability of each edge in a path to indicate whether the path is an accurate and reasonable process. For example, if a path has extremely large jumps between some nodes and low correlation between nodes, its reliability can be judged as low, and thus it can be removed from the list. The formula is:

[0101] ;

[0102] in, For the edge The weight can be calculated using fixed rules (e.g., if the path node information is empty, the weight is directly reduced to 0.1).

[0103] To determine the rule matching degree, the degree of matching for all conditions in the path is calculated. This degree is determined by the proportion of all matching background condition nodes in the path to the total number of condition nodes in the path. The formula is:

[0104] ;

[0105] The denominator represents the total number of rule nodes in the candidate path, and the numerator represents the total number of rule nodes in the candidate path that match the user's query information. If the rule contains time-sensitive content (such as specific keywords), a timeliness check is performed.

[0106] Bonus points for case studies are awarded based on whether there are relevant case studies for reference. The system is encouraged to identify and refer to historical case studies. The formula is as follows:

[0107] ;

[0108] in, Condition coefficients, For the matching degree of the case, The formula is:

[0109] ;

[0110] in, This indicates the number of parameters to be evaluated for the path, including the timeliness, specificity, and frequency of use of the case. Let be the coefficient of the i-th parameter. This represents the value of the i-th parameter (e.g., in this case, it is 1 within the time limit and 0 outside the time limit).

[0111] The initial deviation during path search will be amplified as the path continues. At the same time, an excessively long path also represents too much content, which will lead to more tokens input into the LLM model. This will increase the probability of the model being disturbed and producing illusions. Therefore, it is necessary to place it in the denominator in the design to reduce the weight of excessively long paths.

[0112] The path penalty length, i.e., the number of nodes in the candidate path p, is used as a penalty term, and the formula is:

[0113] ;

[0114] They correspond to as The weighting coefficients can be manually adjusted to adapt to different business scenarios based on actual conditions.

[0115] For example, there is a path from the start of screen flickering to the detection of the connection harness and the end of re-stabilization, and there are other paths such as the path of first detecting the chip pins and then detecting the bypass capacitor and then replacing the chip. Each path is calculated according to the above formula to obtain an evaluation score, and the paths that are lower than the preset score threshold can be eliminated to achieve preliminary fine screening.

[0116] like Figure 5 As shown, this invention proposes a feedback enhancement selection mechanism based on exponential decay. It dynamically updates the prediction success rate of historical case features using an exponentially weighted moving average, allowing old feedback to gradually decay and new feedback to take effect quickly. The update formula is as follows:

[0117] ;

[0118] in, This represents the prediction success rate of this feature after dynamic updates. This represents the prediction success rate of this feature before dynamic updates. This indicates whether the user has adopted the prediction suggestions; 1 represents adoption, and 0 represents non-adoption. This represents the attenuation rate.

[0119] In the path optimization stage, the prediction success rate is introduced as a weighting factor into the selection scoring function. Multiple candidate paths are then selected based on the prediction success rate and the path quality scorer, thus making reasonable use of historical experience. The formula is as follows:

[0120] ;

[0121] in, A score is selected to optimize the candidate paths. This represents the overall quality score of the candidate paths. For feedback adjustment coefficient, This is the feature influence weight, which defaults to 1. The current prediction success rate is a feature. It is a collection of text information in the text materials corresponding to each node in the candidate path.

[0122] This mechanism avoids the path-locking problem caused by static feedback, ensuring the system can adapt to rule changes and business trend evolution. By incorporating a feedback loop—that is, using historical data for feedback—the system can continuously learn from actual business adoption results, understanding which experiences are more effective, thus providing output suggestions more quickly and accurately. It compensates for the limitation of static fitness functions in not being able to perceive real-world effects, and achieves experience-driven intelligent optimization without introducing neural networks, enabling the system to continuously evolve.

[0123] For example, suppose a technician repairs a motherboard and finds that while the power indicator light is normal and the system functions normally, the screen displays a distorted image. The problem is eventually solved by replacing a bypass capacitor in the display section. The success rate for the screen distortion feature was previously 0.8, with a decay rate of 0.2. If the model correctly suggested replacing the bypass capacitor on the first attempt, the success rate for the next prediction would be: (1−0.2)×0.8+0.2×1=0.64+0.2=0.84. The system will be more likely to trust the option of replacing the bypass capacitor due to the screen distortion. If it doesn't correctly suggest replacing the bypass capacitor, the success rate for the next prediction will become (1−0.2)×0.8+0.2×0=0.64+0=0.64, and the system will reduce its preference for this path. Meanwhile, similar features, such as whether the resistor has burn marks or short circuits between the positive and negative terminals, will naturally decrease the prediction success rate due to the lack of feedback this time: (1-0.2)×0.8=0.64. If no feedback is received for a long time (such as none in 10 attempts), the final prediction success rate will decrease as shown below.

[0124] ;

[0125] A prediction success rate close to 0 indicates that the feature has not been validated for a long time and is no longer of reference value. In real-world scenarios, while it is desirable for the system to have a natural decay of feature reference value, it is unacceptable for a once highly successful feature to become completely worthless simply due to time decay. Therefore, the system sets a minimum value. When this minimum value is reached, decay will cease or decay at a smaller exponential rate. When the feature is reactivated, the prediction success rate can be restored.

[0126] This invention can absorb and feedback historical data and experience. Simultaneously, by incorporating an exponentially weighted decay mechanism, it overcomes the problem of relying on past experience and failing to adapt to external changes in a timely manner, thus enabling the system to possess continuous reasoning and evolutionary capabilities. The decay mechanism addresses the problem of path dependence caused by past successful paths being adopted, leading to self-reinforcement with iteration.

[0127] The final enhanced candidate paths are sorted by score, and the Top-K candidate paths are selected. The information corresponding to each node in the candidate path in the text material will be used as background prompts or reference information for LLM to make a comprehensive judgment. At the same time, the calculation information and content of the above paths will be backed up to provide support for data backtracking.

[0128] The background prompts are processed and converted into text information, including: based on the knowledge graph, performing structured semantic merging of the text information corresponding to the nodes in each candidate path of the background prompts; removing the text information of nodes with a repetition degree higher than the repetition threshold according to the degree of repetition between nodes; simplifying the format and symbol information in the information; and obtaining the merged and deduplicated text information.

[0129] Based on the selected LLM model format requirements, the information is converted into the corresponding format and input into the LLM model, allowing the LLM to regenerate natural language that is more in line with everyday expressions and present it to the user.

[0130] When a user asks a question, this invention can generate multiple candidate answers from the aforementioned knowledge graph. The reference content covers both standard text matching paths and historical cases, without overly relying on the LLM's autonomous capabilities, providing the LLM with precise prompts to ensure accuracy. For example, when equipment malfunctions, the intelligent question-answering system can quickly locate the root cause of the fault based on the graph's semantic reasoning capabilities (e.g., through the association path of "screen not lit → equipment operation status check → equipment component damage"), reducing the traditional 2-3 hours of manual troubleshooting to minutes, improving fault response speed by over 80%. Addressing the cost issue of LLM reasoning consuming tokens in actual production environments, the inference path is structured and compressed to reduce redundant and invalid input information. Simultaneously, the complete inference process and model input / output are recorded to ensure interpretability and support a backtracking mechanism to guarantee the traceability and reproducibility of events.

[0131] Finally, a suggestion feedback interface is provided so that users can evaluate the accuracy of the content and record the adoption status in the feedback database. The accuracy evaluation is used to update the prediction success rate of the features involved in the selected paths, thus forming a feedback loop and achieving a logical closed loop.

[0132] This invention can improve operation and maintenance efficiency and reduce overall costs, providing practitioners with a transformation and upgrading path from passive emergency repairs to proactive operation and maintenance. It can also build a core maintenance capability barrier for enterprises through knowledge accumulation and provide technical support capabilities for sustainable development.

[0133] The information retrieval reasoning optimization system based on knowledge graphs and feedback enhancement described in this invention includes:

[0134] The knowledge graph construction module is used to acquire text materials, convert the natural language data in the text materials into directed graphs, construct knowledge graphs based on the directed graphs, and dynamically update them; the knowledge graphs include rule-oriented directed graphs and historical case-oriented directed graphs;

[0135] The matrix and database building module is used to extract features of historical cases based on the directed graph of historical cases in the knowledge graph, construct a sparse matrix of cases, and build a feedback database based on the sparse matrix of cases to record the prediction success rate of the features of historical cases.

[0136] The candidate node module is used to acquire and preprocess user query information, perform semantic matching queries between the preprocessed user query information and nodes in the directed graph, and filter out nodes with higher matching thresholds to form a candidate node set; a path search algorithm is used to obtain the path from the starting node to the ending node in the directed graph from the candidate node set as candidate paths, and all candidate paths are combined into a candidate path set.

[0137] The comprehensive scoring module is used to establish a path quality scorer. It inputs the candidate path set into the path quality scorer and outputs the comprehensive quality score of each candidate path. The path quality scorer comprehensively evaluates path confidence, rule matching degree, case bonus items, and path penalty length.

[0138] The feedback enhancement selection module is used to establish a feedback enhancement selection mechanism based on exponential decay. It dynamically updates the prediction success rate of historical case features using an exponentially weighted moving average method, and calculates the optimal selection score of candidate paths based on the prediction success rate and the path quality scorer.

[0139] The large model output module is used to sort the candidate paths in the candidate path set by their selection scores, select the top-ranked candidate paths as background prompts, process the background prompts into text information, and input them into the large model to generate answers to user questions.

[0140] The computer device of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0141] The computer-readable storage medium of the present invention stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

Claims

1. An information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement, characterized in that, include: The process involves acquiring text materials, converting the natural language data within the text materials into directed graphs, constructing a knowledge graph based on the directed graphs, and dynamically updating the knowledge graph; the knowledge graph includes rule-oriented directed graphs and historical case-oriented directed graphs. Based on the directed graph of historical cases in the knowledge graph, the features of historical cases are extracted and a sparse matrix of cases is constructed. A feedback database is constructed based on a sparse case matrix to record the prediction success rate of historical case features; The system acquires and preprocesses user query information, performs semantic matching queries between the preprocessed user query information and nodes in the directed graph, and filters out nodes with higher matching thresholds to form a candidate node set. A path search algorithm is used to obtain the paths from the starting node to the ending node in the directed graph from the candidate node set as candidate paths, and all candidate paths are combined into a candidate path set. A path quality scorer is established. The candidate path set is input into the path quality scorer, and the comprehensive quality score of each candidate path is output. The path quality scorer comprehensively evaluates the path confidence, rule matching degree, case bonus items, and path penalty length. Establish a feedback-enhanced selection mechanism based on exponential decay, dynamically update the prediction success rate of historical case features using an exponentially weighted moving average method, and calculate the optimal selection score of candidate paths based on the prediction success rate and path quality scorer. The candidate paths in the candidate path set are sorted by their selection scores, and the top-ranked candidate paths are selected as background prompts. The background cues are processed and converted into text information, which is then input into a large model to generate answers to user questions.

2. The information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement according to claim 1, characterized in that, The process of converting natural language data from textual materials into directed graphs, constructing a knowledge graph based on the directed graphs, and dynamically updating it includes: The text materials are classified according to domain relationships and relevance, and multiple tree-like domain classification diagrams are constructed. Extract the target object entities, target object association conditions or rules, and final conclusions from each tree-like domain classification graph of the text material. The target object entities include rule entities and case entities. Use the target object entities, target object association conditions or rules, and final conclusions as nodes of a directed graph. Construct edges of the directed graph based on the relationships between each node to form a directed graph. The directed graph starting with rule entities is called a rule-oriented graph, and the directed graph starting with case entities is called a historical case-oriented graph. Construct all the directed graphs into a knowledge graph. For updated text materials, the knowledge graph is updated synchronously and dynamically.

3. The information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement according to claim 1, characterized in that, The process of extracting historical case features from a directed graph of historical cases in a knowledge graph and constructing a sparse case matrix includes: The historical case features are extracted, classified, and combined according to the set features from the directed graph of historical cases. All historical case features are used as rows of the case sparse matrix, and the numbers of each starting point in the directed graph of historical cases are used as columns of the case sparse matrix. For each directed graph of historical cases, it is analyzed whether there are historical case features. If there are, the corresponding row and column position of the case sparse matrix is ​​1, otherwise it is 0. A feedback database is constructed based on the rows of the sparse case matrix to record the prediction success rate of historical case features.

4. The information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement according to claim 1, characterized in that, The path quality scorer includes: Establish a path quality scorer Each candidate path p in the candidate path set is input into the path quality scorer to calculate the overall quality score of candidate path p. The formula for candidate path p is: ； in, Represents the starting node. Represents the i-th node. represents the edge from node i to node i+1; k is the number of nodes in the candidate path p; Path quality scorer The formula is: ； in, The formula for path confidence is: ； in, For the edge The weights; The formula for rule matching degree is: ； Wherein, the denominator represents the total number of all rule nodes in the candidate path, and the numerator represents the total number of rule nodes in the candidate path that match the user's query information; Bonus points for case studies, the formula is: ； in, Conditional coefficients, The formula for the match degree of the case is: ； in, This indicates the number of parameters to be evaluated for the path, including the timeliness, specificity, and frequency of use of the case. Let be the coefficient of the i-th parameter. This represents the value of the i-th parameter; The path penalty length is the number of nodes in the candidate path p. They correspond to as The weighting coefficients.

5. The information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement according to claim 4, characterized in that, The prediction success rate, achieved by dynamically updating historical case features using an exponentially weighted moving average method, is calculated using the following formula: ； in, The prediction success rate is dynamically updated. The prediction success rate before dynamic updates; The adoption results of users' answers provided by the large model; This represents the attenuation rate.

6. The information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement according to claim 5, characterized in that, The formula for selecting multiple candidate paths based on prediction success rate and path quality scorer is as follows: ； in, A score is selected to optimize the candidate paths. This represents the overall quality score of the candidate paths. For feedback adjustment coefficient, As the feature influence weight, The current prediction success rate is a feature. It is a collection of text information in the text materials corresponding to each node in the candidate path.

7. The information retrieval reasoning optimization method based on knowledge graphs and feedback enhancement according to claim 1, characterized in that, The process of converting background prompts into text information includes: based on a knowledge graph, performing structured semantic merging on the text information corresponding to the nodes in each candidate path of the background prompts; and removing the text information of nodes with a repetition degree higher than the repetition threshold according to the degree of repetition between nodes, thereby obtaining the merged and deduplicated text information.

8. An information retrieval reasoning optimization system based on knowledge graphs and feedback enhancement, characterized in that, include: The knowledge graph construction module is used to acquire text materials, convert the natural language data in the text materials into directed graphs, construct knowledge graphs based on the directed graphs, and dynamically update them; the knowledge graphs include rule-oriented directed graphs and historical case-oriented directed graphs; The matrix and database building module is used to extract features of historical cases and construct a sparse matrix of cases based on the directed graph of historical cases in the knowledge graph. A feedback database is constructed based on a sparse case matrix to record the prediction success rate of historical case features; The candidate node module is used to acquire and preprocess user query information, perform semantic matching queries between the preprocessed user query information and nodes in the directed graph, and filter out nodes with higher matching thresholds to form a candidate node set; a path search algorithm is used to obtain the path from the starting node to the ending node in the directed graph from the candidate node set as candidate paths, and all candidate paths are combined into a candidate path set. The comprehensive scoring module is used to establish a path quality scorer. It inputs the candidate path set into the path quality scorer and outputs the comprehensive quality score of each candidate path. The path quality scorer comprehensively evaluates path confidence, rule matching degree, case bonus items, and path penalty length. The feedback enhancement selection module is used to establish a feedback enhancement selection mechanism based on exponential decay. It dynamically updates the prediction success rate of historical case features using an exponentially weighted moving average method, and calculates the optimal selection score of candidate paths based on the prediction success rate and the path quality scorer. The large model output module is used to sort the candidate paths in the candidate path set by their selection scores and select the top-ranked candidate paths as background prompts. The background cues are processed and converted into text information, which is then input into a large model to generate answers to user questions.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images