Knowledge graph multi-source feedback closed loop updating method and device, equipment and medium

Abstract

The application relates to the technical field of knowledge graphs, in particular to a knowledge graph multi-source feedback closed-loop updating method and device, equipment and a medium. The application obtains multi-source feedback data generated by interaction between a downstream application and a target knowledge graph, and standardizes the processing. Knowledge defects are diagnosed based on a rule matching and machine learning model complementary mode, knowledge defect events are generated, repair work orders are generated and distributed, and after review, the repair and updating of the knowledge graph are performed. The above method collects multi-source feedback in actual application to form a closed-loop updating of the knowledge graph. Known defect patterns are hit through preset rule matching, and potential abnormalities are deeply mined through a machine learning model. The discovery and ordered repair of existing knowledge graph defects are realized. In combination with an updated contrast verification and rollback mechanism, it is ensured that the updating operation effectively improves the service quality of the knowledge graph in a real application environment, and the updating efficiency and reliability of the knowledge graph are improved.

Description

Technical Field

[0001] This application relates to the field of knowledge graph technology, and in particular to a method, apparatus, device, and medium for multi-source feedback closed-loop updating of knowledge graphs. Background Technology

[0002] As a structured semantic knowledge base, knowledge graphs play a core role in knowledge representation and reasoning in vertical fields such as intelligent operation and maintenance of power equipment, industrial fault diagnosis, and intelligent customer service. With the continuous updates to industry standards and the dynamic changes in business needs, knowledge graphs need to respond quickly to the evolution of knowledge forms and maintain consistency with the actual needs of downstream applications.

[0003] Currently, the updating and maintenance of knowledge graphs primarily rely on operations and maintenance personnel manually locating and correcting problems based on business experience or sporadic feedback; or on a scheduled batch update mechanism, centrally processing the knowledge graph according to a preset cycle. However, the manual method depends on individual experience and lacks a systematic approach; the scheduled update method can only be triggered at fixed nodes and cannot respond promptly to sudden knowledge defects exposed in downstream applications. The lack of effective connection between actual feedback from downstream applications and knowledge graph update actions makes it difficult to detect and accurately fix knowledge defects in a timely manner.

[0004] Therefore, how to use multi-source feedback data generated by downstream applications to drive the knowledge graph for error detection and updating is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The main purpose of this application is to provide a method, apparatus, device and medium for multi-source feedback closed-loop updating of knowledge graphs, aiming to solve the technical problem in the prior art of how to use multi-source feedback data generated by downstream applications to drive error detection and updating of knowledge graphs.

[0006] To achieve the above objectives, this application provides a multi-source feedback closed-loop update method for knowledge graphs. This method includes the following steps: acquiring multi-source feedback data generated when downstream applications interact with the target knowledge graph; structuring the multi-source feedback data according to a preset data template to obtain structured feedback data; based on the structured feedback data, performing preliminary defect diagnosis on the structured feedback data using preset defect matching rules, and inputting structured feedback data not matched by the defect matching rules into an anomaly detection model for in-depth defect diagnosis, obtaining knowledge defect events based on the preliminary and in-depth diagnosis results; generating knowledge graph repair work orders based on the knowledge defect events; reviewing the knowledge update work orders and generating knowledge graph update operation content based on the reviewed knowledge update work orders; and repairing and updating the target knowledge graph according to the update operation content to obtain the updated knowledge graph.

[0007] Optionally, the above-mentioned defect diagnosis based on structured feedback data, using a complementary approach of rule matching and machine learning models, to obtain knowledge defect events includes: matching structured feedback data according to preset defect matching rules, and determining the first type of defect event in the target knowledge graph based on the matching results, wherein the preset defect matching rules are used to screen typical defects based on statistical indicators in the structured feedback data; inputting structured feedback data that is not matched by the preset defect matching rules into an anomaly detection model for identification, and determining the second type of defect event in the target knowledge graph based on the output results of the anomaly detection model, wherein the anomaly detection model is used to extract multi-dimensional features to identify potential defects; and summarizing the first type of defect event and the second type of defect event to obtain knowledge defect events.

[0008] Optionally, the above-mentioned matching of structured feedback data according to preset defect matching rules, and determination of the first type of defect event of the target knowledge graph based on the matching results, includes: statistically analyzing the interaction result performance data of the corresponding knowledge points in the structured feedback data of the target knowledge graph based on the knowledge point identifiers and quantitative performance indicators carried in the structured feedback data; determining that the corresponding knowledge point in the target knowledge graph has a knowledge content deviation defect when the accuracy reflected by the interaction result performance data is lower than a first quality threshold; statistically analyzing the query result quality score of the corresponding knowledge point in the target knowledge graph based on the knowledge point identifiers and satisfaction association fields carried in the structured feedback data; determining that the corresponding knowledge point in the target knowledge graph has a knowledge coverage gap defect when the frequency of occurrence of the query result quality score being lower than a second quality threshold reaches a preset frequency threshold; and summarizing the knowledge points in the target knowledge graph that have knowledge content deviation defects and the knowledge points that have knowledge coverage gap defects to obtain the first type of defect event.

[0009] Optionally, the above-mentioned input of structured feedback data that is not matched by the preset defect matching rules into the anomaly detection model for identification, and the determination of the second type of defect event of the target knowledge graph based on the output of the anomaly detection model, includes: extracting multi-dimensional features from the structured feedback data that is not matched by the rules, wherein the multi-dimensional features include at least the behavioral statistical features of the feedback and the structural features of the associated nodes in the target knowledge graph; inputting the multi-dimensional features into the anomaly detection model to obtain the confidence level of the existence of defects in the target knowledge graph, and the probability distribution under each preset defect type; when the confidence level exceeds the preset model threshold, determining the defect knowledge location by the knowledge point identifier corresponding to the feedback data that is not matched by the preset defect matching rules, and taking the defect type with the highest probability in the probability distribution as the actual defect type; and determining the second type of defect event based on the actual defect type, the corresponding defect confidence level, the defect knowledge location, and the original interaction record corresponding to the feedback data that is not matched by the preset defect matching rules.

[0010] Optionally, the above-mentioned inputting feedback data that is not matched by the rules into the anomaly detection model for identification to determine the second type of defect event of the target knowledge graph includes: extracting multi-dimensional features based on the feedback data that is not matched by the rules, wherein the multi-dimensional features include at least the behavioral statistical features of the feedback and the structural features of the associated nodes in the target knowledge graph; inputting the multi-dimensional features into the anomaly detection model to obtain the confidence level of the existence of defects in the target knowledge graph and the probability distribution under each preset defect type; when the confidence level exceeds the preset model threshold, determining the defect knowledge location by the knowledge point identifier corresponding to the feedback data that is not matched by the rules, and taking the defect type with the highest probability in the probability distribution as the actual defect type; and determining the second type of defect event based on the actual defect type and the corresponding defect confidence level.

[0011] Optionally, the above-mentioned generation of a knowledge graph repair work order based on a knowledge defect event includes: determining the associated graph nodes from the target knowledge graph based on the defect knowledge location in the knowledge defect event; matching the corresponding suggested operation type and candidate update content from a preset repair strategy library based on the defect type in the knowledge defect event; determining the trigger source field of the knowledge graph repair work order based on the trigger source information carried by the knowledge defect event; and encapsulating the trigger source field, the problem description corresponding to the defect type, the associated graph nodes, the defect type, the suggested operation type, and the candidate update content into structured work order data to obtain the knowledge graph repair work order.

[0012] Optionally, the above-mentioned update operation content for generating the knowledge graph based on the reviewed knowledge update work order includes: determining the entity nodes or relation edges to be updated in the target knowledge graph based on the defect knowledge location and repair suggestions carried in the reviewed knowledge update work order; generating corresponding graph atomic operation instructions based on the repair operation type indicated in the repair suggestions, the types of graph atomic operation instructions including entity attribute value modification, relation edge addition / deletion, or triple replacement; and encapsulating the graph atomic operation instructions into update operation content executable by the target knowledge graph.

[0013] Optionally, the above-mentioned multi-source feedback closed-loop update method for knowledge graphs further includes: extracting the interaction questions corresponding to the feedback data generated by the knowledge defect event and constructing a test question set; dividing users accessing the target knowledge graph into a control group and an experimental group, with the control group using the knowledge graph before the update and the experimental group using the updated knowledge graph; obtaining the performance indicators of the control group and the experimental group for the test question set within a preset testing period; determining that the updated knowledge graph has passed verification when the performance indicators of the experimental group are better than those of the control group and meet the preset improvement conditions; determining that the updated knowledge graph has failed verification when the performance indicators of the experimental group do not meet the preset improvement conditions, and rolling back the target knowledge graph to the version before the update.

[0014] Furthermore, to achieve the above objectives, this application also proposes a knowledge graph multi-source feedback closed-loop update device, which includes: a data acquisition module, a data preprocessing module, a defect diagnosis module, and a knowledge graph repair module. Specifically, The data acquisition module is used to acquire multi-source feedback data generated when downstream applications interact with the target knowledge graph; The data preprocessing module is used to process the multi-source feedback data output by the feedback data acquisition module in a structured manner according to a preset data template to obtain structured feedback data. The defect diagnosis module is used to perform preliminary defect diagnosis on structured feedback data based on pre-set defect matching rules, and input structured feedback data that is not matched by the defect matching rules into the anomaly detection model for in-depth defect diagnosis. Based on the preliminary diagnosis results and the in-depth diagnosis results, knowledge defect events are obtained. The knowledge graph repair module is used to generate knowledge graph repair work orders based on the knowledge defect events output by the defect diagnosis module; to review the knowledge update work orders output by the work order generation module and generate knowledge graph update operation content based on the reviewed knowledge update work orders; and to repair and update the target knowledge graph based on the update operation content output by the work order review module to obtain the updated knowledge graph.

[0015] Furthermore, to achieve the above objectives, this application also proposes a knowledge graph multi-source feedback closed-loop update device, which includes: a memory, a processor, and a knowledge graph multi-source feedback closed-loop update program stored in the memory and executable on the processor. The knowledge graph multi-source feedback closed-loop update program is configured to implement the steps of the knowledge graph multi-source feedback closed-loop update method described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium storing a knowledge graph multi-source feedback closed-loop update program, which, when executed by a processor, implements the steps of the knowledge graph multi-source feedback closed-loop update method described above.

[0017] The one or more technical solutions proposed in this application have at least the following technical effects: By acquiring multi-source feedback data generated by downstream application interactions, the update trigger signal directly originates from the actual application scenario, overcoming the problem of delayed discovery of update requirements. By structuring the multi-source feedback data, heterogeneous data is converted into a standard structure, avoiding diagnostic blind spots caused by format differences. A complementary approach of rule matching and machine learning models is adopted for defect diagnosis, achieving rapid capture of typical defects and deep identification of hidden defects, reducing the risk of missed detections. Repair work orders are generated based on knowledge defect events, transforming diagnostic results into structured repair tasks, reducing the workload of manual sorting and positioning. Work orders are reviewed and update operation content is generated, introducing manual oversight on top of automated processing to avoid erroneous updates. Repair updates are executed according to the update operation content, enabling orderly closed-loop repair of knowledge defects and ensuring the consistency between the knowledge graph and downstream application requirements. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the knowledge graph multi-source feedback closed-loop update method provided in this application embodiment. Figure 1 ; Figure 2 A flowchart illustrating the knowledge graph multi-source feedback closed-loop update method provided in this application embodiment. Figure 2 ; Figure 3 A flowchart illustrating the knowledge graph multi-source feedback closed-loop update method provided in this application embodiment. Figure 3 ; Figure 4 This is a schematic diagram of the structure of the knowledge graph multi-source feedback closed-loop update device provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the knowledge graph multi-source feedback closed-loop update device provided in the embodiments of this application. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] As described in the background section, how to utilize multi-source feedback data generated by downstream applications to drive error detection and updates of knowledge graphs is a pressing technical problem that needs to be solved.

[0024] In view of this, this application provides a multi-source feedback closed-loop update method for knowledge graphs. By acquiring multi-source feedback data generated by downstream application interactions, the update trigger signal directly originates from the actual application scenario, overcoming the problem of delayed discovery of update requirements. By structuring the multi-source feedback data, heterogeneous data is converted into a standard structure, avoiding diagnostic blind spots caused by format differences. A complementary approach of rule matching and machine learning models is used for defect diagnosis, achieving rapid capture of typical defects and deep identification of hidden defects, reducing the risk of missed detections. Repair work orders are generated based on knowledge defect events, transforming diagnostic results into structured repair tasks, reducing the workload of manual sorting and positioning. The work orders are reviewed and update operation content is generated, introducing manual oversight on top of automated processing to avoid erroneous updates. Repair updates are executed according to the update operation content, enabling orderly closed-loop repair of knowledge defects and ensuring consistency between the knowledge graph and downstream application requirements.

[0025] The execution subject of this application embodiment can be a knowledge graph multi-source feedback closed-loop update device, or a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a thermal management device capable of realizing the above functions of a knowledge graph multi-source feedback closed-loop update device, etc. This embodiment does not specifically limit it in this way. The following uses a knowledge graph multi-source feedback closed-loop update device as the execution subject as an example to describe this embodiment and the following embodiments.

[0026] Step S10: Obtain multi-source feedback data generated when downstream applications interact with the target knowledge graph.

[0027] It can be explained that downstream applications refer to business systems that call upon the target knowledge graph to provide services, specifically including intelligent question-answering systems, simulation training and evaluation systems, and fault diagnosis assistance systems. The target knowledge graph refers to a structured semantic knowledge base that needs to be maintained and updated. Multi-source feedback data refers to a series of problematic signals reflecting knowledge quality generated by downstream applications during operation due to the use of the target knowledge graph. Examples include missed answers in intelligent question-answering systems (where users did not receive satisfactory answers), low satisfaction records where users rated returned answers below preset standards, and statistical records of a high collective error rate for a specific knowledge point in trainee assessments in simulation training and evaluation systems.

[0028] Understandably, by continuously acquiring multi-source feedback data from downstream applications, potential defects in the knowledge graph can be proactively exposed in real-world application scenarios. Compared to relying on manual, periodic literature reviews or scheduled batch inspections to discover knowledge gaps, obtaining feedback data from downstream applications can directly capture real knowledge issues occurring in frontline business operations, making the discovery of update needs more timely and targeted. For example, when an intelligent question-answering system repeatedly returns "no answer found" for the same type of user question, this feedback data can serve as an important signal that the target knowledge graph may have gaps in knowledge coverage.

[0029] It should be understood that when acquiring feedback data from multiple sources, attention must be paid to the diversity of data sources and the completeness of coverage. Relying solely on feedback from a single application channel may result in overlooking deficiencies exposed in other application scenarios, causing some knowledge gaps to remain undiscovered for a long time.

[0030] Step S20: The multi-source feedback data is structured according to a preset data template to obtain structured feedback data.

[0031] It can be explained that a data template refers to a predefined set of unified data format specifications, which stipulates the standard fields and data structures that multi-source feedback data must possess before entering subsequent processing. Structured feedback data means that, after template mapping and transformation, feedback data from various sources are expressed with unified field names, data types, and hierarchical structures, eliminating the differences in format between the original data.

[0032] Understandably, feedback data from different downstream applications may vary significantly in recording methods, field naming, and data granularity. Directly analyzing the raw, heterogeneous data would increase processing difficulty and easily lead to omissions. This step uses a pre-defined data template to structure the multi-source feedback data, converting various feedback sources such as miss records from the intelligent question-answering system and error rate statistics from the training system into a standardized structure. This provides a consistent data input foundation for subsequent defect diagnosis, allowing the diagnostic logic to focus on judging the knowledge deficiency itself without having to deal with the adaptation issues of the underlying data format.

[0033] It should be understood that data templates, when designed, should take into account both common fields and unique information from feedback data from different sources. They should ensure complete mapping of core diagnostic fields while also reserving space for extended fields to accommodate the increase in downstream application types or the expansion of feedback data dimensions. In actual processing, abnormal data that cannot fully match the template should be recorded separately and handled in a degraded manner to avoid blocking the entire processing flow due to a small number of data format anomalies.

[0034] Step S30: Based on the structured feedback data, perform a preliminary diagnosis of defects in the structured feedback data using preset defect matching rules, and input the structured feedback data that is not matched by the defect matching rules into the anomaly detection model for in-depth defect diagnosis. Based on the preliminary diagnosis results and the in-depth diagnosis results, obtain the knowledge defect event.

[0035] To clarify, rule matching refers to comparing structured feedback data against pre-defined defect judgment rules. When a quantitative indicator in the feedback data meets the rule triggering condition, the corresponding knowledge defect is directly determined. A machine learning model is an anomaly detection model trained on historical feedback data and known defect annotations, used to identify complex or hidden defect patterns not covered by the rules. A knowledge defect event refers to the system's output of identified knowledge defects as structured events after diagnosing the feedback data, including information such as defect location, defect type, and confidence level.

[0036] Understandably, this step employs a complementary approach of rule matching and machine learning models for defect diagnosis, aiming to balance the accuracy and coverage of defect detection. Rule matching excels at quickly capturing typical defects with clearly defined features; for example, when the error rate of a knowledge point exceeds a preset threshold multiple times during training, it can be directly identified as a suspected knowledge content deviation. Machine learning models, on the other hand, can learn implicit defect patterns from the behavioral statistical features and graph structure features of feedback data, identifying complex anomalies that are difficult to predefine using rules. In practical application, these two approaches are complemented: first, rules provide accurate matching, and then the model performs in-depth analysis, ensuring that defect diagnosis retains both the efficiency and controllability of rules and the discovery capabilities of models.

[0037] It should be understood that the complementary order of rule matching and machine learning models should maintain the execution logic of rules first, then model. This is because rule matching can quickly screen out clear defects, while the model performs in-depth analysis on the remaining ambiguous data, avoiding the model from repeatedly calculating simple defects that have already been clearly identified by rules, thus improving diagnostic efficiency. In addition, the threshold parameters in the rule conditions need to be set reasonably based on the actual business scenario. Too high a threshold may lead to missed detections, while too low a threshold may generate a large number of invalid alarms.

[0038] Step S40: Generate a knowledge graph repair work order based on the knowledge defect event.

[0039] It should be noted that a knowledge graph repair work order refers to transforming knowledge defect events output from the diagnostic process into structured repair task orders that can be manually reviewed and executed. A repair work order must include at least the trigger source, problem description, associated knowledge graph nodes, defect type, and suggested action type, allowing review experts to fully understand the defect situation without having to trace back to the original feedback data.

[0040] Understandably, automatically generating structured remediation work orders from knowledge deficiency events can efficiently transform diagnostic results into actionable remediation tasks. Compared to manually sorting through and locating remediation needs from scattered feedback records, automatically generating work orders reduces the workload of experts in information collection and problem identification, allowing them to directly enter the review and decision-making stages, thus improving the overall efficiency of update processing.

[0041] It should be understood that the process of generating repair work orders should ensure the completeness and accuracy of information transmission, avoiding the need for experts to repeatedly query original data during review due to missing information. The recommended operation type in the work order should be matched from a pre-set repair strategy library based on the defect type, rather than relying entirely on real-time inference, in order to maintain the consistency and reliability of the repair recommendations.

[0042] Step S50: Review the knowledge update work order and generate the knowledge graph update operation content based on the reviewed knowledge update work order.

[0043] It can be explained that review refers to the process by which domain experts manually confirm, correct, supplement, or reject automatically generated knowledge update work orders. Update operations refer to the set of atomic operation instructions that, after expert review and confirmation, are transformed into executable elements of the target knowledge graph. These specifically include operations such as modifying entity node attribute values, adding or deleting relation edges, and replacing triples.

[0044] Understandably, this step introduces a manual review process on top of the automatically generated repair work orders. The aim is to strike a balance between the efficiency of automated processing and the accuracy of knowledge updates. While automated diagnosis and work order generation can significantly improve efficiency, the accuracy of the knowledge graph directly impacts the service quality of downstream applications; erroneous knowledge updates can lead to more serious consequences. Through expert review, the correctness of defect location and the rationality of suggested actions in the work orders can be further verified and corrected, preventing erroneous updates from polluting the knowledge graph.

[0045] It should be understood that the review process should be designed to allow experts flexible handling of work orders, including multiple processing methods such as confirmation and approval, approval after modification, and rejection with explanations. Rejected work orders and the reasons for rejection should be recorded as important feedback data for subsequent optimization of defect diagnosis rules and models.

[0046] In one embodiment, the above-mentioned review of the knowledge update work order and generation of knowledge graph update operation content based on the reviewed knowledge update work order includes: determining the entity nodes or relation edges to be updated in the target knowledge graph based on the defect knowledge location and repair processing suggestions carried in the reviewed knowledge update work order; generating corresponding graph atomic operation instructions based on the repair operation type indicated in the repair processing suggestions, wherein the types of graph atomic operation instructions include entity attribute value modification, relation edge addition / deletion, or triple replacement; and encapsulating the graph atomic operation instructions into update operation content executable by the target knowledge graph.

[0047] It can be explained that defect knowledge localization refers to the specific location identifier in the target knowledge graph that needs to be repaired, recorded in a knowledge defect event. This could be a unique identifier for an entity node or the start and end nodes and relation type of a relation edge. Repair suggestions refer to the repair direction and specific measures specified in the work order after expert review and confirmation. Graph atomic operation instructions refer to the smallest unit of operation executed on the knowledge graph, and their operation types include modifying entity attribute values, adding or deleting relation edges, or replacing triples.

[0048] Understandably, translating the reviewed repair suggestions into graph atomic operation instructions is to ensure that update operations can be executed accurately and securely within the knowledge graph database. Atomic operation instructions directly correspond to the data manipulation primitives of the knowledge graph, avoiding the secondary conversion deviation between natural language suggestions and actual execution, thus ensuring that the update action remains consistent with the repair intent confirmed by experts.

[0049] It should be understood that when mapping repair operation types to atomic operation instructions, the validity of the operation object must be verified. For example, performing attribute modification on a non-existent entity node or repeatedly adding existing constraints should be subject to validity checks before encapsulation to avoid generating invalid or conflicting operation instructions.

[0050] Step S60: Repair and update the target knowledge graph according to the update operation content to obtain the updated knowledge graph.

[0051] To clarify, repair and update refers to executing verified and legitimate update operations line by line on the target knowledge graph, actually correcting any deficient entities, attributes, or relationships. A completed knowledge graph refers to a version of the graph where the original knowledge defects have been corrected or supplemented after the repair process.

[0052] Understandably, this step outputs the complete process of diagnosis, work order generation, and review confirmation, ultimately ensuring its implementation and creating a closed loop from front-end awareness of knowledge deficiencies to back-end repair. Each update has clear operational guidelines and review records, ensuring that every change to the knowledge graph is traceable and auditable, thus guaranteeing the long-term reliable operation of the knowledge base.

[0053] It should be understood that atomicity should be guaranteed when performing update operations, meaning that each update instruction should either be executed completely successfully or fail entirely, to avoid inconsistencies in the knowledge graph caused by partial execution. If an anomaly occurs during the update process, a failure log should be recorded, and the graph versions before and after the repair should be retained for backtracking.

[0054] In one embodiment, the above-mentioned knowledge graph multi-source feedback closed-loop update method further includes: extracting the interaction questions corresponding to the feedback data generated by the knowledge defect event, and constructing a test question set; dividing users accessing the target knowledge graph into a control group and an experimental group, with the control group using the knowledge graph before the update and the experimental group using the updated knowledge graph; obtaining the performance indicators of the control group and the experimental group for the test question set within a preset test period; determining that the updated knowledge graph has passed verification when the performance indicators of the experimental group are better than those of the control group and meet the preset improvement conditions; determining that the updated knowledge graph has failed verification when the performance indicators of the experimental group do not meet the preset improvement conditions, and rolling back the target knowledge graph to the version before the update.

[0055] It should be noted that the test question set refers to the set of interactive questions extracted from the original feedback data that triggered this update, used to verify the effectiveness of the update. The control group refers to the user group that continues to use the pre-update version of the knowledge graph in downstream applications. The experimental group refers to the user group that uses the updated knowledge graph. Performance metrics refer to quantitative parameters used to measure the service quality of the knowledge graph in downstream applications. For example, in an intelligent question-answering system, these may include the question resolution rate and user satisfaction rating; in a simulated training evaluation system, they may include the group error rate of knowledge points.

[0056] Understandably, this embodiment introduces an effectiveness verification step after the update is completed. The purpose is to confirm whether the update truly resolved the original problem that triggered the update. By applying the knowledge graphs before and after the update to the control and experimental groups respectively, and comparing the differences in performance metrics under the same test question set, the effectiveness of the update can be verified quantitatively. When the experimental group performs significantly better than the control group, it indicates that the update has achieved its intended purpose; otherwise, it suggests that the update may not have resolved the original problem or has introduced new problems. In this case, an automatic rollback to the previous version is performed to ensure that the service quality of downstream applications is not affected.

[0057] It should be understood that the preset test period and enhancement conditions for rollback testing should be reasonably set according to the actual business scenario. A period that is too short may result in insufficient statistical samples, affecting the accuracy of judgment; conditions that are too strict may cause valid updates to be mistakenly judged as invalid. Rollback operations must also adhere to atomicity requirements to ensure that the knowledge graph can be completely restored to its state before the update and continue to provide services.

[0058] This embodiment acquires multi-source feedback data generated by the interaction between downstream applications and the target knowledge graph. The data is then structured according to a preset template to obtain standard structured feedback data. A complementary approach of rule matching and machine learning models is used for defect diagnosis and to generate knowledge defect events. Repair work orders are automatically generated based on these defect events. After expert review, update operation content is generated, and the repair update is finally executed. Following the update, the effectiveness is verified by dividing the system into a control group and an experimental group; if the results are not satisfactory, the system automatically rolls back.

[0059] In summary, this method overcomes the problem of delayed update requirement discovery in traditional methods by collecting multi-source feedback data from downstream applications in real time, ensuring that update trigger signals originate directly from actual business scenarios. Through complementary diagnostics of rule matching and machine learning models, it achieves rapid capture of typical defects and deep identification of hidden defects. By combining automatic work order generation with manual review, it ensures update efficiency while preventing erroneous updates from polluting the knowledge graph. Through post-update effect verification and automatic rollback mechanisms, it ensures that each update positively impacts downstream applications. Ultimately, it achieves accurate and efficient closed-loop updates to the knowledge graph, guaranteeing continuous consistency between knowledge base content and business requirements.

[0060] Combination Figure 1 Please refer to Figure 2 In the knowledge graph multi-source feedback closed-loop update method provided in this application embodiment, the above step S30 includes steps S301 to S303: Step S301: Match the structured feedback data according to the preset defect matching rules, and determine the first type of defect event of the target knowledge graph based on the matching results.

[0061] It should be noted that the preset defect matching rules refer to a set of judgment conditions predefined by domain experts and operations personnel based on common knowledge defect patterns. Each rule consists of a trigger condition and a corresponding defect type. The first type of defect event refers to a knowledge defect event that is directly determined through rule matching, and its defect type and defect location are explicitly given by the rule conditions.

[0062] Understandably, rule matching, as the first stage of defect diagnosis, aims to quickly and accurately capture typical defects with clear characteristics and mature judgment conditions. Since the rules are predefined and validated, once feedback data matches the rule conditions, a defect conclusion can be directly drawn without complex model reasoning. For example, setting a rule "a certain knowledge point has a group error rate exceeding X% for N consecutive times in simulated training," when the statistical result of a certain feedback data meets this condition, it can be directly determined that the knowledge point has a suspected content deviation defect. Rule matching is characterized by fast response speed and strong interpretability of results.

[0063] It should be understood that the preset defect matching rules need to be reviewed and updated regularly to adapt to changes in business scenarios and the emergence of new types of defects. The quantification thresholds of the rule conditions should be calibrated based on the data distribution of the actual application to avoid missed detections due to overly strict thresholds or frequent false alarms due to overly lenient thresholds.

[0064] In a specific embodiment, the above-mentioned matching of structured feedback data based on preset defect matching rules to determine the first type of defect event in the target knowledge graph includes steps A10 to A50: Step A10: Based on the knowledge point identifiers and quantitative performance indicators carried in the structured feedback data, statistically analyze the interaction result performance data of the knowledge points corresponding to the feedback data in the target knowledge graph.

[0065] It can be explained that a knowledge point identifier refers to a unique identifier recorded in the feedback data that corresponds to a knowledge point node in the target knowledge graph. Quantitative performance indicators refer to application performance parameters recorded in numerical form in the feedback data, such as the satisfaction score of a single question-and-answer session or the correctness of a single training assessment. Interaction result performance data refers to the comprehensive performance parameters obtained by summarizing and statistically analyzing the quantitative performance indicators of all feedback data for a specific knowledge point within a preset statistical period, such as the average accuracy of answers or the cumulative number of errors for that knowledge point.

[0066] Understandably, this step aggregates and statistically analyzes scattered feedback data according to knowledge point identifiers. The aim is to transform discrete, single feedback responses into performance data that reflects the overall quality status of the knowledge point. Taking a simulated training assessment system as an example, the answers to a certain knowledge point in multiple assessments are scattered across different feedback records. By aggregating and statistically analyzing the cumulative number of correct and incorrect responses according to the knowledge point identifier, the accuracy level of that knowledge point can be obtained, providing a data foundation for subsequent quality threshold determination.

[0067] Step A20: When the accuracy of the interactive result data is lower than the first quality threshold, it is determined that there is a knowledge content deviation defect in the corresponding knowledge point in the target knowledge graph.

[0068] It should be noted that the first quality threshold refers to the minimum permissible standard for judging whether the accuracy of knowledge point content is acceptable. Knowledge content deviation defects refer to discrepancies between the content recorded for a certain knowledge point in the target knowledge graph and actual facts or the latest procedures, leading downstream applications to provide incorrect answers or cause misleading information when providing services based on that knowledge point.

[0069] Understandably, when the accuracy of the interaction results for a certain knowledge point consistently falls below the first quality threshold, it indicates that the answers given based on the current knowledge graph content are frequently inconsistent with the correct results, suggesting that the content of that knowledge point may be biased. For example, in an intelligent question-answering scenario, if the accuracy of the answer regarding a device operation specification is consistently low, it may mean that the version of the specification recorded in the knowledge graph is outdated. By setting clear quality thresholds and automatically determining them, such content defects can be captured in a timely manner.

[0070] It should be understood that the setting of the first quality threshold needs to be differentiated based on the business importance and tolerable error range of different knowledge points. A higher threshold can be set for key knowledge such as safety operation procedures, while a more lenient threshold can be set for routine knowledge.

[0071] Step A30: Based on the knowledge point identifiers and satisfaction-related fields carried in the structured feedback data, calculate the quality score of the query results for the corresponding knowledge points in the structured feedback data of the target knowledge graph.

[0072] It can be explained that satisfaction-related fields refer to fields in the feedback data that reflect the user's satisfaction with the knowledge graph service results, such as user ratings of the returned answers, whether the user continues to ask follow-up questions, or exits directly. The query result quality score is a score calculated by combining multiple satisfaction-related fields, used to characterize the service quality of the knowledge graph for a query request for a specific knowledge point.

[0073] Understandably, satisfaction-related fields can reflect the service quality of a knowledge graph from the perspective of user behavior. For example, if a user directly accepts the system's returned answer and ends the conversation after asking a question, it usually indicates high satisfaction; while if a user repeatedly asks follow-up questions or gives a low rating, it indicates that the returned results fail to meet their needs. By statistically analyzing the satisfaction-related fields of a knowledge point over a period of time and calculating the quality score of the query results, the service coverage capability of that knowledge point can be quantitatively evaluated.

[0074] Step A40: When the frequency of occurrence of query result quality scores below the second quality threshold reaches a preset frequency threshold, it is determined that the corresponding knowledge point in the target knowledge graph has a knowledge coverage deficiency defect.

[0075] It should be noted that the second quality threshold refers to the minimum acceptable standard for the quality score of the query results. The preset frequency threshold refers to the upper limit of the cumulative number of times the quality score is lower than the second quality threshold. Knowledge coverage gap deficiency refers to the insufficient coverage of a certain knowledge point in the target knowledge graph, which fails to meet the diverse query needs of users for that knowledge point in downstream applications.

[0076] Understandably, unlike knowledge content bias defects, knowledge coverage gap defects focus on the breadth and completeness of knowledge points. A knowledge point may be correct in content, but due to insufficient coverage, some reasonable queries may fail to yield satisfactory answers. For example, if a knowledge point on equipment fault diagnosis only covers common fault types, and queries for intermittent faults consistently return low-quality results, it indicates a coverage gap in that knowledge point. By monitoring the frequency of quality scores falling below a threshold, an alert can be triggered when the coverage gap problem accumulates to a certain severity.

[0077] It should be understood that the setting of the preset frequency threshold should distinguish between occasional low quality and continuous insufficient coverage. A small number of occasional low quality may be due to the special way users ask questions, while frequent occurrences likely reflect a gap in the coverage of the knowledge points themselves.

[0078] Step A50: Summarize the knowledge points in the target knowledge graph that have knowledge content deviation defects and knowledge points that have knowledge coverage gap defects to obtain the first type of defect event.

[0079] It can be explained that the first type of defect event refers to the complete set of the two types of defect events determined by rule matching mentioned above. Each event includes defect knowledge location, defect type, and quantitative basis for triggering the determination.

[0080] Understandably, the purpose of aggregating the two types of defects into a unified first type of defect event set is to pass all diagnostic results from the rule matching stage to the subsequent processing stage in a consistent data structure, so that subsequent model diagnosis and work order generation can be processed uniformly on this basis.

[0081] It should be understood that the original triggering data source and quantitative indicators of each defect event should be retained when summarizing, so as to provide sufficient evidence support and expert review reference when generating subsequent work orders.

[0082] Step S302: Input the structured feedback data that is not matched by the preset defect matching rules into the anomaly detection model for identification, and determine the second type of defect event of the target knowledge graph based on the output of the anomaly detection model.

[0083] It can be explained that the feedback data not matched by the rules refers to the remaining feedback data that was not matched by any preset rules after the rule matching in step S301. The anomaly detection model is a machine learning model trained based on historical feedback data and known defect-labeled samples, used to identify complex or hidden defect patterns that cannot be covered by the rules. The second type of defect event refers to the knowledge defect event identified and output by the anomaly detection model.

[0084] Understandably, while pre-defined rules can accurately capture known typical defects, their coverage is limited for newly emerging, complex, or hidden defects caused by multiple intertwined factors. Handing over feedback data that is not matched by rules to an anomaly detection model leverages the model's ability to learn hidden defect patterns from high-dimensional features, effectively supplementing rule-based diagnosis, making overall defect detection more comprehensive, and reducing the risk of missed detections.

[0085] It should be understood that anomaly detection models need to be trained on a sufficient number of high-quality labeled samples. After the model goes online, feedback from expert review should be continuously collected as incremental training data so that its defect identification capability can be gradually improved as the system runs.

[0086] In a specific embodiment, the above-mentioned input of feedback data that is not matched by the rule into the anomaly detection model for identification, to determine the second type of defect event of the target knowledge graph, includes steps B10 to B40: Step B10: Extract multi-dimensional features based on the structured feedback data that is not matched by the rules.

[0087] It can be explained that multi-dimensional features include at least the behavioral statistical features of feedback and the structural features of related nodes in the target knowledge graph. The behavioral statistical features of feedback refer to the statistics extracted from the feedback data that reflect user behavior, such as the frequency of similar feedback, the concentration of feedback time distribution, and the number of users involved in the feedback. The structural features of related nodes refer to the topological structure information of the knowledge points corresponding to the feedback data in the target knowledge graph, such as the number of associations of the node, attribute completeness, and the activity level of upstream and downstream nodes.

[0088] Understandably, a single piece of feedback data may not be sufficient to directly expose knowledge deficiencies, but a combination of multi-dimensional features can reveal potential anomaly patterns. For example, if the frequency of user feedback on a certain knowledge point suddenly increases recently but satisfaction decreases, and at the same time, this node has many missing attribute fields in the knowledge graph, the combination of these features may suggest that the knowledge point has incomplete or outdated content. The purpose of extracting multi-dimensional features is to transform the raw feedback data into a high-dimensional vector representation that the model can recognize, enabling the model to comprehensively consider information from multiple perspectives to make anomaly judgments.

[0089] It should be understood that the selection of feature dimensions should strike a balance between coverage and computational cost. Too many redundant features will increase the overhead of model training and inference, while too few may miss key anomaly signals. Feature selection should be determined by combining domain knowledge and analysis of historical defect cases.

[0090] Step B20: Input the multi-dimensional features into the anomaly detection model to obtain the confidence level of the target knowledge graph having defects, as well as the probability distribution under each preset defect type.

[0091] To clarify, confidence level refers to the probability that the anomaly detection model judges the knowledge point corresponding to the current feedback data to have a defect, and its value is usually between 0 and 1. Preset defect types refer to the set of defect categories that the model can identify, such as contradictory knowledge content, outdated knowledge, and incomplete knowledge. Probability distribution refers to the distribution of the model's predicted probability values ​​for input samples under each preset defect type.

[0092] Understandably, anomaly detection models calculate the confidence level of a defect's existence and the probability distribution of defect types by analyzing the multi-dimensional features of the input, providing a quantitative basis for subsequent defect judgment. Higher confidence levels indicate that the model is more certain that the knowledge point contains a defect. The probability distribution of each defect type further indicates the specific nature of the defect; for example, the highest probability being "knowledge outdated" means the model believes that the knowledge point is most likely outdated.

[0093] It should be understood that the model's prediction results are affected by the quality of the training data and the model structure. High confidence does not mean absolute correctness. Therefore, the model's output results still need to be manually confirmed in the subsequent work order review process, rather than being directly used as the final basis for update execution.

[0094] Step B30: When the confidence level exceeds the preset model threshold, the defect knowledge location is determined by the knowledge point identifier corresponding to the feedback data that is not matched by the preset defect matching rules, and the defect type with the highest probability in the probability distribution is taken as the actual defect type.

[0095] It can be explained that the preset model threshold refers to the lowest threshold value used to determine whether the confidence level of the model output meets the standard. Defect knowledge localization refers to determining the specific node location of the defect in the knowledge graph based on the knowledge point identifier carried by the feedback data that triggers defect judgment. The actual defect type refers to the defect category with the highest probability value in the probability distribution.

[0096] Understandably, the purpose of setting a preset model threshold is to filter out judgments with low confidence from the model, and only convert defect judgments with high model confidence into formal second-type defect events. Using the defect type with the highest probability as the actual defect type means selecting the most likely one from the multiple possibilities predicted by the model as the final judgment, ensuring that subsequent remediation operations are targeted.

[0097] It should be understood that the setting of preset model thresholds needs to balance recall and precision. A higher threshold can reduce false positives but may miss some real defects, while a lower threshold has the opposite effect. In practical applications, the threshold can be flexibly adjusted according to the business's tolerance for false positives and false negatives.

[0098] Step B40: Based on the actual defect type, the corresponding defect confidence level, defect knowledge localization, and the original interaction records corresponding to the feedback data that was not matched by the preset defect matching rules, determine the second type of defect event.

[0099] It can be noted that the structure of the second type of defect event is consistent with that of the first type of defect event, including information such as defect knowledge location, defect type and confidence level. The difference is that the defect type it labels comes from model prediction rather than rule determination.

[0100] Understandably, encapsulating the model-identified results into defect events with the same standard structure as rule-based diagnosis enables defect events from both sources to be processed uniformly and without distinction in the subsequent work order generation process, thus simplifying the complexity of process design.

[0101] It should be understood that the confidence level in the second type of defect event can be used as one of the reference criteria for prioritizing work orders. Defect events with higher confidence levels can be prioritized for generating work orders and pushing them for review, so as to improve the processing efficiency of high-probability defects.

[0102] Step S303: Summarize the first type of defect events and the second type of defect events to obtain knowledge defect events.

[0103] It can be explained that knowledge defect events refer to the complete set of defect events after merging the first type of defect events output by the rule matching stage and the second type of defect events output by the anomaly detection model stage.

[0104] Understandably, this step summarizes the outputs of the two diagnostic phases to obtain all knowledge defect events after the current round of feedback data has undergone complete diagnosis. Both types of defect events maintain a consistent data format and are then aggregated and uniformly entered into the work order generation stage. This ensures that both explicit defects discovered by rules and hidden defects discovered by the model are included in the remediation process, forming complete defect discovery coverage.

[0105] It should be understood that if the same knowledge point is judged as defective by both rules and models during the summary process, the rule judgment result should prevail, because the judgment conditions of the rules have been manually verified and have higher certainty, thus avoiding the repeated generation of multiple repair work orders for the same knowledge point.

[0106] In this embodiment, standard structure feedback data is matched using preset defect matching rules. Based on knowledge point identifiers and quantitative performance indicators, the interaction result performance data is statistically analyzed. When the accuracy is lower than the first quality threshold, it is determined to be a knowledge content deviation defect. When the frequency of query result quality scores being lower than the second quality threshold reaches a preset frequency threshold, it is determined to be a knowledge coverage gap defect. These are then aggregated to form the first type of defect event. Feedback data that is not matched by the rules is input into the anomaly detection model. After extracting multi-dimensional features, the defect confidence and probability distribution of each defect type are obtained. When the confidence exceeds the preset model threshold, the defect type with the highest probability is determined to be the second type of defect event. Finally, the two types of defect events are aggregated to obtain a complete set of knowledge defect events.

[0107] In summary, this embodiment achieves a balance between the breadth and accuracy of defect discovery by using a phased complementary diagnosis of rule matching and machine learning models. First, the rules accurately capture typical defects, and then the model deeply mines hidden anomalies. The results of rule judgment and model recognition are summarized and output in a unified defect event format to ensure the consistency of processing in the subsequent work order generation process, so that different types of knowledge defects can be included in the closed-loop repair process.

[0108] In combination with the above Figure 1 Please refer to Figure 3 The above step S40 includes steps S401 to S404: Step S401: Based on the location of defective knowledge in the knowledge defect event, determine the associated graph nodes from the target knowledge graph.

[0109] It can be explained that defect knowledge localization refers to the identification of problematic knowledge points recorded in the knowledge defect event within the target knowledge graph. Related graph nodes refer to adjacent nodes in the graph that have a direct relationship with the defective knowledge point, including superior nodes that have attribute affiliation relationships with the defective entity and sibling nodes with relational edges.

[0110] Understandably, knowledge points in a knowledge graph do not exist in isolation; a defect in one knowledge point often relates to its adjacent nodes. For example, if an attribute value of a device entity is incorrect, fault phenomenon nodes and repair solution nodes related to that device may also be affected. This step identifies related graph nodes from the target knowledge graph. Its purpose is to provide complete contextual node information for subsequent repair work orders, enabling experts to comprehensively assess the scope of the defect's impact and the impact of the repair solution on related nodes during review.

[0111] It should be understood that the determination of associated nodes in the graph should control the scope of association, generally limited to directly adjacent one-hop nodes, to avoid excessive association scope leading to redundancy of work order information and affecting the efficiency of expert review.

[0112] Step S402: Based on the defect type in the knowledge defect event, match the corresponding suggested operation type and candidate update content from the preset repair strategy library.

[0113] It should be noted that the preset repair strategy library refers to a knowledge base that pre-configures common defect types with their corresponding repair operation types and candidate update content templates. The suggested operation type refers to the recommended graph repair operation category for that defect type, such as adding entities, modifying attribute values, adding relationship edges, and deleting outdated entries. Candidate update content refers to the preliminary draft repair content automatically generated by the system based on the repair strategy library and historical similar cases.

[0114] Understandably, pre-matching defect types with remediation strategies can provide valuable remediation suggestions for expert review, reducing the cognitive burden on experts to conceive remediation solutions from scratch. For example, when the defect type is "knowledge content deviation," the system automatically matches the suggested operation "modify attribute value" and generates candidate update content. Experts only need to confirm, correct, or supplement the content, without having to write remediation instructions from scratch.

[0115] It should be understood that the repair strategy library should be continuously updated and refined based on the defect repair experience accumulated in actual business, so that the accuracy of matching suggested operation types and candidate update content gradually improves as the system is used for a longer period of time.

[0116] Step S403: Determine the trigger source field of the knowledge graph repair work order based on the trigger source information carried by the knowledge defect event.

[0117] It should be noted that "trigger source information" refers to the source identifier of the original feedback data recorded in a knowledge defect event that triggered the defect determination, including the source system name and feedback data type. The "trigger source field" is a formatted field in the repair work order used to indicate the defect discovery channel.

[0118] Understandably, explicitly recording the trigger source field in the remediation ticket aims to allow review experts to clearly understand which downstream application the defect originated from, facilitating the assessment of its urgency and scope of impact. For example, a defect with a high miss rate from an intelligent question-answering system may have a different remediation priority than a defect with an occasionally high error rate from a training system. The trigger source field also provides a traceable basis for subsequent effectiveness verification.

[0119] It should be understood that when the same defect event is triggered by feedback data from multiple downstream applications, the trigger source field should record all source information in full to comprehensively reflect the exposure path of the defect.

[0120] Step S404: Encapsulate the trigger source field, the problem description corresponding to the defect type, the associated graph node, the defect type, the suggested operation type, and the candidate update content into structured work order data to obtain the knowledge graph repair work order.

[0121] It should be noted that the problem description corresponding to the defect type refers to a natural language summary automatically generated based on the defect type, which concisely describes the manifestation and possible causes of the defect. Structured work order data refers to the data object obtained by organizing and encapsulating the above fields according to a preset work order template format.

[0122] Understandably, encapsulating multiple pieces of information, such as the trigger source, problem description, related graph nodes, defect type, suggested operation type, and candidate update content, into a structured repair work order allows experts to obtain the complete background, diagnostic conclusion, and repair suggestions for the defect at once on the review platform, without having to switch between multiple systems, thus significantly improving review efficiency.

[0123] It should be understood that the fields in the work order should maintain consistency and accuracy of information. In particular, the candidate update content should come from the matching results of the repair strategy library rather than temporary inferences, so as to avoid increasing the workload of expert correction due to the low quality of candidate content.

[0124] In this embodiment, the associated graph nodes are determined from the target knowledge graph based on the defect knowledge location in the knowledge defect event. The suggested operation type and candidate update content are matched from the preset repair strategy library according to the defect type. The trigger source field of the work order is determined according to the trigger source information. Finally, the trigger source field, problem description, associated graph nodes, defect type, suggested operation type and candidate update content are encapsulated into structured work order data to generate a knowledge graph repair work order.

[0125] In summary, this application's embodiments automatically transform diagnostic output knowledge defect events into structured repair work orders containing complete contextual information, enabling experts to quickly understand the full picture of the defect and make decisions based on system recommendations during review. This reduces the workload of manually sorting out and locating repair needs from scattered feedback data, achieving efficient connection from defect discovery to repair task generation, and providing a standardized task carrier for the closed-loop update process of the knowledge graph.

[0126] This application also provides a knowledge graph multi-source feedback closed-loop update device. Please refer to... Figure 4 The knowledge graph multi-source feedback closed-loop update device includes: Data acquisition module 10 is used to acquire multi-source feedback data generated when downstream applications interact with the target knowledge graph; The data preprocessing module 20 is used to process the multi-source feedback data output by the feedback data acquisition module in a structured manner according to a preset data template to obtain structured feedback data. The defect diagnosis module 30 is used to diagnose defects based on the structured feedback data output by the data structuring module, and to obtain knowledge defect events by using a complementary approach of rule matching and machine learning models. The knowledge graph repair module 40 is used to generate knowledge graph repair work orders based on the knowledge defect events output by the defect diagnosis module; to review the knowledge update work orders output by the work order generation module and generate knowledge graph update operation content based on the reviewed knowledge update work orders; and to repair and update the target knowledge graph based on the update operation content output by the work order review module to obtain the updated knowledge graph.

[0127] The knowledge graph multi-source feedback closed-loop update device provided in this application, employing the knowledge graph multi-source feedback closed-loop update method described in the above embodiments, can solve the technical problem of how to use multi-source feedback data generated by downstream applications to drive error detection and updating of the knowledge graph. Compared with the prior art, the beneficial effects of the knowledge graph multi-source feedback closed-loop update device provided in this application are the same as those of the knowledge graph multi-source feedback closed-loop update method described in the above embodiments, and other technical features in this knowledge graph multi-source feedback closed-loop update device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0128] This application provides a knowledge graph multi-source feedback closed-loop update device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the knowledge graph multi-source feedback closed-loop update method in the first embodiment described above.

[0129] The following is for reference. Figure 5 This document illustrates a structural schematic diagram of a knowledge graph multi-source feedback closed-loop update device suitable for implementing embodiments of this application. The knowledge graph multi-source feedback closed-loop update device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The knowledge graph multi-source feedback closed-loop update device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0130] like Figure 5 As shown, the knowledge graph multi-source feedback closed-loop update device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the knowledge graph multi-source feedback closed-loop update device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the knowledge graph multi-source feedback closed-loop update device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows a knowledge graph multi-source feedback closed-loop update device with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0131] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0132] The knowledge graph multi-source feedback closed-loop update device provided in this application, employing the knowledge graph multi-source feedback closed-loop update method described in the above embodiments, can solve the technical problem of how to use multi-source feedback data generated by downstream applications to drive error detection and updating of the knowledge graph. Compared with the prior art, the beneficial effects of the knowledge graph multi-source feedback closed-loop update device provided in this application are the same as those of the knowledge graph multi-source feedback closed-loop update method provided in the above embodiments, and other technical features in this knowledge graph multi-source feedback closed-loop update device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0133] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0134] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0135] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the knowledge graph multi-source feedback closed-loop update method in the above embodiments.

[0136] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0137] The aforementioned computer-readable storage medium may be included in the knowledge graph multi-source feedback closed-loop update device; or it may exist independently and not be assembled into the knowledge graph multi-source feedback closed-loop update device.

[0138] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the knowledge graph multi-source feedback closed-loop update device, the knowledge graph multi-source feedback closed-loop update device performs the following actions: acquires multi-source feedback data generated when downstream applications interact with the target knowledge graph; performs structured processing on the multi-source feedback data according to a preset data template to obtain structured feedback data; performs defect diagnosis based on the structured feedback data using a complementary approach of rule matching and machine learning models to obtain knowledge defect events; generates knowledge graph repair work orders based on the knowledge defect events; reviews the knowledge update work orders and generates knowledge graph update operation content based on the reviewed knowledge update work orders; and repairs and updates the target knowledge graph according to the update operation content to obtain the updated knowledge graph.

[0139] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0141] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0142] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described knowledge graph multi-source feedback closed-loop update method. This solves the technical problem of how to use multi-source feedback data generated by downstream applications to drive error detection and updates of the knowledge graph. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the knowledge graph multi-source feedback closed-loop update method provided in the above embodiments, and will not be repeated here.

[0143] The computer program product provided in this application can solve the technical problem of multi-source feedback closed-loop update of knowledge graphs. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the multi-source feedback closed-loop update method of knowledge graphs provided in the above embodiments, and will not be repeated here.

[0144] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A multi-source feedback closed-loop update method for knowledge graphs, characterized in that, The knowledge graph multi-source feedback closed-loop update method includes: Acquire multi-source feedback data generated when downstream applications interact with the target knowledge graph; The multi-source feedback data is structured according to a preset data template to obtain structured feedback data; Based on the structured feedback data, a preliminary diagnosis of defects is performed on the structured feedback data using preset defect matching rules, and the structured feedback data that is not matched by the defect matching rules is input into the anomaly detection model for in-depth defect diagnosis. Knowledge defect events are obtained based on the preliminary diagnosis results and the in-depth diagnosis results. Based on the knowledge defect events, a knowledge graph repair work order is generated; The knowledge update work order is reviewed, and the knowledge graph update operation content is generated based on the reviewed knowledge update work order. The target knowledge graph is repaired and updated according to the update operation content to obtain the updated knowledge graph.

2. The knowledge graph multi-source feedback closed-loop update method according to claim 1, characterized in that, Based on the structured feedback data, a preliminary diagnosis of defects is performed on the structured feedback data using preset defect matching rules. Structured feedback data not matched by the defect matching rules is then input into an anomaly detection model for in-depth defect diagnosis. Knowledge defect events are obtained based on the preliminary and in-depth diagnosis results, including: The structured feedback data is matched according to a preset defect matching rule, and the first type of defect event of the target knowledge graph is determined based on the matching result. The preset defect matching rule is used to screen typical defects based on statistical indicators in the structured feedback data. Structured feedback data that is not matched by the preset defect matching rules is input into the anomaly detection model for identification. The second type of defect event of the target knowledge graph is determined based on the output of the anomaly detection model. The anomaly detection model is used to extract multi-dimensional features to identify potential defects. The knowledge defect events are obtained by summarizing the first type of defect events and the second type of defect events.

3. The knowledge graph multi-source feedback closed-loop update method according to claim 2, characterized in that, The step of matching the structured feedback data according to preset defect matching rules and determining the first type of defect event of the target knowledge graph based on the matching results includes: Based on the knowledge point identifiers and quantitative performance indicators carried in the structured feedback data, the interaction result performance data of the knowledge points corresponding to the structured feedback data in the target knowledge graph are statistically analyzed. When the accuracy of the interaction result data is lower than the first quality threshold, it is determined that the corresponding knowledge point in the target knowledge graph has a knowledge content deviation defect. Based on the knowledge point identifiers and satisfaction-related fields carried in the structured feedback data, the quality scores of the query results for the corresponding knowledge points in the structured feedback data in the target knowledge graph are calculated. When the frequency of the query result quality score being lower than the second quality threshold reaches a preset frequency threshold, it is determined that the corresponding knowledge point in the target knowledge graph has a knowledge coverage deficiency defect. The first type of defect event is obtained by summarizing the knowledge points in the target knowledge graph that have knowledge content deviation defects and the knowledge points that have knowledge coverage defects.

4. The knowledge graph multi-source feedback closed-loop update method according to claim 2, characterized in that, The step of inputting structured feedback data that is not matched by the preset defect matching rules into the anomaly detection model for identification, and determining the second type of defect event of the target knowledge graph based on the output of the anomaly detection model, includes: Based on the structured feedback data that is not matched by the rule, multi-dimensional features are extracted. The multi-dimensional features include at least the behavioral statistical features of the feedback and the structural features of the associated nodes in the target knowledge graph. The multi-dimensional features are input into the anomaly detection model to obtain the confidence level that the target knowledge graph has defects, as well as the probability distribution under each preset defect type; When the confidence level exceeds the preset model threshold, the defect knowledge location is determined by the knowledge point identifier corresponding to the feedback data that is not hit by the preset defect matching rule, and the defect type with the highest probability in the probability distribution is taken as the actual defect type. The second type of defect event is determined based on the actual defect type, the corresponding defect confidence level, the defect knowledge location, and the original interaction record corresponding to the feedback data that was not matched by the preset defect matching rule.

5. The knowledge graph multi-source feedback closed-loop update method according to claim 1, characterized in that, The step of generating a knowledge graph repair work order based on the knowledge defect event includes: Based on the location of defective knowledge in the aforementioned knowledge defect events, related graph nodes are determined from the target knowledge graph; Based on the defect type in the knowledge defect event, the corresponding suggested operation type and candidate update content are matched from the preset repair strategy library; Based on the trigger source information carried by the knowledge defect event, determine the trigger source field of the knowledge graph repair work order; The trigger source field, the problem description corresponding to the defect type, the associated graph node, the defect type, the suggested operation type, and the candidate update content are encapsulated into structured work order data to obtain the knowledge graph repair work order.

6. The knowledge graph multi-source feedback closed-loop update method according to claim 1, characterized in that, The update operation of generating a knowledge graph based on the reviewed knowledge update work order includes: Based on the defect knowledge location and repair suggestions carried in the reviewed knowledge update work order, determine the entity nodes or relationship edges in the target knowledge graph that need to be updated. Based on the repair operation type indicated in the repair processing suggestion, a corresponding graph atomic operation instruction is generated. The types of graph atomic operation instructions include entity attribute value modification, relation edge addition / deletion, or triple replacement. The atomic operation instructions of the graph are encapsulated into update operations that can be executed by the target knowledge graph.

7. The knowledge graph multi-source feedback closed-loop update method according to any one of claims 1 to 6, characterized in that, The knowledge graph multi-source feedback closed-loop update method also includes: Based on the feedback data generated by the knowledge defect event, extract the corresponding interactive questions from the feedback data and construct a test question set; Users accessing the target knowledge graph are divided into a control group and an experimental group. The control group uses the knowledge graph before the update, while the experimental group uses the updated knowledge graph. Within a preset testing period, the performance indicators of the control group and the experimental group for the test question set are obtained respectively; When the performance indicators of the experimental group are better than those of the control group and meet the preset improvement conditions, the updated knowledge graph is determined to have passed the verification. When the performance indicators of the experimental group do not meet the preset improvement conditions, it is determined that the updated knowledge graph has failed the verification, and the target knowledge graph is rolled back to the version before the update.

8. A knowledge graph multi-source feedback closed-loop update device, characterized in that, The knowledge graph multi-source feedback closed-loop update device includes: The data acquisition module is used to acquire multi-source feedback data generated when downstream applications interact with the target knowledge graph; The data preprocessing module is used to perform structured processing on the multi-source feedback data output by the feedback data acquisition module according to a preset data template to obtain structured feedback data. The defect diagnosis module is used to perform preliminary defect diagnosis on the structured feedback data based on the structured feedback data using preset defect matching rules, and input the structured feedback data that is not matched by the defect matching rules into the anomaly detection model for in-depth defect diagnosis, and obtain knowledge defect events based on the preliminary diagnosis results and in-depth diagnosis results. The knowledge graph repair module is used to generate knowledge graph repair work orders based on the knowledge defect events output by the defect diagnosis module. The knowledge graph repair module is also used to review the knowledge update work orders output by the work order generation module, and generate knowledge graph update operation content based on the reviewed knowledge update work orders. The graph repair module is also used to repair and update the target knowledge graph according to the update operation content output by the work order review module, so as to obtain the updated knowledge graph.

9. A knowledge graph multi-source feedback closed-loop update device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the knowledge graph multi-source feedback closed-loop update method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by the processor, they implement the knowledge graph multi-source feedback closed-loop update method as described in any one of claims 1 to 7.

Images