A Fault Location System and Method Based on Knowledge Graph Data Platform

Abstract

This invention discloses a data platform fault location system and method based on knowledge graphs, belonging to the field of data analysis technology. The method includes the following steps: defining core indicators and entities based on the e-commerce order data platform link; collecting historical sample data before and after platform transformation; constructing a data platform knowledge graph using a graph database after standardization processing; quantifying the matching degree between historical samples and the current scenario based on standardized assets; obtaining an effective historical sample library by classifying according to thresholds; when indicators are abnormal, locking the initial entity and tracing upstream to form a root cause candidate set; using an entity vulnerability index to screen high-risk candidates; sorting by final score; and outputting the top N items as the root causes of the fault; periodically supplementing effective samples and refining fault templates to achieve continuous optimization. This invention can effectively improve the situation in existing technologies where historical data is difficult to effectively utilize due to differing reference values ​​when locating faults after platform transformation.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, specifically a fault location system and method for a knowledge graph-based data platform. Background Technology

[0002] In the field of data platform fault localization, knowledge graphs are an important technical tool. They can structurally integrate the core elements of a data platform, covering key process nodes on the business side, various processing components on the technical side, and event information such as historical faults and configuration changes. By defining the dependencies and influences between entities, a visualized entity relationship network is constructed. For data platforms, the core challenge of fault localization lies in the complexity of the chain—platforms often involve multi-stage collaboration, and a single abnormal indicator may be associated with multiple potential problem points. Knowledge graphs can make this complex chain explicit, supporting reverse tracing from abnormal indicators to related entities, providing a structured tracing framework for fault analysis, helping technical personnel quickly clarify the fault propagation path, and are one of the key foundational technologies for improving fault localization efficiency.

[0003] However, existing data platform fault location technologies have significant shortcomings in platform transformation scenarios. Platform transformation is often accompanied by architectural adjustments, changes in load scale (such as an increase in daily data volume), and updates to task configurations. Existing technologies often directly reuse all historical fault samples without quantifying the matching degree between historical samples and the current scenario. This results in a large number of historical samples with significant differences in structure, load, and configuration being deemed to have low reference value or even causing interference, making them unable to be effectively screened and eliminated. Consequently, this leads to redundancy in the candidate set and decreased accuracy when locating the root cause of faults, making it difficult to adapt to the fault location requirements of the post-transformation scenario. Summary of the Invention

[0004] The purpose of this invention is to provide a fault location system and method for a knowledge graph-based data platform to solve the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a fault location method for a knowledge graph-based data platform, the method comprising the following steps:

[0006] Step 1: Define core indicators and entities, collect historical sample data, and construct a knowledge graph for the data platform using a graph database after standardization.

[0007] Step 2: Based on standardized assets, calculate the credible migration factor to quantify the matching degree between historical samples and the current scenario, and classify them according to the threshold to obtain an effective historical sample library;

[0008] Step 3: When the indicators are abnormal, lock the initial entity and trace the upstream to form a root cause candidate set. Use the entity vulnerability index to screen high-risk candidates. For high-risk candidates, sort them by the final score and output the top N items as the root causes of the failure.

[0009] Step 4: Regularly replenish valid samples and refine fault templates to achieve continuous optimization.

[0010] In step 1, based on the core link of the e-commerce order data platform (order creation → payment → fulfillment), a core indicator set M is defined, including business indicators (such as order processing volume and fulfillment delay) and technical indicators (such as task success rate and queue backlog) as well as preset abnormal thresholds for each indicator;

[0011] Define the core entity set E: including business entities, technical entities, and event entities;

[0012] Business entities include data tables (such as order details tables and payment transaction tables) and BI metrics; among which, the BI metrics come from the core metric set M;

[0013] Technical entities include processing tasks (such as Flink stream processing tasks and Spark batch ETL tasks) and services (such as Kafka queue services and storage services).

[0014] Event entities: including historical failures and change logs;

[0015] Collect historical sample data: data sources corresponding to event entities, including historical failures before and after the transformation, entity change records, and entity attribute snapshots;

[0016] Historical failures before and after the transformation: This refers to failure case data that occurred in the data platform before the transformation (such as the original business architecture stage) and after the transformation (such as the stage of adding new business links). It includes the structured characteristics of the failures: specifically, the unique identifier (ID) of the failure, the time of occurrence, the triggered abnormal indicators (such as a sudden drop in order volume, low payment success rate), the description of the failure phenomenon (such as task synchronization delay, service timeout), the associated entities (such as the processing tasks and data tables involved in the failure), the verified root cause of the failure and the corresponding handling solutions (such as restarting the service, adjusting the task configuration). This data is the core source of experience for subsequent extraction of failure fragment templates and calculation of Trusted Transfer Factor (CTF).

[0017] Entity Change Records: This refers to the trajectory data of changes to the core entity set (business entities, technical entities) before and after the transformation. Specifically, it includes the unique identifier (ID) of the change, the time of the change, the type of change (such as adding / deleting data table fields, adjusting the resource configuration of processing tasks, and upgrading service interfaces), the entities involved in the change (such as adding health attribute fields to the product data table and adjusting the frequency of order synchronization tasks), and the scope of the change's impact (such as whether the change affects the calculation logic of indicators and entity dependencies). This data is used to associate the causal relationship of change failures and support the calculation of the number of changes dimension in the subsequent entity vulnerability index.

[0018] Entity attribute snapshot: This refers to the snapshot data of the attribute status of the core entity set at key time nodes (such as when a historical failure occurs, or before and after an entity change). Specifically, it includes the entity's unique identifier (ID), snapshot timestamp, and core entity attributes (such as the field structure and business semantic mapping of the data table, the resource configuration of the processing task (memory / CPU quota), and the service timeout threshold). This data is used to trace the differences in entity structure / configuration / attributes before and after the transformation and is the basic data for calculating structural similarity and configuration similarity in Trusted Migration Factor (CTF).

[0019] Historical sample data includes three categories: "historical faults," "entity change records," and "entity attribute snapshots," which must be aligned with specific entity types in the core entity set.

[0020] Historical fault sample alignment event entities (historical fault class): The original historical fault data (such as fault logs and review reports) is decomposed into structured features (fault ID, occurrence time, triggering abnormal indicators, associated entities, root cause, and handling plan) according to the attribute requirements of "event entity (historical fault)" to ensure that the data dimensions are consistent with the definition of historical fault class event entities;

[0021] Aligning Entity Change Record Samples with Event Entities (Change Record Class): Extracting structured information (change ID, change time, change type, involved entities, and scope of impact) from the original change data (such as configuration modification logs and table structure change notifications) according to the attribute requirements of "Event Entity (Change Record)" to ensure that the data dimensions match the definition of the event entity of the change record class;

[0022] Aligning Entity Attribute Snapshot Samples with Business / Technical Entities: Based on the content of the snapshot records, determine the entity type to which it belongs (e.g., a snapshot recording the field structure of a data table → aligns with the business entity (data table); a snapshot recording the resource configuration of a processing task → aligns with the technical entity (processing task)). Then, supplement the snapshot information according to the core attribute requirements of the corresponding business / technical entity (e.g., data tables need to be supplemented with "field business semantic mapping," and processing tasks need to be supplemented with "memory / CPU quota") to ensure that the snapshot data conforms to the attribute definition of the corresponding entity.

[0023] Standardize entity data: Assign a unique ID, version number, and timestamp to each entity, and supplement core attributes (such as business semantic mapping of data table fields and resource configuration of processing tasks).

[0024] Standardize data relationships: Divide the relationships between entities into dependency relationships (such as "task depends on data table"), generation relationships (such as "task generates BI indicators"), and change impact relationships (such as "change record affects data table"), and record them using triples (depending party ID, dependent party ID, relationship type); it also includes association relationships, which represent the binding of historical faults and related entities (such as "fault A is associated with task B"), and record them using triples (historical fault ID, associated entity ID, association relationship);

[0025] Standardize historical faults: Decompose historical faults into structured features (fault ID, occurrence time, phenomenon, associated entities, root cause, handling actions, etc.).

[0026] Integrate to form standardized data assets, including structured datasets of entity attributes, relationship records, and historical fault characteristics;

[0027] Based on standardized data assets, a knowledge graph for the data platform is constructed using graph databases:

[0028] Entity layer: includes business entities, technical entities, and event entities (all associated with standardized core attributes);

[0029] Relationship layer: includes dependency relationships, production relationships, change-affected relationships, and association relationships;

[0030] Building a knowledge graph for a data platform involves: transforming business, technology, and event entities in standardized data assets into Neo4j nodes; converting dependencies, generation, change impacts, and relationships into edges between nodes; creating nodes and edges using Cypher statements to form a knowledge graph (entity relationship graph) for the data platform; and leveraging Neo4j's graph storage features to achieve efficient traversal of entity relationships, providing structured support for entity tracing during fault location.

[0031] In step 2, a Trusted Migration Factor (CTF) is defined to screen valid historical samples;

[0032] CTF is used to quantify the matching degree between historical samples h and the current scene c (values ​​range from [0,1], with higher values ​​indicating stronger reference value), and is calculated from four dimensions:

[0033] CTF{h,c}=w s ·S s +w w ·S w +w g ·S g +wt ·S t ;

[0034] Among them, based on the entity attributes in standardized data assets, S s Structural similarity represents the proportion of overlap between the data table fields and BI metric calculation logic involved in historical samples and the current scenario (including semantic mapping, such as fields with different names but the same meaning being considered a match); S w Load similarity represents the cosine similarity between the processing scale of historical samples (average daily data volume, query frequency) and the processing scale of the current scenario; S g To configure similarity, it represents the degree of overlap between the task / service configurations (such as resource allocation) involved in historical samples and the current scenario; S t The time decay factor represents the time difference Δt between historical samples and the current scene, calculated according to e. -λΔt Calculate; λ represents the time decay coefficient (recent samples have higher weight); w s w w w g w t These represent the structural similarity weight, load similarity weight, configuration similarity weight, and time decay factor weight (the sum of which is 1).

[0035] For each historical sample, extract structural, load, configuration, and temporal features from standardized data assets, compare them with current scene features, and calculate CTF.

[0036] Set classification thresholds θ1 and θ2 (0 < θ2 < θ1 < 1): When CTF ≥ θ1, it is defined as a strong reference sample, indicating that the historical sample is highly similar to the current scene; when θ2 ≤ CTF < θ1, it is defined as a weak reference sample; when CTF < θ2, it is defined as an invalid sample and is directly removed.

[0037] A valid historical sample library is obtained, which includes strong reference samples and weak reference samples, and is associated with entities in the knowledge graph of the data platform;

[0038] The effective historical sample library is linked and stored with the knowledge graph of the data platform, serving as a source of historical experience for fault root cause localization.

[0039] In step 3, a judgment is made based on the preset abnormal threshold for each indicator. When an indicator in the core indicator set M triggers an abnormality;

[0040] Extract abnormal information, including abnormal indicators (from the core indicator set M), abnormal time, and fluctuation range;

[0041] By querying the generation chain of this indicator through the knowledge graph of the data platform, the initial entity can be further identified;

[0042] Starting from the initial entity, the upstream dependent entities of layer a are traced back through the knowledge graph of the data platform to form a root cause candidate set;

[0043] Using the Entity Vulnerability Index VI e Screening root cause candidate set (VI) e VI measures the probability of physical failure, taking values ​​[0,1]): e =w1·N(f e )+w2·N(c e )+w3·(1-t e )+w4·N(d e );

[0044] Among them, f e This indicates the number of failures of entity e within time period T; c e This indicates the number of times entity e changes within time period T; t e This represents the test coverage of entity e; d e The number of direct dependent entities of entity e is represented by N(), which is the normalization function; w1, w2, w3, and w4 represent their respective weights; entity e is any entity in the root cause candidate set.

[0045] Set a threshold θ3 to remove VI. e Low-risk candidates with <θ3 are retained, while high-risk candidates are preserved.

[0046] For high-risk candidates after screening, the final score FS is used. n Evaluation (higher scores are more likely to be true causes): FS n =W1·I(n)+W2·S p +W3·max(CTF{h,n});

[0047] Wherein, I() represents the normalized impact of candidate entity n on the abnormal indicator; the impact is calculated based on the dependency path from candidate entity n to the initial entity corresponding to the abnormal indicator. This dependency path is the upstream-to-downstream dependency path from candidate entity n to the initial entity, and the path range is consistent with the range of "tracing upstream entities from the initial entity" when constructing the root cause candidate set; for each entity on this dependency path, there is a corresponding path length weight, which follows the rule of "the shorter the path length, the greater the weight" (path length refers to the number of entity levels traversed from candidate entity n to the initial entity; the closer the hierarchical distance between the entity and the initial entity, the higher the fault transmission intensity of its impact on the alarm indicator, and therefore the greater the corresponding weight); the impact is calculated as follows: for all entities on the dependency path, calculate the entity vulnerability index VI of that entity. e The product of the path length weights corresponding to the path level where the entity resides, and then the sum of all the product results, i.e., I(n) = Σ(VI). e× path length weight), where Σ represents the summation of the product of all entities on the dependent path; finally, it is normalized to [0,1]; S p S represents the matching score between candidate entity n and the fault fragment template (the fault fragment template is obtained based on valid historical samples, including triggering indicators, log features, etc.). p The value ranges from [0,1]); max(CTF{h,n}) represents the highest CTF value among the valid historical samples associated with candidate entity n; entity n is any entity among the high-risk candidates; W1, W2, and W3 represent their respective weights;

[0048] According to FS n Sort the data in descending order, select the top N items as the root causes of the fault, and output them.

[0049] In step 4, new failure cases are periodically broken down into standardized samples, CTFs are calculated and classified, and then added to the effective historical sample library.

[0050] Regularly extract new fault segment templates from high-frequency faults and optimize template features (such as adding typical log keywords).

[0051] A fault location system for a knowledge graph-based data platform, comprising a data construction module, a sample screening module, a root cause localization module, and an optimization iteration module;

[0052] The data construction module is used to define core indicators and entities based on the e-commerce order data platform link, collect historical sample data before and after the platform transformation, and construct a knowledge graph of the data platform using a graph database after standardization processing. The sample screening module is used to calculate the trust migration factor based on standardized assets to quantify the matching degree between historical samples and the current scenario, and obtain an effective historical sample library by classification according to thresholds. The root cause localization module is used to lock the initial entity and trace upstream to form a root cause candidate set when the indicators are abnormal. It uses the entity vulnerability index to screen high-risk candidates. For high-risk candidates, it uses the final score to sort and outputs the top N items as the root causes of the failure. The optimization iteration module is used to regularly supplement effective samples and refine failure templates to achieve continuous optimization.

[0053] The data construction module includes an indicator definition unit, an entity definition unit, a data collection unit, a data standardization unit, and a graph construction unit;

[0054] The indicator definition unit is used to define core business indicators and technical indicators and their abnormal thresholds; the entity definition unit is used to define the types of business entities, technical entities and event entities; the data collection unit is used to collect historical fault data, change records and entity attribute snapshots; the data standardization unit is used to assign unique IDs to entities / relationships and structure fault characteristics; the graph construction unit is used to build entity layer and relation layer networks in the graph database.

[0055] The sample screening module includes a feature extraction unit, a matching degree calculation unit, and a sample classification unit;

[0056] The feature extraction unit is used to extract structural / load / configuration / time features from standardized assets; the matching degree calculation unit is used to calculate the reliable migration factor between historical samples and the current scenario; the sample classification unit is used to construct an effective historical sample library based on the matching degree threshold.

[0057] The root cause localization module includes an anomaly handling unit, an entity locking unit, a candidate generation unit, a risk screening unit, and a scoring and ranking unit.

[0058] The anomaly handling unit is used to parse the abnormal information of the indicators; the entity locking unit is used to locate the initial abnormal entity through the data network; the candidate generation unit is used to trace the upstream entities in reverse to form a root cause candidate set; the risk screening unit is used to screen high-risk candidates based on the entity vulnerability index; and the scoring and sorting unit is used to calculate the final score and output the TOP-N root causes of failure.

[0059] The optimization iteration module includes a sample replenishment unit and a template extraction unit;

[0060] The sample supplementation unit is used to standardize new fault cases and supplement them to the sample library; the template extraction unit is used to extract and optimize fault fragment templates from high-frequency faults.

[0061] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention defines a Trusted Transfer Factor (CTF) to quantify the matching degree between historical samples and the current scenario from four dimensions: structural similarity, load similarity, configuration similarity, and time decay. It also classifies samples into strong references, weak references, and invalid samples according to thresholds, retaining only the knowledge graph associated with valid samples. In principle, this achieves "precise screening of valid historical experience," avoiding interference from invalid samples while fully reusing historical fault patterns that match the current scenario. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating a fault location system for a knowledge graph-based data platform according to the present invention.

[0063] Figure 2 This is a flowchart illustrating a fault location method for a knowledge graph-based data platform according to the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] Example: Figure 1 As shown, the present invention provides a technical solution, a fault location method for a knowledge graph-based data platform, the method comprising the following steps:

[0066] Step 1: Define core indicators and entities, collect historical sample data, and construct a knowledge graph for the data platform using a graph database after standardization.

[0067] Step 2: Based on standardized assets, calculate the credible migration factor to quantify the matching degree between historical samples and the current scenario, and classify them according to the threshold to obtain an effective historical sample library;

[0068] Step 3: When the indicators are abnormal, lock the initial entity and trace the upstream to form a root cause candidate set. Use the entity vulnerability index to screen high-risk candidates. For high-risk candidates, sort them by the final score and output the top N items as the root causes of the failure.

[0069] Step 4: Regularly replenish valid samples and refine fault templates to achieve continuous optimization.

[0070] In step 1, based on the core link of the e-commerce order data platform (order creation → payment → fulfillment), a core indicator set M is defined, including business indicators (such as order processing volume and fulfillment delay) and technical indicators (such as task success rate and queue backlog) as well as preset abnormal thresholds for each indicator;

[0071] Define the core entity set E: including business entities, technical entities, and event entities;

[0072] Business entities include data tables (such as order details tables and payment transaction tables) and BI metrics; among which, the BI metrics come from the core metric set M;

[0073] Technical entities include processing tasks (such as Flink stream processing tasks and Spark batch ETL tasks) and services (such as Kafka queue services and storage services).

[0074] Event entities: including historical failures and change logs;

[0075] Collect historical sample data: data sources corresponding to event entities, including historical failures before and after the transformation, entity change records, and entity attribute snapshots;

[0076] Standardize entity data: Assign a unique ID, version number, and timestamp to each entity, and supplement core attributes (such as business semantic mapping of data table fields and resource configuration of processing tasks).

[0077] Standardize data relationships: Divide the relationships between entities into dependency relationships (such as "task depends on data table"), generation relationships (such as "task generates BI indicators"), and change impact relationships (such as "change record affects data table"), and record them using triples (depending party ID, dependent party ID, relationship type); it also includes association relationships, which represent the binding of historical faults and related entities (such as "fault A is associated with task B"), and record them using triples (historical fault ID, associated entity ID, association relationship);

[0078] Standardize historical faults: Decompose historical faults into structured features (fault ID, occurrence time, phenomenon, associated entities, root cause, handling actions, etc.).

[0079] Integrate to form standardized data assets, including structured datasets of entity attributes, relationship records, and historical fault characteristics;

[0080] Based on standardized data assets, a knowledge graph for the data platform is constructed using graph databases:

[0081] Entity layer: includes business entities, technical entities, and event entities (all associated with standardized core attributes);

[0082] Relationship layer: includes dependency relationships, generation relationships, change-affected relationships, and association relationships.

[0083] In step 2, a Trusted Migration Factor (CTF) is defined to screen valid historical samples;

[0084] CTF is used to quantify the matching degree between historical samples h and the current scene c (values ​​range from [0,1], with higher values ​​indicating stronger reference value), and is calculated from four dimensions: CTF{h,c}=w s ·S s +w w ·S w +w g ·S g +w t ·S t ;

[0085] Among them, based on the entity attributes in standardized data assets, S s Structural similarity represents the proportion of overlap between the data table fields and BI metric calculation logic involved in historical samples and the current scenario (including semantic mapping, such as fields with different names but the same meaning being considered a match); S wLoad similarity represents the cosine similarity between the processing scale of historical samples (average daily data volume, query frequency) and the processing scale of the current scenario; S g To configure similarity, it represents the degree of overlap between the task / service configurations (such as resource allocation) involved in historical samples and the current scenario; S t The time decay factor represents the time difference Δt between historical samples and the current scene, calculated according to e. -λΔt Calculate; λ represents the time decay coefficient (recent samples have higher weight); w s w w w g w t These represent the structural similarity weight, load similarity weight, configuration similarity weight, and time decay factor weight (the sum of which is 1).

[0086] For each historical sample, extract structural, load, configuration, and temporal features from standardized data assets, compare them with current scene features, and calculate CTF.

[0087] Set classification thresholds θ1 and θ2 (0 < θ2 < θ1 < 1): When CTF ≥ θ1, it is defined as a strong reference sample, indicating that the historical sample is highly similar to the current scene; when θ2 ≤ CTF < θ1, it is defined as a weak reference sample; when CTF < θ2, it is defined as an invalid sample and is directly removed.

[0088] A valid historical sample library is obtained, which includes strong reference samples and weak reference samples, and is associated with entities in the knowledge graph of the data platform;

[0089] The effective historical sample library is linked and stored with the knowledge graph of the data platform, serving as a source of historical experience for fault root cause localization.

[0090] In step 3, a judgment is made based on the preset abnormal threshold for each indicator. When an indicator in the core indicator set M triggers an abnormality;

[0091] Extract abnormal information, including abnormal indicators (from the core indicator set M), abnormal time, and fluctuation range;

[0092] By querying the generation chain of this indicator through the knowledge graph of the data platform, the initial entity can be further identified;

[0093] Starting from the initial entity, the upstream dependent entities of layer a are traced back through the knowledge graph of the data platform to form a root cause candidate set;

[0094] Using the Entity Vulnerability Index VI e Screening root cause candidate set (VI) e VI measures the probability of physical failure, taking values ​​[0,1]): e =w1·N(f e )+w2·N(ce )+w3·(1-t e )+w4·N(d e );

[0095] Among them, f e This indicates the number of failures of entity e within time period T; c e This indicates the number of times entity e changes within time period T; t e This represents the test coverage of entity e; d e The number of direct dependent entities of entity e is represented by N(), which is the normalization function; w1, w2, w3, and w4 represent their respective weights; entity e is any entity in the root cause candidate set.

[0096] Set a threshold θ3 to remove VI. e Low-risk candidates with <θ3 are retained, while high-risk candidates are preserved.

[0097] For high-risk candidates after screening, the final score FS is used. n Evaluation (higher scores are more likely to be true causes): FS n =W1·I(n)+W2·S p +W3·max(CTF{h,n});

[0098] Where I() represents the normalized influence of candidate entity n on the anomaly index (entity VI on the dependency path). e Weighted sum, the shorter the path, the higher the VI. e The higher the value, the greater the influence (e.g., the weight is set to 1 for a path length of 1, 0.8 for a length of 2, and 0.6 for a length of 3), i.e., I(n) = Σ(VI). e ×path length weight), finally normalized to [0,1]; S p S represents the matching score between candidate entity n and the fault fragment template (the fault fragment template is obtained based on valid historical samples, including triggering indicators, log features, etc.). p The value ranges from [0,1]); max(CTF{h,n}) represents the highest CTF value among the valid historical samples associated with candidate entity n; entity n is any entity among the high-risk candidates; W1, W2, and W3 represent their respective weights;

[0099] According to FS n Sort the data in descending order, select the top N items as the root causes of the fault, and output them.

[0100] In step 4, new failure cases are periodically broken down into standardized samples, CTFs are calculated and classified, and then added to the effective historical sample library.

[0101] Regularly extract new fault segment templates from high-frequency faults and optimize template features (such as adding typical log keywords).

[0102] A fault location system for a knowledge graph-based data platform, comprising a data construction module, a sample screening module, a root cause localization module, and an optimization iteration module;

[0103] The data construction module is used to define core indicators and entities based on the e-commerce order data platform link, collect historical sample data before and after the platform transformation, and construct a knowledge graph of the data platform using a graph database after standardization processing. The sample screening module is used to calculate the trust migration factor based on standardized assets to quantify the matching degree between historical samples and the current scenario, and obtain an effective historical sample library by classification according to thresholds. The root cause localization module is used to lock the initial entity and trace upstream to form a root cause candidate set when the indicators are abnormal. It uses the entity vulnerability index to screen high-risk candidates. For high-risk candidates, it uses the final score to sort and outputs the top N items as the root causes of the failure. The optimization iteration module is used to regularly supplement effective samples and refine failure templates to achieve continuous optimization.

[0104] The data construction module includes an indicator definition unit, an entity definition unit, a data collection unit, a data standardization unit, and a graph construction unit;

[0105] The indicator definition unit is used to define core business indicators and technical indicators and their abnormal thresholds; the entity definition unit is used to define the types of business entities, technical entities and event entities; the data collection unit is used to collect historical fault data, change records and entity attribute snapshots; the data standardization unit is used to assign unique IDs to entities / relationships and structure fault characteristics; the graph construction unit is used to build entity layer and relation layer networks in the graph database.

[0106] The sample screening module includes a feature extraction unit, a matching degree calculation unit, and a sample classification unit;

[0107] The feature extraction unit is used to extract structural / load / configuration / time features from standardized assets; the matching degree calculation unit is used to calculate the reliable migration factor between historical samples and the current scenario; the sample classification unit is used to construct an effective historical sample library based on the matching degree threshold.

[0108] The root cause localization module includes an anomaly handling unit, an entity locking unit, a candidate generation unit, a risk screening unit, and a scoring and ranking unit.

[0109] The anomaly handling unit is used to parse the abnormal information of the indicators; the entity locking unit is used to locate the initial abnormal entity through the data network; the candidate generation unit is used to trace the upstream entities in reverse to form a root cause candidate set; the risk screening unit is used to screen high-risk candidates based on the entity vulnerability index; and the scoring and sorting unit is used to calculate the final score and output the TOP-N root causes of failure.

[0110] The optimization iteration module includes a sample replenishment unit and a template extraction unit;

[0111] The sample supplementation unit is used to standardize new fault cases and supplement them to the sample library; the template extraction unit is used to extract and optimize fault fragment templates from high-frequency faults.

[0112] In this embodiment, an e-commerce company's original core business was product A (traditional snacks from brand A, such as fried potato chips and sandwich cookies). In January 2024, due to a shift towards healthier consumption trends, it added product B (healthier snacks from brand B, such as whole-wheat cookies and low-sugar nut crisps). The business processes for both are similar (both are "user places order → payment → order data synchronization → fulfillment → after-sales service"), but there are differences:

[0113] Data structure: Add a "Health Attribute (Low Sugar / High Fiber)" field to the B product data table;

[0114] Load characteristics: Product A has an average of 100,000 orders per day, while Product B initially had an average of 20,000 orders per day (which increased to 30,000 orders per day in March).

[0115] Configuration differences: The initial frequency of the order synchronization task for product B was every 10 minutes (later adjusted to 5 minutes).

[0116] Building standardized data assets and knowledge graphs (data platform knowledge graphs):

[0117] Defining the core indicator set and the core entity set:

[0118] Core indicator set (including outlier thresholds):

[0119] Business metrics: BI_001 (Order volume of product A); Total number of successfully paid orders for product A per day; A anomaly threshold: ±30% (average 100,000);

[0120] Business metrics: BI_002 (Order volume of product B); Total number of successfully paid orders for product B per day; Abnormal threshold for product B: ±40% (average 30,000);

[0121] Business metrics: BI_003 (Payment Success Rate); Number of successful payments / Total number of payment requests; A. Anomaly threshold: ≤95%; B. Anomaly threshold: ≤94%;

[0122] Technical Specifications: TI_001 (Order Synchronization Delay); Time taken from order generation to data writing to the table; A. Abnormal Threshold: >30s; B. Abnormal Threshold: >25s;

[0123] Technical Specifications: TI_002 (Service Response Time); Average response time for the order interface service; A. Anomaly threshold: >500ms; B. Anomaly threshold: >400ms;

[0124] Core Entity Set E: Categorized into "Business Entities, Technical Entities, and Event Entities," some key entities are as follows:

[0125] Business Entity:

[0126] Data tables: E_BT_A01 (Basic table for Product A: id / name / specification / unit price), E_BT_B01 (Basic table for Product B: id / name / specification / unit price / health attributes), E_BT_B02 (Order table for Product B: order id / user id / Product B id / order time / payment amount), E_BT_B03 (Payment table for Product B: payment id / order id / payment status);

[0127] BI indicators: E_BI_001 (corresponding to BI_001), E_BI_002 (corresponding to BI_002);

[0128] Technical Entities:

[0129] Processing tasks: E_TT_B02 (B order data synchronization task), E_TT_B04 (B product information entry task);

[0130] Services: E_TS_S02 (Payment Interface Service), E_TS_S04 (B Order Data Acquisition Service);

[0131] Event Entity:

[0132] Historical faults: E_ET_F01 (On October 15, 2023, the order volume of product A suddenly dropped by 35%, root cause E_TT_A01 synchronization delay), E_ET_F02 (On November 20, 2023, the payment success rate of product A was 92%, root cause E_TS_S02 timeout).

[0133] Change log: E_ET_C01 (E_BT_B01 went online on January 10, 2024), E_ET_C02 (the synchronization frequency of E_TT_B02 was adjusted to 5 minutes on March 1, 2024).

[0134] Collection and standardization of historical sample data:

[0135] Before the transformation (Product A): Faults (F01, F02) from September 2023 to December 2023, change records (such as E_ET_C03: update of E_BT_A01 field on September 5, 2023), entity attribute snapshots (such as the field structure of E_BT_A01 in September 2023);

[0136] After the transformation (Product B): Faults from 2024.1 to 2024.2 (such as F03: 2024.2.15 Product B order synchronization delay), change records (C01, C02).

[0137] Standardization process:

[0138] Entity data standardization: Assign a unique ID + version + timestamp to each entity, supplementing core attributes. For example: E_BT_B01: id=E_BT_B01, version=V1.1, timestamp=20240110, dataSource=e-commerce ERP, updateFreq=daily.

[0139] Data relation standardization: Use triples to record four types of relations. Example:

[0140] Dependency relationship: (E_BT_B02, dependency, E_BT_B01) (B order table depends on B product base table);

[0141] Generation relation: (E_TT_B02, Generation, E_BT_B02) (Synchronous task generates B order table data);

[0142] Impact of the change: (E_ET_C02, Impact of the change, E_TT_B02) (Frequency adjustment affects synchronization tasks);

[0143] Association relationship: (E_ET_F01, Association, E_TT_A01) (A product failure association synchronization task).

[0144] Historical fault standardization: decomposed into structured features, such as F01: {Fault ID: F01, Time: 202310151400, Anomaly index: E_BI_001, Anomaly amplitude: -35%, Root cause entity: E_TT_A01, Fault duration: 2h}.

[0145] Building a knowledge graph for a data platform using Neo4j:

[0146] The knowledge graph, created by Cypher statements, contains 28 nodes (12 business entities, 8 technical entities, and 8 event entities) and 45 edges (18 dependencies, 12 generation, 10 change impacts, and 5 associations).

[0147] Calculate the Trusted Transfer Factor (CTF) to screen for valid historical sample libraries:

[0148] Set CTF parameters:

[0149] Weights: ws=0.3 (structural similarity), ww=0.2 (load similarity), wg=0.3 (configuration similarity), wt=0.2 (time decay);

[0150] Time decay coefficient: λ=0.1 (in months, reflecting the higher reference value of recent samples);

[0151] Classification thresholds: θ1=0.8 (strong reference sample), θ2=0.6 (weak reference sample).

[0152] Calculate the CTF of a typical historical sample: Example:

[0153] Sample h=F01 (Order volume of product A suddenly drops, root cause E_TT_A01);

[0154] Ss (structural similarity): F01 involves 100% overlap of fields between order table A (E_BT_A02) and order table B (E_BT_B02), and the BI indicator calculation logic (successful order count) is completely consistent, therefore Ss=1.0;

[0155] Sw (load similarity): At F01, product A has an average of 100,000 orders per day, and product B currently has 30,000 orders. The single-dimensional cosine similarity Sw = 1.0.

[0156] Sg (Configuration Similarity): E_TT_A01 (A Synchronization Task) and E_TT_B02 (B Synchronization Task) have completely identical configurations (synchronization frequency of 5 minutes, timeout of 30 seconds), Sg=1.0;

[0157] St (time decay): Δt = 5 months (October 2023 - March 2024), St = e^(-0.1*5) ≈ 0.607;

[0158] CTF = 0.3*1.0 + 0.2*1.0 + 0.3*1.0 + 0.2*0.607 = 0.9214; greater than θ1, strong reference sample).

[0159] Forming an effective historical sample database:

[0160] Strong reference samples (CTF≥0.8): F01, F02, F03 (5 in total);

[0161] Weak reference sample (0.6≤CTF<0.8): 1 (e.g., F04: Inventory synchronization error for product 2023.5A, CTF=0.648);

[0162] Invalid samples (CTF<0.6): 2 (e.g., the old system failure of product 2023.3A with large structural differences) are directly removed.

[0163] Fault localization (taking the sudden drop in order volume for product B on March 10, 2024 as an example):

[0164] At 15:00 on March 10, 2024, an anomaly was triggered in BI_002 (order quantity of product B):

[0165] Abnormal indicator: E_BI_002; Abnormal time: 202403101500; Fluctuation range: -40% (from 30,000 orders / day to 18,000 orders / day).

[0166] Query the generation chain of BI_002 using the knowledge graph:

[0167] E_TT_B02 (Synchronization Task) → E_BT_B02 (B Order Table) → E_BI_002 (B Order Quantity Indicator);

[0168] The direct upstream of the abnormal indicator is E_BT_B02 (B order table), so the initial entity is E_BT_B02.

[0169] Tracing upstream to form the root cause candidate set:

[0170] Set the number of tracing layers a=2 (to avoid an excessively large range), and trace back from E_BT_B02:

[0171] Upstream of Layer 1: E_TT_B02 (Generate B Order Table), E_BT_B01 (B Product Basic Table), E_BT_B03 (B Payment Table);

[0172] Upstream of Layer 2: E_TS_S04 (Data Acquisition Service, providing data for E_TT_B02), E_TT_B04 (Data Entry Service, generating E_BT_B01), E_TS_S02 (Payment Service, generating E_BT_B03);

[0173] Root cause candidate set:

[0174] {E_TT_B02,E_BT_B01,E_BT_B03,E_TS_S04,E_TT_B04,E_TS_S02} (6 entities in total).

[0175] Using the Entity Vulnerability Index (VI) e Screening for high-risk candidates:

[0176] Setting VI e Parameters: Time period T = 30 days (February 10, 2024 - March 10, 2024); Weights: w1 = 0.3 (number of failures), w2 = 0.2 (number of changes), w3 = 0.3 (1 - test coverage), w4 = 0.2 (number of dependent entities);

[0177] The normalization function is N(x) = x / x_max (where x_max is the maximum value of this dimension over 30 days, such as the maximum number of failures = 5).

[0178] The screening threshold θ3 = 0.3 (removing low-risk entities).

[0179] Calculate the VI of the candidate entitye The high-risk candidate set obtained after calculation is: {E_TT_B02,E_TS_S04,E_TT_B04,E_TS_S02}.

[0180] Calculate the final score (FSn) and output the root cause:

[0181] Set the FSn parameters: weights: W1=0.4 (influence), W2=0.3 (template matching score), W3=0.3 (highest CTF);

[0182] Fault segment template: TPL_001 (order volume drop template) extracted from valid samples: {Abnormal characteristics: order volume drops by ≥30%; Root cause characteristics: synchronous task / collection service abnormality; Impact path: service → task → order table → indicator};

[0183] Calculate the high-risk candidate FSn, set N=2, and output the first two root causes:

[0184] E_TS_S02 (Payment Interface Service): The disconnection of bank interface A caused 35% of payment failures, indirectly leading to a decrease in order volume;

[0185] E_TS_S04 (B Order Collection Service): Front-end interface timeout resulted in 20% of order data not being collected, directly causing a decrease in order volume.

[0186] Sample supplementation and parameter optimization:

[0187] Supplementing valid samples:

[0188] In April 2024, a new fault F05 was added for product B (order volume for product B decreased by 25% on March 20th, root cause E_TT_B02 insufficient retries):

[0189] Standardized sample: {Fault ID:F05, Time:202403201000, Root cause:E_TT_B02, Characteristics: 2 retries → data packet loss};

[0190] Calculate CTF=0.9208 (strong reference sample) and add it to the effective sample library.

[0191] Refine the new template:

[0192] Combining F05 and F01, a new template TPL_002 (Sudden Drop in Order Volume - Abnormal Parameters) is added: {Root Cause Characteristics: Insufficient Retry Count for Synchronization Task / Too Short Timeout; Impact Path: Parameters → Tasks → Order Table → Metrics}.

[0193] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A fault location method for a knowledge graph-based data platform, characterized in that: The method includes the following steps: Step 1: Define the core indicator set based on the e-commerce order data platform link; define the core entity set based on the core indicator set and the e-commerce order data platform link; collect historical sample data before and after the platform transformation, align the historical sample data with the entity types of the core entity set, and standardize the aligned historical sample data to obtain standardized data assets; construct a knowledge graph for the data platform using a graph database based on the standardized data assets; the historical sample data includes historical faults, entity change records, and entity attribute snapshots before and after the platform transformation. Step 2: Based on standardized data assets, calculate the trusted migration factor to quantify the matching degree between historical sample data and the platform transformation scenario, and classify them according to the threshold to obtain an effective historical sample library; the trusted migration factor is determined based on the structural similarity, load similarity, configuration similarity and time decay factor between historical samples and the platform transformation scenario. Step 3: Based on the preset abnormal thresholds of each indicator in the core indicator set, determine the abnormality of the indicator. When the indicator is abnormal, determine the initial entity based on the knowledge graph of the data platform and trace the upstream to form a root cause candidate set; use the entity vulnerability index to screen high-risk candidates based on the root cause candidate set; use the final score to sort the high-risk candidates and output the top N items as the root causes of the failure; the entity vulnerability index is determined based on the number of failures and changes of a certain entity in the root cause candidate set within the time period T, as well as the test coverage and the number of directly dependent entities of a certain entity, where T is the set time period; N is a natural number, and N is not greater than the number of entities in the high-risk candidates; Step 4: Regularly replenish valid samples, refine fault segment templates, and feed them back to Step 3 to update the fault segment templates; In step 1, the e-commerce order data platform link refers to the business flow path of an e-commerce order from its generation to its completion; The core indicator set refers to a set of quantitative indicators used to monitor the operational status of the e-commerce order data platform; the core indicator set includes business indicators, technical indicators, and preset abnormal thresholds for each indicator; The platform transformation refers to the changes that occur in the e-commerce order data platform due to business needs, resulting in differences in the structure, load, and configuration of historical data compared to the current scenario. In step 3, the initial entity is determined based on the generation link of abnormal indicators queryed from the knowledge graph of the data platform; The root cause candidate set is formed by tracing back upstream dependent entities at layer a based on the knowledge graph of the data platform, starting from the initial entity; In step 4, the fault fragment template refers to a structured template formed by screening faults and breaking down common features based on valid historical samples.

2. The fault location method for a knowledge graph-based data platform according to claim 1, characterized in that: In step 1, based on the order chain of the e-commerce platform, a core indicator set M is defined, including business indicators, technical indicators, and preset abnormal thresholds for each indicator; Define the core entity set E: including business entities, technical entities, and event entities; Business entities: including data tables and BI metrics; Based on the core indicator set combined with the e-commerce order data platform link definition, the core entity set refers to BI indicators derived from the core indicator set M; Technical entities: including task processing and services; Event entities: including historical failures and change logs; Collect historical sample data: data sources corresponding to event entities, including historical failures before and after the transformation, entity change records, and entity attribute snapshots; Aligning historical sample data with the entity types of the core entity set specifically includes: matching historical fault data with event entities; matching entity change records with event entities; and matching entity attribute snapshots with corresponding business or technical entities. The standardization process includes: Standardize entity data: Assign a unique ID, version number, and timestamp to each entity, and supplement core attributes; Standardize data relationships: classify relationships between entities into dependency relationships, generation relationships, and change impact relationships, and record them using triples; also include association relationships, which represent the binding of historical faults with associated entities, and record them using triples; Standardize historical faults: Decompose historical faults into structured features; Integrate to form standardized data assets, including structured datasets of entity attributes, relationship records, and historical fault characteristics; Based on standardized data assets, a knowledge graph for the data platform is constructed using graph databases: Entity layer: includes business entities, technical entities, and event entities; Relationship layer: includes dependency relationships, production relationships, change-affected relationships, and association relationships; The construction of a data platform knowledge graph using a graph database specifically includes: transforming business, technology, and event entities in standardized data assets into Neo4j nodes; transforming dependencies, generation, change effects, and associations into edges between nodes; and creating nodes and edges using Cypher statements to form a data platform knowledge graph.

3. The fault location method for a knowledge graph-based data platform according to claim 2, characterized in that: Step 2 specifically includes: defining a Trusted Transfer Factor (CTF) to screen valid historical samples; CTF is used to quantify the matching degree between historical samples h and the current scene c, and is calculated from four dimensions: CTF{h,c}=w s ·S s +w w ·S w +w g ·S g +w t ·S t ; Among them, based on the entity attributes in standardized data assets, S s Structural similarity represents the proportion of overlap between the data table fields and BI metric calculation logic involved in historical samples and the current scenario; w Let S be the load similarity, representing the cosine similarity between the processing scale of historical samples and the processing scale of the current scene; g To configure similarity, it represents the degree of overlap between the task / service configurations involved in historical samples and the current scenario; S t The time decay factor represents the time difference Δt between historical samples and the current scene, calculated according to e. -λΔt Calculate; λ represents the time decay coefficient; w s w w w g w t These represent the structural similarity weight, load similarity weight, configuration similarity weight, and time decay factor weight, respectively. For each historical sample, extract structural, load, configuration, and temporal features from standardized data assets, compare them with current scene features, and calculate CTF. Set classification thresholds θ1 and θ2: when CTF≥θ1, it is defined as a strong reference sample; when θ2≤CTF<θ1, it is defined as a weak reference sample; when CTF<θ2, it is defined as an invalid sample and is directly removed. A valid historical sample library is obtained, which includes strong reference samples and weak reference samples, and is associated with entities in the knowledge graph of the data platform.

4. The fault location method for a knowledge graph-based data platform according to claim 3, characterized in that: In step 3, a judgment is made based on the preset abnormal threshold for each indicator. When an indicator in the core indicator set M is judged to be abnormal; Extract abnormal information, including abnormal indicators, abnormal time, and fluctuation range; By querying the generation chain of this indicator through the knowledge graph of the data platform, the initial entity can be further identified; Starting from the initial entity, the upstream dependent entities of layer a are traced back through the knowledge graph of the data platform to form a root cause candidate set; Using the Entity Vulnerability Index VI e Screening root cause candidate sets: VI e =w1·N(f e )+w2·N(c e )+w3·(1-t e )+w4·N(d e ); Among them, f e This indicates the number of failures of entity e within time period T; c e This indicates the number of times entity e changes within time period T; t e This represents the test coverage of entity e; d e The number of direct dependent entities of entity e is represented by N(), which is the normalization function; w1, w2, w3, and w4 represent their respective weights; entity e is any entity in the root cause candidate set. Set a threshold θ3 to remove VI. e Low-risk candidates with <θ3 are retained, while high-risk candidates are preserved. For high-risk candidates after screening, the final score FS is used. n Assessment: FS n =W1·I(n)+W2·S p +W3·max(CTF{h,n}); Where I() represents the normalized influence of candidate entity n on the anomaly index; S p The matching score between candidate entity n and the fault fragment template is represented; max(CTF{h,n}) represents the highest CTF value among the valid historical samples associated with candidate entity n; entity n is any entity among the high-risk candidates; W1, W2, and W3 represent their respective weights. According to FS n Sort the data in descending order, select the top N items as the root causes of the fault, and output them. Where N is no greater than the number of entities in the high-risk candidates.

5. The fault location method for a knowledge graph-based data platform according to claim 4, characterized in that: In step 4, new failure cases are periodically broken down into standardized samples, CTFs are calculated and classified, and then added to the effective historical sample library. Periodically extract new fault fragment templates from high-frequency faults and feed them back to step 3 to update the fault fragment templates.

6. A knowledge graph-based data platform fault location system, applied to the knowledge graph-based data platform fault location method described in any one of claims 1-5, characterized in that: The system includes a data construction module, a sample screening module, a root cause localization module, and an optimization iteration module; The data construction module is used to define core indicators and entities based on the e-commerce order data platform link, collect historical sample data before and after the platform transformation, and construct a data platform knowledge graph using a graph database after standardization processing. The sample screening module is used to calculate the credible migration factor based on standardized assets to quantify the matching degree between historical samples and the current scenario, and to obtain an effective historical sample library by classifying them according to a threshold. The root cause localization module is used to lock the initial entity and trace upstream to form a root cause candidate set when the indicators are abnormal. It uses the entity vulnerability index to screen high-risk candidates. For high-risk candidates, it uses the final score to sort them and outputs the top N items as the root causes of the failure. The optimization iteration module is used to periodically supplement effective samples, refine failure templates and provide feedback.

7. A fault location system for a knowledge graph-based data platform according to claim 6, characterized in that: The data construction module includes an indicator definition unit, an entity definition unit, a data collection unit, a data standardization unit, and a graph construction unit; The indicator definition unit is used to define core business indicators and technical indicators and their abnormal thresholds; the entity definition unit is used to define the types of business entities, technical entities and event entities; the data collection unit is used to collect historical fault data, change records and entity attribute snapshots; the data standardization unit is used to assign unique IDs to entities / relationships and structure fault characteristics; the graph construction unit is used to build entity layer and relation layer networks in the graph database.

8. The fault location system for a knowledge graph-based data platform according to claim 7, characterized in that: The sample screening module includes a feature extraction unit, a matching degree calculation unit, and a sample classification unit; The feature extraction unit is used to extract structural / load / configuration / time features from standardized assets; the matching degree calculation unit is used to calculate the reliable migration factor between historical samples and the current scenario; the sample classification unit is used to construct an effective historical sample library based on the matching degree threshold.

9. A fault location system for a knowledge graph-based data platform according to claim 8, characterized in that: The root cause localization module includes an anomaly handling unit, an entity locking unit, a candidate generation unit, a risk screening unit, and a scoring and ranking unit. The anomaly handling unit is used to parse the indicator anomaly information; the entity locking unit is used to locate the initial abnormal entity through the data network. The candidate generation unit is used to trace upstream entities in reverse to form a root cause candidate set; The risk screening unit is used to screen high-risk candidates based on the entity vulnerability index; The scoring and sorting unit is used to calculate the final score and output the TOP-N root causes of failure.

10. A fault location system for a knowledge graph-based data platform according to claim 9, characterized in that: The optimization iteration module includes a sample replenishment unit and a template extraction unit; The sample supplementation unit is used to standardize new fault cases and supplement them to the sample library; the template extraction unit is used to extract and optimize fault fragment templates from high-frequency faults.

Images