Automated Construction Methods for Industry Knowledge Graphs Based on Multi-Source Heterogeneous Data

Abstract

This invention discloses an automated method for constructing industry knowledge graphs from multi-source heterogeneous data, belonging to the field of knowledge engineering technology. It supports structured, semi-structured, unstructured, and distributed data, and uses Kafka to cache high-concurrency data. Preprocessing and standardization are performed, including cleaning, field unification, text conversion, and format conversion. A pre-trained model and rule engine are employed to cover entities and relationships across multiple industries. A dynamic schema is designed, stored in a graph database, and indexed. Evaluation and correction are conducted from multiple dimensions. Dynamic updates and maintenance are implemented, with incremental updates based on data changes, supporting version management and knowledge services. This invention achieves automated construction of knowledge graphs from multi-source heterogeneous data, improving construction efficiency and quality, supporting cross-domain integration, adapting to dynamic updates, and facilitating intelligent applications across multiple industries.

Description

Technical Field

[0001] This invention relates to the field of knowledge engineering technology, and in particular to an automated method for constructing industry knowledge graphs for multi-source heterogeneous data. Background Technology

[0002] With the deepening of digital transformation across industries, sectors such as finance, manufacturing, and healthcare have accumulated massive amounts of multi-source, heterogeneous data. This data is scattered across different systems, encompassing structured database tables, semi-structured XML / JSON files, and unstructured text reports and PDF documents. Knowledge graphs, as a core carrier for integrating scattered knowledge and realizing intelligent applications, are increasingly in demand for construction. The financial sector needs to integrate customer, account, and transaction data through knowledge graphs to achieve risk warnings; the manufacturing sector needs to link equipment, factory, and production order data to support operational decisions; and the healthcare sector needs to integrate disease, symptom, and treatment plan data to assist in clinical diagnosis. However, the diversity and complexity of multi-source, heterogeneous data have become the main bottleneck for the large-scale construction of knowledge graphs.

[0003] Existing knowledge graph construction methods have many limitations: the data access stage relies on customized development, requiring the writing of dedicated parsing scripts for different data sources, resulting in poor scalability and difficulty in adapting to scenarios with concurrent access from 100+ data sources; the preprocessing stage lacks quantitative assessment of data heterogeneity, and resource allocation is based on experience, leading to insufficient preprocessing of highly heterogeneous data and waste of resources for low-heterogeneity data; entity and relation extraction largely relies on a single pre-trained model, failing to fully integrate industry expert knowledge, resulting in insufficient extraction accuracy for low-resource industries (such as new energy and new materials), and entity alignment relies solely on string similarity, ignoring semantic and contextual features, leading to high matching errors across data sources. Schema designs are mostly static and fixed structures, unable to dynamically adapt to newly added categories and relationships of industry knowledge. When emerging concepts such as "industrial robot arms" or "new drug clinical trials" appear, the schema needs to be redesigned manually, which can take several weeks. Updates and maintenance often adopt a full reconstruction approach. In the face of high-frequency scenarios where financial transactions are updated hourly and equipment data is updated daily, this is not only time-consuming and labor-intensive, but can also lead to service interruptions and affect real-time applications. In addition, existing methods are mostly limited to the integration of knowledge from a single industry and lack cross-domain integration mechanisms. Knowledge silos are formed between financial customer data and medical patient data, and between manufacturing equipment data and logistics cargo data, which cannot support the needs of cross-industry correlation analysis.

[0004] These issues directly lead to long construction cycles, high costs, and unstable quality in knowledge graph construction. Traditional methods require several months to half a year to build an industry knowledge graph, with core entity coverage often below 85%, attribute conflict rates exceeding 5%, and difficulty adapting to dynamic knowledge growth and high-frequency update requirements. In scenarios with stringent timeliness and accuracy requirements, such as financial risk early warning and real-time maintenance of manufacturing equipment, knowledge graphs built using existing methods often lag behind business needs, failing to fully leverage their knowledge support value and hindering the large-scale deployment and in-depth application of knowledge graphs across various industries. Summary of the Invention

[0005] The present invention proposes an automated method for constructing industry knowledge graphs based on multi-source heterogeneous data, in order to solve the problems mentioned in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an automated method for constructing industry knowledge graphs based on multi-source heterogeneous data, comprising:

[0007] Step 1: Unified access to multi-source heterogeneous data: Structured data is synchronized in real time using JDBC / ODBC protocol, semi-structured data is parsed and extracted using XPath / JSONPath, and unstructured data is parsed using Apache Tika; distributed data access is supported, high-concurrency data is cached through Kafka message queue, and access nodes are dynamically expanded;

[0008] Step 2, Data Preprocessing and Standardization: Perform cleaning operations; standardize field names and data types of structured data according to industry data dictionaries; process unstructured text through word segmentation, stop word removal, and word vector conversion; convert semi-structured data into JSON-LD format;

[0009] Step 3, Joint Entity and Relation Extraction: A pre-trained model + rule engine fusion architecture is adopted. The BERT-BiLSTM-CRF model identifies entities, covering 100+ core industry categories; the Relation-BERT model based on the Attention mechanism classifies relationships; the rule engine integrates industry expert knowledge to correct model extraction errors.

[0010] Step 4: Knowledge Graph Construction and Storage: Design an industry-standard schema that supports dynamically expanding classes and attributes; store extracted triples and entity attributes in a graph database; build an entity index that supports fuzzy queries and semantic retrieval.

[0011] Step 5, Graph Quality Assessment and Optimization: Evaluation is conducted from four dimensions: accuracy, completeness, consistency, and timeliness; anomaly detection algorithms are used to identify erroneous triples, which are then corrected through manual review via an interface; attributes are added to sparse entities;

[0012] Step 6, Dynamic Update and Maintenance of the Atlas: Set incremental update trigger conditions based on data change rate; adopt differential update mechanism; establish version management mechanism to support version rollback; provide knowledge services to external systems through API interface.

[0013] Furthermore, it also includes:

[0014] Data heterogeneity measurement steps: Quantify the heterogeneity of multi-source data to guide preprocessing resource allocation. First, label each data source by type: structured data is labeled as type 1, semi-structured data as type 2, and unstructured data as type 3. Calculate the type difference coefficient T for the i-th data source. i Next, the data source structure is compared with the industry standard structure, and the structural difference coefficient S is calculated. i =1 - (number of matching fields / total number of standard fields); then calculate the semantic similarity x of the data fields using the Word2Vec model, and construct the semantic difference function Sem. i (x) = 1 - x, for Sem i (x) is integrated over the interval [0,1] to obtain the semantic difference integral value; finally, the overall heterogeneity is calculated using the following formula: Where H represents the overall data heterogeneity; n represents the total number of data sources;

[0015] Entity alignment optimization steps: First, calculate the string similarity Sim between entity names. str Using the Levenshtein edit distance algorithm, Sim str =1 - (edit distance / Max); then generate entity word vectors using an industry-specific word vector model and calculate the cosine similarity Sim. sem = (vector A • vector B) / (||A|| • ||B||); then extract entity context features and calculate the context matching degree Sim using Jaccard similarity. ctx = (Context Intersection / Context Union); Finally, the alignment confidence score is calculated using the formula: , where C is the alignment confidence score; α=0.3, β=0.4, γ=0.3 are the weights.

[0016] Furthermore, step 2 of the data preprocessing also includes deep cleaning and conflict resolution strategies. For numerical data, the mean μ and standard deviation σ are calculated first, and the 3σ criterion is used. Outliers are filtered out through box squaring. Figure 2 Second verification.

[0017] Furthermore, in step 3, entity and relation extraction adopts a collaborative optimization mechanism between models. For the new energy and new materials industries, transfer learning is used to transfer weights from the pre-trained model of the general knowledge graph, freeze 60% of the bottom layer parameters, and fine-tune the top layer parameters with industry data.

[0018] Furthermore, in step 4, the knowledge graph schema adopts a dynamic adaptation design. During the initial schema construction phase, when referring to international / national industry standards, the semantic distance with existing classes is calculated using WordNet thesaurus. After schema changes, consistency verification tools are used to check whether there are conflicts in class hierarchy and attribute constraints. A schema version log is established to record the class / relation name, change time, and operator for each change, supporting version backtracking.

[0019] Furthermore, in step 5, the map quality assessment employs a multi-dimensional quantitative model, introducing feature importance calculation to optimize the assessment weights. First, four core assessment features are defined: accuracy, completeness, consistency, and timeliness. Then, 1000 labeled samples are selected, and their quality levels are manually labeled. A quality prediction model is constructed, using the four features as input and the manually labeled quality levels as output, and the model is trained. The importance weight of each feature is calculated using the model, with the following formula: , where I k The importance weight of the k-th feature; M=1000 is the number of samples; P m F represents the predicted quality probability of the m-th sample; k,m Let ∂P be the value of the k-th feature of the m-th sample; m / ∂F k,m This is the partial derivative of the prediction probability with respect to the eigenvalues.

[0020] Furthermore, in step 6, the incremental update adopts a triggering and parallel processing mechanism, and the triggering condition is set with a dual threshold mechanism; the update task processing adopts the Spark distributed framework, splitting tasks according to data source type + entity category, and scheduling between tasks through DAG dependency; and adopts a read-write separation + dual-database switching architecture.

[0021] Furthermore, it also includes:

[0022] Cross-domain knowledge fusion steps: First, construct a cross-domain semantic mapping table, with domain experts participating in the formulation of mapping rules, including mapping of the same identifier class, synonym class, and attribute; then, use the TransH model for cross-domain entity alignment, mapping entities from the source and target domains to a hyperplane, calculating the L2 norm distance of entity vectors, and the similarity Sim. cross =1 / ||hr-tr||2, where hr is the source domain entity vector and tr is the target domain entity vector; after fusion, a cross-domain knowledge index is constructed to support cross-industry related queries; connectivity is evaluated by calculating the proportion of connected components in the graph;

[0023] Knowledge service optimization steps: At the interface design level, both RESTful API and GraphQL interfaces are provided. The RESTful API supports standardized queries, with parameters supporting pagination and filtering; the GraphQL interface supports custom queries. At the performance optimization level, batch queries adopt an asynchronous processing mechanism. The knowledge reasoning interface integrates dual inference engines: the rule-based inference engine loads industry-specific inference rules; and the distributed representation inference engine predicts missing relationships. At the monitoring and operation level, an API monitoring dashboard is built to display real-time interface call volume (TPS), success rate, and average response time, with threshold alarms set and records of caller IP, query content, and processing results. The optimized knowledge service is adapted to question-and-answer, decision support, and risk warning application scenarios.

[0024] Compared with existing technologies, the beneficial effects of this invention are:

[0025] In the data access and preprocessing stages, the unified access framework supports flexible access to multiple types of data sources, adapting to structured, semi-structured, and unstructured data without custom development, significantly improving access scalability and efficiency. The data heterogeneity measurement mechanism enables quantitative assessment of heterogeneity, allowing preprocessing resources to be allocated on demand, avoiding resource waste and insufficient processing. The deep cleaning and conflict resolution strategies design differentiated processing solutions for different types of data, combining industry synonym unification and data source credibility weighting, greatly improving data quality and laying a reliable foundation for subsequent extraction stages.

[0026] At the entity relationship extraction and schema design level, a multi-model collaborative optimization mechanism integrates pre-trained models, remote supervision, and industry rule engines. This enhances the model's adaptability to industry knowledge and improves extraction accuracy in low-resource industries. Entity alignment incorporates multi-dimensional features such as strings, semantics, and context, significantly reducing cross-data source matching errors. Dynamic schema adaptation design can automatically identify newly added categories and relationships, completing structural adjustments without manual intervention. This flexibly adapts to the dynamic growth of industry knowledge, addressing the pain point of poor scalability in traditional static schemas.

[0027] In the knowledge graph maintenance and fusion phase, incremental updates employ intelligent triggering and parallel processing mechanisms, processing only newly added or modified data. Combined with read-write separation and dual-database switching, update efficiency is significantly improved, and service interruption time is minimized, meeting the needs of high-frequency data update scenarios. The cross-domain knowledge fusion mechanism breaks down industry knowledge silos through semantic mapping and entity alignment, enabling the integration of knowledge from different domains and expanding the application boundaries of knowledge graphs.

[0028] Knowledge service optimization further enhances the practicality of the knowledge graph. The dual-interface design adapts to the query needs of different application scenarios. The integration of batch asynchronous processing and the inference engine improves service response efficiency and functional richness. The monitoring panel and anomaly alarm mechanism ensure service stability. Overall, this invention shortens the knowledge graph construction cycle, improves graph quality and maintenance efficiency, breaks down knowledge silos, and provides high-quality, highly available knowledge support for industry applications such as financial risk early warning, manufacturing operation and maintenance decision-making, and medical clinical assistance, effectively promoting the large-scale deployment and in-depth application of knowledge graphs. Attached Figure Description

[0029] Figure 1 This is a schematic block diagram of the automated construction method for industry knowledge graphs based on multi-source heterogeneous data proposed in this invention.

[0030] Figure 2 This diagram illustrates the performance comparison of different data source access methods for the automated construction method of industry knowledge graphs for multi-source heterogeneous data proposed in this invention.

[0031] Figure 3 This diagram illustrates the incremental update performance comparison of the automated construction method for industry knowledge graphs based on multi-source heterogeneous data proposed in this invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0034] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified. Furthermore, the terms "installed," "connected," and "linked" should be interpreted broadly; for example, they may refer to a fixed connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. The invention will now be described in further detail with reference to the accompanying drawings.

[0035] Reference Figures 1 to 3 This application presents an automated method for constructing industry knowledge graphs from multi-source heterogeneous data. The method is applied to knowledge integration and intelligent applications in industries such as finance, manufacturing, and healthcare. It belongs to the fields of knowledge engineering and data mining technology. The method includes the following steps:

[0036] Step 1: Unified access to multi-source heterogeneous data: For structured data (relational database tables, CSV files), JDBC / ODBC protocol is used for real-time synchronization; semi-structured data (XML, JSON, HTML web pages) is parsed and extracted using XPath / JSONPath; unstructured data (text reports, PDF documents, image OCR text) is parsed using Apache Tika; distributed data access (HDFS, S3 object storage) is supported, and high-concurrency data is cached through Kafka message queues. Access nodes can be dynamically expanded (a single node supports 100+ concurrent data source accesses, with a data throughput ≥100MB / s).

[0037] Step 2: Data Preprocessing and Standardization: Perform cleaning operations (remove null values, redundant and duplicate data, and filter special characters based on regular expressions); standardize field names and data types of structured data according to industry data dictionaries (such as the "Customer ID-Account Type-Transaction Amount" standard in the financial industry); process unstructured text through word segmentation (jieba / StanfordCoreNLP), stop word removal (based on industry stop word lists), and word vector conversion (Word2Vec / GloVe, adjustable from 100 to 300 dimensions); convert semi-structured data to JSON-LD format to ensure semantic consistency across different data sources.

[0038] Step 3: Joint Entity and Relation Extraction: A fusion architecture of "pre-trained model + rule engine" is adopted. The BERT-BiLSTM-CRF model identifies entities (such as "equipment model-manufacturer-manufacture date" in the manufacturing industry), covering more than 100 core industry categories. The Relation-BERT model based on the Attention mechanism classifies relations (such as "equipment-belongs to-factory" and "product-related-raw materials"). The rule engine integrates industry expert knowledge (such as the syntactic rules of "credit card-related-cardholder" in the financial industry) to correct the model extraction error. The F1 score for entity recognition is ≥0.92 and the F1 score for relation extraction is ≥0.88.

[0039] Step 4: Knowledge Graph Construction and Storage: Design an industry-standard schema (containing a hierarchical structure of classes, attributes, and relationships, such as the "disease-symptom-treatment plan" hierarchy in the medical industry), supporting dynamic expansion of classes and attributes; store the extracted triples (entity 1-relationship-entity 2) and entity attributes (entity-attribute-value) in a graph database (Neo4j / OrientDB, supporting storage of 1 billion triples, with a query response time ≤100ms); build an entity index (based on Elasticsearch), supporting fuzzy queries and semantic retrieval.

[0040] Step 5: Graph Quality Assessment and Optimization: Evaluation is conducted across four dimensions: accuracy (entity / relation extraction accuracy), completeness (core entity coverage), consistency (attribute value conflict rate), and timeliness (data update delay). Anomaly detection algorithms (Isolation Forest / LOF) are used to identify erroneous triples, which are then corrected via a manual review interface (supporting batch annotation). Attributes are supplemented for sparse entities (missing attributes are predicted based on the TransE knowledge reasoning model), achieving a core entity coverage rate ≥95% and an attribute conflict rate ≤2%.

[0041] Step 6: Dynamic Update and Maintenance of the Knowledge Map: Set incremental update trigger conditions based on data change rate (e.g., financial transaction data is updated hourly, manufacturing equipment data is updated daily); adopt a differential update mechanism (only update the triples corresponding to newly added / changed data) to avoid full reconstruction; establish a version management mechanism (retain historical versions of the last 12 months) to support version rollback; provide knowledge services to external systems (intelligent question answering, decision support platform) through API interface, with an interface call success rate of ≥99.9%.

[0042] This invention also includes:

[0043] Data heterogeneity measurement steps: Quantify the heterogeneity of multi-source data to guide preprocessing resource allocation. The specific process is as follows: First, label each data source by type. Structured data (such as MySQL tables and Oracle views) is labeled as type 1, semi-structured data (such as API returns in JSON format and e-commerce platform XML orders) is labeled as type 2, and unstructured data (such as medical case PDFs and manufacturing equipment maintenance reports in TXT format) is labeled as type 3. Calculate the type difference coefficient T for the i-th data source. i (Type 1 uses 0.1, Type 2 uses 0.5, and Type 3 uses 1.0); Next, the data source structure is compared with the industry standard structure (e.g., the financial industry refers to the FIBO standard, and the medical industry refers to the SNOMEDCT standard), and the structural difference coefficient S is calculated. i =1 - (number of matching fields / total number of standard fields), with a value range of 0-1; then, the semantic similarity x of the data fields is calculated using the Word2Vec model (trained based on industry corpus, 200 dimensions), and the semantic difference function Sem is constructed. i (x) = 1 - x (the larger x is, the greater the semantic difference in the novel), for Sem i (x) is integrated over the interval [0,1] to obtain the semantic difference integral value; finally, the overall heterogeneity is calculated using the following formula: Where H represents the overall data heterogeneity (0-1); n represents the total number of data sources; w t =0.4、w s =0.3、w s em=0.3 is the weighting coefficient (determined through the analytic hierarchy process). Priorities are determined based on the H value: H≥0.7 is high priority (allocated 40% of computing resources), 0.4≤H<0.7 is medium priority (35% of resources), and H<0.4 is low priority (25% of resources), ensuring accurate allocation of preprocessing resources;

[0044] Entity alignment optimization steps: Improve the accuracy of cross-data source entity matching, specifically by: first calculating the string similarity Sim between entity names. str The Levenshtein edit distance algorithm (the smaller the edit distance, the higher the similarity) is used. Sim str =1 - (edit distance / Max(entity 1 length, entity 2 length)); Secondly, entity word vectors are generated using an industry-specific word vector model (a GloVe model trained on a corpus of 100,000+ industry entities, with a window size of 10), and cosine similarity Sim is calculated. sem = (Vector A • Vector B) / (||A|| • ||B||); then extract the entity context features (such as other entities associated with the entity, attribute values, such as "Manufacturer B" and "Date of manufacture 2023-01" associated with "Device A"), and calculate the context matching degree Sim using Jaccard similarity. ctx= (Context Intersection / Context Union); Finally, the alignment confidence score is calculated using the following formula: Where C represents the alignment confidence score (0-1); α=0.3, β=0.4, and γ=0.3 are weights (determined based on entity alignment sample training). A three-level processing mechanism is set up: entities with C≥0.8 are automatically merged (retaining attributes from each data source, and conflicting attributes are marked for review); entities with 0.5≤C<0.8 are pushed to the manual review platform (supporting batch annotation and manual selection of conflicting attributes); and entities with C<0.5 are judged as different entities. Through this strategy, the entity alignment accuracy is improved to ≥0.94, which is 15%-20% higher than traditional single-feature alignment.

[0045] In this invention, the data preprocessing in step 2 also includes deep cleaning and conflict resolution strategies. Specifically, for numerical data (such as financial transaction amounts, manufacturing equipment temperatures, and medical testing indicators), the mean μ and standard deviation σ are first calculated, and the 3σ criterion is used (data with |x-μ|>3σ is marked as outliers). Outliers are then filtered through box-shaped data structures. Figure 2 Secondary verification (values ​​exceeding 1.5 times the interquartile range are considered anomalies). When the outlier rate is ≤5%, linear interpolation is used for imputation (xinterp=xprev+(xnext-xprev)*(tinterp-tprev) / (tnext-tprev)); when the rate is >5%, the KNN algorithm (k=5) is used. For text-based data, an industry-specific thesaurus is constructed (e.g., in the financial field, "savings card - debit card" and "wealth management product - asset management plan"; in the medical field, "myocardial infarction - acute myocardial infarction" and "hypertension - elevated blood pressure"). A dual determination of similarity is made through string matching and semantic similarity (≥0.85). Synonyms are uniformly mapped to standard terminology. For conflicting data (inconsistent attribute values ​​for the same entity across different data sources, such as a customer's "account opening date" being 2023-01-01 in the bank's core system but 2023-01-02 in the mobile banking logs), the credibility of the data source is first assessed: authoritative databases (such as the bank's core system or hospital HIS system) are weighted at 0.8, business systems (such as OA systems or ERP systems) at 0.5, and log data at 0.2. The weighted average value Cdata = ∑(wj・Dj) is calculated, and the attribute value closest to Cdata is taken as the final value. If the differences are the same, the value from the authoritative data source is preferred. The accuracy of the deeply cleaned data is ≥0.96, providing a high-quality data foundation for subsequent extraction.

[0046] In this invention, the entity and relation extraction in step 3 adopts a multi-model collaborative optimization mechanism. The specific process is as follows: In the first stage, a BERT-LSTM-CRF entity extraction model is constructed based on a pre-trained BERT model (industry-adjusted versions, such as financial BERT and medical BERT). The BERT layer outputs word vectors (768 dimensions), the LSTM layer has a two-layer bidirectional structure (256 hidden layer dimensions), and the CRF layer outputs entity labels (such as B-DEVICE, I-DEVICE, O), initially extracting a candidate entity set; In the second stage, remote supervised annotation: the triples in the existing knowledge graph (such as "equipment A-manufacturer-manufacturer B") are matched with unstructured text, and the text simultaneously appears Sentences containing "equipment A" and "manufacturer B" and keywords such as "production" are labeled as positive examples, with 5000+ new samples added for each entity class to expand the training set; the third stage is adversarial training optimization: Gaussian noise (mean 0, standard deviation 0.1) is added to the BERT word vector layer, and adversarial examples are generated using the FGSM algorithm to enhance the model's robustness to noise; the fourth stage is rule engine filtering: industry syntax rules (such as "entity X + verb 'production' + entity Y → X-manufacturer-Y" and "entity A + 'located at' + entity B → A-location-B") are integrated to correct erroneous relationships extracted by the model (such as "manufacturer B-production-equipment A" being mistakenly extracted as "equipment A-production-manufacturer B"). For low-resource industries such as new energy and new materials, transfer learning is adopted: weights are transferred from pre-trained models of general knowledge graphs (such as KnowBERT), 60% of the bottom-level parameters are frozen, and the top-level parameters are fine-tuned with a small amount of industry data (1000+ samples). The F1 score for entity recognition is improved by 10%-15%, and the F1 score for relation extraction is improved by 8%-12%.

[0047] In this invention, the knowledge graph schema in step 4 adopts a dynamic adaptation design. Specifically, in the initial schema construction phase, referring to international / national industry standards (FIBO (Financial Industry Business Ontology) for the financial industry, ISO 8373 (Robotics and Automation Terminology) for the manufacturing industry, and ICD-11 (International Classification of Diseases) for the medical industry), core classes (such as "customer," "account," and "transaction" in finance, and "equipment," "factory," and "production order" in manufacturing), attributes (such as "customer-age-integer" and "equipment-model-string"), and relationships (such as "customer-owner-account" and "equipment-participate-production order") are defined to form a hierarchical structure (such as "account" with subclasses "debit card account" and "credit card account"); the schema adapts... During the configuration phase, new data sources are monitored in real time: When an undefined class appears (e.g., adding an "industrial robot arm" class in the manufacturing industry), the semantic distance between this class and existing classes is calculated using WordNet thesaurus (path distance = number of nodes - 1, semantic distance = 1 / path distance). If the semantic distance with "industrial robot" is ≤0.3, it is classified as a subclass; otherwise, a new top-level class is added. When the frequency of an undefined relationship is ≥1000 (e.g., "equipment-maintenance-engineer"), a new "maintenance" relationship type is added, supplementing the domain (equipment class) and value domain (personnel class). After schema changes, consistency verification tools (e.g., Protégé's inference engine) are used to check for conflicts in class hierarchy and attribute constraints; the change takes effect when the conflict rate is ≤1%. A schema version log is established to record the class / relationship name, change time, and operator for each change, supporting version rollback. Dynamic adaptation response time is ≤5 minutes, supporting the expansion of 100+ industry categories to meet the dynamic growth needs of industry knowledge.

[0048] In this invention, step 5 of the graph quality assessment adopts a multi-dimensional quantitative model, introducing feature importance calculation to optimize the assessment weights. The specific process is as follows: First, define four core assessment features: accuracy (Acc, number of correct entity / relation extractions / total number of extractions), completeness (Com, number of core entity / relation coverages / number of industry standard cores), consistency (Con, number of conflict-free attributes / total number of attributes), and timeliness (Tim, 1 - data update delay / standard delay threshold); then select 1000 labeled samples (covering different industries and data types), and manually label the sample quality levels (1-5 points, with 5 points being optimal); construct a quality prediction model (such as a random forest regression model, with 100 trees and a maximum depth of 15), using the four features as input and the manual quality level as output, and train the model; calculate the importance weight of each feature through the model, using the formula: Among them, I k The importance weight of the k-th feature is (0-1); M=1000 is the number of samples; Pm F represents the predicted quality probability (0-1) of the m-th sample. k,m Let ∂P be the value of the k-th feature of the m-th sample; m / ∂F k,m The partial derivatives of the predicted probabilities with respect to the eigenvalues ​​were calculated using gradient descent. The optimized weights were calculated as: Acc=0.4, Com=0.25, Con=0.2, Tim=0.15, which better reflects the actual quality impact than equal weights (each 0.25). The final quality score is Q=0.4×Acc+0.25×Com+0.2×Con+0.15×Tim, where Q≥0.9 is excellent, 0.7≤Q<0.9 is acceptable, and Q<0.7 indicates areas for improvement. The consistency between the evaluation results and human judgment is ≥0.93, a 10% improvement over traditional evaluation methods.

[0049] In this invention, the incremental update in step 6 adopts an intelligent triggering and parallel processing mechanism, specifically designed as follows: A dual-threshold triggering mechanism is set. The quantitative threshold is triggered when the amount of newly added data is ≥1000 records (or the number of newly added triples is ≥500). The qualitative threshold is triggered when the update frequency of key entities (such as "wealth management products" and "credit customers" in finance, and "new drugs" and "difficult cases" in medicine) is ≥5 times / hour. Key entities are determined by industry expert annotation + TF-IDF keyword ranking (TOP10% entities). The update task processing adopts the Spark distributed framework, splitting tasks according to "data source type + entity category": structured data update, semi-structured data update, and unstructured data update are divided into three main tasks, each further divided into... Entity categories (such as equipment and personnel) are broken down into subtasks, each allocated independent computing resources (CPU ≥ 4 cores, memory ≥ 4GB, disk ≥ 10GB). Tasks are scheduled via DAG dependencies (e.g., an extraction task is triggered after the parsing task is completed). A "read-write separation + dual-database switching" architecture is adopted: the production database (providing query services) and the update database (processing incremental data) are deployed independently. Incremental data is first written to the update database. After extraction, alignment, and storage, master-slave switching is achieved through ZooKeeper (switching time ≤ 500ms), and service interruption time ≤ 1 second. After the update is completed, a consistency check is performed: the attribute values ​​and relationships of core entities before and after the update are compared. If the pass rate is ≥ 99.9%, the update is confirmed as successful; otherwise, a rollback is triggered. This mechanism supports incremental updates of 1000+ triples per second, meeting the needs of high-concurrency data scenarios.

[0050] This invention also includes:

[0051] Cross-domain knowledge fusion steps: To achieve the integration of sub-graphs from different industries, the specific process is as follows: First, construct a cross-domain semantic mapping table, with domain experts participating in the formulation of mapping rules: mapping of the same identifier class (e.g., "ID number" corresponds to "customer ID" in the financial sub-graph and "patient ID" in the medical sub-graph), mapping of synonym classes (e.g., "enterprise" is called "legal entity" in the financial sub-graph and "manufacturing enterprise" in the manufacturing sub-graph), and mapping of attributes (e.g., "establishment time" is called "registration time" in the financial sub-graph and "factory establishment time" in the manufacturing sub-graph); Then, use the TransH model for cross-domain entity alignment: map the entities of the source domain (e.g., the financial sub-graph) and the target domain (e.g., the medical sub-graph) to a hyperplane, calculate the L2 norm distance of the entity vectors, and the similarity Simcross=1 / ||hr-tr||2 (hr is the entity vector of the source domain, tr is the similarity of the target domain). For the target domain entity vector), a "cross-domain association" relationship is established when Simcross ≥ 0.75; conflict detection and resolution during the fusion process: when the same identifier corresponds to different entities (e.g., ID number "110101XXXX" is "Zhang San" in the financial subgraph and "Li Si" in the medical subgraph), the data source is sorted by authority level (government database > industry core system > business log), and information from high-authority data sources is given priority; after fusion, a cross-domain knowledge index is constructed (based on cross-index query of Elasticsearch), supporting cross-industry association queries (e.g., "query the association analysis between 'Zhang San's' bank wealth management product purchase records and medical expenditures"); connectivity is evaluated by calculating the proportion of connected components in the graph (number of nodes in connected components / total number of nodes), and the connectivity after fusion is ≥ 0.85, which is 30%-40% higher than before fusion;

[0052] Knowledge service optimization steps: Improve the ease of use of graph applications, specifically implemented as follows: At the interface design level, provide dual interfaces of RESTful API and GraphQL: The RESTful API supports standardized queries (such as GET / entity / id to get entity attributes, POST / relation / type to query entity pairs with a specified relationship), and parameters support pagination and filtering; The GraphQL interface supports custom queries (such as "query the model, manufacturer and associated production order number of 'device A'"), reducing redundant data returns. In terms of performance optimization, batch queries adopt an asynchronous processing mechanism: after the client submits a batch query task (maximum 100 queries / time), it obtains the task ID. The system background processes the query in parallel through a thread pool (20 core threads, 50 maximum threads). After processing, the results are pushed via WebSocket. The batch query response time is ≤200ms. The knowledge reasoning interface integrates dual reasoning engines: a rule reasoning engine (such as Drools) loads industry reasoning rules (such as "If A → belongs to B, B → belongs to C, then A → belongs to C" and "If equipment → fault type = short circuit, then the repair plan = replace fuse + test circuit"); and a distributed representation reasoning engine (TransE model, vector dimension 100, training iteration 1000 rounds) predicts missing relationships (such as "Given A → production → B, B → raw material → C, predict A → use → C"). The reasoning accuracy is ≥0.85. At the monitoring and operations level, an API monitoring dashboard is built: It displays real-time API call volume (TPS), success rate, and average response time, and allows setting threshold alarms (triggering email and SMS alarms when success rate < 99% or response time > 500ms); it supports API call log querying (retained for 3 months), recording the caller's IP address, query content, and processing results for easy troubleshooting. The optimized knowledge service is adaptable to diverse application scenarios such as intelligent question answering (e.g., customer service robots), decision support (e.g., enterprise risk assessment), and risk warning (e.g., financial fraud detection).

[0053] Example 1: Automated Construction of Customer Knowledge Graph in the Financial Industry

[0054] This embodiment uses the construction of a customer knowledge graph of a joint-stock bank as a scenario, integrating heterogeneous data from multiple sources such as customers, accounts, transactions, and credit to support intelligent risk control and precision marketing. The specific implementation process is as follows:

[0055] 1. Unified access to multi-source heterogeneous data

[0056] The data sources covered include three categories: ① Structured data: Customer tables (containing 20 fields such as customer ID, name, and ID number), account tables (containing 15 fields such as account number, account type, and account opening date), and transaction tables (containing 25 fields such as transaction amount, transaction time, and counterparty) from the bank's core system (Oracle19c), synchronized in real time via JDBC protocol at a frequency of 5 minutes per synchronization; ② Semi-structured data: Mobile banking APP logs (JSON format, containing 12 fields such as user operation behavior and device information) and credit approval XML files (containing 18 fields such as approval opinions and collateral information), extracted via JSONPath / XPath parsing with a parsing latency of ≤1 second; ③ Unstructured data: Customer credit reports (PDF format) and customer service call transcripts (TXT format), parsed using Apache Tika, with OCR recognition accuracy of ≥98% for scanned PDFs. All data is cached through a Kafka cluster (3 nodes, single node throughput 50MB / s), with a total access throughput of 120MB / s, supporting concurrent access from 150+ data sources.

[0057] 2. Data Preprocessing and Heterogeneity Measurement

[0058] Preprocessing steps: ① Cleaning: Remove null values ​​from the transaction table (0.3%) and duplicate records from the customer table (duplicated based on ID number, 0.5% duplication rate), and filter special characters (such as "*" and "#") in transaction remarks using regular expressions; ② Standardization: Standardize field names according to the banking data dictionary (e.g., "custno" is standardized to "customer ID", "txamt" is standardized to "transaction amount"), and standardize data types (transaction amount is converted to decimal(18,2), and account opening date is converted to yyyy-MM-dd format); ③ Text processing: Credit reports are segmented using jieba (loading a financial professional thesaurus containing 3000+ terms such as "non-performing loans" and "collateral ratio"), and stop words are removed (based on a financial stop word list containing 500+ words such as "of" and "and"), and converted into word vectors using the GloVe model (trained based on 1 million financial texts, 200 dimensions).

[0059] Data heterogeneity measurement: Ten data sources are accessed (n=10). The following parameters are calculated for each data source: Ti (structured 0.1, semi-structured 0.5, unstructured 1.0), Si (matching degree with FIBO standard structure, e.g., core system Si=0.1, customer service text Si=0.8), and semantic difference integral ∫0. 1 Semi(x)dx (core system 0.1, customer service text 0.9), substitute into the formula:

[0060] H = (1 / 10) × [0.4 × (3 × 0.1 + 4 × 0.5 + 3 × 1.0) + 0.3 × (3 × 0.1 + 4 × 0.6 + 3 × 0.8) + 0.3 × (3 × 0.1 + 4 × 0.5 + 3 × 0.9)] = 0.62. Resources are allocated based on H value: High priority (H ≥ 0.7, 3 data sources including customer service text) allocates 40% GPU resources; medium priority (0.4 ≤ H < 0.7, 4 data sources including APP logs) allocates 35% resources; low priority (H < 0.4, 3 data sources including core systems) allocates 25% resources.

[0061] 3. Entity Relationship Extraction and Alignment

[0062] Entity Relationship Extraction: A financial-tuned version of the BERT-BiLSTM-CRF model (pre-trained weights based on 100,000 financial texts) is used to identify 30 entity types, including customers (F1=0.94), accounts (F1=0.95), transactions (F1=0.93), and collateral (F1=0.91). The Relation-BERT model is used to classify relationships, such as "customer-owner-account" (F1=0.90), "account-occurrence-transaction" (F1=0.89), and "customer-collateral-collateral" (F1=0.87). The rule engine integrates expert rules: "same customer ID → same customer" and "account number prefix 6222 → savings card account," correcting model mis-extraction (such as misclassifying "counterparty" as "customer"). The overall entity F1 score is improved to 0.95, and the relationship F1 score is improved to 0.90.

[0063] Entity alignment: Calculate the confidence score for customer entity alignment. For example, for "Zhang San" (core system) and "Zhang San" (credit report): Simstr=1.0 (edit distance 0), Simsem=0.98 (word vector cosine similarity), Simctx=0.95 (both associated with "account 622208XXXX"), substituting into the formula C=0.3×1.0+0.4×0.98+0.3×0.95=0.97≥0.8, automatically merged; for "Zhang Xiaosan" (APP log) and "Zhang San" (core system), C=0.65, pushed for manual review, the final alignment accuracy is 0.96.

[0064] 4. Schema Construction and Graph Storage

[0065] The schema design is based on the FIBO standard: core classes include "Person" (customer), "FinancialAccount" (account), and "FinancialTransaction" (transaction), with a class hierarchy of "Person→IndividualCustomer" and "FinancialAccount→SavingsAccount / CreditAccount"; attribute definitions include "Person.hasID" (data type string, required) and "FinancialTransaction.hasAmount" (decimal, required); relationship definitions include "Person.owns.FinancialAccount" (domain Person, value domain FinancialAccount). A new class "DigitalChannel" (entity added to APP logs) is added, and its semantic distance from "Person" is calculated to be 0.25≤0.3, classifying it as an associated class of "Person". The graph is stored in a Neo4j cluster (3 masters and 3 slaves), storing 120 million triples. The response time for a single entity attribute query is 80ms, and the response time for a join query is 100ms. Elasticsearch builds the entity index, supporting "fuzzy search for customer name" and "query by transaction amount range", with a retrieval response time of ≤50ms.

[0066] 5. Quality Assessment and Optimization

[0067] Multi-dimensional evaluation: accuracy (entity extraction accuracy 0.95, relationship accuracy 0.90), completeness (core customer coverage 0.98), consistency (attribute conflict rate 0.8%), and timeliness (data update delay 5 minutes). The Isolation Forest algorithm was used to identify erroneous triples (e.g., "Customer A - owns - Account B" but Account B was opened by Customer C), with an identification rate of 0.92. Corrections were made through a manual review platform (supporting batch annotation, processing 1000 records per batch). For sparse entities (e.g., newly added customers only contain name and ID number), the TransE model predicted missing attributes (e.g., age, occupation), with a prediction accuracy of 0.85. Feature importance calculation: IAcc=0.42, ICom=0.24, ICon=0.20, ITim=0.14. After weight optimization, the quality score Q=0.92, indicating a high-quality graph.

[0068] 6. Incremental Updates and Knowledge Services

[0069] Incremental updates are triggered when there are ≥1000 new transactions per hour or ≥50 new customers per hour. Spark distributed processing (4 nodes, 8 cores per node, 16GB) splits tasks: transaction data parsing (2 cores, 4GB), customer entity extraction (4 cores, 8GB), and relationship alignment (2 cores, 4GB), with parallel processing taking 15 seconds. Read / write separation is used, with the production database providing services and the update database processing data; the switching time is 300ms, and service interruption is ≤1 second. The knowledge service provides a RESTful API: querying customer-related accounts ( / api / entity / customer / id / accounts) and counterparty analysis ( / api / relation / transaction / id / counterpart), supporting batch queries (100 records / time) with a response time of 150ms. The GraphQL interface supports custom queries (e.g., "query customer Zhang San's savings card account and transactions in the last 3 months"). It integrates rule-based reasoning ("customer has 3 overdue payments → high-risk customer") and TransE reasoning (predicting "customer - potential demand - financial products"), with an inference accuracy of 0.88.

[0070] Table 1: Comparison of the effects of constructing financial customer knowledge graphs

[0071]

[0072] Table 1 shows that this invention significantly shortens the construction cycle (to only 1 / 6 of the traditional method), significantly improves entity recognition accuracy and coverage, increases incremental update efficiency by 480 times, and enhances API service stability. The intelligent risk control system supported by knowledge graphs improves the early detection rate of non-performing loans by 30% and the conversion rate of targeted marketing by 25%, fully demonstrating its application value in the financial industry.

[0073] Example 2: Automated Construction of Equipment Knowledge Graph in the Manufacturing Industry

[0074] This embodiment uses the construction of an equipment knowledge graph for an automobile manufacturer as a scenario. It integrates multi-source data such as equipment, production, operation and maintenance, and materials to support predictive maintenance of equipment and optimization of production scheduling. The specific implementation process is as follows:

[0075] 1. Unified access to multi-source heterogeneous data

[0076] The data sources include: ① Structured data: Equipment status table (equipment ID, temperature, pressure, etc., 18 fields) from the PLC control system (S7-1500), production order table (order number, product model, etc., 12 fields) from the MES system (SQL Server 2019), and material table (material code, name, etc., 10 fields) from the ERP system, synchronized in real time via ODBC protocol, with a data synchronization frequency of 1 second / time; ② Semi-structured data: Equipment operation and maintenance logs (JSON format, including fault description, maintenance record, etc., 15 fields), and production process XML files (including process parameters, cycle time, etc., 20 fields), parsed via JSONPath / XPath, with a parsing success rate of ≥99%; ③ Unstructured data: Equipment manuals (PDF format), and OCR text of maintenance work orders (including handwritten maintenance records), parsed using Apache Tika + Baidu OCR, with an OCR accuracy rate of ≥95%. Data is cached through a Kafka cluster (4 nodes), with a total throughput of 110MB / s, supporting concurrent access from 120+ data sources.

[0077] 2. Data Preprocessing and Conflict Resolution

[0078] Preprocessing steps: ① Numerical cleaning: Outliers (such as a sudden temperature spike of 999℃) are removed from equipment temperature data using the 3σ criterion, with an outlier rate of 0.6%. Missing values ​​are filled using linear interpolation (continuous data collection interval ≤ 5 seconds); ② Text processing: Maintenance logs are segmented using Stanford CoreNLP (loading a manufacturing-specific thesaurus containing 2000+ terms such as "bearing wear" and "belt loosening"). After stop words are removed, a Word2Vec model (trained based on 500,000 maintenance texts) is used. ③ Conflict resolution: The "manufacturing date" of the same device is 2020-01-10 in the PLC system and 2020-01-15 in the ERP system. The data source credibility weights are PLC=0.8 and ERP=0.5. Cdata=(0.8×20200110+0.5×20200115) / (0.8+0.5)≈20200112. Take 2020-01-10 (closer to the weighted value and PLC has higher credibility).

[0079] Data heterogeneity measurement: n=8 data sources, T i (Structured 0.1, Semi-structured 0.5, Unstructured 1.0), S i (Compatibility with ISO8373 standard, PLC system S) i =0.2, instruction manual S i=0.9), semantic integral (PLC system 0.2, manual 0.9), calculate H=0.65, resource allocation: high priority (manual and other 2 data sources) 40% resources, medium priority (maintenance logs and other 4) 35%, low priority (PLC and other 2) 25%.

[0080] 3. Entity Relationship Extraction and Alignment

[0081] Entity Relation Extraction: A manufacturing-tuned version of the BERT-BiLSTM-CRF model was used to identify 25 entity types, including equipment (F1=0.94), parts (F1=0.93), production orders (F1=0.92), and materials (F1=0.91). Relation-BERT classification relationships were also used, such as "equipment-contains-parts" (F1=0.90), "equipment-execution-production order" (F1=0.88), and "production order-consumption-material" (F1=0.87). The rule engine integrated expert rules: "equipment model prefix 'EQ' → production equipment" and "part name containing 'bearing' → rotating parts," correcting model mis-extraction (such as misclassifying "material pallet" as "equipment"). The entity F1 value was improved to 0.96, and the relation F1 value was improved to 0.91.

[0082] Entity alignment: Equipment "EQ001" (PLC system) and "EQ-001" (MES system): Sim str =0.9 (edit distance 1), Sim sem =0.99 (word vector similarity), Sim ctx =0.96 (all associated with “Production Order PO202509”), C=0.3×0.9+0.4×0.99+0.3×0.96=0.97≥0.8, automatically merged; alignment accuracy 0.95.

[0083] 4. Schema Construction and Graph Storage

[0084] The schema design is based on the ISO 8373 standard: core classes include "Equipment", "Component", and "ProductionOrder", with a class hierarchy of "Equipment → ProductionEquipment / MaintenanceEquipment" and "Component → RotatingComponent / StaticComponent". Attribute definitions include "Equipment.hasModel" (string, required) and "ProductionOrder.hasQuantity" (int, required). Relationship definitions include "Equipment.contains.Component" and "ProductionOrder.uses.Equipment". A new class, "AIInspectionRecord" (AI inspection record), is added (new entity added for device detection). Its semantic distance from "Equipment" is 0.28 ≤ 0.3, classifying it as an association class. The graph is stored in an OrientDB cluster (2 master, 2 slave), storing 80 million triples. Device status query response time is 70ms, and production order association query response time is 90ms. Elasticsearch is used to build the index, supporting "querying fault records by device model", with a retrieval response time ≤ 40ms.

[0085] 5. Quality Assessment and Optimization

[0086] Multi-dimensional evaluation: accuracy (entity accuracy 0.96, relationship accuracy 0.91), completeness (core equipment coverage 0.97), consistency (attribute conflict rate 0.7%), and timeliness (data update delay 3 minutes). The LOF algorithm identified incorrect triples (e.g., "device EQ001-contains-part C002" but C002 belongs to EQ002), with a recognition rate of 0.93. After manual correction, the accuracy improved to 0.98. The TransE model predicted sparse equipment attributes (e.g., "maintenance cycle") with a prediction accuracy of 0.86. Feature importance calculation: IAcc=0.41, ICom=0.25, ICon=0.21, ITim=0.13, quality score Q=0.91, indicating a high-quality map.

[0087] 6. Incremental Updates and Knowledge Services

[0088] Incremental updates are triggered when device status changes ≥5 times / hour or production orders are added ≥20 times / hour. Spark distributed processing (3 nodes, 8 cores / 16GB per node) splits tasks: device data parsing, component relationship alignment, etc., with parallel processing taking 12 seconds. Read / write separation switching time is 250ms, and service interruption is ≤1 second. The knowledge service provides a RESTful API: querying device component composition ( / api / entity / equipment / id / components) and production order associated devices ( / api / relation / order / id / equipment), with a batch query response time of 120ms. The GraphQL interface supports custom queries (e.g., "query the bearing components of EQ001 and fault records for the past month"). Integrating rule-based reasoning ("bearing temperature ≥80℃ → requires maintenance") and TransE reasoning (predicting "equipment - requires replacement - component"), the reasoning accuracy is 0.89.

[0089] Table 2: Comparison of the effects of knowledge graph construction for manufacturing equipment

[0090]

[0091] Table 2 shows that the construction cycle of this invention is only 1 / 6 of that of traditional methods, entity recognition accuracy is improved by 15%, core equipment coverage is improved by 14%, incremental update efficiency is improved by 450 times, and inference accuracy is improved by 17%. Based on this map, the predictive maintenance system reduces equipment downtime by 40% and improves production scheduling efficiency by 30%, fully verifying its application value in the manufacturing industry.

[0092] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for automatically constructing industry knowledge graphs for multi-source heterogeneous data, characterized in that, include: Step 1: Unified access to multi-source heterogeneous data: Structured data is synchronized in real time using JDBC / ODBC protocol, semi-structured data is parsed and extracted using XPath / JSONPath, and unstructured data is parsed using Apache Tika; distributed data access is supported, high-concurrency data is cached through Kafka message queue, and access nodes are dynamically expanded; Step 2, Data Preprocessing and Standardization: Perform cleaning operations; standardize field names and data types of structured data according to industry data dictionaries; process unstructured text through word segmentation, stop word removal, and word vector conversion; convert semi-structured data into JSON-LD format; Step 3, Joint Entity and Relation Extraction: A pre-trained model + rule engine fusion architecture is adopted. The BERT-BiLSTM-CRF model identifies entities, covering 100+ core industry categories; the Relation-BERT model based on the Attention mechanism classifies relationships; the rule engine integrates industry expert knowledge to correct model extraction errors. Step 4: Knowledge Graph Construction and Storage: Design an industry-standard schema that supports dynamically expanding classes and attributes; store extracted triples and entity attributes in a graph database; build an entity index that supports fuzzy queries and semantic retrieval. Step 5, Graph Quality Assessment and Optimization: Evaluation is conducted from four dimensions: accuracy, completeness, consistency, and timeliness; anomaly detection algorithms are used to identify erroneous triples, which are then corrected through manual review via an interface; attributes are added to sparse entities; Step 6, Dynamic Update and Maintenance of the Map: Set incremental update trigger conditions based on the data change rate; It adopts a differential update mechanism; establishes a version management mechanism to support version rollback; and provides knowledge services to external systems through API interfaces. Data heterogeneity measurement steps: Quantify the heterogeneity of multi-source data to guide preprocessing resource allocation. First, label each data source by type: structured data is labeled as type 1, semi-structured data as type 2, and unstructured data as type 3. Calculate the type difference coefficient T for the i-th data source. i ; Next, the data source structure is compared with the industry standard structure, and the structural difference coefficient S is calculated. i =1 - (number of matching fields / total number of standard fields); then calculate the semantic similarity x of the data fields using the Word2Vec model, and construct the semantic difference function Sem. i (x) = 1 - x, for Sem i (x) is integrated over the interval [0,1] to obtain the semantic difference integral value; finally, the overall heterogeneity is calculated using the following formula: Where H represents the overall data heterogeneity; n represents the total number of data sources; Entity alignment optimization steps: First, calculate the string similarity Sim between entity names. str Using the Levenshtein edit distance algorithm, Sim str =1 - (edit distance / Max); then generate entity word vectors using an industry-specific word vector model and calculate the cosine similarity Sim. sem = (vector A • vector B) / (||A|| • ||B||); then extract entity context features and calculate the context matching degree Sim using Jaccard similarity. ctx = (Context Intersection / Context Union); Finally The alignment confidence score is calculated using the following formula: , where C is the alignment confidence score; α=0.3, β=0.4, γ=0.3 are the weights.

2. The method for automatically constructing industry knowledge graphs for multi-source heterogeneous data according to claim 1, characterized in that, Step 2 data preprocessing also includes deep cleaning and conflict resolution strategies. For numerical data, the mean μ and standard deviation σ are calculated first, and the 3σ criterion is adopted. Outliers are verified a second time through box plots.

3. The method for automated construction of industry knowledge graphs for multi-source heterogeneous data according to claim 1, characterized in that, In step 3, entity and relation extraction adopts a collaborative optimization mechanism between models. For the new energy and new materials industries, transfer learning is used to transfer weights from the pre-trained model of the general knowledge graph, freeze 60% of the bottom layer parameters, and fine-tune the top layer parameters with industry data.

4. The method for automated construction of industry knowledge graphs for multi-source heterogeneous data according to claim 1, characterized in that, In step 4, the knowledge graph schema adopts a dynamic adaptation design. During the initial schema construction phase, when referring to international / national industry standards, the semantic distance with existing classes is calculated using WordNet thesaurus. After schema changes, consistency verification tools are used to check for conflicts in class hierarchy and attribute constraints. A schema version log is established to record the class / relation name, change time, and operator for each change.

5. The method for automated construction of industry knowledge graphs for multi-source heterogeneous data according to claim 1, characterized in that, Step 5 employs a multi-dimensional quantitative model for map quality assessment, introducing feature importance calculation to optimize assessment weights. First, four core assessment features are defined: accuracy, completeness, consistency, and timeliness. Then, 1000 labeled samples are selected, and their quality levels are manually labeled. A quality prediction model is constructed, using the four features as input and the manually labeled quality levels as output, and the model is trained. The importance weight of each feature is calculated using the model, with the formula: , where I k The importance weight of the k-th feature; M=1000 is the sample size; P m F represents the predicted quality probability of the m-th sample; k,m Let ∂P be the value of the k-th feature of the m-th sample; m / ∂F k,m This is the partial derivative of the prediction probability with respect to the eigenvalues.

6. The method for automatically constructing industry knowledge graphs for multi-source heterogeneous data according to claim 1, characterized in that, In step 6, incremental updates employ a triggering and parallel processing mechanism, with a dual threshold mechanism for triggering conditions. Update task processing utilizes the Spark distributed framework, splitting tasks by data source type and entity category, with tasks scheduled via DAG dependencies. A read-write separation + dual-database switching architecture is adopted.

7. The method for automatically constructing industry knowledge graphs for multi-source heterogeneous data according to claim 1, characterized in that, Also includes: Cross-domain knowledge fusion steps: First, construct a cross-domain semantic mapping table, with domain experts participating in the formulation of mapping rules, including mapping of the same identifier class, synonym class, and attribute; then, use the TransH model for cross-domain entity alignment, mapping entities from the source and target domains to a hyperplane, calculating the L2 norm distance of entity vectors, and the similarity Sim. cross =1 / ||hr-tr||2, where hr is the source domain entity vector and tr is the target domain entity vector; after fusion, a cross-domain knowledge index is constructed to support cross-industry related queries; connectivity is evaluated by calculating the proportion of connected components in the graph.

8. The method for automated construction of industry knowledge graphs for multi-source heterogeneous data according to claim 1, characterized in that, Also includes: Knowledge service optimization steps: At the interface design level, provide dual interfaces of RESTful API and GraphQL. RESTful API supports standardized queries, and parameters support pagination and filtering. The GraphQL interface supports custom queries; In terms of performance optimization, batch queries adopt an asynchronous processing mechanism; the knowledge reasoning interface integrates dual reasoning engines, with the rule reasoning engine loading industry reasoning rules; the distributed representation reasoning engine predicts missing relationships; in terms of monitoring and operation and maintenance, an API monitoring panel is built to display the interface call volume TPS, success rate, and average response time in real time, set threshold alarms, and record the caller IP, query content, and processing results; the optimized knowledge service is adapted to question answering, decision support, and risk warning application scenarios.

Images