Rare earth industry-oriented big data semantic retrieval method and system based on knowledge graph

Abstract

The invention discloses a rare earth industry-oriented big data semantic retrieval method and system based on a knowledge graph, and belongs to the technical field of knowledge management and intelligent retrieval. In order to solve the problems that traditional keyword retrieval is insufficient in accuracy, incapable of understanding semantics and high in use threshold, the method comprises the following steps: establishing a rare earth industry field knowledge graph, defining entities such as materials, processes, equipment and suppliers and association relationships thereof, and extracting data from a manufacturing execution platform, a resource planning platform and a supplier collaboration platform; carrying out intention recognition and entity extraction on the natural language retrieval request by adopting a deep language understanding model, and converting the intention recognition and entity extraction into a graph query statement; semantic reasoning and associated query are performed based on the knowledge graph, a related result is returned in combination with a vector retrieval technology, and intelligent recommendation is provided. According to the method, the retrieval accuracy is improved by more than six percent, a user can perform natural language retrieval without knowing a data structure, and the method is suitable for scenes such as rare earth enterprise knowledge query, data analysis assistance and intelligent question and answer.

Description

Technical Field

[0001] This invention belongs to the field of knowledge management and intelligent retrieval technology, specifically involving a big data semantic retrieval scheme based on domain knowledge graph for rare earth enterprises. It can be widely applied to knowledge query, data analysis assistance, intelligent question answering, knowledge recommendation and other scenarios in systems such as manufacturing execution platforms, resource planning platforms and supplier collaboration platforms of rare earth enterprises. Background Technology

[0002] Rare earth enterprises' information systems have accumulated a large amount of business data, covering material information, process parameters, equipment ledgers, supplier information, order records, etc. Users face numerous difficulties when searching for information within this data. Existing data retrieval technologies have significant shortcomings: Limitations of keyword retrieval: Traditional retrieval is based on keyword matching, which cannot understand the user's true search intent, resulting in insufficient accuracy of search results; High barrier to entry: Users need to understand the database structure and field names to perform effective queries, making it difficult for non-technical personnel to use; Difficulty in performing joins: Cross-table and cross-system joins require writing complex query statements that ordinary users cannot complete; Knowledge is difficult to accumulate: Business knowledge is scattered across various systems, lacking unified knowledge organization and management; Lack of intelligent recommendations: Search results lack related recommendations, making it difficult for users to find relevant information.

[0003] The aforementioned limitations prevent existing technologies from achieving an integrated design of "knowledge graph + semantic understanding + intelligent retrieval," making it difficult to meet the demands of rare earth enterprises for intelligent knowledge retrieval. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of traditional data retrieval methods, such as insufficient accuracy, high usage barriers, and difficulty in related queries, this invention proposes a method specifically for sparse data retrieval. A big data semantic retrieval solution based on domain knowledge graphs for the land industry aims to improve retrieval accuracy by more than 60% and support self-service. It offers language retrieval and provides intelligent related recommendations.

[0005] (II) Technical Solution The core idea of ​​this invention is to construct an integrated knowledge retrieval system that combines "domain knowledge graph + semantic understanding + intelligent retrieval". The specific technical path is as follows: 1. Construction of a knowledge graph for the rare earth industry Body design: Entity category definition: Material: Material code, material name, material type, specifications, unit of measurement Process: Process code, process name, process type, process parameters Equipment: Equipment code, equipment name, equipment type, technical parameters Supplier: Supplier code, supplier name, contact information, rating Customer: Customer code, customer name, industry, credit rating Order: Order number, order date, order status, order amount Quality Inspection Report (QCReport): Report number, testing date, testing results, and tester. Association type definition: Materials - [Used for] -> Process Process - [Use] -> Equipment Supplier - [Supply] -> Materials Customer - [Place Order] -> Orders Orders - [Includes] -> Materials Materials - [Inspection] -> Quality Inspection Report Entity extraction: Extract from the manufacturing execution platform: equipment, processes, and quality inspection reports; Extract the following from the resource planning platform: materials, customers, and orders; Extract from the supplier collaboration platform: suppliers and purchase orders; Entity deduplication: Deduplication based on encoding or name similarity; Association establishment: Relationships are established based on foreign key associations (such as order-material); Inference based on business rules (such as inferring applicable processes from material types). Implicit associations were discovered based on co-occurrence analysis; Map storage: Use a graph database for storage; Establish node label index and attribute index.

[0006] 2. Linking Business Data with the Map Data mapping: Establish a mapping relationship between graph entities and business data tables; Entity attributes correspond to table fields; Data synchronization: Incremental synchronization: Monitor changes in business data and refresh the graph in real time; Full synchronization: Regular full refresh ensures data consistency; Query related terms: The results of the graph query can be linked to query detailed business data; It supports jumping from a graph node to the business system page.

[0007] 3. Natural Language Semantic Understanding Intent recognition: Train a deep language understanding model to identify user intent categories: Query intent: To find a specific entity or information; Statistical purposes: to count quantities, sum, average, etc. Comparison Intent: To compare the attributes of multiple entities; Recommendation intent: Requesting relevant recommendations; Example: "Who are the suppliers of neodymium iron boron?" → Search Intent "How much rare earth raw material was purchased last month?" → Statistical purpose Entity extraction: Train a named entity recognition model to extract entities from user input: Material entity: Material name, Material type; Supplier Entity: Supplier Name; Time entity: Time range; Numerical entities: quantity, amount; Entity Linking: Linking extracted entities to nodes in the graph; Query generation: Generate graph query statements based on intent and entities: / / User input: "What are some suppliers of neodymium iron boron magnets?" / / Generate query: MATCH (m:Material {name: 'NdFeB'})<-[:supply]-(s:Supplier) RETURN s.name, s.contact 4. Graph Retrieval and Intelligent Recommendation Graph query execution: Execute the generated graph query statement; Supports multi-hop queries (e.g., supplier → material → process → equipment); The query results include node attributes and associated paths; Vector retrieval supplement: For fuzzy queries, vector similarity retrieval is used; Vectorize entity descriptions (based on sentence embedding models); Calculate the similarity between the query vector and the entity vector; Return the top K most similar entities; Results merge and sort: Combining graph matching degree and vector similarity; Sort by overall score; Intelligent Recommendation: Recommending related entities based on graph association; Example: When searching for materials, recommend suppliers, applicable processes, and related orders for that material; Recommend content that users may be interested in based on their historical search history.

[0008] (III) Beneficial Effects 1. Significantly improved search accuracy: Semantic understanding improves search accuracy by more than 60%; 2. Significantly lower barrier to entry: Supports natural language retrieval, requiring no understanding of data structures; 3. Convenient related queries: The knowledge graph supports complex related queries and multi-hop reasoning; 4. Effective knowledge accumulation: The knowledge graph organizes scattered business data into structured knowledge; 5. Intelligent Recommendation Enhancement: Graph-based recommendations help users discover more relevant information. Attached Figure Description

[0009] Appendix Figure 1 This is a schematic diagram of the knowledge graph semantic retrieval system architecture.

[0010] Appendix Figure 2 This is a schematic diagram of a knowledge graph structure. Detailed Implementation

[0011] The following uses a knowledge retrieval system of a rare earth enterprise as an example to illustrate the specific implementation process of this invention: (a) Implementation preparation 1. Data Environment: Manufacturing Execution Platform: Includes 200 equipment records, 50 process specifications, and 5,000 quality inspection reports; Resource planning platform: Includes 3,000 basic material data entries, 500 customers, and 100,000 order records; Supplier collaboration platform: includes 200 suppliers and 50,000 purchase orders; 2. System Deployment: Graph database: Single-node deployment, 16GB of RAM; Semantic understanding service: Based on a deep language understanding model, deployed on a graphics processing unit server; Search services: Enterprise application framework services; Front-end interface: Front-end framework + component library; 3. Model Training: Intent recognition model: based on a pre-trained Chinese language model, with 2000 training data entries; Entity extraction model: Based on the named entity recognition model, with 3000 training data entries.

[0012] (II) Implementation Steps 1. Knowledge Graph Construction: Ontology design: Defines 7 entity categories and 10 association types; Entity extraction: Extract 3,000 material entities and 500 customer entities from the resource planning platform; Extract 200 supplier entities from the supplier collaboration platform; Extract 200 equipment entities and 50 process entities from the manufacturing execution platform; Association establishment: Supplier-Supply-Materials: 800 related items; Material - Used in - Process: 500 related entries; Process-Use-Equipment: 150 related items; Graph size: 3950 nodes and 1450 associations.

[0013] 2. Semantic understanding model training: Intent recognition: Training data: 2000 labeled samples (60% query, 20% statistics, 10% comparison, 10% recommendation); Model performance: Accuracy 92%; Entity extraction: Training data: 3000 labeled samples; Model performance: F1 score 87%; 3. Deployment of search service: Deploy a graph query service and encapsulate graph query statements; Deploy a vector retrieval service, based on a vector index library; Deploy the semantic understanding service and load the trained model; Integrate various services to provide a unified search interface.

[0014] 4. User Interface Development: Develop a natural language input box that supports voice input; The development results are displayed in a list, with support for viewing details; Develop a graph visualization that supports interactive exploration; Develop an intelligent recommendation area to present relevant recommendations.

[0015] (III) Verification of Implementation Results 1. Query accuracy verification: Test set: 100 retrieval questions; Traditional keyword search accuracy: 35%; The system's search accuracy rate is 89%. Accuracy improvement: 154%; 2. User experience verification: User survey: 50 business personnel tried it for one week; Satisfaction: 85% of users found it easier to use than the original system; Typical feedback: "Previously, finding supplier information required logging into multiple systems, but now it can be done with just one sentence." 3. Query efficiency verification: Average search response time: 1.2 seconds; Response time for complex related queries: 2.5 seconds (e.g., "Which orders are the materials supplied by supplier A used for?"). 4. Business Value Validation: Information search time: reduced from an average of 15 minutes to 30 seconds; Data utilization: More business personnel are able to query data independently, resulting in a 200% increase in data access volume; Knowledge discovery: Three potential supplier alternatives were discovered through graph association.

[0016] This implementation case fully verifies the feasibility and advancement of the present invention. It realizes intelligent retrieval of rare earth enterprise knowledge through knowledge graph and semantic retrieval. The ontology and model can be expanded according to different industry and business needs, and it has good versatility and scalability.

Claims

1. A knowledge graph-based big data semantic retrieval method for the rare earth industry, characterized in that, Includes the following steps: a. Rare Earth Industry Knowledge Graph Construction: Ontology Design: Define entity categories and association types in the rare earth industry; Entity Categories: Materials (raw materials, semi-finished products, finished products), processes (smelting, extraction, sintering), equipment (calcining kilns, extraction towers), suppliers, customers, orders, quality inspection reports; Association Types: Materials-for-process, Process-for-equipment, Supplier-supply-materials, Order-include-materials; Entity Extraction: Extract entity data from the databases of the Manufacturing Execution Platform, Resource Planning Platform, and Supplier Collaboration Platform; Association Establishment: Establish relationships between entities based on business rules and data associations; Graph Storage: Store entities and associations in a graph database; b. Business Data and Graph Association: Establish a mapping relationship between business data and graph entities; Automatically refresh graph entity attributes when business data changes; Supports graph-based querying and analysis of business data; c. Natural Language Semantic Understanding: Intent Recognition: Employs a deep language understanding model to identify user search intent (query, statistics, comparison, recommendation); Entity Extraction: Employs a named entity recognition model to extract entities from user input (material name, supplier name, time range, etc.); Association Recognition: Identifies entity associations implicit in user queries; Query Generation: Converts semantic understanding results into graph query statements; d. Graph Retrieval and Intelligent Recommendation: Graph Query: Executes graph query statements to obtain matching graph nodes and associations; Vector Retrieval: For text-based queries, uses vector similarity retrieval to supplement results; Result Fusion: Combines graph query and vector retrieval results and sorts them by relevance; Intelligent Recommendation: Recommends related entities based on graph associations (e.g., recommending related suppliers when querying materials). result Presentation: The system returns structured results and visualizes the knowledge graph. Its features include: a knowledge graph construction module supporting visualized ontology design, automatic entity extraction, and associative reasoning, with a graph scale supporting millions of entities; a data synchronization module supporting real-time data synchronization and batch data import, with synchronization latency of less than 5 minutes; a semantic understanding module based on a deep language understanding model, achieving intent recognition accuracy of over 90% and entity extraction F1 score of over 85%; a retrieval engine module supporting graph query and vector retrieval, with a query response time of less than 2 seconds; and an interactive interface module providing a natural language input box, a result list, graph visualization, and an intelligent recommendation area. The system is characterized in that, in step 1, the knowledge graph supports multi-hop reasoning, capable of answering complex questions requiring multiple steps of association, such as "What products are the materials provided by supplier A used for?"; in step 3, the semantic understanding supports multi-turn dialogue, enabling the system to understand user follow-up questions and supplements based on context, improving the interactive experience; and the system supports continuous learning of the knowledge graph, optimizing retrieval ranking and recommendation algorithms based on user feedback (clicks, acceptance).

Images