One-stop knowledge graph construction method and system, medium, program product and equipment

Abstract

The invention provides a one-stop knowledge graph construction method and system, a medium, a program product and equipment, and belongs to the technical field of data processing. The method comprises the following steps: processing dynamically acquired map service data from a structured or unstructured data source to a set relational database, and performing primary processing of the map service data; creating schema information of the atlas, and mapping data fields and entity attribute information or relation attribute information to complete migration binding of data; migrating the migrated and bound relational data to a graph database of a specified type to complete graph construction; according to the method, the problem that the source of supporting data of an existing atlas construction system is single is solved, and the method is adaptive to currently common relational databases such as MySQL, Oracle and SQL Server, so that the system can directly fetch data from different relational databases during atlas data migration, and the data processing work in the earlier stage of atlas construction is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and specifically to a one-stop knowledge graph construction method, system, medium, program product and equipment. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] As an important part of artificial intelligence technology, knowledge graph technology realizes the effective organization and expression of real-world knowledge by building a large-scale network of entities and entity relationships. This technology not only helps people obtain information and knowledge more effectively, but also provides a foundation for intelligent information applications. At present, the way to build a knowledge graph is often to extract entities and relationships in the text through a series of natural language processing algorithms or rule processing methods, transfer them to relational databases such as MySQL, Oracle, etc., and then migrate them to the graph database to complete the construction of the graph.

[0004] The existing knowledge graph construction system has the following shortcomings: (1) The supported data sources are relatively single. For example, the business data of the graph is often stored in relational databases such as MySQL and Oracle. The graph construction system only supports data retrieval from a certain type of relational database. This will result in the need to migrate and convert business data before graph construction can be performed when building graphs from different data sources; (2) When it comes to multi-table data fusion and data processing, a lot of data cleaning work is required before the data can be migrated to the graph database; there is a lack of graph incremental construction logic. The existing graph construction system often uses a scheduled full data refresh to complete the migration and construction of incremental data, which will bring downtime risks to the related business systems; there is a lack of a common graph API, and the supported graph database types are single. Summary of the invention

[0005] In order to address the deficiencies in the prior art, the present invention provides a one-stop knowledge graph construction method, system, medium, program product and device, which solves the problem of a single data source supported by the existing graph construction system, and is compatible with currently commonly used relational databases, such as MySQL, Oracle, and SQL Server, so that the system can directly retrieve data from different relational databases when performing graph data migration, reducing the data processing work in the early stage of graph construction.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: In a first aspect, the present invention provides a one-stop knowledge graph construction method.

[0007] A one-stop knowledge graph construction method includes the following processes: Process the dynamically collected graph business data from structured or unstructured data sources into a set relational database to perform primary processing of the graph business data; Create the schema information of the graph, map the data fields with the entity attribute information or relationship attribute information to complete the migration and binding of the data; Migrate the bound relational data to the specified type of graph database to complete the graph construction.

[0008] As a further limitation of the first aspect of the present invention, the schema information includes: a data ontology, a data entity, and a relationship between the data ontology and the data entity.

[0009] As a further limitation of the first aspect of the present invention, before the step of creating the schema information of the graph, the following process is also included: The graph business data after primary processing is subjected to secondary processing, which at least includes: multi-table data fusion operation, data interception operation and data calculation operation. After the secondary processing is completed, the step of creating the graph schema information is entered.

[0010] As a further limitation of the first aspect of the present invention, before the step of creating the schema information of the graph, the following process is also included: In the incremental data processing logic, the data monitoring service listens to the database log to obtain incremental data, and pushes the incremental data to the message queue. The incremental data in the message queue is used to enter the next step of creating the schema information of the graph, or to enter the secondary processing step.

[0011] As a further limitation of the first aspect of the present invention, a dynamic data collection mechanism is adopted to determine the data source that needs to be collected, determine the update frequency of the data source that needs to be collected, perform real-time collection according to the determined update frequency, and extract the attribute information of the data fields, entities and relationships of the collected data source data in real time, and form a data table or data view according to the attribute information of the data fields, entities and relationships; complete the pre-extraction of data from the data table or data view through a predefined SQL statement, and submit the predefined SQL statement to the database to perform the data extraction operation when executing the collection.

[0012] In a second aspect, the present invention provides a one-stop knowledge graph construction system.

[0013] A one-stop knowledge graph construction system, including: The primary processing unit is configured to: process the dynamically collected graph business data from the structured or unstructured data source into a set relational database to perform primary processing of the graph business data; The migration binding unit is configured to: create schema information of the graph, map data fields with entity attribute information or relationship attribute information to complete data migration binding; The graph generation unit is configured to migrate the bound relationship data to a graph database of a specified type to complete graph construction.

[0014] As a further limitation of the second aspect of the present invention, in the migration binding unit, the schema information includes: a data ontology, a data entity, and a relationship between the data ontology and the data entity.

[0015] As a further limitation of the second aspect of the present invention, in the primary processing unit, before the step of creating the schema information of the graph, the following process is also included: The graph business data after primary processing is processed twice, which at least includes: multi-table data fusion operation, data interception operation and data calculation operation. After the secondary processing is completed, the step of creating the graph schema information is entered; In the incremental data processing logic, the data monitoring service listens to the database log to obtain incremental data, and pushes the incremental data to the message queue. The incremental data in the message queue is used to enter the next step of creating the schema information of the graph, or to enter the secondary processing step.

[0016] As a further limitation of the first aspect of the present invention, a dynamic data collection mechanism is adopted to determine the data source that needs to be collected, determine the update frequency of the data source that needs to be collected, perform real-time collection according to the determined update frequency, and extract the attribute information of the data fields, entities and relationships of the collected data source data in real time, and form a data table or data view according to the attribute information of the data fields, entities and relationships; complete the pre-extraction of data from the data table or data view through a predefined SQL statement, and submit the predefined SQL statement to the database to perform the data extraction operation when executing the collection.

[0017] In a third aspect, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium; a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the one-stop knowledge graph construction method as described in the first aspect of the present invention is implemented.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by a processor and executing the one-stop knowledge graph construction method as described in the first aspect of the present invention.

[0019] In a fifth aspect, the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the one-stop knowledge graph construction method as described in the first aspect of the present invention.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention solves the problem of single data source supported by existing graph construction systems, and is compatible with currently commonly used relational databases, such as MySQL, Oracle, and SQL Server, so that the system can directly retrieve data from different relational databases when migrating graph data, reducing the data processing work in the early stage of graph construction.

[0021] 2. In order to reduce the data fusion and data processing problems before graph construction, the present invention introduces a data acquisition mechanism, which dynamically processes data when graph data is acquired, reducing the large-scale data processing work in the early stage of graph construction.

[0022] 3. The present invention designs and implements the logic of incremental graph construction, reduces the workload of existing incremental data migration, greatly improves the stability of related businesses, and supports multiple graph databases. Currently supported graph databases include Nebula and Neo4j. It has graph API capabilities, abstracts and encapsulates commonly used graph query scenarios, and provides an efficient query interface.

[0023] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0025] Figure 1 A schematic diagram showing the principle of a one-stop graph construction method for integrating multiple data sources provided in Example 1 of the present invention; Figure 2 An application architecture diagram of a one-stop graph construction method integrating multiple data sources provided in Example 1 of the present invention; Figure 3 A schematic diagram of incremental data processing provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0028] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0029] Embodiment 1: This implementation proposes a one-stop knowledge graph construction method, such as Figure 1 Specifically, the process includes the following steps: S1: Process the graph business data from structured (i.e. business data) or unstructured data (such as PDF, HTML or TEXT, etc.) sources to relational databases such as MySQL, Oracle, SQL Server, etc. (i.e. primary processing of business data), and complete the primary processing of graph business data. If necessary for the business, the data can be processed again, such as multi-table data fusion, data interception, data calculation, etc. Specifically, in step S1 of this implementation, the graph business data is processed from a structured or unstructured data source to a relational database such as MySQL, Oracle, or SQL Server, including the following process: S1.1: Data extraction process, specifically, including: Unstructured data: Extract the required data fields by writing scripts or using tools to parse unstructured files; Structured data: Read the data source directly, such as extracting data from another database using SQL queries.

[0030] S1.2: Data cleaning and conversion process, specifically including: Cleaning: remove or correct erroneous data, duplicate data, format date, time, number and other data; Conversion: Convert the extracted data into a form suitable for the target database structure, including defining the data table structure, field type, primary key, foreign key, etc.; Standardization: Convert data into a unified format and unit, such as date format, currency unit, etc.

[0031] S1.3: Data loading process, specifically, includes: Design database structure: Design database table structure according to business requirements and data characteristics; Data import: Use SQL statements, database management tools, or ETL tools (such as Talend, Informatica, Apache NiFi, etc.) to load cleaned and converted data into relational databases such as MySQL, Oracle, and SQL Server; Verify: Check whether the data is loaded successfully and verify the completeness and accuracy of the data.

[0032] S1.4: Maintenance and optimization process, specifically including: Regular updates: Set up regular tasks or triggers to update data in the database; Performance optimization: optimize database indexes and query statements to improve data retrieval efficiency; Security: Ensure that security measures such as database access control and data encryption are in place.

[0033] In this implementation, a dynamic data collection mechanism is adopted to determine the data sources (including structured data and unstructured data) that need to be collected, determine the update frequency of the data sources that need to be collected, perform real-time collection according to the determined update frequency, and extract the attribute information of the data fields, entities and relationships of the collected data source data in real time, and convert it into a data table or data view; when performing data collection specifically, pre-extraction of data from one table (or one view) or multiple tables (or multiple views) is completed through pre-defined SQL statements, and the pre-defined SQL statements are submitted to the database to perform data extraction when the system executes the collection.

[0034] S2: Create the graph schema information (such as entities and relationships) on the graph construction service platform, execute entity and relationship creation, and map the data fields of the data source data extracted and collected in real time with the attribute fields of the entities and relationships to complete the data migration and binding.

[0035] S3: Migrate data to a specified type of graph database through data binding relationships to complete graph construction (supports multiple graph databases such as Nebula and Neo4j).

[0036] Figure 1 In the context of ETL, ETL refers to ETL (Extract, Transform, Load) data processing, which describes the process of extracting, transforming, and loading data from the source to the target. The main function of the ETL tool is to clean, integrate, transform, and load scattered and messy data, helping enterprises to aggregate data distributed in different systems into a data warehouse, and provide accurate and timely data support for enterprise decision-making; MQ stands for message queue.

[0037] For the constructed graph, the present invention provides a general data query interface, such as path discovery query, N-degree path query starting from a specified node, etc.; in addition, the present invention provides incremental data processing logic based on a message queue, which can automatically perform incremental graph data migration.

[0038] like Figure 2 As shown in the figure, it is an application architecture diagram of a one-stop graph construction method integrating multiple data sources, including: data processing layer, graph construction service layer, graph API interface service layer, and graph application layer. More specifically, it includes the following contents: (1) The data processing layer is the foundation of the entire architecture. It is responsible for processing raw data from different channels and formats and converting it into a standard format suitable for graph construction. Specifically, it includes: PDF: Process various academic papers, technical documents and other PDF files, and extract structured, semi-structured or unstructured data from them; HTML: crawl data from web pages, especially semi-structured data such as tables and lists, and process plain text content in web pages; TEXT: Processes plain text files, such as log files, reports, etc., and extracts useful information through natural language processing technology; System business data: Integrate business data from internal enterprise systems, such as structured data in databases.

[0039] (2) The graph construction service layer is the core layer, responsible for converting the processed data into components of the knowledge graph, including: Create a graph: initialize the graph structure and define the basic framework and metadata of the graph; Ontology construction: Build domain ontology based on business needs and data characteristics, and define entities, attributes, and relationships between entities; Data collection: obtain processed data from the data processing layer and perform necessary cleaning and formatting; Knowledge mapping: Map the collected data into the ontology and establish the mapping relationship between entities, attributes and relationships; Graph construction: Build a knowledge graph based on mapping relationships, including adding entities, establishing relationships, etc.

[0040] In this implementation, the purpose of knowledge mapping is to bind and configure database fields with attribute fields of entities or relationships, and the data in the database information table is mapped to the attributes of entities or relationships. Through data mapping, the data in the database table can be written into the graph database in the form of entity or relationship objects.

[0041] For example, when the entity is a company, the entity attributes include: company name, main business, unified social credit code, company address, legal representative, etc. The database table is a company information table, and the table fields include: company name, main business, unified social credit code, company address, legal representative, etc.; When the relationship is shareholding, the relationship attributes include: starting entity primary key field, ending entity primary key field, shareholding ratio, shareholding time, number of shares held, etc. The database table is a shareholding relationship table, and the table fields include: starting entity primary key field, ending entity primary key field, shareholding ratio, shareholding time, number of shares held, etc.

[0042] (3) The graph API interface service layer provides an external service interface to support graph writing and query operations, including: Graph writing interface: provides an API interface that allows external systems or users to write new entities, relationships, and attributes into the graph; Graph query interface: provides a rich query interface that supports entity-based query, relationship-based query, path query, complex query, etc. to meet different business needs.

[0043] (4) The graph application layer is a variety of application scenarios built on the knowledge graph, such as intelligent question answering, visual analysis, etc., including: Intelligent question answering: Utilize the rich information of knowledge graph to build an intelligent question answering system that supports natural language query and answer; Visual analysis: Through visualization technology, the entities and relationships in the knowledge graph are displayed in a graphical way, which is convenient for users to understand and analyze intuitively.

[0044] like Figure 3 As shown, in the incremental data processing logic, the data monitoring service monitors the log (MySQL log is binlog) of the database (taking MySQL as an example, MySQL primary processing data) to obtain incremental data, and pushes it to the message queue (MQ). The incremental data in the message queue is finally consumed to the MySQL secondary processing database or graph database.

[0045] In this implementation, the path query logic process specifically includes: Submit it to the graph query service through the specified interface; the graph query service fills the parameters received by the interface into the graph query language for the specified query scenario (similar to SQL, a query language for graph databases); the graph query service submits the graph query language with the query parameters filled in to the graph database, and the graph database executes the query logic and returns the query results; the graph query service parses the query results returned by the graph database, designs a unified and user-friendly query result return data structure, and returns the results to the query client through the interface.

[0046] More specifically, this implementation method abstracts and encapsulates commonly used graph query scenarios, provides an efficient query interface, and implements the path discovery query process, including: (A) Input processing, specifically, including: (A-1) Receiving query requests: The interface receives query requests from users or systems. The request contains query statements (such as graph query languages ​​such as SPARQL and Cypher) and possible additional parameters (such as paging parameters, sorting parameters, etc.); (A-2) Parsing query statements: Parse the received query statements to extract key information such as query conditions, target nodes, and relationship types. This usually requires the use of a parser or interpreter corresponding to the query language; (A-3) Verify input parameters, verify the legality and validity of query statements and additional parameters, and ensure that they comply with the specifications and requirements of the query interface.

[0047] (B) Query execution, specifically, including: The user submits query parameters or query statements with parameters to the backend through the interface. The backend will fill the parameters into the predefined graph database path query statement, and then submit the query statement to the graph database. The graph database completes the query and returns the query results. This system parses the query results and encapsulates them into a unified data structure.

[0048] (B-1) Processing optimization: During the query execution process, some optimization strategies may be applied to improve query efficiency, such as index lookup, cache utilization, query plan rewriting, etc.

[0049] (C) Result processing, specifically including: (C-1) Filtering and sorting: Filtering and sorting the query results according to the filter conditions and sorting parameters in the query conditions, which can ensure that the results returned to the user are accurate and orderly; (C-2) Paging: If the query request contains paging parameters, the query results need to be paging, which usually involves splitting the result set into multiple smaller subsets and returning the subset requested by the user; (C-3) Result conversion: converting query results from internal representation to external representation (such as JSON, XML, etc.) to facilitate parsing and processing by users or systems.

[0050] (D) Output generation, specifically, including: (D-1) Construct a response: Construct a response message based on the processed query result. The response message should include the query result, status code, error message (if any), etc. (D-2) Send a response, sending the constructed response message to the requester, which usually involves operations such as network programming and protocol processing.

[0051] (E) Subsequent processing, specifically including: (E-1) Logging, recording detailed information of query requests and responses to facilitate subsequent monitoring, analysis and debugging; (E-2) Performance evaluation: evaluate the performance of the query interface, including indicators such as response time and throughput, and make necessary optimization and adjustments to the interface based on the evaluation results; (E-3) Feedback processing: collecting and processing feedback from users or systems, and making improvements and optimizations based on issues raised in the feedback.

[0052] This implementation also provides an N-degree path query interface starting from a specified node. The implementation process includes: (a) Input processing, specifically, including: (a-1) Receiving a query request: The interface receives a query request from a user or a system. The request includes a starting node ID, a path length N (i.e., a query depth), and possible additional conditions (such as relationship type restrictions, node type restrictions, etc.).

[0053] (a-2) Verify the input parameters, verify the validity of parameters such as the starting node ID and path length N, and ensure that they comply with the specifications of the knowledge graph and the requirements of the interface.

[0054] (b) The inquiry process, specifically, includes: The user submits query parameters or query statements with parameters to the backend through the interface. The backend will fill the parameters into the predefined graph database path query statement, and then submit the query statement to the graph database. The graph database completes the query and returns the query results. This system parses the query results and encapsulates them into a unified data structure.

[0055] (c) Result processing and output, including: (c-1) Result filtering and sorting: Filter and sort the collected paths as needed. Filtering can remove paths that do not meet specific requirements, and sorting can be based on path length, weight or other indicators. (c-2) Construct a response message, convert the processed path result into an external representation (such as JSON, XML, etc.), and construct a response message. The response message should include the query result, status code, error information (if any), etc.; (c-3) Send a response, sending the constructed response message to the requester, which usually involves operations such as network programming and protocol processing.

[0056] (d) Performance optimization and error handling, including: Throughout the entire process, attention should be paid to performance optimization, such as using indexes to accelerate node lookups, caching calculated path results, etc. At the same time, a complete error handling mechanism should be designed to handle possible abnormal situations, such as query timeouts, node non-existence, etc., and return clear error information to the user or system.

[0057] (e) Subsequent processing, specifically, including: recording detailed information of query requests and responses to facilitate subsequent monitoring, analysis and debugging; making necessary improvements and optimizations to the interface based on user or system feedback.

[0058] The present invention supports multiple relational databases as data sources and supports multiple graph databases to store graph data. The invention provides a data collection mechanism, incremental data processing capabilities, and graph query API capabilities, making the graph construction process simple, convenient, and efficient.

[0059] Embodiment 2: This implementation provides a one-stop knowledge graph construction system, including: The primary processing unit is configured to: process the dynamically collected graph business data from the structured or unstructured data source into a set relational database to perform primary processing of the graph business data; The migration binding unit is configured to: create schema information of the graph, map data fields with entity attribute information or relationship attribute information to complete data migration binding; The graph generation unit is configured to migrate the bound relationship data to a graph database of a specified type to complete graph construction.

[0060] The specific construction method is shown in the process of Example 1 and will not be repeated here.

[0061] It is understandable that the above-mentioned units can be separately or completely combined into one or several other units to constitute, or one (some) of the units can be further divided into multiple functionally smaller units to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the one-stop knowledge graph construction system may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0062] According to another embodiment of the present application, the system described in this embodiment can be constructed and the one-stop knowledge graph construction method of the embodiment of the present application can be implemented by running a computer program (including program code) capable of executing the steps involved in the corresponding method described in Example 1 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0063] Embodiment 3: This implementation provides an electronic device, which includes a processor, a communication interface, and a computer-readable storage medium, wherein the processor, the communication interface, and the computer-readable storage medium may be connected via a bus or other means.

[0064] Among them, the communication interface is used to receive and send data, the computer-readable storage medium can be stored in the memory of the electronic device, the computer-readable storage medium is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer-readable storage medium.

[0065] A processor (or CPU (Central Processing Unit)) is the computing core and control core of an electronic device, which is suitable for implementing one or more instructions, and specifically suitable for loading and executing one or more instructions to implement corresponding method flows or corresponding functions.

[0066] The processor is configured to perform the following process: Process the dynamically collected graph business data from structured or unstructured data sources into a set relational database to perform primary processing of the graph business data; Create the schema information of the graph, map the data fields with the entity attribute information or relationship attribute information to complete the migration and binding of the data; Migrate the bound relational data to the specified type of graph database to complete the graph construction.

[0067] The specific construction method is shown in the process of Example 1 and will not be repeated here.

[0068] Embodiment 4: This implementation provides a computer-readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The computer-readable storage medium provides a storage space that stores the processing system of the electronic device.

[0069] In addition, the storage space also stores one or more instructions suitable for being loaded and executed by the processor, and these instructions may be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it may also be at least one computer-readable storage medium located away from the aforementioned processor.

[0070] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to implement the following process: Process the dynamically collected graph business data from structured or unstructured data sources into a set relational database to perform primary processing of the graph business data; Create the schema information of the graph, map the data fields with the entity attribute information or relationship attribute information to complete the migration and binding of the data; Migrate the bound relational data to the specified type of graph database to complete the graph construction.

[0071] The specific construction method is shown in the process of Example 1 and will not be repeated here.

[0072] Embodiment 5: The present implementation provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the following process: Process the dynamically collected graph business data from structured or unstructured data sources into a set relational database to perform primary processing of the graph business data; Create the schema information of the graph, map the data fields with the entity attribute information or relationship attribute information to complete the migration and binding of the data; Migrate the bound relational data to the specified type of graph database to complete the graph construction.

[0073] The specific construction method is shown in the process of Example 1 and will not be repeated here.

[0074] A person skilled in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0075] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by the computer or a data processing device such as a server, a data center, etc. that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A one-stop knowledge graph construction method, It is characterized in that The process includes: Process the dynamically collected graph business data from structured or unstructured data sources into a set relational database to perform primary processing of the graph business data; Create the schema information of the graph, map the data fields with the entity attribute information or relationship attribute information to complete the migration and binding of the data; Migrate the bound relational data to the specified type of graph database to complete the graph construction.

2. The one-stop knowledge graph construction method according to claim 1, It is characterized in that Before creating the schema information of the graph, the following process is also included: The graph business data after primary processing is subjected to secondary processing, which at least includes: multi-table data fusion operation, data interception operation and data calculation operation. After the secondary processing is completed, the step of creating the graph schema information is entered.

3. The one-stop knowledge graph construction method according to claim 1, It is characterized in that Before creating the graph's schema information, The process includes: In the incremental data processing logic, the data monitoring service listens to the database log to obtain incremental data, and pushes the incremental data to the message queue. The incremental data in the message queue is used to enter the next step of creating the schema information of the graph, or to enter the secondary processing step.

4. The one-stop knowledge graph construction method according to any one of claims 1 to 3, It is characterized in that The following processes are also included: Adopt a dynamic data collection mechanism to determine the data source to be collected, determine the update frequency of the data source to be collected, perform real-time collection according to the determined update frequency, and extract the data field, entity and relationship attribute information of the collected data source data in real time, and form a data table or data view according to the data field, entity and relationship attribute information; The pre-extraction of data from a data table or a data view is completed by using a pre-defined SQL statement, and when performing the collection, the pre-defined SQL statement is submitted to the database to perform the data extraction operation.

5. A one-stop knowledge graph construction system, It is characterized in that include: The primary processing unit is configured to: process the dynamically collected graph business data from the structured or unstructured data source into a set relational database to perform primary processing of the graph business data; The migration binding unit is configured to: create schema information of the graph, map data fields with entity attribute information or relationship attribute information to complete data migration binding; The graph generation unit is configured to migrate the bound relationship data to a graph database of a specified type to complete graph construction.

6. The one-stop knowledge graph construction system according to claim 6, It is characterized in that In the primary processing unit, before the step of creating the schema information of the graph, the following process is also included: The graph business data after primary processing is processed twice, which at least includes: multi-table data fusion operation, data interception operation and data calculation operation. After the secondary processing is completed, the step of creating the graph schema information is entered; In the incremental data processing logic, the data monitoring service listens to the database log to obtain incremental data, and pushes the incremental data to the message queue. The incremental data in the message queue is used to enter the next step of creating the schema information of the graph, or to enter the secondary processing step.

7. The one-stop knowledge graph construction system according to claim 5 or 6, It is characterized in that Adopt a dynamic data collection mechanism to determine the data source to be collected, determine the update frequency of the data source to be collected, perform real-time collection according to the determined update frequency, and extract the data field, entity and relationship attribute information of the collected data source data in real time, and form a data table or data view according to the data field, entity and relationship attribute information; The pre-extraction of data from a data table or a data view is completed by using a pre-defined SQL statement, and when performing the collection, the pre-defined SQL statement is submitted to the database to perform the data extraction operation.

8. A computer device, It is characterized in that include: A processor and a computer readable storage medium; a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the one-stop knowledge graph construction method as described in any one of claims 1 to 4 is implemented.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing the one-stop knowledge graph construction method as described in any one of claims 1 to 4.

10. A computer program product, It is characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the one-stop knowledge graph construction method as described in any one of claims 1 to 4.

Images