A knowledge graph processing method and device based on distributed database

Abstract

The present application discloses a knowledge graph processing method based on a distributed database, the method comprising: obtaining an SQL processing statement for the knowledge graph; generating a target programming language code corresponding to the SQL processing statement according to the SQL processing statement; using the target programming language code, calling a target function corresponding to the target programming language code; using the target function to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph. This embodiment can realize that the reading and storage of the graph data of the knowledge graph and the graph calculation can be performed in the same distributed database, thereby improving the processing efficiency of the knowledge graph.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a knowledge graph processing method and device based on a distributed database. Background Art

[0002] With the development of big data and artificial intelligence technology, the idea of ​​the Internet of Everything has been gradually applied in many fields, and knowledge graph technology has emerged. Knowledge graph is a modern theory that combines the theories and methods of applied mathematics, graphics, information visualization technology, information science and other disciplines with methods such as metrology citation analysis and co-occurrence analysis, and uses visual graphs to vividly display the core structure, development history, frontier fields and overall knowledge architecture of the discipline to achieve the purpose of multidisciplinary integration. It displays complex knowledge fields through data mining, information processing, knowledge metrology and graphics drawing, reveals the dynamic development laws of knowledge fields, and provides practical and valuable references for disciplinary research.

[0003] The knowledge graph is a graph-based data structure consisting of vertices V and edges E. In the knowledge graph, each vertex represents an "entity" that exists in the real world, and each edge is a "relationship" between entities. The knowledge graph is the most effective way to represent relationships, and it depicts our real world in the form of a graph.

[0004] Due to the particularity of graph data, graph data is often stored and calculated using dedicated graph databases. Commonly used graph databases include ArangoDB, OrientDB, JanusGraph, HugeGraph, etc. These graph databases are different from relational databases. The graph-based data loading and calculation methods are quite different from storage and query, and their distribution is not directly achieved on the big data framework. Therefore, common data is stored in big data, and the data used for graph calculation needs to be taken out and stored separately in the graph database. Therefore, its disadvantages are: data redundancy is generated and additional data synchronization is required; graph calculation does not have the ability to inherit the existing parallel capabilities on the big data platform, and additional framework support (such as Plato, etc.) is required to achieve distributed parallelism. Summary of the invention

[0005] The present application provides a knowledge graph processing method based on a distributed database, so that the reading and storage of graph data and graph calculation of the knowledge graph can be performed in the same database, thereby improving the processing efficiency of the knowledge graph.

[0006] In a first aspect, the present application provides a knowledge graph processing method based on a distributed database, the method is applied to a distributed database, the distributed database includes a node table and a relationship table of a knowledge graph, the method includes:

[0007] Obtaining SQL processing statements for the knowledge graph;

[0008] According to the SQL processing statement, generate a target programming language code corresponding to the SQL processing statement;

[0009] Using the target programming language code, calling the target function corresponding to the target programming language code;

[0010] The objective function is used to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph.

[0011] In a second aspect, the present application provides a knowledge graph processing device based on a distributed database, the device is applied to a distributed database, the distributed database includes a node table and a relationship table of a knowledge graph, and the device includes:

[0012] An acquisition unit, used to acquire an SQL processing statement for the knowledge graph;

[0013] A generating unit, configured to generate a target programming language code corresponding to the SQL processing statement according to the SQL processing statement;

[0014] A calling unit, configured to use the target programming language code to call a target function corresponding to the target programming language code;

[0015] A processing unit is used to process the graph data in the node table and the relationship table using the objective function to obtain a processing result of the knowledge graph.

[0016] In a third aspect, the present application provides a readable medium comprising execution instructions. When a processor of an electronic device executes the execution instructions, the electronic device executes any method as described in the first aspect.

[0017] In a fourth aspect, the present application provides an electronic device, comprising a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor executes any method described in the first aspect.

[0018] It can be seen from the above technical scheme that the present application provides a knowledge graph processing method based on a distributed database, the method is applied to a distributed database, the distributed database includes a node table and a relationship table of a knowledge graph, the method includes: obtaining an SQL processing statement for the knowledge graph; generating a target programming language code corresponding to the SQL processing statement according to the SQL processing statement; using the target programming language code, calling the target function corresponding to the target programming language code; using the target function to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph. It should be noted that in this embodiment, since the graph data of the knowledge graph is stored in the distributed database in the form of a node table and a relationship table, and the distributed database stores functions with multiple functions, such as a function for reading stored data, a function for calculating data, etc., therefore, after generating the target programming language code corresponding to the SQL processing statement, the target programming language code can be used to call the target function corresponding to the target programming language code, so that the target function can be used to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph. In this way, this embodiment can realize that the reading and storage of graph data and graph calculation of the knowledge graph can be performed in the same distributed database, thereby improving the processing efficiency of the knowledge graph.

[0019] The further effects of the above-mentioned non-conventional preferred manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 A schematic diagram of a process flow of a knowledge graph processing method based on a distributed database provided in one embodiment of the present application;

[0022] Figure 2 A schematic diagram of the system architecture of a knowledge graph processing system based on a distributed database provided in one embodiment of the present application;

[0023] Figure 3 A schematic diagram of the structure of a knowledge graph processing device based on a distributed database provided in one embodiment of the present application;

[0024] Figure 4A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0026] At present, common data is stored in the big data, and the data used for graph computing needs to be taken out and stored separately in the graph database. Therefore, its disadvantages are: data redundancy occurs and additional data synchronization is required; graph computing does not have the ability to inherit the existing parallel capabilities on the big data platform, and additional framework support (such as Plato) is required to achieve distributed parallelism.

[0027] The present application provides a knowledge graph processing method based on a distributed database, the method is applied to a distributed database, the distributed database includes a node table and a relationship table of a knowledge graph, the method includes: obtaining an SQL processing statement for the knowledge graph; generating a target programming language code corresponding to the SQL processing statement according to the SQL processing statement; using the target programming language code, calling a target function corresponding to the target programming language code; using the target function to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph. It should be noted that in this embodiment, since the graph data of the knowledge graph is stored in the distributed database in the form of a node table and a relationship table, and the distributed database stores functions with multiple functions, such as a function for reading stored data, a function for calculating data, etc., therefore, after generating the target programming language code corresponding to the SQL processing statement, the target programming language code can be used to call the target function corresponding to the target programming language code, so that the target function can be used to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph. In this way, this embodiment can realize that the reading and storage of graph data of the knowledge graph and the graph calculation can be performed in the same distributed database, thereby improving the processing efficiency of the knowledge graph. It should be noted that the embodiments of the present application can be applied to electronic devices (such as mobile phones, tablets, computers, etc.) or servers. It should be noted that in addition to the above-mentioned methods, other implementation methods can also be used, which are not limited here.

[0028] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0029] See also Figure 1 , shows a knowledge graph processing method based on a distributed database in an embodiment of the present application. In this embodiment, the method is applied to a distributed database, and the distributed database includes a node table and a relationship table of the knowledge graph. The method may include the following steps:

[0030] S101: Obtain SQL processing statements for the knowledge graph.

[0031] In this embodiment, when it is necessary to process the graph data of the knowledge graph stored in the distributed database, such as storage, reading, and calculation, the corresponding processing instructions can be input to generate the corresponding SQL processing statements, wherein the SQL processing statements include relevant information on the processing method, for example, the method of processing data and the object data to be processed. In this way, the distributed database can obtain the SQL processing statements for the knowledge graph. That is to say, in this embodiment, only the distributed database needs to

[0032] S102: Generate a target programming language code corresponding to the SQL processing statement according to the SQL processing statement;

[0033] S103: Using the target programming language code, calling a target function corresponding to the target programming language code;

[0034] In this embodiment, the distributed database may be a GreenPlum database system, and the underlying layer of the GreenPlum database system is a Postgresql cluster. Because GreenPlum's distributed database system can both efficiently read and write PB-level data and support AI engines, the community discovery algorithm has distributed capabilities when executed in Plpython. The data that the function execution depends on are all in the GreenPlum table, and no additional storage or data synchronization is required. At the same time, this application uses the world's most efficient open source distributed database system to optimize the graph computing community discovery algorithm from ordinary single-machine memory computing to distributed database parallel computing.

[0035] The storage form of graph data in the node table and relationship table of the knowledge graph in the distributed database can be in the form of triples; triples are an organizational form of graph data, including vertices, relationship pointers, and respective attributes; the basic forms of triples mainly include entity 1-relationship-entity 2 and entity-attribute-attribute value, etc. The vertex table includes the self-incrementing sequence ID of the graph data, the entity name, and the entity attribute, wherein the entity attribute is in json format, which can be understood as the attribute of the entity can be any content in json format, so any number of labels or dimensions can be appended to the attribute; the relationship table includes the self-incrementing sequence ID of the graph data, the relationship pointer starting point, the relationship pointer end point, and the relationship attribute, wherein the relationship attribute is in json format, which can be understood as the relationship attribute can be any content in json format, so any number of labels or dimensions can be appended to the attribute. It can be understood that since Postgresql / GreenPlum has built-in Json operators, Json format data can realize arbitrary extension and combination of attributes, and can perform advanced queries according to keys to realize big data analysis combined with other ordinary tables. Figure 2 As shown in the figure, after obtaining the knowledge graph data, the graph data of the knowledge graph data can be stored in a triple structure to form a vertex table and a relationship table, and then the vertex table and the relationship table are stored in a distributed database. The triples are stored in the madlib of pg / gp (GreenPlum), and the existing functions of madlib or the plpython procedural language are used to write functions for graph calculation of the knowledge graph data. At the same time, the attribute columns of the vertex table or the relationship table can be used for big data query, filtering, association, analysis, partitioning and other operations. The attribute columns are stored in json format, and the full-text search or gin index of json supported by pg / gp is used for efficient retrieval of json, which can integrate big data and knowledge graph calculation into one, and the implementation process of all business logic becomes a SQL statement. After the vertex table and the relationship table are written into the GreenPlum database, they will be evenly distributed on each segment node in the GreenPlum cluster. In the Master-Segment mode, the master node performs task distribution management and scheduling, and the child nodes implement parallel analysis and processing, and finally the results are landed in a table or returned result set.

[0036] In this embodiment, the distributed database first needs to use plpython to implement the function of the procedural programming language. Its script is similar to a stored procedure, but plpython can use python code in the database function, that is, the target programming language code can be python code. It should be noted that Plpython is a procedural programming language based on Postgresql / GreenPlum, and the calling method is similar to the SQL stored procedure. The target function is a plpython function obtained by using plpython to write a procedural programming language. It can be understood that it can run as long as the version is compatible; that is, a section of python code can be converted into a plpython function to be called by the database, and the table name and column name can be passed in to perform query calculations. The stored procedure of Pg / GreenPlum itself is language plpgsql, and plpython is lunguageplpythonu or plpython3u.

[0037] As an example, after the distributed database obtains the SQL processing statement, it can generate the target programming language code corresponding to the SQL processing statement according to the SQL processing statement. For example, after the distributed database obtains the SQL processing statement, it can first analyze the SQL processing statement, and then convert the SQL processing statement into the target programming language code supported by the distributed database.

[0038] S103: Using the target programming language code, call the target function corresponding to the target programming language code.

[0039] In this embodiment, Plpython can be used in advance to convert algorithms or modules that implement various functions into built-in function parameters. In this way, after obtaining the target programming language code, the target function corresponding to the target programming language code can be called according to the content of the target programming language code. For example, the modularity-based Louvain algorithm can be used to realize community discovery. Plpython is used on GreenPlum to transform and implement it according to the Louvain algorithm definition. The built-in function parameters are passed in as database table names and the column names of attribute columns can be specified. By submitting SQL statements, passing in vertex tables, relationship tables and attribute columns, the algorithm for discovering knowledge graph communities can be implemented on GreenPlum's distributed database by calling the function corresponding to the Louvain algorithm.

[0040] S104: Using the objective function to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph.

[0041] Since each function has corresponding functions, such as storing, reading, and calculating graph data, after calling the target function, the target function can be used to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph.

[0042] It can be seen that the present application provides a knowledge graph processing method based on a distributed database, the method is applied to a distributed database, the distributed database includes a node table and a relationship table of a knowledge graph, the method includes: obtaining an SQL processing statement for the knowledge graph; generating a target programming language code corresponding to the SQL processing statement according to the SQL processing statement; using the target programming language code, calling the target function corresponding to the target programming language code; using the target function to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph. It should be noted that in this embodiment, since the graph data of the knowledge graph is stored in the distributed database in the form of a node table and a relationship table, and the distributed database stores functions with multiple functions, such as a function for reading stored data, a function for calculating data, etc., therefore, after generating the target programming language code corresponding to the SQL processing statement, the target programming language code can be used to call the target function corresponding to the target programming language code, so that the target function can be used to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph. In this way, this embodiment can realize the reading and storage of graph data of the knowledge graph, and graph calculation can be performed in the same distributed database, thereby improving the processing efficiency of the knowledge graph. It should be noted that the embodiments of the present application can be applied to electronic devices (such as mobile phones, tablets, computers, etc.) or servers. It should be noted that in addition to the above-mentioned methods, other implementation methods can also be used, which are not limited here. It should be noted that GreenPlum+Madlib / Plpython is more lightweight to deploy, and only one copy of the data can be stored to implement various operations of big data (filtering associated partitions and full-text indexing), and graph algorithms can be performed directly in the data table, and the final results of the business analysis can be output in the form of a data table. There is no data redundancy in the middle, and you only need to operate the data table. The calling method is also simpler. After the Plpython function is implemented, you only need to enter the SQL statement and pass in the correct parameters to complete the entire business process.

[0043] Figure 1 The following is only a basic example of the method described in this application. Figure 1 A scenario embodiment of the method shown. In this embodiment, the method specifically includes the following steps:

[0044] S201: The distributed database obtains an SQL processing statement for the knowledge graph.

[0045] The SQL processing statement includes the table name of the node table and the table name of the relationship table.

[0046] S202: The distributed database generates a target programming language code for querying the table name of the node table and the table name of the relationship table according to the SQL processing statement.

[0047] S203: The distributed database uses the target programming language code to call the networkx component and the python-lovain component corresponding to the target programming language code.

[0048] The objective function includes a networkx component and a python-lovain component, and the python-lovain component is preset with a community discovery algorithm.

[0049] S204: The distributed database uses the networkx component and the python-lovain component to query the graph data in the node table and the relationship table to obtain all vertex data in the node table and all relationship data in the relationship table; and uses all vertex data in the node table and all relationship data in the relationship table as the processing result of the knowledge graph.

[0050] Taking GreenPlum as an example, pip installs the components of the supporting python code under the highest authority gpadmin (root authority installation is invalid), such as networkx and python-louvain, and calls them in plpython after the installation is complete. That is, the distributed database includes networkx components and python-lovain components, and the python-lovain component is preset with a community discovery algorithm.

[0051] If the analyzed SQL processing statement includes the table name of the node table and the table name of the relationship table, a target programming language code for querying the table name of the node table and the table name of the relationship table is generated. Then, the distributed database uses the target programming language code to call the networkx component and the python-lovain component corresponding to the target programming language code. The distributed database uses the networkx component and the python-lovain component to query the graph data in the node table and the relationship table, and obtains all vertex data in the node table and all relationship data in the relationship table; all vertex data in the node table and all relationship data in the relationship table are used as the processing result of the knowledge graph. That is, networkx and python-louvain are directly called to query the graph data in the node table and the relationship table, and the query results are stored as list_v all vertices and list_e all relationships, so that the calculation results of the community discovery algorithm can be completed, sorted into a table output and queried externally. This solution solves the problem that it is difficult to realize efficient parallel computing of knowledge graphs in the existing big data framework, and at the same time solves the problem that big data and knowledge graph algorithm frameworks are difficult to merge and data are difficult to be homologous. This solution has been developed and implemented in the GreenPlum6.1.0+Plpython2 / Plpython3 environment and successfully executed. The verification results are completely consistent with the results of the traditional Louvain algorithm, and the computing efficiency is basically linearly related to the number of distributed nodes and resource configuration. Therefore, dynamically expanding resources can improve the execution efficiency of the algorithm.

[0052] In one implementation of this embodiment, in the distributed database spark, the distributed implementation is performed simultaneously on different data storage nodes, resulting in changes in the partitioning results due to parallelism. In the plpython code, the execution process is controlled by the python code, which can guarantee the order. For example, list_v and list_e are directly stored in the upstream filtering query, so the array order can be guaranteed after sorting and other processing during the query process, thereby ensuring the order of the list input and output.

[0053] Figure 1 The following is only a basic example of the method described in this application. Figure 1 A scenario embodiment of the method shown. In this embodiment, the method specifically includes the following steps:

[0054] S301: The distributed database obtains an SQL processing statement for the knowledge graph.

[0055] The SQL processing statement includes an entity object query condition, wherein the entity object query condition may be a query condition set by a user, such as males who have contacted user A in the past week.

[0056] S302: The distributed database generates a target programming language code for querying entity objects that meet the entity object query condition according to the SQL processing statement.

[0057] S303: The distributed database uses the target programming language code to call the python-lovain component.

[0058] The objective function includes a python-lovain component, and the python-lovain component is preset with a community discovery algorithm.

[0059] S304: The distributed database uses the community discovery algorithm in the python-lovain component and the entity object query condition to screen the graph data in the node table and the relationship table to obtain entity objects that meet the entity object query condition; and uses the entity objects that meet the entity object query condition as the processing result of the knowledge graph.

[0060] If the analyzed SQL processing statement includes an entity object query condition, a target programming language code is generated for querying entity objects that meet the entity object query condition. Then, the distributed database uses the target programming language code to call the python-lovain component. The distributed database uses the python-lovain component to perform a community discovery algorithm operation on the graph data in the node table and the relationship table according to the entity object query condition, that is, the community discovery algorithm in the python-lovain component and the entity object query condition are used to perform data screening on the graph data in the node table and the relationship table to obtain entity objects that meet the entity object query condition, so that the entity objects that meet the entity object query condition can be used as the processing result of the knowledge graph.

[0061] Figure 1 The following is only a basic example of the method described in this application. Figure 1 A scenario embodiment of the method shown. In this embodiment, the method specifically includes the following steps:

[0062] S401: The distributed database obtains an SQL processing statement for the knowledge graph.

[0063] The SQL processing statement includes an index field, wherein the index field may be a query field set by a user, such as "diabetes".

[0064] S402: The distributed database generates a target programming language code for retrieving graph data including the index field according to the SQL processing statement.

[0065] S403: The distributed database uses the target programming language code to call the Gin index component.

[0066] The objective function includes a Gin index component. Figure 2 As shown in the figure, the Gin index component can be used to implement a full-text search method based on Postgresql / GreenPlum, and can be used in combination with Json type fields to achieve efficient query and calculation.

[0067] S404: The distributed database uses the Gin index component to retrieve the graph data in the node table and the relationship table to obtain the graph data including the index field; and uses the graph data including the index field as the processing result of the knowledge graph.

[0068] If the analyzed SQL processing statement includes an index field, a target programming language code is generated for retrieving the graph data including the index field. Then, the distributed database uses the target programming language code to call the Gin index component. The distributed database uses the Gin index component to retrieve the graph data in the node table and the relationship table to obtain the graph data including the index field, so that the graph data including the index field can be used as the processing result of the knowledge graph.

[0069] It should be noted that the attributes of vertices and relationships are json type fields. For this type of field, the introduction of Gin index in GreenPlum provides efficient full-text search capabilities. This solution is seamlessly connected with GreenPlum-based big data technology, and the knowledge graph data table can also perform attribute association analysis with other tables, thus combining the advantages of big data and knowledge graph algorithms.

[0070] like Figure 3 As shown, a specific embodiment of a knowledge graph processing device based on a distributed database described in this application is shown. The device described in this embodiment is a physical device for executing the method described in the above embodiment. Its technical solution is essentially consistent with the above embodiment. The device is applied to a distributed database, and the distributed database includes a node table and a relationship table of the knowledge graph. The device described in this embodiment includes:

[0071] An acquisition unit 301 is used to acquire an SQL processing statement for the knowledge graph;

[0072] A generating unit 302 is used to generate a target programming language code corresponding to the SQL processing statement according to the SQL processing statement;

[0073] A calling unit 303 is used to call a target function corresponding to the target programming language code by using the target programming language code;

[0074] The processing unit 304 is used to use the objective function to process the graph data in the node table and the relationship table to obtain the processing result of the knowledge graph.

[0075] Optionally, the distributed database is a GreenPlum database system.

[0076] Optionally, the storage format of the graph data in the node table and relationship table of the knowledge graph is triple form; the vertex table includes the auto-increment sequence ID of the graph data, the entity name and the entity attributes, wherein the entity attributes are in json format; the relationship table includes the auto-increment sequence ID of the graph data, the relationship pointing to the starting point, the relationship pointing to the end point, and the relationship attributes, wherein the relationship attributes are in json format.

[0077] Optionally, the target programming language code is a python code; and the target function is a plpython function written in a procedural programming language using plpython.

[0078] Optionally, the objective function includes a networkx component and a python-lovain component, and the python-lovain component is preset with a community discovery algorithm;

[0079] The SQL processing statement includes the table name of the node table and the table name of the relationship table; the generating unit 302 is specifically used to:

[0080] Generate a target programming language code for querying the table name of the node table and the table name of the relationship table according to the SQL processing statement;

[0081] Accordingly, the calling unit 303 is used to:

[0082] Using the target programming language code, calling the networkx component and the python-lovain component corresponding to the target programming language code;

[0083] Accordingly, the processing unit 304 is configured to:

[0084] Utilizing the networkx component and the python-lovain component to query the graph data in the node table and the relationship table, obtaining all vertex data in the node table and all relationship data in the relationship table;

[0085] All vertex data in the node table and all relationship data in the relationship table are taken as processing results of the knowledge graph.

[0086] Optionally, the objective function includes a python-lovain component, and the python-lovain component is preset with a community discovery algorithm;

[0087] The SQL processing statement includes an entity object query condition; the generating unit 302 is specifically used to:

[0088] generating, according to the SQL processing statement, a target programming language code for querying entity objects that satisfy the entity object query condition;

[0089] Accordingly, the calling unit 303 is specifically used for:

[0090] Using the target programming language code, calling the python-lovain component;

[0091] Accordingly, the processing unit 304 is specifically configured to:

[0092] Using the community discovery algorithm in the python-lovain component and the entity object query condition, data screening is performed on the graph data in the node table and the relationship table to obtain entity objects that meet the entity object query condition;

[0093] The entity objects that meet the entity object query conditions are used as the processing results of the knowledge graph.

[0094] Optionally, the objective function includes a Gin index component;

[0095] The SQL processing statement includes an index field; the generating unit 302 is specifically used to:

[0096] Generate a target programming language code for retrieving graph data including the index field according to the SQL processing statement;

[0097] Accordingly, the calling unit 303 is specifically used for:

[0098] Using the target programming language code, calling the Gin index component;

[0099] Accordingly, the processing unit 304 is specifically configured to:

[0100] The Gin index component is used to retrieve the graph data in the node table and the relationship table to obtain the graph data including the index field; the graph data including the index field is used as the processing result of the knowledge graph.

[0101] Figure 4 It is a structural diagram of an electronic device provided in an embodiment of the present application. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device may also include hardware required for other services.

[0102] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0103] The memory is used to store execution instructions. Specifically, the execution instructions are computer programs that can be executed. The memory may include internal memory and non-volatile memory, and provides execution instructions and data to the processor.

[0104] In one possible implementation, the processor reads the corresponding execution instructions from the non-volatile memory into the memory and then runs them, and can also obtain the corresponding execution instructions from other devices to form a knowledge graph processing device based on a distributed database at the logical level. The processor executes the execution instructions stored in the memory to implement the knowledge graph processing method based on a distributed database provided in any embodiment of the present application through the executed execution instructions.

[0105] The above application Figure 1The method performed by the knowledge graph processing device based on a distributed database provided in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0106] The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0107] An embodiment of the present application also proposes a readable storage medium, which stores execution instructions. When the stored execution instructions are executed by a processor of an electronic device, the electronic device can execute the knowledge graph processing method based on a distributed database provided in any embodiment of the present application, and is specifically used to execute the above-mentioned method described in the knowledge graph processing based on a distributed database.

[0108] The electronic device described in the above embodiments may be a computer.

[0109] Those skilled in the art should understand that the embodiments of the present application can be provided as methods or computer program products. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware.

[0110] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0111] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0112] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A knowledge graph processing method based on a distributed database, characterized in that: The method is applied to a distributed database, the distributed database includes a node table and a relationship table of a knowledge graph, and the method includes: Obtaining SQL processing statements for the knowledge graph; According to the SQL processing statement, generate a target programming language code corresponding to the SQL processing statement; Using the target programming language code, calling the target function corresponding to the target programming language code; Using the objective function to process the graph data in the node table and the relationship table, to obtain the processing result of the knowledge graph; The target programming language code is a python code; the target function is a plpython function written in a procedural programming language using plpython; The objective function includes a networkx component and a python-lovain component, and the python-lovain component is preset with a community discovery algorithm; The SQL processing statement includes the table name of the node table and the table name of the relationship table; and generating the target programming language code corresponding to the SQL processing statement according to the SQL processing statement includes: Generate a target programming language code for querying the table name of the node table and the table name of the relationship table according to the SQL processing statement; Accordingly, the using the target programming language code to call the target function corresponding to the target programming language code includes: Using the target programming language code, calling the networkx component and the python-lovain component corresponding to the target programming language code; Correspondingly, the process of processing the graph data in the node table and the relationship table using the objective function to obtain the processing result of the knowledge graph includes: Utilizing the networkx component and the python-lovain component to query the graph data in the node table and the relationship table, obtaining all vertex data in the node table and all relationship data in the relationship table; All vertex data in the node table and all relationship data in the relationship table are taken as processing results of the knowledge graph.

2. The method according to claim 1, characterized in that: The distributed database is a GreenPlum database system.

3. The method according to claim 1, characterized in that The storage form of the graph data in the node table and relationship table of the knowledge graph is triple form; the node table includes the self-incrementing sequence ID of the graph data, the entity name and the entity attributes, wherein the entity attributes are in json format; the relationship table includes the self-incrementing sequence ID of the graph data, the relationship pointing to the starting point, the relationship pointing to the end point, and the relationship attributes, wherein the relationship attributes are in json format.

4. The method according to claim 1, characterized in that: The objective function includes a python-lovain component, and the python-lovain component is preset with a community discovery algorithm; The SQL processing statement includes an entity object query condition; and generating a target programming language code corresponding to the SQL processing statement according to the SQL processing statement includes: generating, according to the SQL processing statement, a target programming language code for querying entity objects that satisfy the entity object query condition; Accordingly, the using the target programming language code to call the target function corresponding to the target programming language code includes: Using the target programming language code, calling the python-lovain component; Correspondingly, the process of processing the graph data in the node table and the relationship table using the objective function to obtain the processing result of the knowledge graph includes: Using the community discovery algorithm in the python-lovain component and the entity object query condition, data screening is performed on the graph data in the node table and the relationship table to obtain entity objects that meet the entity object query condition; The entity objects that meet the entity object query conditions are used as the processing results of the knowledge graph.

5. The method according to claim 1, characterized in that The objective function includes a Gin index component; The SQL processing statement includes an index field; and generating a target programming language code corresponding to the SQL processing statement according to the SQL processing statement includes: Generate a target programming language code for retrieving graph data including the index field according to the SQL processing statement; Accordingly, the using the target programming language code to call the target function corresponding to the target programming language code includes: Using the target programming language code, calling the Gin index component; Correspondingly, the process of processing the graph data in the node table and the relationship table using the objective function to obtain the processing result of the knowledge graph includes: Using the Gin index component to search the graph data in the node table and the relationship table to obtain the graph data including the index field; The graph data including the index field is used as the processing result of the knowledge graph.

6. A knowledge graph processing device based on a distributed database, characterized in that: The device is applied to a distributed database, the distributed database includes a node table and a relationship table of a knowledge graph, and the device includes: An acquisition unit, used to acquire an SQL processing statement for the knowledge graph; A generating unit, configured to generate a target programming language code corresponding to the SQL processing statement according to the SQL processing statement; A calling unit, configured to use the target programming language code to call a target function corresponding to the target programming language code; A processing unit, used to process the graph data in the node table and the relationship table using the objective function to obtain a processing result of the knowledge graph; The target programming language code is a python code; the target function is a plpython function written in a procedural programming language using plpython; The objective function includes a networkx component and a python-lovain component, and the python-lovain component is preset with a community discovery algorithm; The SQL processing statement includes the table name of the node table and the table name of the relationship table; and generating the target programming language code corresponding to the SQL processing statement according to the SQL processing statement includes: Generate a target programming language code for querying the table name of the node table and the table name of the relationship table according to the SQL processing statement; Accordingly, the using the target programming language code to call the target function corresponding to the target programming language code includes: Using the target programming language code, calling the networkx component and the python-lovain component corresponding to the target programming language code; Correspondingly, the process of processing the graph data in the node table and the relationship table using the objective function to obtain the processing result of the knowledge graph includes: Utilizing the networkx component and the python-lovain component to query the graph data in the node table and the relationship table, obtaining all vertex data in the node table and all relationship data in the relationship table; All vertex data in the node table and all relationship data in the relationship table are taken as the processing results of the knowledge graph.

7. A readable medium, characterized in that: The readable medium includes execution instructions. When a processor of an electronic device executes the execution instructions, the electronic device executes the method according to any one of claims 1 to 5.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory storing execution instructions. When the processor executes the execution instructions stored in the memory, the processor executes the method according to any one of claims 1 to 5.

Images