Graph data processing method and device, storage medium, and electronic device
By receiving graph calculation requests, dividing and calculating large-scale graph data, the problem of slow graph calculation speed in the prior art is solved, the efficiency and accuracy of graph data processing are improved, and the development cost is reduced.
Patent Information
- Application Number
- CN201811085994.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2018-09-18
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2038-09-18
AI Technical Summary
In the prior art, a stand-alone graph computing platform cannot meet the scalability requirements of large-scale graph data processing, and the computing speed of the distributed graph computing framework is slow, resulting in poor graph computing performance.
By receiving the graph calculation request, the graph division algorithm is called to divide the full graph to calculate, multiple sub-graphs are obtained, and the division results are stored in the hive database. Then the graph calculation algorithm is called to calculate the sub-graph, and finally the calculation is verified by the graph prediction algorithm Accuracy of the results.
The speed of graph division and graph calculation is improved, the accuracy and efficiency of the calculation algorithm are enhanced, the development cost is reduced, and the coupling between algorithm applications and development packages is decoupled.
Smart Images

Figure CN110909191B_ABST
Abstract
Description
Background Art
[0002] Currently, graph data based on the connections between data is playing an increasingly important role. Extracting useful information from related data sets can be used to mine the network characteristics of data. Therefore, correlation calculation has become the core of big data. Furthermore, by obtaining the correlation of data, useful information can be extracted from massive data with a lot of noise. For example, by obtaining the correlation of data, useful information can be extracted from social networks, e-commerce networks, and web links with a lot of noise.
[0003] As data grows, the number of nodes and edges in large graph models can often reach hundreds of millions or even billions; therefore, stand-alone graph computing platforms often cannot meet the scalability requirements of graph computing, and simply using existing distributed graph computing frameworks cannot provide good performance and has a slow computing speed.
[0004] Therefore, it is necessary to provide a new graph data processing method and device.
[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention
[0006] The purpose of the present invention is to provide a graph data processing method, a graph data processing device, a computer-readable storage medium and an electronic device, thereby overcoming the problem of slow computing speed caused by the limitations and defects of related technologies at least to a certain extent.
[0007] According to one aspect of the present disclosure, a graph data processing method is provided, comprising:
[0008] Receiving a graph computing request including a full graph to be computed, and in response to the graph computing request, calling a graph partitioning algorithm corresponding to the full graph to be computed;
[0009] Using the graph partitioning algorithm to partition the full graph to be calculated, to obtain a partitioning result;
[0010] A graph calculation algorithm corresponding to the division result is called to calculate the division result to obtain a graph calculation result.
[0011] In an exemplary embodiment of the present disclosure, calling a graph partitioning algorithm corresponding to the full graph to be calculated includes:
[0012] According to the application configuration of the full graph to be calculated, a graph partitioning algorithm corresponding to the full graph to be calculated is called from a graph partitioning algorithm package.
[0013] In an exemplary embodiment of the present disclosure, the full graph to be calculated is partitioned using the graph partitioning algorithm, and the partitioning result includes:
[0014] The graph partitioning algorithm is used to partition the full graph to be calculated to obtain multiple subgraphs.
[0015] In an exemplary embodiment of the present disclosure, after obtaining the division result, the graph data processing method further includes:
[0016] The division results are stored in the hive database.
[0017] In an exemplary embodiment of the present disclosure, calling a graph calculation algorithm corresponding to the division result to calculate the division result to obtain the graph calculation result includes:
[0018] Calling a graph computing algorithm corresponding to the partitioning result from a graph computing algorithm package;
[0019] The graph calculation algorithm is used to calculate the partition result to obtain the graph calculation result.
[0020] In an exemplary embodiment of the present disclosure, after obtaining the graph calculation result, the graph data processing method further includes:
[0021] Calling a graph prediction algorithm corresponding to the calculation result from a graph prediction algorithm package;
[0022] The graph prediction algorithm is used to verify whether the graph calculation result is correct.
[0023] In an exemplary embodiment of the present disclosure, storing the partitioning result in a hive database includes:
[0024] The division results are stored in the hive database, and the names corresponding to the division results are stored in the table corresponding to the hive database.
[0025] According to one aspect of the present disclosure, a graph data processing device is provided, comprising:
[0026] A calling module, used to receive a graph computing request including a full graph to be calculated, and in response to the graph computing request, call a graph partitioning algorithm corresponding to the full graph to be calculated;
[0027] A partitioning module, used to partition the full graph to be calculated using the graph partitioning algorithm to obtain a partitioning result;
[0028] The calculation module is used to call the graph calculation algorithm corresponding to the division result to calculate the division result to obtain the graph calculation result.
[0029] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for processing graph data described in any one of the above is implemented.
[0030] According to one aspect of the present disclosure, there is provided an electronic device, including:
[0031] Processor; and
[0032] A memory, configured to store executable instructions of the processor;
[0033] Wherein, the processor is configured to perform any one of the above-mentioned graph data processing methods by executing the executable instructions.
[0034] An embodiment of the present invention provides a graph data processing method and device, which calls a graph partitioning algorithm corresponding to a full graph to be calculated, and then uses the graph partitioning algorithm to partition the full graph to be calculated to obtain a partitioning result; finally, calls a graph computing algorithm corresponding to the partitioning result to calculate the partitioning result to obtain a graph computing result; on the one hand, by first calling the graph partitioning algorithm to partition the full graph to be calculated, and then calling the graph computing algorithm corresponding to the partitioning result to calculate the partitioning result, the problem of slow computing speed caused by the use of distributed graph partitioning and graph computing framework in the prior art is solved, and the speed of graph partitioning and graph computing is improved; on the other hand, by calling the graph computing algorithm corresponding to the partitioning result to calculate the partitioning result, the accuracy of calling the graph computing algorithm is improved, and the accuracy of the graph computing result is also improved.
[0035] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present invention, and together with the specification are used to explain the principles of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0037] Figure 1 An example diagram of graph data is schematically shown.
[0038] Figure 2 A flowchart of a graph data processing method is schematically shown.
[0039] Figure 3 A diagram schematically illustrates an example application scenario of a graph data processing method.
[0040] Figure 4 A flowchart schematically shows another method for processing graph data.
[0041] Figure 5 A block diagram of a graph data processing device is schematically shown.
[0042] Figure 6 An example block diagram of an electronic device for implementing the above-mentioned graph data processing method is schematically shown.
[0043] Figure 7 A computer-readable storage medium for implementing the above-mentioned graph data processing method is schematically shown. DETAILED DESCRIPTION
[0044] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present invention will be more comprehensive and complete, and the concept of the example embodiments will be fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solutions of the present invention may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present invention.
[0045] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0046] First, for the sake of explanation and description, the graph database is explained and described here. Figure 1As shown in the figure, the graph database (hive database) is a new database system that efficiently processes complex relational networks based on the algorithms of graph theory in mathematics. A graph database is a database that uses points, edges and attribute information to represent and store data, and responds to queries using a graph structure. The points in a graph database represent entities, such as people, bank accounts, or any traceable items. Edges, also called relationships, are connections between points. Edges are a key concept in graph databases and are an abstraction that cannot be directly implemented in other database systems. Attributes are information that is closely related to nodes. For example, the attributes of an account node include event timestamps, connected branch IDs, etc. The relationships between nodes in a graph database are directly stored in edge-type tables. If a node has a relationship with another node, there is a pointer between the two that directly points to each other, and there is no need to scan the entire table by looking up the primary key of a node to find a matching node.
[0047] This exemplary embodiment first provides a graph data processing method, which can be run on a server, server cluster or cloud server, etc. Of course, those skilled in the art can also run the method of the present invention on other platforms as required, which is not specifically limited in this exemplary embodiment. Figure 2 As shown, the graph data processing method may include steps S210 to S230. Among them:
[0048] In step S210, a graph computing request including a full graph to be computed is received, and in response to the graph computing request, a graph partitioning algorithm corresponding to the full graph to be computed is called.
[0049] In this example implementation, reference Figure 3 As shown, first, the graph computing framework 301 in the server receives a graph computing request including a full graph to be calculated sent from the graph application 302 in the terminal; secondly, when the graph computing framework 301 of the server receives the graph computing request, it responds to the graph computing request and calls the graph partitioning algorithm corresponding to the full graph to be calculated from the graph partitioning engine 3011. Among them, calling the graph partitioning algorithm corresponding to the full graph to be calculated may include: calling the graph partitioning algorithm corresponding to the full graph to be calculated from the graph partitioning algorithm package according to the application configuration of the full graph to be calculated. In detail:
[0050] Here, the graph data of the application scenario of e-commerce anti-fraud is used as the full graph to be calculated for explanation. For example, when the graph computing framework 301 in the server receives the above-mentioned graph computing request, it first calls the graph partitioning algorithm corresponding to the full graph to be calculated for the application scenario of e-commerce anti-fraud from the graph partitioning algorithm package according to the application configuration of the full graph to be calculated for the application scenario of e-commerce anti-fraud. In addition, the above-mentioned application scenarios can also include insurance fraud, non-performing asset collection, anti-money laundering, etc., and this example does not impose special restrictions on this.
[0051] Furthermore, the above-mentioned graph partitioning algorithm package may include a global graph partitioning algorithm and a distributed graph partitioning algorithm; wherein, the global graph partitioning algorithm may include the Laplace graph eigenvalue spectrum bisection method and the multi-layer graph partitioning algorithm, etc.; the distributed graph partitioning algorithm may include a static graph partitioning algorithm and a dynamic graph partitioning algorithm, and the static graph partitioning algorithm may include a hash partitioning, a BHP (balanced Hash partition, load-balanced hash data partitioning) algorithm, a static Mizan algorithm, and a BLP algorithm, etc.; the dynamic graph partitioning algorithm may include a Mizan algorithm and an xDGP algorithm. It should be additionally explained here that since the above is based on the application scenario of e-commerce anti-fraud graph data as the full graph to be calculated for partitioning, a distributed graph partitioning algorithm can be selected for partitioning.
[0052] In step S220, the graph partitioning algorithm is used to partition the full graph to be calculated to obtain a partitioning result.
[0053] In this example implementation, continue to refer to Figure 3 As shown, when the graph computing engine 3012 calls the graph partitioning algorithm corresponding to the full graph to be calculated, the full graph to be calculated is partitioned using the graph partitioning algorithm. Partitioning the full graph to be calculated may include: partitioning the full graph to be calculated using the graph partitioning algorithm to obtain multiple subgraphs. In detail:
[0054] A graph partitioning algorithm (for example, the above-mentioned distributed graph partitioning algorithm) is used to divide the full graph to be calculated for the above-mentioned application scenario of e-commerce anti-fraud into multiple subgraphs. It should be noted here that the above-mentioned graph partitioning algorithm can be used to divide the full graph to be calculated into multiple subgraphs of as uniform size as possible. In this way, the problem of different subgraph sizes causing different times required in the subsequent calculation process, which affects the overall calculation speed, is avoided, and the calculation speed is further improved.
[0055] Further, in order to facilitate the subsequent calculation of each subgraph, after obtaining multiple subgraphs, the divided subgraphs can be stored in the hive database, which can specifically include: storing the division results in the hive database 3014, and storing the names corresponding to the division results in the table corresponding to the hive database. Among them, the hive database is a data warehouse infrastructure built on Hadoop; it provides a series of tools that can be used for data extraction, transformation and loading (ETL), and is a mechanism that can store, query and analyze large-scale data stored in Hadoop. Since the hive database has no requirements for the format of data storage, it is convenient to store the above-mentioned subgraphs, which improves storage efficiency. Further, through the above storage method, each subgraph can be calculated in turn according to the name, and the name corresponding to the calculated subgraph can be marked, avoiding the problem of incorrect calculation results caused by some subgraphs not being calculated and some subgraphs having been calculated multiple times, and further improving the accuracy of the calculation results.
[0056] In step S230, a graph calculation algorithm corresponding to the division result is called to calculate the division result to obtain a graph calculation result.
[0057] In this example implementation, after obtaining multiple divided subgraphs, it is necessary to call the graph calculation algorithm corresponding to the division result to calculate the division result to obtain the graph calculation result. Among them, calling the graph calculation algorithm corresponding to the division result to calculate the division result to obtain the graph calculation result may include: calling the graph calculation algorithm corresponding to the division result from the graph calculation algorithm package; using the graph calculation algorithm to calculate the division result to obtain the graph calculation result. In detail:
[0058] First, call the graph computing algorithm corresponding to the above graph partitioning result from the graph computing algorithm package; after calling the graph computing algorithm corresponding to the graph partitioning result, use the graph computing algorithm to calculate the graph partitioning result, and then obtain the calculation results corresponding to each subgraph. Among them, the graph algorithm can include: various traversal algorithms (these traversals are similar to tree traversals), algorithms for finding the shortest path, algorithms for finding the lowest cost path in the network, etc., and can also include algorithms for answering some simple related questions (for example, whether the graph is connected, what is the shortest path between two vertices in the graph, etc.). Graph algorithms can be applied to many occasions, such as: optimizing pipelines, routing tables, express services, communication websites, etc.
[0059] Furthermore, in order to facilitate the subsequent verification of the calculation results, the calculation results can be stored in the hive database after the calculation results are obtained, which can specifically include: storing the calculation results in the hive database, and storing the names corresponding to the calculation results in the table corresponding to the hive database. Furthermore, through the above storage method, each calculation result can be verified in turn according to the name, and the names corresponding to the verified calculation results can be marked, avoiding that some calculation results have been verified while others have not been verified, further improving the accuracy of the calculation result verification; thereby improving the accuracy of the calculation results.
[0060] Furthermore, after obtaining the above calculation results, in order to ensure the accuracy of the calculation results, the calculation results can also be verified. Figure 4 As shown, the graph data processing method may further include step S410 and step S420. Among them:
[0061] In step S410, a graph prediction algorithm corresponding to the calculation result is called from a graph prediction algorithm package.
[0062] In step S420, the graph prediction algorithm is used to verify whether the graph calculation result is correct.
[0063] Next, step S410 and step S420 are explained and illustrated. First, from the graph prediction algorithm package configured in the graph prediction engine 3013, the corresponding prediction algorithm is called according to the application configuration; then the prediction algorithm is used to predict the above-mentioned full amount graph to be calculated, and the prediction result is obtained; finally, the prediction result is compared to see whether it is consistent with the above-mentioned graph calculation result to determine the correctness of the above-mentioned graph calculation result; if they are consistent, it means that the above-mentioned graph calculation result is correct; if they are inconsistent, the above-mentioned full amount graph to be calculated can be re-divided and recalculated.
[0064] Furthermore, the graph data processing method provided by the present disclosure can be connected to the source data of any business system in e-commerce. Common business types are: order data, login data, and registration data. The source data is stored in the hive table. This system does not support real-time business for the time being, but only for graph calculation of offline data. However, it extracts the data of the day from the stream processing framework Storm every day, processes it, and saves it to the hive table.
[0065] The above graph data processing method, on the one hand, solves the problem of slow operation speed caused by the use of distributed graph partitioning and graph computing framework in the prior art by first calling the graph partitioning algorithm to partition the full graph to be calculated, and then calling the graph computing algorithm corresponding to the partitioning result to calculate the partitioning result, thereby improving the speed of graph partitioning and graph computing; on the other hand, by calling the graph computing algorithm corresponding to the partitioning result to calculate the partitioning result, the accuracy of the graph computing algorithm call is improved, and the accuracy of the graph computing result is also improved. Furthermore, traditional graph partitioning and graph computing are often separate systems. Even if the graph can be partitioned before graph computing to reduce the amount of data for graph computing, the same graph computing algorithm often needs to try multiple partitioning methods, and the graph computing application party may try multiple graph computing algorithms. This framework decouples the coupling between the algorithm application party and the algorithm development kit through the configuration center, reduces development costs, and allows algorithm engineers to focus on algorithm development.
[0066] The present disclosure also provides a graph data processing device. Figure 5 As shown, the graph data processing device may include a calling module 510, a partitioning module 530 and a calculation module.
[0067] The calling module 510 may be configured to receive a graph computing request including a full graph to be computed, and in response to the graph computing request, call a graph partitioning algorithm corresponding to the full graph to be computed.
[0068] The partitioning module 520 may be configured to partition the full graph to be calculated using the graph partitioning algorithm to obtain a partitioning result.
[0069] The calculation module 530 may be configured to call a graph calculation algorithm corresponding to the division result to calculate the division result to obtain a graph calculation result.
[0070] In an example implementation of the present disclosure, the calling module 510 may also be configured to: call a graph partitioning algorithm corresponding to the full graph to be calculated from a graph partitioning algorithm package according to an application configuration of the full graph to be calculated.
[0071] In an example implementation of the present disclosure, the above-mentioned partitioning module may also be configured to: partition the full graph to be calculated using the graph partitioning algorithm to obtain a plurality of sub-graphs.
[0072] In an example implementation of the present disclosure, the graph data processing apparatus may further include: a storage module, which may be used to store the partitioning result in a hive database.
[0073] In an example implementation of the present disclosure, the above-mentioned calculation module may also be configured to: call a graph calculation algorithm corresponding to the partition result from a graph calculation algorithm package; and calculate the partition result using the graph calculation algorithm to obtain the graph calculation result.
[0074] In an exemplary implementation of the present disclosure, the above-mentioned graph data processing apparatus may further include:
[0075] The prediction algorithm calling module can be used to call the graph prediction algorithm corresponding to the calculation result from the graph prediction algorithm package.
[0076] A verification module is used to verify whether the graph calculation result is correct using the graph prediction algorithm.
[0077] In an example implementation of the present disclosure, the storage module may also be configured to: store the division result in the hive database, and store the name corresponding to the division result in a table corresponding to the hive database.
[0078] The specific details of each module in the above-mentioned graph data processing device have been described in detail in the corresponding graph data processing method, so they will not be repeated here.
[0079] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.
[0080] In addition, although the steps of the method of the present invention are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.
[0081] Through the description of the above implementation modes, it is easy for those skilled in the art to understand that the example implementation modes described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation mode of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation mode of the present invention.
[0082] In an exemplary embodiment of the present invention, an electronic device capable of implementing the above method is also provided.
[0083] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".
[0084] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present invention is described. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0085] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610).
[0086] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 610 can perform the following steps: Figure 2 Step S210 shown in: receiving a graph calculation request including a full graph to be calculated, and in response to the graph calculation request, calling a graph partitioning algorithm corresponding to the full graph to be calculated; step S220: using the graph partitioning algorithm to partition the full graph to be calculated to obtain a partitioning result; step S230: calling a graph calculation algorithm corresponding to the partitioning result to calculate the partitioning result to obtain a graph calculation result.
[0087] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .
[0088] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0089] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0090] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 660. As shown, the network adapter 660 communicates with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0091] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present invention.
[0092] In an exemplary embodiment of the present invention, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of this specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to perform the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of this specification.
[0093] refer to Figure 7 As shown, a program product 710 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.
[0094] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0095] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0096] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0097] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0098] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0099] Other embodiments of the invention will readily occur to those skilled in the art after considering the specification and practicing the invention invented herein. This application is intended to cover any variations, uses or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art that are not invented by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.
Claims
1. A graph data processing method, characterized in that: include: Receiving a graph computing request including a full graph to be computed, and in response to the graph computing request, calling a graph partitioning algorithm corresponding to the full graph to be computed from a graph partitioning engine, wherein the graph computing framework includes the graph partitioning engine, the graph computing engine, and the graph database; Using the graph partitioning algorithm to partition the full graph to be calculated, obtaining a partitioning result, storing the partitioning result in the graph database, and storing the name corresponding to the partitioning result in a table corresponding to the graph database; Calling a graph calculation algorithm corresponding to the partition result from the graph calculation engine to calculate the partition result to obtain a graph calculation result; Calling a graph prediction algorithm corresponding to the graph calculation result from a graph prediction algorithm package; The graph prediction algorithm is used to verify whether the graph calculation result is correct.
2. The graph data processing method according to claim 1, characterized in that: Calling a graph partitioning algorithm corresponding to the full graph to be calculated includes: According to the application configuration of the full graph to be calculated, a graph partitioning algorithm corresponding to the full graph to be calculated is called from a graph partitioning algorithm package.
3. The graph data processing method according to claim 1, characterized in that: The graph partitioning algorithm is used to partition the full graph to be calculated, and the partitioning results include: The graph partitioning algorithm is used to partition the full graph to be calculated to obtain multiple subgraphs.
4. The graph data processing method according to claim 1, characterized in that: After obtaining the division result, storing the division result in the graph database includes: The division results are stored in the hive database.
5. The graph data processing method according to claim 1, characterized in that: Calling a graph calculation algorithm corresponding to the division result to calculate the division result, and obtaining the graph calculation result includes: Calling a graph computing algorithm corresponding to the partitioning result from a graph computing algorithm package; The graph calculation algorithm is used to calculate the partition result to obtain the graph calculation result.
6. The graph data processing method according to claim 4, characterized in that: Storing the partitioning results in the hive database includes: The division results are stored in the hive database, and the names corresponding to the division results are stored in the table corresponding to the hive database.
7. A graph data processing device, characterized in that: include: A calling module, used to receive a graph computing request including a full graph to be computed, and in response to the graph computing request, call a graph partitioning algorithm corresponding to the full graph to be computed from a graph partitioning engine, wherein the graph computing framework includes the graph partitioning engine, the graph computing engine, and the graph database; A partitioning module, used to partition the full graph to be calculated using the graph partitioning algorithm to obtain a partitioning result; A storage module, used to store the division results in the graph database, and store the names corresponding to the division results in a table corresponding to the graph database; A calculation module, used to call a graph calculation algorithm corresponding to the division result from the graph calculation engine to calculate the division result to obtain a graph calculation result; A prediction algorithm calling module, used to call a graph prediction algorithm corresponding to the graph calculation result from a graph prediction algorithm package; A verification module is used to verify whether the graph calculation result is correct using the graph prediction algorithm.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the graph data processing method described in any one of claims 1 to 6 is implemented.
9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the graph data processing method described in any one of claims 1-6 by executing the executable instructions.
