Method and device for generating business logic rule graph
By generating knowledge graphs and display graphs, and automatically processing the business logic rules of data warehouses, the problems of low efficiency and poor accuracy of manual sorting in the existing technology are solved, and the efficiency and accuracy of sorting business logic rules of data warehouses are improved.
Patent Information
- Application Number
- CN202111416543.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-25
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-11-25
AI Technical Summary
In the prior art, the business logic rules sorting of data warehouses mainly relies on manual labor, resulting in low efficiency and poor accuracy. As the complexity of business logic rules increases, the difficulty and time cost of manual sorting also increase.
By obtaining data table identification, business functions and association rules in the data warehouse, a knowledge graph is generated, and a business logic rule graph is automatically generated, and a data table identification list and display graph are provided to help users select and query data tables and simplify the business logic of the data warehouse. Rule sorting process.
It improves the efficiency and accuracy of business logic rules sorting in data warehouses, reduces manual intervention, simplifies the display and query of business logic rules of data warehouses, and improves intuitiveness and clarity.
Smart Images

Figure CN114090702B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of big data technology, and in particular to a method and device for generating a business logic rule graph. Background Art
[0002] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section.
[0003] The data warehouse includes: data source layer, source layer, basic layer, main layer, and application layer; each layer has different functions and roles. When migrating the data warehouse, you first need to sort out the business logic rules in the original data warehouse, and then develop in the new data warehouse based on the sorted business logic rules.
[0004] Currently, the business logic rules of data warehouses are mainly combed manually. However, enterprises will continuously adjust and update the business logic rules in the data warehouse according to their own business needs. As time goes by, the business logic rules in the enterprise will become more and more complex, and the program source code in the data warehouse is developed in groups. Therefore, it is difficult to comb the business logic rules of the data warehouse manually, and the efficiency and accuracy of the business logic rules combing of the data warehouse are low. Summary of the invention
[0005] The embodiment of the present invention provides a method for generating a business logic rule graph, which is used to automatically generate a knowledge graph of business logic rules, improve the efficiency of combing business logic rules in a data warehouse, and improve the accuracy of combing business logic rules in a data warehouse. The method includes:
[0006] Obtaining a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table; wherein the first data table includes a data table of an application layer in the data warehouse;
[0007] Determine, according to the association rule table, a second data table identifier of a second data table associated with the first data table, and a generation order of the second data table and the first data table;
[0008] Generate a knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table;
[0009] Generate a data table identifier list according to the first data table identifier and the second data table identifier included in the knowledge graph, and provide the data table identifier list to the user; wherein the data table identifier list is used for the user to select a data table identifier to be queried;
[0010] A display graph is generated according to a data table identifier to be queried by a user and a knowledge graph; wherein the display graph includes an upstream data table identifier and / or a downstream data table identifier corresponding to the data table identifier to be queried by the user.
[0011] The embodiment of the present invention also provides a device for generating a business logic rule graph, which is used to automatically generate a knowledge graph of business logic rules, improve the efficiency of combing business logic rules in a data warehouse, and improve the accuracy of combing business logic rules in a data warehouse. The device includes:
[0012] An acquisition module, used to acquire a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table; wherein the first data table includes a data table of an application layer in the data warehouse;
[0013] A first processing module, configured to determine, according to the association rule table, a second data table identifier of a second data table associated with the first data table, and a generation order of the second data table and the first data table;
[0014] A knowledge graph generation module, used to generate a knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table;
[0015] The second processing module is used to generate a data table identifier list according to the first data table identifier and the second data table identifier contained in the knowledge graph, and provide the data table identifier list to the user; wherein the data table identifier list is used for the user to select the data table identifier to be queried;
[0016] A display graph generation module is used to generate a display graph based on the data table identifier to be queried by the user and the knowledge graph; wherein the display graph contains the upstream data table identifier and / or downstream data table identifier corresponding to the data table identifier to be queried by the user.
[0017] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for generating a business logic rule graph when executing the computer program.
[0018] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating the above-mentioned business logic rule graph is implemented.
[0019] An embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for generating the above-mentioned business logic rule graph is implemented.
[0020] In an embodiment of the present invention, a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table are obtained; wherein the first data table includes a data table of an application layer in the data warehouse; according to the association rule table, a second data table identifier of a second data table associated with the first data table and a generation order of the second data table and the first data table are determined; a knowledge graph is generated according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table; a data table identifier list is generated according to the first data table identifier and the second data table identifier included in the knowledge graph, and the data table identifier list is provided to a user; wherein the data table identifier list is used for the user to select a data table identifier to be queried; a display graph is generated according to the data table identifier to be queried by the user and the knowledge graph; wherein the display graph includes an upstream data table identifier and / or a downstream data table identifier corresponding to the data table identifier to be queried by the user. In this way, a knowledge graph of business logic rules can be automatically generated, the efficiency of combing business logic rules of the data warehouse can be improved, and the accuracy of combing business logic rules of the data warehouse can be improved. Furthermore, according to the data table identifier that the user wants to query, the upstream data table identifier and / or downstream data table identifier of the data table identifier can be displayed to the user, so as to simplify the display of the knowledge graph and make it more intuitive and clear. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0022] Figure 1 A flowchart of a method for generating business logic rules provided in an embodiment of the present invention;
[0023] Figure 2 A flow chart of a method for obtaining a first data table identifier of a first data table in a data warehouse and storing an association rule table of association rules of the first data table provided in an embodiment of the present invention;
[0024] Figure 3 A flow chart of a method for obtaining a business function involved in a first data table in a data warehouse provided in an embodiment of the present invention;
[0025] Figure 4 This is an example diagram of a knowledge graph provided in an embodiment of the present invention;
[0026] Figure 5 This is an example diagram showing a graph provided in an embodiment of the present invention;
[0027] Figure 6 An example diagram of a knowledge graph and its corresponding edge graph provided in an embodiment of the present invention;
[0028] Figure 7 This is an example diagram of an upstream data table identification display content provided in an embodiment of the present invention;
[0029] Figure 8 This is an example diagram of a device for generating business logic rules provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0030] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.
[0031] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.
[0032] In the description of this specification, the terms "include", "including", "have", "contain", etc. are all open terms, which mean including but not limited to. The descriptions with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. The order of steps involved in each embodiment is used to schematically illustrate the implementation of the present application, and the order of steps is not limited and can be appropriately adjusted as needed.
[0033] According to research, the data warehouse includes: data source layer, source layer, basic layer, main layer, and application layer; each layer has different functions and roles. When migrating the data warehouse, the business logic rules in the original data warehouse must be sorted out first, and then developed in the new data warehouse according to the sorted out business logic rules. At present, the business logic rules of the data warehouse are mainly sorted out manually. However, enterprises will continuously adjust and update the business logic rules in the data warehouse according to their own business needs. As time goes by, the business logic rules in the enterprise will become more and more complex, and the program source code in the data warehouse is developed in groups. Therefore, it is difficult to sort out the business logic rules of the data warehouse manually, and the efficiency and accuracy of sorting out the business logic rules of the data warehouse are low.
[0034] In view of the above research, an embodiment of the present invention provides a method for generating a business logic rule graph, such as Figure 1 As shown, including:
[0035] S101: Obtain a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table; wherein the first data table includes a data table of an application layer in the data warehouse;
[0036] S102: Determine, according to the association rule table, a second data table identifier of a second data table associated with the first data table, and a generation order of the second data table and the first data table;
[0037] S103: Generate a knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table;
[0038] S104: Generate a data table identifier list according to the first data table identifier and the second data table identifier included in the knowledge graph, and provide the data table identifier list to the user; wherein the data table identifier list is used for the user to select a data table identifier to be queried;
[0039] S105: Generate a display graph according to the data table identifier to be queried by the user and the knowledge graph; wherein the display graph includes the upstream data table identifier and / or downstream data table identifier corresponding to the data table identifier to be queried by the user.
[0040] In an embodiment of the present invention, a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table are obtained; wherein the first data table includes a data table of an application layer in the data warehouse; according to the association rule table, a second data table identifier of a second data table associated with the first data table and a generation order of the second data table and the first data table are determined; a knowledge graph is generated according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table; a data table identifier list is generated according to the first data table identifier and the second data table identifier included in the knowledge graph, and the data table identifier list is provided to a user; wherein the data table identifier list is used for the user to select a data table identifier to be queried; a display graph is generated according to the data table identifier to be queried by the user and the knowledge graph; wherein the display graph includes an upstream data table identifier and / or a downstream data table identifier corresponding to the data table identifier to be queried by the user. In this way, a knowledge graph of business logic rules can be automatically generated, the efficiency of combing business logic rules of the data warehouse can be improved, and the accuracy of combing business logic rules of the data warehouse can be improved. Furthermore, according to the data table identifier that the user wants to query, the upstream data table identifier and / or downstream data table identifier of the data table identifier can be displayed to the user, so as to simplify the display of the knowledge graph and make it more intuitive and clear.
[0041] The above S101 to S105 are described in detail below.
[0042] With respect to the above S101, the first data table includes, for example, a data table of an application layer in a data warehouse.
[0043] like Figure 2 As shown, a flow chart of a method for obtaining a first data table identifier of a first data table in a data warehouse and storing an association rule table of association rules of the first data table provided by an embodiment of the present invention includes:
[0044] S201: Perform keyword screening on the Perl script of the data warehouse, extract the content after the first preset keyword and before the second preset keyword to obtain a first data table identifier.
[0045] Exemplarily, the first preset keyword includes, for example: "CREATE TABLE" or "CREATE TABLE IFNOT EXISTS", and the second preset keyword includes, for example: "AS"; the Perl script of the data warehouse is screened for keywords, and for the first data table identifier, the part after "CREATE TABLE" or "CREATE TABLE IF NOT EXISTS" and before "AS" is extracted. This part corresponds to the name of the created data table according to the SQL syntax, that is, the first data table identifier of the first data table.
[0046] S202: using a regular expression and a third preset keyword related to the association rule, obtaining an association rule table from a corresponding position in the Perl script.
[0047] Exemplarily, the third preset keyword includes, for example, at least one of: "JOIN", "INNER JOIN", "LEFT JOIN", "RIGHTJOIN", "UNION (UNION ALL)", etc. The association rule table generally appears before the third preset keyword or after the third preset keyword, or exists in the SQL statement after the third preset keyword. A regular expression is used to determine the position of the association rule table according to the third preset keyword, and the association rule table is extracted from the corresponding position of the association rule table.
[0048] In addition, if Figure 3 FIG. 1 is a flow chart of a method for obtaining a business function involved in a first data table in a data warehouse provided in an embodiment of the present invention, including:
[0049] S301: traverse the comment part of the Perl script of each first data table according to the corpus and the stop words to extract keywords related to the business function.
[0050] Here, the corpus is generated according to the business functions involved in the data warehouse; the stop words include, for example, at least one of the following: time, date, name, email address, and phone number.
[0051] S302: Obtain the priority of each extracted keyword based on the frequency of occurrence of the extracted keyword in the annotation part and the frequency of occurrence in the corpus, and determine the business function corresponding to the keyword with a priority greater than a preset priority threshold as the business function involved in the first data table.
[0052] Here, when obtaining the priority of each extracted keyword based on the frequency of occurrence of the extracted keyword in the annotation part and the frequency of occurrence in the corpus, for example, the (term frequency-inverse document frequency, TF-IDF) algorithm can be used, wherein TF is the frequency of occurrence of the extracted keyword in the annotation part, and IDF is the frequency of occurrence of the extracted keyword in the corpus. The priority of the extracted keyword increases in direct proportion to the number of times the keyword appears in the annotation part, but at the same time decreases in inverse proportion to the frequency of occurrence of the keyword in the corpus.
[0053] With respect to the above S102, there is more than one association rule table in the data warehouse, and each association rule table has a sequence, and the sequence of the association rule tables is related to the keywords related to the association rules corresponding to the association rule tables. Therefore, for example, the following method can be used to determine the second data table identifier of the second data table associated with the first data table, and the generation sequence of the second data table and the first data table according to the association rule table: determine the query priority of each association rule table according to the keywords related to the association rules corresponding to the association rule table; query each association rule table in turn according to the query priority of each association rule table, and obtain the second data table identifier of the second data table associated with the first data table, and the generation sequence of the second data table and the first data table from each association rule table.
[0054] With respect to the above S103, when generating a knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business functions involved in the first data table, for example, the following method can be used: a mesh data graph in RDF data format is generated according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business functions involved in the first data table, wherein the first data table identifier and the second data table identifier are used as nodes in the mesh data graph, and the data table identifier is used as the corresponding node identifier; each node is connected according to the generation order of the second data table and the first data table, and the connecting lines are used as edges in the mesh data graph, and the business functions involved in the first data table are marked next to the nodes corresponding to the first data table.
[0055] For example, Figure 4 As shown, it is an example diagram of a knowledge graph provided by an embodiment of the present invention. Data Table 4 is obtained according to Data Table 1, Data Table 5 is obtained according to Data Table 2, Data Table 8 is obtained according to the combination of Data Table 3 and Data Table 5, Data Table 6 is obtained according to Data Table 4, Data Table 7 is obtained according to the combination of Data Table 4 and Data Table 5, Data Table 9 and Data Table 10 are obtained according to Data Table 7, and Data Table 11 is obtained according to Data Table 10. The lines between the data table identifiers are edges, and the connection relationship between the data table identifiers is determined by the generation order between the data tables. Among them, the functions included in Data Table 6 are: loan, the functions included in Data Table 8 are: living payment, the functions included in Data Table 9 are: transfer, remittance, and the functions included in Data Table 11 are: investment and financial management. Data Table 6, Data Table 8, Data Table 9, and Data Table 11 are, for example, the first data table (data table of the application layer).
[0056] With respect to the above S104, the data warehouse contains a huge number of data tables, and the generated knowledge graph contains the generation order between the data tables in the data warehouse, as well as the association relationship between the data tables. Therefore, the generated knowledge graph contains a huge number of nodes, and the connection relationship between the nodes is also relatively complex. Therefore, the data table identifiers of the data tables contained in the knowledge graph can be listed into a data table identifier list, and the data table identifier list is used for the user to select the data table identifier to be queried, and the source and / or destination of the data table identifier (that is, the upstream data table identifier and / or downstream data table identifier corresponding to the data table identifier) is displayed according to the data table identifier to be queried selected by the user.
[0057] For example, Figure 4 The knowledge graph shown generates a list of data identifiers: Data Table 1, Data Table 2, Data Table 3, Data Table 4, Data Table 5, Data Table 6, Data Table 7, Data Table 8, Data Table 9, Data Table 10, and Data Table 11.
[0058] For the above S105, the display graph contains the upstream data table identifier and / or the downstream data table identifier corresponding to the data table identifier to be queried by the user. For example, for Figure 4 The following is a knowledge graph. The data table that the user wants to query is identified as data table 10. The corresponding display graph is as follows: Figure 5 As shown, data table 1, data table 2, data table 4, data table 5, and data table 7 are upstream data identifiers of data table 10, and data table 11 is downstream data identifier of data table 10.
[0059] In addition, some nodes in the knowledge graph can be reused. For the reusable nodes, when obtaining the display graph corresponding to the downstream nodes, the upstream data of the downstream nodes above the reused nodes are the same, for example Figure 4In the knowledge graph shown, data table 7 is a reusable node. When display graphs are generated for data table 9 and data table 10 respectively, the display graph of data table 9 and the display graph of data table 10 are the same in the upstream display content above data table 7 (that is, the upstream data table identifier above data table 7, and the relationship between the data table identifiers). If the knowledge graph needs to be traversed once each time a display graph is generated, it will cause a waste of computing resources. Therefore, in one embodiment of the present invention, after the knowledge graph is generated according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business functions involved in the first data table, the edges in the knowledge graph are converted into points in the edge graph, and the knowledge graph is converted into points in the edge graph. Two edges with the same node are converted into two points connected by the edge in the edge graph; multiple communities are obtained according to the structural similarity between every two points in the edge graph, each community contains at least one point, and the structural similarity between points in the same community is greater than a preset structural similarity threshold; the cosine similarity between communities is calculated according to the adjacency matrix of each community, two communities whose cosine similarity is greater than a preset cosine similarity threshold are merged, and corresponding labels are generated for the merged community; the merged community contains multiple points; the label corresponding to each merged community is used as the label of the corresponding edge of the point in the merged community in the knowledge graph; for the common nodes connected by the edges with the same label, the upstream data table identification display content of the node is generated.
[0060] Specifically, for example, the Louvain algorithm of the Spark GraphX framework can be used to calculate the edge graph converted from the knowledge graph (calculate the structural similarity between points and the cosine similarity between communities), merge communities based on the calculation results, and generate labels for the merged communities.
[0061] For example, Figure 6 The knowledge graph shown in a contains edge A, edge B, edge C, edge D, edge E, and edge F, where edge A, edge B, edge C, and edge D are connected to the same node, and the node shared by edge A, edge B, edge C, and edge D is a reused node. The edges A, edge B, edge C, edge D, edge E, and edge F in the knowledge graph are converted into position Figure 6 In the point graph shown in b, there are lines connecting points A, B, C, D, E and F. Two points with a structural similarity greater than a preset structural similarity threshold are regarded as a community. Each community contains at least one point. For example, after calculation, Figure 6In the edge graph shown in b, the structural similarity between point A and point B is greater than the preset structural similarity threshold, and the structural similarity between point C and point D is greater than the preset structural similarity threshold, so four communities are obtained. Community 1 contains: point A and point B, community 2 contains points C and point D, community 3 contains point E, and community 4 contains point F. The cosine similarity between the communities is calculated, and the communities whose cosine similarity is greater than the preset cosine similarity threshold are merged. For example, the cosine similarity between community 1 and community 2 is greater than the preset cosine similarity threshold, and finally three communities are obtained: the merged community 1 after the merger of community 1 and community 2, community 3, and community 4. The corresponding label 1 is generated for the points in the merged community 1, and label 1 is used as Figure 6 The labels of edge A, edge B, edge C, and edge D in the knowledge graph shown in a.
[0062] In this way, when a display graph is generated based on the data table identifier that the user wants to query and the knowledge graph, for example, the upstream data and / or downstream data of the data table identifier that the user wants to query is obtained from the knowledge graph; wherein the upstream data includes the upstream data table identifier and the edge connecting the upstream data table identifier; the downstream data includes the upstream data table identifier and the edge connecting the downstream data table identifier; a display graph is generated based on the upstream data and the downstream data; wherein, when obtaining the upstream data of the node corresponding to the data table identifier that the user wants to query, if there is a label in the upstream data, for the upstream data above the upstream data table identifier connected by the edge corresponding to the label, the display content of the upstream data table identifier corresponding to the label is directly obtained.
[0063] For example Figure 4 In the knowledge graph shown, node 7 is a reused node, and the labels of edges (7,9) and (7,10) are the same. Then the upstream data table corresponding to data table 7 is directly generated to display the content (such as Figure 7 As shown), when generating the display graph of data table 9, data table 10, or data table 11, when obtaining upstream data above data table 7, directly obtain the upstream data table identification display content corresponding to data table 7, without repeatedly traversing the knowledge graph, reducing the waste of computing resources.
[0064] The present invention also provides a device for generating a business logic rule map, as described in the following embodiments. Since the principle of the device for solving the problem is similar to the method for generating a business logic rule map, the implementation of the device can refer to the implementation of the method for generating a business logic rule map, and the repeated parts will not be repeated.
[0065] like Figure 8As shown, it is an example diagram of a device for generating a business logic rule graph provided by an embodiment of the present invention, comprising: an acquisition module 801, a first processing module 802, a knowledge graph generation module 803, a second processing module 804, and a display graph generation module 805; wherein,
[0066] The acquisition module 801 is used to acquire a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table; wherein the first data table includes a data table of an application layer in the data warehouse;
[0067] A first processing module 802 is used to determine, according to the association rule table, a second data table identifier of a second data table associated with the first data table, and a generation order of the second data table and the first data table;
[0068] The knowledge graph generation module 803 is used to generate a knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table;
[0069] The second processing module 804 is used to generate a data table identifier list according to the first data table identifier and the second data table identifier contained in the knowledge graph, and provide the data table identifier list to the user; wherein the data table identifier list is used for the user to select the data table identifier to be queried;
[0070] The display graph generation module 805 is used to generate a display graph based on the data table identifier to be queried by the user and the knowledge graph; wherein the display graph contains the upstream data table identifier and / or downstream data table identifier corresponding to the data table identifier to be queried by the user.
[0071] In one possible implementation, the acquisition module is specifically used to perform keyword screening on the Perl script of the data warehouse, extract the content after the first preset keyword and before the second preset keyword to obtain the first data table identifier; use regular expressions and the third preset keyword involving the association rules to obtain the association rule table from the corresponding position in the Perl script.
[0072] In a possible implementation, the first processing module is specifically used to determine the query priority of each association rule table according to the keywords related to the association rules corresponding to the association rule table; query each association rule table in turn according to the query priority of each association rule table, and obtain the second data table identifier of the second data table associated with the first data table, as well as the generation order of the second data table and the first data table from each association rule table.
[0073] In a possible implementation, the acquisition module is specifically used to extract keywords related to business functions based on a corpus and a comment part of a Perl script that traverses each first data table using stop words; the corpus is generated based on the business functions involved in the data warehouse; the stop words include at least one of the following: time, date, name, email, and phone number; based on the frequency of occurrence of the extracted keywords in the comment part and the frequency of occurrence in the corpus, the priority of each extracted keyword is obtained, and the business function corresponding to the keyword having a priority greater than a preset priority threshold is determined as the business function involved in the first data table.
[0074] In one possible implementation, the knowledge graph generation module is specifically used to generate a mesh data graph in RDF data format according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business functions involved in the first data table, wherein the first data table identifier and the second data table identifier are used as nodes in the mesh data graph, and the data table identifier is used as the corresponding node identifier; each node is connected according to the generation order of the second data table and the first data table, and the connecting line is used as the edge in the mesh data graph, and the business functions involved in the first data table are marked next to the node corresponding to the first data table.
[0075] In a possible implementation, it also includes: a third processing module, which is used to convert the edges in the knowledge graph into points in the edge graph, and convert two edges with the same node in the knowledge graph into two points connected by the edge in the edge graph; obtain multiple communities according to the structural similarity between every two points in the edge graph, each community contains at least one point, and the structural similarity between points in the same community is greater than a preset structural similarity threshold; calculate the cosine similarity between communities according to the adjacency matrix of each community, merge two communities with cosine similarity greater than the preset cosine similarity threshold, and generate corresponding labels for the merged community; the merged community contains multiple points; use the label corresponding to each merged community as the label of the corresponding edge of the point in the merged community in the knowledge graph; for common nodes connected by edges with the same label, generate the upstream data table identification display content of the node;
[0076] A display graph generation module is specifically used to obtain upstream data and / or downstream data of a data table identifier that a user wants to query from a knowledge graph; wherein the upstream data includes an upstream data table identifier and an edge connecting the upstream data table identifier; the downstream data includes an upstream data table identifier and an edge connecting the downstream data table identifier; a display graph is generated based on the upstream data and the downstream data; wherein, when obtaining the upstream data of a node corresponding to the data table identifier that a user wants to query, if there is a label in the upstream data, for the upstream data above the upstream data table identifier connected to the edge corresponding to the label, the display content of the upstream data table identifier corresponding to the label is directly obtained.
[0077] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for generating a business logic rule graph when executing the computer program.
[0078] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating the above-mentioned business logic rule graph is implemented.
[0079] An embodiment of the present invention further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method for generating the above-mentioned business logic rule graph is implemented.
[0080] In an embodiment of the present invention, a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table are obtained; wherein the first data table includes a data table of an application layer in the data warehouse; according to the association rule table, a second data table identifier of a second data table associated with the first data table and a generation order of the second data table and the first data table are determined; a knowledge graph is generated according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table; a data table identifier list is generated according to the first data table identifier and the second data table identifier included in the knowledge graph, and the data table identifier list is provided to a user; wherein the data table identifier list is used for the user to select a data table identifier to be queried; a display graph is generated according to the data table identifier to be queried by the user and the knowledge graph; wherein the display graph includes an upstream data table identifier and / or a downstream data table identifier corresponding to the data table identifier to be queried by the user. In this way, a knowledge graph of business logic rules can be automatically generated, the efficiency of combing business logic rules of the data warehouse can be improved, and the accuracy of combing business logic rules of the data warehouse can be improved. Furthermore, according to the data table identifier that the user wants to query, the upstream data table identifier and / or downstream data table identifier of the data table identifier can be displayed to the user, so as to simplify the display of the knowledge graph and make it more intuitive and clear.
[0081] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0082] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0083] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0085] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A method for generating a business logic rule graph, characterized in that: include: Obtaining a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table; wherein the first data table includes a data table of an application layer in the data warehouse; Determine, according to the association rule table, a second data table identifier of a second data table associated with the first data table, and a generation order of the second data table and the first data table; Generate a knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table; Generate a data table identifier list according to the first data table identifier and the second data table identifier included in the knowledge graph, and provide the data table identifier list to the user; wherein the data table identifier list is used for the user to select a data table identifier to be queried; Generate a display graph according to the data table identifier to be queried by the user and the knowledge graph; wherein the display graph contains the upstream data table identifier and / or the downstream data table identifier corresponding to the data table identifier to be queried by the user; After generating the knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table, the method further includes: Converting the edges in the knowledge graph into points in the edge graph, and converting two edges with the same node in the knowledge graph into two points connected by the edge in the edge graph; According to the structural similarity between every two points in the edge graph, multiple communities are obtained, each community contains at least one point, and the structural similarity between the points in the same community is greater than a preset structural similarity threshold; The cosine similarity between communities is calculated based on the adjacency matrix of each community, two communities whose cosine similarity is greater than a preset cosine similarity threshold are merged, and corresponding labels are generated for the merged community; the merged community contains multiple points; The label corresponding to each merged community is used as the label of the edge corresponding to the point in the merged community in the knowledge graph; For common nodes connected by edges with the same label, generate the upstream data table identification display content of the node; Generate a display graph based on the data table identifier that the user wants to query and the knowledge graph, including: Obtaining upstream data and / or downstream data of the data table identifier to be queried by the user from the knowledge graph; wherein the upstream data includes the upstream data table identifier and the edge connecting the upstream data table identifier; the downstream data includes the upstream data table identifier and the edge connecting the downstream data table identifier; Generate a display graph according to the upstream data and the downstream data; Among them, when obtaining the upstream data of the node corresponding to the data table identifier that the user wants to query, if there is a label in the upstream data, for the upstream data above the upstream data table identifier connected by the edge corresponding to the label, directly obtain the display content of the upstream data table identifier corresponding to the label.
2. The generation method according to claim 1, characterized in that: Acquiring a first data table identifier of a first data table in a data warehouse and an association rule table storing association rules of the first data table includes: Perform keyword screening on the Perl script of the data warehouse, extract the content after the first preset keyword and before the second preset keyword to obtain a first data table identifier; By using the regular expression and the third preset keyword related to the association rule, the association rule table is obtained from the corresponding position in the Perl script.
3. The generation method according to claim 2, characterized in that: Determining, according to the association rule table, a second data table identifier of a second data table associated with the first data table, and a generation order of the second data table and the first data table, includes: Determine the query priority of each association rule table according to the keywords involved in the association rule corresponding to the association rule table; The association rule tables are queried in sequence according to their query priorities, and the second data table identifier of the second data table associated with the first data table and the generation order of the second data table and the first data table are obtained from the association rule tables.
4. The generation method according to claim 1, characterized in that: Get the business functions involved in the first data table in the data warehouse, including: Extracting keywords related to business functions according to the corpus and the comment part of the Perl script of each first data table using stop words; the corpus is generated according to the business functions involved in the data warehouse; the stop words include at least one of the following: time, date, name, email address, telephone number; The priority of each extracted keyword is obtained according to the frequency of occurrence of the extracted keyword in the annotation part and the frequency of occurrence in the corpus, and the business function corresponding to the keyword with a priority greater than a preset priority threshold is determined as the business function involved in the first data table.
5. The generation method according to claim 1, characterized in that: Generate a knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business functions involved in the first data table, including: A mesh data graph in RDF data format is generated according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business functions involved in the first data table, wherein the first data table identifier and the second data table identifier are used as nodes in the mesh data graph, and the data table identifier is used as the corresponding node identifier; the nodes are connected according to the generation order of the second data table and the first data table, and the connecting lines are used as edges in the mesh data graph, and the business functions involved in the first data table are marked next to the nodes corresponding to the first data table.
6. A device for generating a business logic rule graph, characterized in that: include: An acquisition module, used to acquire a first data table identifier of a first data table in a data warehouse, a business function involved in the first data table, and an association rule table storing association rules of the first data table; wherein the first data table includes a data table of an application layer in the data warehouse; A first processing module, configured to determine, according to the association rule table, a second data table identifier of a second data table associated with the first data table, and a generation order of the second data table and the first data table; A knowledge graph generation module, used to generate a knowledge graph according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business function involved in the first data table; The second processing module is used to generate a data table identifier list according to the first data table identifier and the second data table identifier contained in the knowledge graph, and provide the data table identifier list to the user; wherein the data table identifier list is used for the user to select the data table identifier to be queried; A display graph generation module, used to generate a display graph according to the data table identifier to be queried by the user and the knowledge graph; wherein the display graph contains the upstream data table identifier and / or the downstream data table identifier corresponding to the data table identifier to be queried by the user; It also includes a third processing module, which is used to: convert the edges in the knowledge graph into points in the edge graph, and convert two edges with the same node in the knowledge graph into two points connected by the edge in the edge graph; According to the structural similarity between every two points in the edge graph, multiple communities are obtained, each community contains at least one point, and the structural similarity between the points in the same community is greater than a preset structural similarity threshold; The cosine similarity between communities is calculated based on the adjacency matrix of each community, two communities whose cosine similarity is greater than a preset cosine similarity threshold are merged, and corresponding labels are generated for the merged community; the merged community contains multiple points; The label corresponding to each merged community is used as the label of the edge corresponding to the point in the merged community in the knowledge graph; For common nodes connected by edges with the same label, generate the upstream data table identification display content of the node; The display graph generation module is specifically used to obtain the upstream data and / or downstream data of the data table identifier to be queried by the user from the knowledge graph; wherein the upstream data includes the upstream data table identifier and the edge connecting the upstream data table identifier; the downstream data includes the upstream data table identifier and the edge connecting the downstream data table identifier; Generate a display graph according to the upstream data and the downstream data; Among them, when obtaining the upstream data of the node corresponding to the data table identifier that the user wants to query, if there is a label in the upstream data, for the upstream data above the upstream data table identifier connected by the edge corresponding to the label, directly obtain the display content of the upstream data table identifier corresponding to the label.
7. The generating device according to claim 6, characterized in that An acquisition module, specifically used to perform keyword screening on the Perl script of the data warehouse, extract the content after the first preset keyword and before the second preset keyword to obtain a first data table identifier; By using the regular expression and the third preset keyword related to the association rule, the association rule table is obtained from the corresponding position in the Perl script.
8. The generating device according to claim 7, characterized in that The first processing module is specifically used to determine the query priority of each association rule table according to the keywords related to the association rule corresponding to the association rule table; The association rule tables are queried in sequence according to their query priorities, and the second data table identifier of the second data table associated with the first data table and the generation order of the second data table and the first data table are obtained from the association rule tables.
9. The generating device according to claim 6, characterized in that: The acquisition module is specifically used to extract keywords related to business functions by traversing the annotation part of the Perl script of each first data table according to the corpus and the stop words; the corpus is generated according to the business functions involved in the data warehouse; the stop words include at least one of the following: time, date, name, email address, and phone number; The priority of each extracted keyword is obtained according to the frequency of occurrence of the extracted keyword in the annotation part and the frequency of occurrence in the corpus, and the business function corresponding to the keyword with a priority greater than a preset priority threshold is determined as the business function involved in the first data table.
10. The generating device according to claim 6, characterized in that The knowledge graph generation module is specifically used to generate a mesh data graph in RDF data format according to the generation order of the second data table and the first data table, the first data table identifier, the second data table identifier, and the business functions involved in the first data table, wherein the first data table identifier and the second data table identifier are used as nodes in the mesh data graph, and the data table identifier is used as the corresponding node identifier; each node is connected according to the generation order of the second data table and the first data table, and the connecting line is used as the edge in the mesh data graph, and the business functions involved in the first data table are marked next to the node corresponding to the first data table.
11. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
