Data model table construction method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of data processing, discloses a data model table construction method and device, computer equipment and a storage medium, and can solve the problems that a currently constructed data model table cannot be accurately matched with business requirements and data importance is not prominent. And the efficiency of subsequent data processing based on the data model table is low. Comprising the following steps: determining a business theme domain corresponding to an original data table, and splitting the business theme domain of the original data table to obtain an intermediate data table corresponding to each business theme domain; for each business theme domain, constructing a graph structure according to the intermediate data table corresponding to the business theme domain, and obtaining a hub node table of the business theme domain according to the graph structure; and constructing a data model table according to a preset service requirement and the hub node table.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, computer equipment and storage medium for constructing a data model table. Background Art

[0002] In the era of big data, processing massive amounts of data is a very important step. For example, banks have a large amount of data (customers, assets, loans, time and amount of entry and exit, etc.), and insurance companies have a large amount of data (insurance product number, customers, term, transaction amount, etc.). Based on this, the use of data warehouse modeling to build data model tables came into being, but the data model tables built by traditional data warehouse modeling cannot accurately match business needs, which reduces the efficiency of subsequent data processing based on the data model tables. In addition, the data model tables built by traditional data warehouse modeling simply list the data, the importance of the data is not highlighted, and unimportant data is also processed together, which also reduces the efficiency of subsequent data processing based on the data model tables. Summary of the invention

[0003] In view of this, the present invention provides a method, device, computer equipment and storage medium for constructing a data model table, which can solve the technical problems that the currently constructed data model table cannot accurately match business needs and the importance of data is not highlighted, resulting in low efficiency of subsequent data processing based on the data model table.

[0004] According to a first aspect of the present invention, a method for constructing a data model table is provided, the method comprising:

[0005] Determine the business subject domain corresponding to the original data table, split the original data table into the business subject domains, and obtain an intermediate data table corresponding to each of the business subject domains;

[0006] For each of the business subject domains, a graph structure is constructed according to the intermediate data table corresponding to the business subject domain, and a hub node table of the business subject domain is obtained according to the graph structure;

[0007] A data model table is constructed according to preset business requirements and the hub node table.

[0008] According to a second aspect of the present invention, there is provided a device for constructing a data model table, the device comprising:

[0009] A determination module, used to determine the business subject domain corresponding to the original data table, split the original data table into the business subject domains, and obtain an intermediate data table corresponding to each of the business subject domains;

[0010] A first construction module is used to construct a graph structure for each of the business subject domains according to the intermediate data table corresponding to the business subject domain, and obtain a hub node table of the business subject domain according to the graph structure;

[0011] The second construction module is used to construct a data model table according to preset business requirements and the hub node table.

[0012] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for constructing the above-mentioned data model table when executing the computer program.

[0013] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the method for constructing the above-mentioned data model table are implemented.

[0014] Through the above technical scheme, the present invention provides a method, device, computer equipment and storage medium for constructing a data model table. First, the business subject domain corresponding to the original data table is determined, and the original data table is split into the business subject domains to obtain an intermediate data table corresponding to each of the business subject domains; then, for each of the business subject domains, a graph structure is constructed according to the intermediate data table corresponding to the business subject domain, and a hub node table of the business subject domain is obtained according to the graph structure; finally, a data model table is constructed according to preset business requirements and the hub node table. Through the technical solution of the present invention, the original data table is split based on the business subject domain, and the intermediate data table corresponding to each business subject domain is obtained. In this way, the intermediate data table that distinguishes the businesses is obtained from the original data table that does not distinguish between different businesses. Therefore, when constructing the data model table, the business subject domain is matched with the preset business requirements to obtain the intermediate data table that matches the preset business requirements. However, each business subject domain corresponds to multiple intermediate data tables. In order to further highlight the importance of the data, for each business subject domain, it is necessary to obtain a more important hub node table from the multiple intermediate data tables. When constructing the data model table, the hub node table that matches the preset business requirements and highlights the importance is used instead of the intermediate data table that only matches the preset business requirements, thereby improving the efficiency of subsequent data processing based on the data model table.

[0015] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation on the local application. In the drawings:

[0017] Figure 1 A schematic diagram showing a flow chart of a method for constructing a data model table provided in an embodiment of the present invention;

[0018] Figure 2 A schematic diagram showing a flow chart of another method for constructing a data model table provided in an embodiment of the present invention;

[0019] Figure 3 A schematic diagram of the structure of a device for constructing a data model table provided in an embodiment of the present invention is shown;

[0020] Figure 4 A schematic diagram showing the structure of another device for constructing a data model table provided in an embodiment of the present invention is shown;

[0021] Figure 5 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0022] Figure 6 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0024] This embodiment provides a method for constructing a data model table, such as Figure 1 As shown, the method includes:

[0025] 101. Determine the business subject domain corresponding to the original data table, split the original data table into the business subject domains, and obtain an intermediate data table corresponding to each of the business subject domains.

[0026] As an implementation method, the business subject domain can be pre-set. For example, the business subject domain can be set to customers, assets, contracts, etc. Each original data table includes at least one business subject domain. The original data tables belonging to the same business subject domain are split out to obtain multiple intermediate data tables. For example, the intermediate data table corresponding to the customer business subject domain includes: customer name, customer phone number, customer gender, customer ID number, etc.

[0027] An intermediate data table that distinguishes different businesses is obtained from the original data table that does not distinguish different businesses. Therefore, when constructing the data model table, the business subject domain is matched with the preset business requirements to obtain the intermediate data table that matches the preset business requirements.

[0028] 102. For each of the business subject domains, a graph structure is constructed according to the intermediate data table corresponding to the business subject domain, and a hub node table of the business subject domain is obtained according to the graph structure.

[0029] As an implementation method, since the graph structure represents all nodes and the relationships between all nodes, by taking each intermediate data table as a node in the graph structure, the relationships between all intermediate data tables in the same business subject domain can be constructed based on the graph structure, thereby obtaining the more important intermediate data table, namely the hub node table.

[0030] In order to further highlight the importance of data, for each business subject domain, it is necessary to obtain more important hub node tables from multiple intermediate data tables. When constructing the data model table, the hub node table that matches the preset business requirements and highlights the importance is used instead of the intermediate data table that only matches the preset business requirements, thereby improving the efficiency of subsequent data processing based on the data model table.

[0031] 103. Construct a data model table according to preset business requirements and the hub node table.

[0032] In a specific application scenario, for example, the preset business requirement is to analyze the customer value in the mobile phone customer base, then it is necessary to obtain data related to the customer value in the mobile phone customer base from massive data. Specifically, first determine the business subject domain related to the customer value in the mobile phone customer base from the business subject domain. For example, the business subject domain related to the customer value in the mobile phone customer base is: mobile phone, customer, purchase time, then, determine the hub node table as mobile phone model and mobile phone price in all intermediate data tables in the business subject domain of mobile phone, determine the hub node table as customer ID in all intermediate data tables in the business subject domain of customer, and determine the hub node table as purchase time interval in all intermediate data tables in the business subject domain of purchase time. Finally, the data corresponding to the mobile phone model, mobile phone price, customer ID and purchase time interval are listed as a data model table, so that the customer value in the mobile phone customer base can be analyzed based on the data model table later.

[0033] The present invention provides a method, device, computer equipment and storage medium for constructing a data model table. First, the business subject domain corresponding to the original data table is determined, and the original data table is split into the business subject domains to obtain an intermediate data table corresponding to each of the business subject domains; then, for each of the business subject domains, a graph structure is constructed according to the intermediate data table corresponding to the business subject domain, and a hub node table of the business subject domain is obtained according to the graph structure; finally, a data model table is constructed according to preset business requirements and the hub node table. Through the technical solution of the present invention, the original data table is split based on the business subject domain, and the intermediate data table corresponding to each business subject domain is obtained. In this way, the intermediate data table that distinguishes the businesses is obtained from the original data table that does not distinguish between different businesses. Therefore, when constructing the data model table, the business subject domain is matched with the preset business requirements to obtain the intermediate data table that matches the preset business requirements. However, each business subject domain corresponds to multiple intermediate data tables. In order to further highlight the importance of the data, for each business subject domain, it is necessary to obtain a more important hub node table from the multiple intermediate data tables. When constructing the data model table, the hub node table that matches the preset business requirements and highlights the importance is used instead of the intermediate data table that only matches the preset business requirements, thereby improving the efficiency of subsequent data processing based on the data model table.

[0034] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another method for constructing a data model table is provided, such as Figure 2 As shown, the method includes:

[0035] 201. Obtain at least one first business key field corresponding to each original data table, and calculate a first feature vector of the first business key field.

[0036] For this embodiment, there are multiple original data tables, each of which corresponds to at least one first business key field. Metadata such as table names, field names, and comments are extracted from all original data tables. After unified processing, the first business key field of each original data table and the first business key fields of all original data tables are obtained. For example, all original data tables include original data tables 1, 2, and 3, original data table 1 includes first business key fields 1, 2, and 3, original data table 2 includes first business key fields 2 and 3, and original data table 3 includes first business key fields 3 and 4. Then the first business key fields included in all original data tables are 1, 2, 3, and 4.

[0037] According to the data of each first business key field in the original data table where it is located, its first feature vector is calculated. For example, if the first business key field 1 is in the original data table 1, then the data of the first business key field 1 is obtained from the original data table 1, and its first feature vector is calculated. If the first business key field 2 is in the original data tables 1 and 2, then the data of the first business key field 2 is obtained from any one of the original data tables 1 and 2, and its first feature vector is calculated.

[0038] The first feature vector may be calculated using algorithms such as term frequency-inverse document frequency, which are not limited here.

[0039] 202. Classify the first feature vector using a clustering algorithm to obtain at least one business subject domain and a first business key field included in each of the business subject domains, and determine the business subject domain corresponding to the original data table according to the first business key field included in each of the business subject domains.

[0040] Among them, clustering algorithms such as DBSCAN are suitable for high-dimensional data. DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a density-based spatial clustering algorithm. The algorithm divides areas with sufficient density into clusters and finds clusters of arbitrary shapes in a spatial database with noise. It defines a cluster as the maximum set of density-connected points.

[0041] For this embodiment, DBSCAN is used to classify the first feature vector, that is, to classify the sample set with the maximum density connection, and at least one business theme domain is obtained, each business theme domain is a cluster, and each cluster includes at least one first business key field. Following the example of step 201 of the embodiment, the first business key fields 1, 2, 3, and 4 are classified as follows: the first business key fields 1 and 2 are business theme domain 1, and the first business key fields 3 and 4 are business theme domain 2. Correspondingly, the original data table 1 corresponds to business theme domains 1 and 2, the original data table 2 corresponds to business theme domains 1 and 2, and the original data table 3 corresponds to business theme domain 2.

[0042] 203. Split the original data table into the business subject domains to obtain an intermediate data table corresponding to each of the business subject domains.

[0043] For this embodiment, since the original data table is composed of the first business key field and the business subject domain includes at least one first business key field, the original data table is split into business subject domains, that is, the original data table is split based on the first business key field included in each business subject domain.

[0044] Continuing with the example of step 202 of the embodiment, that is, original data table 1 includes first business key fields 1, 2, and 3, original data table 2 includes first business key fields 2 and 3, original data table 3 includes first business key fields 3 and 4, and the first business key fields 1 and 2 are business subject domain 1, and the first business key fields 3 and 4 are business subject domain 2, then the first business key field 1 of original data table 1, the first business key field 2 of original data table 1, and the first business key field 2 of original data table 2 correspond to business subject domain 1, and the first business key field 3 of original data table 1, the first business key field 3 of original data table 2, the first business key field 3 of original data table 3, and the first business key field 4 of original data table 3 correspond to business subject domain 2.

[0045] The original data table is split into business subject domains, including: splitting the data corresponding to the first business key field 1 of the original data table 1 from the original data table 1 as an intermediate data table for the business subject domain 1, splitting the data corresponding to the first business key field 2 of the original data table 1 from the original data table 1 as an intermediate data table for the business subject domain 1, splitting the data corresponding to the first business key field 2 of the original data table 2 from the original data table 2 as an intermediate data table for the business subject domain 1, thereby obtaining three intermediate data tables for the business subject domain 1; similarly, four intermediate data tables for the business subject domain 2 can be obtained.

[0046] 204. For each of the business subject domains, the first business key field included in each of the business subject domains is used as the second business key field, and the second feature vector of the second business key field is calculated. The edge weights between different intermediate data tables are calculated based on the second feature vector of the intermediate data table, and each of the intermediate data tables is used as a node. A graph structure is constructed based on the edge weights and the nodes.

[0047] It should be noted that implementation steps 204-206 are performed for each business subject domain. For this implementation, since the business subject domain includes multiple intermediate data tables, in order to highlight the more important intermediate data tables, each intermediate data table is taken as a node, and the degree of association between different intermediate data tables is calculated by the second eigenvector of the intermediate data table, wherein the degree of association is measured by the edge weight. The larger the edge weight, the greater the degree of association. All intermediate data tables in each business subject domain can be constructed into a graph structure by nodes and edges, and thus connected, so as to obtain the more important intermediate data tables, i.e., the hub node table, in implementation steps 205 and 206.

[0048] Specifically, the original data table corresponds to the first business key field. In order to distinguish, the first business key field from the original data table corresponding to the intermediate data table is named the second business key field. Since the second business key field can find corresponding data in the original data table, the second feature vector can be calculated according to the data corresponding to each second business key field. For example, the three intermediate data tables of business subject domain 1 are:

[0049] Intermediate data table 1 (the first business key field 1 of the original data table 1, the first business key field 1 is named as the second business key field 1, wherein the second business key field 1 is the same as the first business key field 1, for example, both are customer names), intermediate data table 2 (the first business key field 2 of the original data table 1, the first business key field 2 is named as the second business key field 2, wherein the second business key field 2 is the same as the first business key field 2, for example, both are customer phone numbers), intermediate data table 3 (the first business key field 2 of the original data table 2, the first business key field 2 is named as the second business key field 2, wherein the second business key field 2 is the same as the first business key field 2, for example, both are customer phone numbers). If the second business key field 1 is in the original data table 1, then the data of the second business key field 1 is obtained from the original data table 1, and its second eigenvector is calculated. If the second business key field 2 is in the original data tables 1 and 2, then the data of the second business key field 2 is obtained from any one of the original data tables 1 and 2, and its second eigenvector is calculated.

[0050] 205. Obtain a preset centrality index, wherein the preset centrality index includes: at least one of degree centrality, betweenness centrality, closeness centrality, and eigenvector centrality.

[0051] 206. Determine the number of other nodes connected to each node according to the graph structure, calculate the value of the preset centrality index according to the number and / or the edge weight of the edge connected to each node in the graph structure, and determine the node when the value of the preset centrality index is optimal as the hub node table in the intermediate data table.

[0052] For steps 205 and 206 of the embodiment, in order to determine the hub node table in the intermediate data table based on the graph structure, some indicators are needed to characterize the importance of the intermediate data table. These indicators can be at least one of degree centrality, betweenness centrality, closeness centrality, and eigenvector centrality. For example, the number of other nodes connected to each node is determined according to the graph structure. If a certain node has the largest number of corresponding nodes, then the degree centrality of the node is the largest. When the degree centrality is used as the preset centrality indicator, the degree centrality is the largest and the representative value is the best. For another example, according to the edge weights of the edges connected to each node in the graph structure, the average value of the edge weight of each node is calculated. The node with a larger average value has a larger closeness centrality. When the closeness centrality is used as the preset centrality indicator, the closeness centrality is the largest and the representative value is the best.

[0053] 207. Determine at least one target business subject domain required by preset business requirements in the business subject domain, and combine the hub node tables corresponding to all the target business subject domains to obtain a data model table.

[0054] For this embodiment, after the preset business requirements are determined, at least one target business subject domain that matches the preset business requirements can be selected from a large number of business subject domains, so that the data model table obtained by combining the hub node tables corresponding to the target subject domains is an accurate match with the preset business requirements. For example, the target business subject domains include: customers, assets, and credits, wherein all the target business subject domains correspond to the third business key field, then the third business key field is customers, assets, and credits, and the third business key field carries the identifier of the original data table, so as to obtain the data corresponding to the data model table from the original data table according to the identifier, and delete the third business key field in the data model table that does not meet the preset business requirements according to the data corresponding to the data model table.

[0055] It should be noted that each business subject domain corresponds to a second business key field, and the target business subject domain is a set of at least one business subject domain. Then, the third business key field corresponding to the target business subject domain is a set of second business key fields corresponding to at least one business subject domain. For example, there are two business subject domains required by the preset business needs, which are the two business subject domains in step 202 of the above embodiment. Then these two business subject domains are the target business subject domain. Business subject domain 1 includes: second business key fields 1 and 2, and business subject domain 2 includes: second business key fields 3 and 4. Therefore, the target business subject domain includes third business key fields 1, 2, 3, and 4 (wherein, the second business key field 1 is the same as the third business key field 1, just to distinguish that the second business key field is corresponding to the business subject domain, and the third business key field is corresponding to the target business subject domain. Similarly, the second business key field 2 is the same as the third business key field 2, the second business key field 3 is the same as the third business key field 3, and the second business key field 4 is the same as the third business key field 4).

[0056] Among them, the first business key field in the original data table carries the identifier of the original data table, so that the data source of the first business key field can be clarified and the first eigenvector can be calculated. Since the second business key field comes from the first business key field, the second business key field also carries the identifier of the original data table, so that the data source of the second business key field can be clarified and the second eigenvector can be calculated. Similarly, since the third business key field comes from the second business key field, the third business key field carries the identifier of the original data table. In this way, the data model table can directly obtain data from the original data table, and these data match the preset business requirements.

[0057] Although the data model table matches the preset business requirements, and the data model table is a combination of hub node tables, which has reduced the amount of data, there is still a huge amount of data, and some data does not match the preset business requirements. The data model table needs to be further processed to delete the third business key field in the data model table that does not meet the preset business requirements. The specific processing process is step 208 of the implementation example, using the principal component analysis method, wherein the basic principle of the principal component analysis method is to try to recombine the original variables into a new set of several independent comprehensive indicators to replace the original indicators, that is, a statistical method of dimensionality reduction, which converts multiple indicators into a few comprehensive indicators. In the process of dimensionality reduction, the third business key field that does not match the preset business requirements is deleted, so that a data model table with a higher degree of matching with the preset business requirements can be obtained.

[0058] 208. Obtain the third business key field, calculate the third eigenvector of the third business key field, construct a sample matrix of the data model table according to the third eigenvector, determine at least one principal component of the data model table according to the sample matrix, and evaluate the data model table using the principal component.

[0059] For this embodiment, determining at least one principal component of the data model table based on the sample matrix includes: standardizing the sample matrix to obtain a standardized matrix, calculating the covariance matrix of the standardized matrix, and calculating the eigenvalues ​​and eigenvectors of the covariance matrix; sorting the eigenvalues ​​from large to small to obtain a preset number of target eigenvalues; using the eigenvector corresponding to each of the target eigenvalues ​​as a coefficient of the third business key field to obtain at least one principal component of the data model table.

[0060] Specifically, if there are n third business key fields and the third eigenvector of each third business key field has m values, standardizing the sample matrix means calculating the sample mean and the sample standard deviation for each value of the third eigenvector of each third business key field, then calculating the value minus the corresponding sample mean to obtain the difference, calculating the difference and dividing it by the corresponding sample standard deviation to obtain the standardized value corresponding to the value, which is used as the value in the standardized matrix corresponding to the value, thereby obtaining the standardized matrix.

[0061] The eigenvalues ​​are recorded in descending order as λ1, λ2, ...λ m , the corresponding eigenvectors are denoted as u1, u2...u m , where u j =(u 1j 、u 2j ...u nj ) T , if the first p eigenvalues ​​are selected, the eigenvectors corresponding to the first p eigenvalues ​​can form p principal components.

[0062] y1=u 11 x1+u 21 x2+...+u n1 x n

[0063] y2=u 12 x1+u 22 x2+...+u n2 x n ...

[0065] y p =u 1p x1+u 2p x2+...+unp x n

[0066] Among them, y1 is the first principal component, y2 is the second principal component, ...y p is the pth principal component, x1, x2...x n is the vector in the standardized matrix, x1 is the vector in the standardized matrix corresponding to u1, x2 is the vector in the standardized matrix corresponding to u2, ...x n is u n The corresponding vector in the normalized matrix.

[0067] As an implementation method for evaluating the data model table using the principal component, the data model table is evaluated according to the coefficient of the third business key field, and the third business key field is customer, asset, and credit. y1 = 0.6x1 + 0.4x2 + ... + 0.3x n For example, weights 0.6, 0.4, and 0.3 reflect the contribution of each third business key field to the first principal component. The larger the weight, the greater the influence of the third business key field on the principal component. Therefore, a weight threshold can be set. If the weight corresponding to the third business key field is less than the weight threshold in the principal components greater than or equal to a preset number, it means that the third business key field does not match the preset business requirements. The third business key field and the data corresponding to the third business key field can be deleted, thereby pruning the data model table. The pruned data model table has a high degree of match with the preset business requirements, thereby improving the efficiency of subsequent data processing.

[0068] Through the technical solution of the present invention, the original data table is split based on the business subject domain, and the intermediate data table corresponding to each business subject domain is obtained. In this way, the intermediate data table that distinguishes the business is obtained from the original data table that does not distinguish different businesses. Therefore, when constructing the data model table, the business subject domain is matched with the preset business requirements to obtain the intermediate data table that matches the preset business requirements. However, each business subject domain corresponds to multiple intermediate data tables. In order to further highlight the importance of data, for each business subject domain, it is necessary to obtain a more important hub node table from multiple intermediate data tables. Therefore, when constructing the data model table, a hub node table that matches the preset business requirements and has prominent importance is used instead of an intermediate data table that only matches the preset business requirements. The efficiency of subsequent data processing can be improved. The data model table is evaluated using principal components, so that the data model table with a low degree of matching the preset business requirements is deleted, so that the deleted data model table has a high degree of matching with the preset business requirements, which can further improve the efficiency of subsequent data processing.

[0069] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0070] Further, as Figure 1 and Figure 2 As a specific implementation of the method shown in the figure, an embodiment of the present invention provides a device for constructing a data model table, and the device for constructing a data model table corresponds one-to-one to the method for constructing a data model table in the above embodiment. Figure 3 As shown, the device includes: a determination module 31, a first construction module 32, and a second construction module 33;

[0071] A determination module 31 is used to determine the business subject domain corresponding to the original data table, split the original data table into the business subject domains, and obtain an intermediate data table corresponding to each of the business subject domains;

[0072] A first construction module 32 is used to construct a graph structure for each of the business subject domains according to the intermediate data table corresponding to the business subject domain, and obtain a hub node table of the business subject domain according to the graph structure;

[0073] The second construction module 33 is used to construct a data model table according to preset business requirements and the hub node table.

[0074] In one embodiment, the determination module 31 is specifically configured to:

[0075] Obtain at least one first business key field corresponding to each original data table;

[0076] Calculating a first characteristic vector of the first business key field;

[0077] The first feature vector is classified by using a clustering algorithm to obtain at least one business subject domain and a first business key field included in each of the business subject domains, and the business subject domain corresponding to the original data table is determined according to the first business key field included in each of the business subject domains.

[0078] In one embodiment, the first building module 32 is specifically configured to:

[0079] Using the first business key field included in each of the business subject domains as the second business key field;

[0080] Calculating a second eigenvector of the second business key field, and calculating edge weights between different intermediate data tables according to the second eigenvector of the intermediate data table;

[0081] Each of the intermediate data tables is regarded as a node, and a graph structure is constructed according to the edge weights and the nodes.

[0082] In one embodiment, the first building module 32 is specifically configured to:

[0083] Obtaining a preset centrality index, wherein the preset centrality index includes: at least one of degree centrality, betweenness centrality, closeness centrality, and eigenvector centrality;

[0084] Determine the number of other nodes connected to each of the nodes according to the graph structure;

[0085] The value of the preset centrality index is calculated according to the number and / or the edge weight of the edge connected to each node in the graph structure, and the node when the value of the preset centrality index is optimal is determined as the hub node table in the intermediate data table.

[0086] In one embodiment, the second building module 33 is specifically configured to:

[0087] Determine at least one target business subject domain required by preset business requirements in the business subject domain;

[0088] The hub node tables corresponding to all the target business subject domains are combined to obtain a data model table, wherein all the target business subject domains correspond to a third business key field, and the third business key field carries an identifier of the original data table, so as to obtain data corresponding to the data model table from the original data table according to the identifier, and delete the third business key field in the data model table that does not meet the preset business requirements according to the data corresponding to the data model table.

[0089] In specific application scenarios, such as Figure 4 As shown, a device for constructing a data model table, the device also includes: an evaluation module 34, which can be specifically used to:

[0090] Acquire the third business key field, and calculate a third characteristic vector of the third business key field;

[0091] Constructing a sample matrix of the data model table according to the third eigenvector;

[0092] At least one principal component of the data model table is determined according to the sample matrix, and the data model table is evaluated using the principal component.

[0093] In one embodiment, the evaluation module 34 is specifically configured to:

[0094] Standardizing the sample matrix to obtain a standardized matrix, calculating the covariance matrix of the standardized matrix, and calculating the eigenvalues ​​and eigenvectors of the covariance matrix;

[0095] Sort the eigenvalues ​​from large to small to obtain a preset number of target eigenvalues;

[0096] The eigenvector corresponding to each of the target eigenvalues ​​is used as a coefficient of the third business key field to obtain at least one principal component of the data model table.

[0097] The present invention provides a device for constructing a data model table, which splits an original data table based on a business subject domain to obtain an intermediate data table corresponding to each business subject domain. In this way, an intermediate data table that distinguishes between businesses is obtained from an original data table that does not distinguish between different businesses. Therefore, when constructing a data model table, the business subject domain is matched with a preset business requirement to obtain an intermediate data table that matches the preset business requirement. However, each business subject domain corresponds to multiple intermediate data tables. In order to further highlight the importance of data, for each business subject domain, a more important hub node table needs to be obtained from multiple intermediate data tables. When constructing a data model table, a hub node table that matches the preset business requirement and highlights its importance is used instead of an intermediate data table that only matches the preset business requirement, thereby improving the efficiency of subsequent data processing based on the data model table.

[0098] For the specific limitations on the construction device of the data model table, please refer to the limitations on the construction method of the data model table above, which will not be repeated here. Each module in the above-mentioned construction device of the data model table can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0099] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a method for constructing a data model table.

[0100] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a method for constructing a data model table.

[0101] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0102] Determine the business subject domain corresponding to the original data table, split the original data table into the business subject domains, and obtain an intermediate data table corresponding to each of the business subject domains;

[0103] For each of the business subject domains, a graph structure is constructed according to the intermediate data table corresponding to the business subject domain, and a hub node table of the business subject domain is obtained according to the graph structure;

[0104] A data model table is constructed according to preset business requirements and the hub node table.

[0105] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0106] Determine the business subject domain corresponding to the original data table, split the original data table into the business subject domains, and obtain an intermediate data table corresponding to each of the business subject domains;

[0107] For each of the business subject domains, a graph structure is constructed according to the intermediate data table corresponding to the business subject domain, and a hub node table of the business subject domain is obtained according to the graph structure;

[0108] A data model table is constructed according to preset business requirements and the hub node table.

[0109] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0110] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0111] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0112] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for constructing a data model table, characterized in that: The method comprises: Determine the business subject domain corresponding to the original data table, split the original data table into the business subject domains, and obtain an intermediate data table corresponding to each of the business subject domains; For each of the business subject domains, a graph structure is constructed according to the intermediate data table corresponding to the business subject domain, and a hub node table of the business subject domain is obtained according to the graph structure; A data model table is constructed according to preset business requirements and the hub node table.

2. The method according to claim 1, characterized in that Determining the business subject domain corresponding to the original data table includes: Obtain at least one first business key field corresponding to each original data table; Calculating a first characteristic vector of the first business key field; The first feature vector is classified by using a clustering algorithm to obtain at least one business subject domain and a first business key field included in each of the business subject domains, and the business subject domain corresponding to the original data table is determined according to the first business key field included in each of the business subject domains.

3. The method according to claim 2, characterized in that The constructing of the graph structure according to the intermediate data table corresponding to the business subject domain includes: Using the first business key field included in each of the business subject domains as the second business key field; Calculating a second eigenvector of the second business key field, and calculating edge weights between different intermediate data tables according to the second eigenvector of the intermediate data table; Each of the intermediate data tables is regarded as a node, and a graph structure is constructed according to the edge weights and the nodes.

4. The method according to claim 3, characterized in that The step of obtaining the hub node table of the business subject domain according to the graph structure includes: Obtaining a preset centrality index, wherein the preset centrality index includes: at least one of degree centrality, betweenness centrality, closeness centrality, and eigenvector centrality; Determine the number of other nodes connected to each of the nodes according to the graph structure; The value of the preset centrality index is calculated according to the number and / or the edge weight of the edge connected to each node in the graph structure, and the node when the value of the preset centrality index is optimal is determined as the hub node table in the intermediate data table.

5. The method according to claim 1, characterized in that The step of constructing a data model table according to preset business requirements and the hub node table includes: Determine at least one target business subject domain required by preset business requirements in the business subject domain; The hub node tables corresponding to all the target business subject domains are combined to obtain a data model table, wherein all the target business subject domains correspond to a third business key field, and the third business key field carries an identifier of the original data table, so as to obtain data corresponding to the data model table from the original data table according to the identifier, and delete the third business key field in the data model table that does not meet the preset business requirements according to the data corresponding to the data model table.

6. The method according to claim 5, characterized in that After combining the hub node tables corresponding to all the target business subject domains to obtain a data model table, the method further includes: Acquire the third business key field, and calculate a third characteristic vector of the third business key field; Constructing a sample matrix of the data model table according to the third eigenvector; At least one principal component of the data model table is determined according to the sample matrix, and the data model table is evaluated using the principal component.

7. The method according to claim 6, characterized in that Determining at least one principal component of the data model table according to the sample matrix includes: Standardizing the sample matrix to obtain a standardized matrix, calculating the covariance matrix of the standardized matrix, and calculating the eigenvalues ​​and eigenvectors of the covariance matrix; Sort the eigenvalues ​​from large to small to obtain a preset number of target eigenvalues; The eigenvector corresponding to each of the target eigenvalues ​​is used as a coefficient of the third business key field to obtain at least one principal component of the data model table.

8. A device for constructing a data model table, characterized in that: The device comprises: A determination module, used to determine the business subject domain corresponding to the original data table, split the original data table into the business subject domains, and obtain an intermediate data table corresponding to each of the business subject domains; A first construction module is used to construct a graph structure for each of the business subject domains according to the intermediate data table corresponding to the business subject domain, and obtain a hub node table of the business subject domain according to the graph structure; The second construction module is used to construct a data model table according to preset business requirements and the hub node table.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the processor implements the steps of the method for constructing the data model table as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for constructing a data model table as described in any one of claims 1 to 7 are implemented.

Images