Construction method and device of distributed storage system, electronic equipment and storage medium

Abstract

The application provides a construction method and device of a distributed storage system, electronic equipment and a storage medium. The method is applied to the technical field of financial technology. The historical operation log of user access to a database within a preset time is obtained. According to the historical operation log, a data table associated undirected graph, a user role label and a data table set corresponding to the user role label are determined. The nodes of the data table associated undirected graph are data tables. The associated paths between the nodes are the association degrees between the data tables. The data table associated undirected graph is adjusted. Each data table in the adjusted data table associated undirected graph is clustered to form at least one data table cluster set. For each data table cluster set, the data table cluster set is combined according to the access frequency of the data table cluster set to determine a load-balanced distributed storage system. Thus, a distributed storage system with better load balancing performance is constructed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to a construction method and device of a distributed storage system, an electronic device and a storage medium. BACKGROUND

[0002] With the continuous enrichment of enterprise business, the operation level of enterprise employees, customers and the like accessing enterprise databases in different ways (such as mobile terminals and web terminals) is in a trend of increasing year by year. The increase in the level of data queries, additions, deletions and modifications of the enterprise database system every day, as well as the accumulation of different types of data such as characters, numbers, pictures and videos in the bank database, makes the concurrent efficiency, data consistency and database disk stability of the database face great pressure, thereby further affecting the access efficiency and access experience of users to the enterprise.

[0003] In the prior art, there are various load balancing algorithms based on distributed storage, such as round-robin, random, least connection and the like.

[0004] However, the above methods have universality and cannot accurately adapt to the data access characteristics of the bank system, such as obvious distinction of different business data and high concentration of access operations of different user types. Therefore, how to construct a distributed storage system with better load balancing performance has become a problem to be solved. SUMMARY

[0005] The present application provides a construction method and device of a distributed storage system, an electronic device and a storage medium to solve the problem that the prior art cannot construct a distributed storage system with better load balancing performance.

[0006] In a first aspect, the present application provides a construction method of a distributed storage system, comprising:

[0007] obtaining a historical operation log of user access to a database within a preset time;

[0008] determining a data table associated undirected graph, a user role label and a data table set corresponding to the user role label according to the historical operation log, a node of the data table associated undirected graph being a data table, and an associated path between nodes being an association degree between the data tables;

[0009] adjusting a data table association degree of the data table associated undirected graph according to the data table set to obtain an adjusted data table associated undirected graph;

[0010] clustering each data table in the adjusted data table associated undirected graph to form at least one data table clustering set;

[0011] combining the data table clustering sets according to access frequencies of the data table clustering sets, to generate at least one data table clustering cluster;

[0012] constructing at least one storage node according to the at least one data table clustering cluster, to generate a load-balanced distributed storage system.

[0013] In a possible design of the first aspect, the determining, according to the historical operation log, of a data table association undirected graph comprises the following steps.

[0014] obtaining data table names involved in each operation statement on the database in the historical operation log;

[0015] for each operation statement, when the number of data table names contained in the operation statement is greater than 1, increasing the association degree between each data table contained in the operation statement by 1;

[0016] when the number of data table names contained in the operation statement is 1, not updating the association degree between data tables;

[0017] constructing the data table association undirected graph based on the association degrees between each data table.

[0018] Optionally, the constructing, based on the association degrees between each data table, of the data table association undirected graph comprises the following steps.

[0019] normalizing the association degrees between each data table to obtain association degree values between each data table;

[0020] constructing the data table association undirected graph by taking each data table as a node in the data table association undirected graph and taking the association degree values between each data table as weights of association paths between data tables.

[0021] In another possible design of the first aspect, the determining, according to the historical operation log, of a user role label and a data table set corresponding to the user role label comprises the following steps.

[0022] obtaining data table names and user IDs involved in each operation statement on the database in the historical operation log;

[0023] obtaining a user role label according to the user ID, and obtaining a data table set corresponding to the user role label according to the user role label corresponding to the user ID and the data table names and the user IDs involved in each operation statement on the database.

[0024] Optionally, the obtaining of the user role label according to the user ID and the obtaining of the data table set corresponding to the user role label according to the data table name involved in each operation statement on the database and the user role label corresponding to the user ID include:

[0025] The user role label corresponding to the user ID is obtained, and the correspondence between the user ID and the user role label is obtained according to the information filled in by the user during registration;

[0026] All operation statements corresponding to the user ID are extracted from the historical operation log, and the data table name involved in all operation statements is extracted to form a data table set corresponding to the user ID;

[0027] The correlation degree of each user ID belonging to the same user role label is calculated according to the data table set corresponding to the user ID;

[0028] The clustering analysis of a plurality of user IDs belonging to the same user role label is performed according to the correlation degree between the user IDs belonging to the same user role label, to form at least one clustering set of at least one user ID belonging to the same user role label;

[0029] The secondary user role label is assigned to each clustering set based on the at least one clustering set of at least one user ID belonging to the same user role label;

[0030] The data table set corresponding to each secondary user role label is obtained, and the data table involved in the data table set corresponding to each secondary user role label is taken as the data table set corresponding to the secondary user role label;

[0031] Each secondary user role label is taken as a user role label, and the data table set corresponding to the secondary user role label is taken as the data table set corresponding to the user role label.

[0032] Optionally, the correlation degree calculation of each user ID belonging to the same user role label according to the data table set corresponding to the user ID includes:

[0033] The data table set corresponding to each user ID is obtained for each two user IDs;

[0034] The coincidence degree of the data tables in the two data table sets corresponding to the two user IDs is calculated, and the coincidence degree is taken as the correlation degree of the two user IDs;

[0035] The calculation formula of the coincidence degree r is:

[0036] r = x / 【0.5 * (s1 + s2)】

[0037] x is the number of same data tables in two data table sets, and s1 and s2 are the numbers of data tables in the two data table sets respectively.

[0038] In a further possible design of the first aspect, the data table association degree adjustment on the data table association undirected graph according to the data table set comprises:

[0039] For each user role label, all data table names in the data table set corresponding to the user role label are obtained;

[0040] An average value of all association path weights in the data table association undirected graph is obtained.

[0041] In the data table association undirected graph, nodes corresponding to all data table names in the data table set corresponding to the user role label are found, and association path weights between the nodes are adjusted to obtain an adjusted data table association undirected graph.

[0042] Optionally, the finding, in the data table association undirected graph, of nodes corresponding to all data table names in the data table set corresponding to the user role label and the adjustment of association path weights between the nodes comprise:

[0043] A data table combination formed by each two data tables in the data table set corresponding to the user role label is obtained.

[0044] For a first data table and a second data table in each data table combination, when there is an association path between the first data table and the second data table in the data table association undirected graph, the association path weight is doubled.

[0045] When there is no association path between the first data table and the second data table in the data table association undirected graph, an association path between the first data table and the second data table is established, and the association path weight between the first data table and the second data table is set as an average value of all association path weights.

[0046] In a further possible design of the first aspect, the clustering of each data table in the adjusted data table association undirected graph to form at least one data table cluster set comprises:

[0047] According to data table nodes in the adjusted data table association undirected graph and path weights between the data table nodes, all data tables in the adjusted data table association undirected graph are clustered to form the at least one data table cluster set.

[0048] In a first possible implementation of the first aspect, the combining, for each data table cluster set, the data table cluster set according to the access frequency of the data table cluster set, to generate at least one data table cluster, comprises:

[0049] obtaining a data table name involved in each operation statement of the database in the historical operation log, and counting the access times of each data table according to all operation statements in the historical operation log;

[0050] for each data table cluster set, adding the access times of at least one data table in the data table cluster set to obtain an addition result as the access frequency of each data table cluster set;

[0051] sorting all data table cluster sets according to the access frequency from high to low according to the access frequency of each data table cluster set;

[0052] if the number of all data table cluster sets is even, combining all data table cluster sets in a manner of combining the highest access frequency with the lowest access frequency and combining the second highest access frequency with the second lowest access frequency to form the at least one data table cluster;

[0053] if the number of all data table cluster sets is odd, separately taking the data table cluster set with the highest access frequency as a data table cluster, and combining the remaining data table cluster sets in a manner of combining the highest access frequency with the lowest access frequency and combining the second highest access frequency with the second lowest access frequency to form the at least one data table cluster.

[0054] In a second aspect, an embodiment of the present application provides a construction device of a distributed storage system, comprising:

[0055] an obtaining module configured to obtain a historical operation log of a user accessing a database within a preset time;

[0056] a determining module configured to determine, according to the historical operation log, a data table associated undirected graph, a user role label and a data table set corresponding to the user role label, wherein a node of the data table associated undirected graph is a data table, and an associated path between nodes is an association degree between the data tables;

[0057] an adjusting module configured to adjust a data table association degree of the data table associated undirected graph according to the data table set to obtain an adjusted data table associated undirected graph;

[0058] a clustering module configured to cluster each data table in the adjusted data table associated undirected graph to form at least one data table cluster set;

[0059] a combination module, configured to combine, for each data table cluster, the data table cluster according to a frequency of access of the data table cluster, to generate at least one data table cluster;

[0060] a generation module, configured to construct at least one storage node according to the at least one data table cluster, to generate a load-balanced distributed storage system.

[0061] In a possible design of the second aspect, the determination module is configured to determine a data table associated undirected graph according to the historical operation log, and specifically configured to:

[0062] obtain a data table name involved in each operation statement on the database in the historical operation log;

[0063] for each operation statement, when the data table name contained in the operation statement is greater than 1, increase an association degree between data tables contained in the operation statement by 1;

[0064] when the data table name contained in the operation statement is 1, do not update the association degree between data tables;

[0065] construct the data table associated undirected graph based on the association degrees between data tables.

[0066] Optionally, the determination module is configured to construct the data table associated undirected graph based on the association degrees between data tables, and specifically configured to:

[0067] normalize the association degrees between data tables to obtain association degree values between data tables;

[0068] construct the data table associated undirected graph by taking each data table as a node in the data table associated undirected graph and taking the association degree values between data tables as weights of associated paths between data tables.

[0069] In another possible design of the second aspect, the determination module is configured to determine a user role label and a data table set corresponding to the user role label according to the historical operation log, and specifically configured to:

[0070] obtain a data table name involved in each operation statement on the database in the historical operation log, and a user ID;

[0071] obtain the user role label according to the user ID, and obtain the data table set corresponding to the user role label according to the data table name involved in each operation statement on the database and the user role label corresponding to the user ID.

[0072] Optionally, the determining module acquires a user role label according to the user ID, and acquires a data table set corresponding to the user role label according to a data table name involved in each operation statement on the database and the user role label corresponding to the user ID, and is specifically configured to:

[0073] acquire the user role label corresponding to the user ID, and the correspondence between the user ID and the user role label is obtained according to information filled in by a user during registration;

[0074] extract all operation statements corresponding to the user ID from the historical operation log, and extract data table names involved in all operation statements to form a data table set corresponding to the user ID;

[0075] perform correlation degree calculation on each user ID belonging to the same user role label according to the data table set corresponding to the user ID;

[0076] perform clustering analysis on a plurality of user IDs belonging to the same user role label according to the correlation degree between each user ID belonging to the same user role label, to form at least one clustering set of at least one user ID under the same user role label;

[0077] assign a secondary user role label to each clustering set based on at least one clustering set of at least one user ID under the same user role label;

[0078] acquire a data table set corresponding to all user IDs in each secondary user role label, and take data tables involved in the data table set corresponding to all user IDs in the secondary user role label as a data table set corresponding to the secondary user role label;

[0079] take each secondary user role label as a user role label, and take a data table set corresponding to the secondary user role label as a data table set corresponding to the user role label.

[0080] Optionally, the determining module performs correlation degree calculation on each user ID belonging to the same user role label according to the data table set corresponding to the user ID, and is specifically configured to:

[0081] for each two user IDs, acquire a data table set corresponding to each user ID;

[0082] calculate coincidence degrees of data tables in two data table sets corresponding to the two user IDs, and take the coincidence degrees as correlation degrees of the two user IDs;

[0083] wherein, the calculation formula of the coincidence degree r is:

[0084] r = x / 【0.5 * (s1 + s2)】

[0085] x is the number of same data tables in two data table sets, and s1 and s2 are the numbers of data tables in the two data table sets respectively.

[0086] In a further possible design of the second aspect, the adjusting module is specifically configured to:

[0087] For each user role label, obtain all data table names in the data table set corresponding to the user role label;

[0088] Obtain the average value of all association path weights in the data table association undirected graph;

[0089] In the data table association undirected graph, find nodes corresponding to all data table names in the data table set corresponding to the user role label, and adjust association path weights between the nodes to obtain an adjusted data table association undirected graph.

[0090] Optionally, the adjusting module finds nodes corresponding to all data table names in the data table set corresponding to the user role label in the data table association undirected graph, and adjusts association path weights between the nodes, and is specifically configured to:

[0091] Obtain data table combinations each composed of two data tables in the data table set corresponding to the user role label;

[0092] For a first data table and a second data table in each data table combination, when there is an association path between the first data table and the second data table in the data table association undirected graph, double the association path weight;

[0093] When there is no association path between the first data table and the second data table in the data table association undirected graph, establish an association path between the first data table and the second data table, and set the association path weight between the first data table and the second data table as the average value of all association path weights.

[0094] In a further possible design of the second aspect, the clustering module is specifically configured to:

[0095] According to data table nodes and path weights between data table nodes in the adjusted data table association undirected graph, cluster all data tables in the adjusted data table association undirected graph to form the at least one data table cluster set.

[0096] In a further possible design of the second aspect, the generating module is specifically configured to:

[0097] obtain a data table name involved in each operation statement of the database in the historical operation log, and count an access frequency of each data table according to all operation statements in the historical operation log;

[0098] add at least one data table access frequency in each data table clustering set to obtain an addition result as an access frequency of each data table clustering set;

[0099] sort all data table clustering sets according to the access frequencies from high to low according to the access frequencies of the data table clustering sets;

[0100] if the number of all data table clustering sets is even, combine all data table clustering sets in a manner of combining the highest access frequency with the lowest access frequency and combining the second highest access frequency with the second lowest access frequency to form the at least one data table clustering cluster;

[0101] if the number of all data table clustering sets is odd, separately take the data table clustering set with the highest access frequency as a data table clustering cluster, and combine the remaining data table clustering sets in a manner of combining the highest access frequency with the lowest access frequency and combining the second highest access frequency with the second lowest access frequency to form the at least one data table clustering cluster.

[0102] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, and a memory and a transceiver connected to the processor in communication;

[0103] the memory stores computer execution instructions; and the transceiver is configured to receive and transmit data.

[0104] the processor executes the computer execution instructions stored in the memory to implement the method in the first aspect or any of the manners.

[0105] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the method in the first aspect or any of the manners.

[0106] In a fifth aspect, the present application provides a computer program product, which is executed by a processor to implement the method in the first aspect or any of the manners.

[0107] The application provides a distributed storage system construction method and device, electronic equipment and a storage medium. The method is applied to the field of financial science and technology, historical operation logs of user access to a database within a preset time are acquired, a data table associated undirected graph, a user role label and a data table set corresponding to the user role label are determined according to the historical operation logs, a node of the data table associated undirected graph is a data table, and an associated path between the nodes is an association degree between data tables, the data table associated undirected graph is adjusted according to the data table set, an adjusted data table associated undirected graph is obtained, each data table in the adjusted data table associated undirected graph is clustered, at least one data table cluster set is formed, for each data table cluster set, the data table cluster set is combined according to an access frequency of the data table cluster set, at least one data table cluster is generated, at least one storage node is constructed according to the at least one data table cluster, and a load-balanced distributed storage system is generated. In the technical solution, the association between the user label and the data table is considered, and a distributed storage system with better load balancing performance is constructed. BRIEF DESCRIPTION OF DRAWINGS

[0108] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0109] Figure 1 A flowchart of a distributed storage system construction method provided by an embodiment of the application Figure 1 ;

[0110] Figure 2 A flowchart of a distributed storage system construction method provided by an embodiment of the application Figure 2 ;

[0111] Figure 3 A flowchart of a distributed storage system construction method provided by an embodiment of the application Figure 3 ;

[0112] Figure 4 A flowchart of a distributed storage system construction method provided by an embodiment of the application Figure 4 ;

[0113] Figure 5 A flowchart of a distributed storage system construction method provided by an embodiment of the application Figure 5 ;

[0114] Figure 6 A structural schematic diagram of a distributed storage system construction device embodiment provided by an embodiment of the application

[0115] Figure 7 A structural schematic diagram of an electronic equipment provided by an embodiment of the application.

[0116] The specific embodiments of the present disclosure have been shown by the above drawings, and will be described in more detail hereinafter. The drawings and the written description are not intended to restrict the scope of the present disclosure in any way, but to illustrate the concept of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0117] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative work are within the scope of protection of the present application.

[0118] Before introducing the embodiments of the present application, the application background of the embodiments of the present application is first explained:

[0119] With the continuous enrichment of enterprise business, enterprise employees, customers and the like access enterprise databases in different ways (such as mobile terminal, web terminal), and the operation order of magnitude is in the trend of increasing year by year. The increase of data query, addition, deletion, modification and the like of the enterprise database system every day, and the accumulation of different types of data such as characters, numbers, pictures and videos in the bank database, make the concurrent efficiency, data consistency and database disk stability of the database face great pressure, thereby further affecting the access efficiency and access experience of users to the enterprise.

[0120] In the prior art, there are various load balancing algorithms based on distributed storage, such as round-robin method, random method, minimum connection method and the like.

[0121] However, the above methods have universality and cannot accurately adapt to the data access characteristics of the bank system, such as obvious distinction of various business data, high concentration of access operations of different user types and the like.

[0122] The existing problems of the prior art to be solved by the embodiments of the present application are: how to construct a distributed storage system with better load balancing performance, so that the user access to the database is more efficient, which becomes a problem to be solved urgently.

[0123] In view of the technical problems in the prior art, the inventor's idea is as follows: starting from the historical operation log of user access to the database, the correlation degree between the data tables is constructed, then the data tables are clustered, and then the access frequency of the data table clustering set is used to construct a load-balanced distributed storage system. The data tables in the data table clustering set have correlation in two aspects of access operation and business, which can improve the efficiency of subsequent data access, so that the distributed storage system improves the load balancing performance.

[0124] In the technical solution of the present application, the collection, storage, use, processing, transmission, provision and disclosure of financial data or user data and other information comply with relevant laws and regulations and do not violate public order and good customs.

[0125] The technical solution of the present application will be described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0126] It is worth noting that the application field of the distributed storage system construction method, device, electronic equipment and storage medium of the present disclosure is not limited, which can be financial technology, Internet, etc.

[0127] Among them, the execution subject of the present application is an electronic device, which can be a server, a terminal device, etc.

[0128] Figure 1 The flowchart of the distributed storage system construction method provided by the embodiment of the present application Figure 1 As shown in Figure 1 The method can include the following steps:

[0129] Step 11, obtaining the historical operation log of user access to the database within a preset time.

[0130] In this step, the operation log of multiple users accessing the data table in the database, i.e. the historical operation log, is obtained.

[0131] In order to efficiently and accurately construct the distributed storage system, a period of historical operation log can be used as the data basis, and the historical operation log corresponding to the preset time.

[0132] Optionally, the preset time can be a period of time before the current time, for example, within 1 month, within 15 days, or within 2 months.

[0133] Step 12, determining the data table correlation undirected graph, user role label and data table set corresponding to the user role label according to the historical operation log.

[0134] The nodes of the data table association undirected graph are data tables, and the association paths between the nodes are association degrees between the data tables.

[0135] In this step, the historical operation log records operation logs of multiple users accessing data tables in the database, and therefore, the association between the data tables can be determined according to the historical operation log, that is, the data table association undirected graph can be used to represent the association.

[0136] Further, the user can belong to a certain user role label, and therefore, at least one data table (that is, a data table set) corresponding to the operation of the user role label of different users in the historical operation log can be determined according to the historical operation log, that is, the user role label and the data table set corresponding to the user role label are determined.

[0137] Optionally, the specific implementation of this step can be: obtaining the data table names involved in each operation statement in the historical operation log, and for each operation statement, when the data table names contained in the operation statement are greater than 1, the association degrees between the data tables contained in the operation statement are increased by 1, and when the data table names contained in the operation statement are 1, the association degrees between the data tables are not updated, and finally, the data table association undirected graph is constructed based on the association degrees between the data tables.

[0138] Step 13: adjusting the data table association degrees of the data table association undirected graph according to the data table set, to obtain an adjusted data table association undirected graph.

[0139] In this step, the data table association undirected graph is adjusted by the data table set corresponding to the user role label, so that the association degrees of the data table access can be obtained according to the actual operation statements of the user, that is, the association degrees of the data table business can be obtained according to the user portrait (that is, the business association of different data tables is reflected in the access concentration of different users), so that the construction of the data table association undirected graph is more accurate.

[0140] Optionally, the specific implementation of this step can be: for each user role label, obtaining all data table names in the data table set corresponding to the user role label, obtaining the average value of all association path weights in the data table association undirected graph, searching for nodes corresponding to all data table names in the data table set corresponding to the user role label in the data table association undirected graph, and adjusting the association path weights between the nodes, to obtain an adjusted data table association undirected graph.

[0141] Step 14: clustering the data tables in the adjusted data table association undirected graph to form at least one data table cluster set.

[0142] In this step, clustering analysis is performed on each data table in the adjusted data table association undirected graph to obtain at least one data table clustering set, at this time, the data tables in the data table clustering set have association in both access operation and business levels, and the efficiency of subsequent data access is improved.

[0143] Optionally, the implementation of this step can be clustering all data tables in the adjusted data table association undirected graph according to the data table nodes and the path weights between the data table nodes in the adjusted data table association undirected graph.

[0144] Step 15, for each data table clustering set, combining the data table clustering sets according to the access frequencies of the data table clustering sets to generate at least one data table clustering cluster.

[0145] In this step, for each data table clustering set, the data table names involved in each operation statement on the database in the historical operation log can be obtained, and the access frequencies of each data table are counted according to all operation statements in the historical operation log, and then the access frequencies of the data table clustering sets are obtained, and then all data table clustering sets are combined to generate at least one data table clustering cluster.

[0146] That is, the access frequencies of each data table clustering cluster generated are similar, thereby providing support for constructing a load-balanced distributed storage system.

[0147] Optionally, the specific implementation of this step can be: obtaining the data table names involved in each operation statement on the database in the historical operation log, counting the access frequencies of each data table according to all operation statements in the historical operation log, adding the access frequencies of at least one data table in each data table clustering set to obtain an addition result as the access frequency of each data table clustering set, sorting all data table clustering sets according to the access frequencies from high to low according to the access frequencies of the data table clustering sets, if the number of all data table clustering sets is even, combining all data table clustering sets in pairs in the manner of combining the highest access frequency and the lowest access frequency, combining the second highest access frequency and the second lowest access frequency, to form at least one data table clustering cluster, if the number of all data table clustering sets is odd, separately taking the data table clustering set with the highest access frequency as a data table clustering cluster, and combining the remaining data table clustering sets in pairs in the manner of combining the highest access frequency and the lowest access frequency, combining the second highest access frequency and the second lowest access frequency, to form at least one data table clustering cluster.

[0148] Step 16, constructing at least one storage node according to the at least one data table cluster, to generate a load-balanced distributed storage system.

[0149] In this step, in all the data table clusters generated above, each data table cluster can represent a storage node of the distributed storage system, that is, the frequency of access of each storage node is ensured to be as close as possible, thereby achieving the requirement of load balancing.

[0150] It should be understood that the details of the embodiments not disclosed in this embodiment are described below.

[0151] The method for constructing a distributed storage system provided by the embodiments of the present application comprises the following steps: obtaining a historical operation log of user access to a database within a preset time; determining a data table associated undirected graph, a user role label, and a data table set corresponding to the user role label according to the historical operation log, wherein a node of the data table associated undirected graph is a data table, and an associated path between the nodes is an association degree between the data tables; adjusting the data table association degree of the data table associated undirected graph according to the data table set, to obtain an adjusted data table associated undirected graph; clustering each data table in the adjusted data table associated undirected graph, to form at least one data table cluster set; for each data table cluster set, combining the data table cluster set according to an access frequency of the data table cluster set, to generate at least one data table cluster; and constructing at least one storage node according to the at least one data table cluster, to generate a load-balanced distributed storage system. In this technical solution, the association between the user label and the data table is considered, and a distributed storage system with better load balancing performance is constructed.

[0152] In Figure 1 Based on the embodiments shown in Figure 2 The flowchart of the method for constructing a distributed storage system provided by the embodiments of the present application is shown in Figure 2 As Figure 3 In step 12 above, determining the data table associated undirected graph according to the historical operation log can comprise the following steps:

[0153] Step 21, obtaining a data table name involved in each operation statement on the database in the historical operation log.

[0154] In this step, the historical operation log records each operation statement on the database, and the data table name involved in each operation statement can be obtained by traversing these operation statements.

[0155] In one possible implementation, a data table 1 and a data table 2 are involved in an operation statement Z on the database.

[0156] In another possible implementation, a data table 1 is involved in an operation statement Y on the database.

[0157] Step 22, for each operation statement, when the number of data table names contained in the operation statement is greater than 1, the association degree between each data table contained in the operation statement is increased by 1.

[0158] In this step, for each operation statement on the database in the historical operation log, when the number of data table names contained in the operation statement is greater than 1 (i.e. the number of data table names is greater than 1, i.e. 2, 3 or 4…), the association degree between each data table contained in the operation statement is increased by 1.

[0159] In a possible implementation, a data table 1 and a data table 2 are involved in an operation statement Z on the database, and then the association degree between the data table 1 and the data table 2 is increased by 1 on the basis (the association degree before the current processing) of the previous processing (processing for each operation statement).

[0160] Step 23, when the number of data table names contained in the operation statement is 1, the association degree between the data tables is not updated.

[0161] In this step, for each operation statement on the database in the historical operation log, when the number of data table names contained in the operation statement is equal to 1 (i.e. the number of data table names is equal to 1), the association degree between each data table contained in the operation statement is not updated.

[0162] In a possible implementation, a data table 1 is involved in an operation statement Y on the database, and then the association degree is kept unchanged for each operation statement before the current processing.

[0163] Step 24, constructing a data table association undirected graph based on the association degrees between each data table.

[0164] In this step, after all the operation statements are traversed, a data table association undirected graph is constructed based on the association degrees between each data table.

[0165] Optionally, in this step 24, the specific implementation can be divided into the following two steps:

[0166] Step 1, normalizing the association degrees between each data table to obtain the association degree values between each data table.

[0167] Specifically, in order to compare the association degrees between each data table, the association degrees between each data table can be normalized.

[0168] In a possible implementation, the association degree between the data table 1 and the data table 2 is 10, and the association degree between the data table 1 and the data table 3 is 5. After normalization processing, the association degree between the data table 1 and the data table 2 is 1, and the association degree between the data table 1 and the data table 3 is 0.5.

[0169] In step 2, each data table is taken as a node in a data table association undirected graph, and the association degree value between each data table is taken as the weight of the association path between the data tables, to construct the data table association undirected graph.

[0170] Specifically, the data table association undirected graph takes each data table as a node, and takes the association degree value between each two data tables as the weight of the association path between the data tables, which is a normalized value.

[0171] The method for constructing the distributed storage system provided in the embodiments of the present application includes the following steps: obtaining the data table name involved in each operation statement on a database in a historical operation log; for each operation statement, when the data table name contained in the operation statement is greater than 1, adding 1 to the association degree between each data table contained in the operation statement; when the data table name contained in the operation statement is 1, not updating the association degree between the data tables; and finally constructing a data table association undirected graph based on the association degree between each data table. In the technical solution, the preliminary construction of the data table association undirected graph is implemented from the relationship between the operation statement and the data table, thereby providing a basis for the construction of the distributed storage system with high load balancing performance.

[0172] On the basis of the above-described embodiments, Figure 3 The flowchart of the method for constructing the distributed storage system provided in the embodiments of the present application is shown in Figure 3 As shown in Figure 4 In step 12, the user role label and the data table set corresponding to the user role label are determined according to the historical operation log, which can include the following steps:

[0173] In step 31, the data table name, the user ID (Identity Document, ID) involved in each operation statement on a database in a historical operation log are obtained.

[0174] In this step, the data table name involved in each operation statement on a database in a historical operation log has been determined on the basis of the above-described embodiments. At this time, the user ID involved in each operation statement on a database, i.e., the initiator of the operation statement, also needs to be obtained.

[0175] Step 32, according to the user ID, the user role label is obtained, and according to the data table name involved in each operation statement on the database and the user role label corresponding to the user ID, the data table set corresponding to the user role label is obtained.

[0176] In this step, according to the user ID, the user role label can be obtained, which can be a long-term user, a short-term user, a user with high amount, etc.

[0177] Further, after obtaining the data table name and the user role label corresponding to the user ID, the data can be reorganized, that is, the data table set corresponding to each user role label is determined.

[0178] Optionally, in this step 32, the specific implementation can be realized by the following 6 steps:

[0179] Step 1, the user role label corresponding to the user ID is obtained, and the correspondence between the user ID and the user role label is obtained according to the information filled in by the user during registration;

[0180] That is, during user registration, some related information will be filled in, such as the residence place, the birth month and year, etc., and accordingly, the user with earlier birth month and year can be classified as a kind of user role label, and the user with later birth month and year can be classified as a kind of user role label.

[0181] Step 2, all operation statements corresponding to the user ID are extracted from the historical operation log, and all data table names involved in the operation statements are extracted to form a data table set corresponding to the user ID;

[0182] For example, the data table set corresponding to the short-term user: data table 1, data table 2;

[0183] The data table set corresponding to the long-term user: data table 3, data table 2.

[0184] Step 3, according to the data table set corresponding to the user ID, the correlation degree of each user ID belonging to the same user role label is calculated;

[0185] Specifically, for each two user IDs, the data table set corresponding to each user ID is obtained;

[0186] The coincidence degree of the data tables in the two data table sets corresponding to the two user IDs is calculated, and the coincidence degree is taken as the correlation degree of the two user IDs;

[0187] Wherein, the calculation formula of the coincidence degree r is:

[0188] r=x / 【0.5*(s1+s2)】

[0189] x is the number of the same data table in the two data table sets, s1 and s2 are the number of the respective data table in the two data table sets.

[0190] For example, the number of data tables s1 in the data table set Y is 3, the number of data tables s2 in the data table set Z is 5, and the number of the same data table x in the data table set Y and the data table set Z is 2, then the coincidence degree r (i.e. the association degree of the two user IDs) is 0.5.

[0191] Step 4, for the association degree between the respective user IDs belonging to the same user role label, performing clustering analysis on the multiple user IDs belonging to the same user role label to form at least one clustering set of at least one user ID belonging to the same user role label;

[0192] Step 5, based on the at least one clustering set of at least one user ID belonging to the same user role label, assigning a secondary user role label to each clustering set;

[0193] Step 6, obtaining the data table set corresponding to all user IDs in each secondary user role label, and taking the data table involved in the data table set corresponding to all user IDs in the secondary user role label as the data table set corresponding to the secondary user role label;

[0194] Further, taking each secondary user role label as a user role label, and taking the data table set corresponding to the secondary user role label as the data table set corresponding to the user role label.

[0195] The construction method of the distributed storage system provided by the embodiment of the present application obtains the data table name and the user ID involved in each operation statement on the database in the historical operation log, then obtains the user role label according to the user ID, and obtains the data table set corresponding to the user role label according to the data table name involved in each operation statement on the database and the user role label corresponding to the user ID. In the technical solution, the association between the role label and the data table is analyzed, which provides a condition for adjusting the data table association undirected graph.

[0196] On the basis of the above embodiment, Figure 4 The flowchart of the construction method of the distributed storage system provided by the embodiment of the present application is shown in Figure 5 As shown in Figure 5 The above step 13 can include the following steps:

[0197] Step 41, for each user role label, obtaining all data table names in the data table set corresponding to the user role label.

[0198] The implementation of this step can be based on the above detailed content.

[0199] Step 42, obtaining the average value of all the association path weights in the data table association undirected graph.

[0200] In this step, all the association path weights in the data table association undirected graph are obtained, and then the average value w of all the association path weights is determined.

[0201] Step 43, finding all the nodes corresponding to all the data table names in the data table set corresponding to the user role label in the data table association undirected graph, and adjusting the association path weights between the nodes to obtain an adjusted data table association undirected graph.

[0202] In this step, all the nodes corresponding to all the data table names in the data table set corresponding to each user role involved by all the users in the data table association undirected graph are found, and the association path weights between the nodes are adjusted, the adjustment is implemented as follows, and then an adjusted data table association undirected graph is obtained.

[0203] Optionally, in this step 43, the specific implementation can be divided into the following 3 steps:

[0204] Step 1, obtaining a data table combination composed of every two data tables in the data table set corresponding to the user role label.

[0205] For each user role label, for every two data tables in the data table set corresponding to the user role label, a corresponding data table combination is constructed.

[0206] For example, there are data table 1, data table 2 and data table 3 in the data table set corresponding to a short-term user, and then data table combination 1 (data table 1 and data table 2), data table combination 2 (data table 3 and data table 2), and data table combination 3 (data table 1 and data table 3) are constructed.

[0207] Step 2, for the first data table and the second data table in each data table combination, when there is an association path between the first data table and the second data table in the data table association undirected graph, doubling the association path weight.

[0208] If the two data tables in the data table combination constructed above can find an association path in the data table association undirected graph, the weight of the association path is doubled.

[0209] For example, data table combination 1 (data table 1 and data table 2) has an association path in the data table association undirected graph, and then the weight q of the association path is doubled to 2*q.

[0210] Step 3, when there is no association path between the first data table and the second data table in the data table association undirected graph, an association path between the first data table and the second data table is established, and the association path weight between the first data table and the second data table is set as the average of all association path weights.

[0211] If the above constructed data table combination cannot find an association path between two data tables in the data table association undirected graph, the association path weight between the first data table and the second data table is set as the average of all association path weights.

[0212] For example, the data table combination 1 (data table 1 and data table 3) has no association path in the data table association undirected graph, and the association path weight is set as the average value w.

[0213] The method for constructing a distributed storage system provided by the embodiment of the application comprises the following steps: acquiring all data table names in a data table set corresponding to a user role tag for each user role tag; acquiring an average of all association path weights in a data table association undirected graph; finding nodes corresponding to all data table names in the data table set corresponding to the user role tag in the data table association undirected graph; and adjusting association path weights between the nodes to obtain an adjusted data table association undirected graph. In the technical solution, the data table association undirected graph is adjusted by using the user role tag and the data table set corresponding to the user role tag, so that the construction of the data table association undirected graph is more accurate.

[0214] On the basis of the above embodiment, Figure 6 The method for constructing a distributed storage system provided by the embodiment of the application comprises the following steps: acquiring all data table names in a data table set corresponding to a user role tag for each user role tag; acquiring an average of all association path weights in a data table association undirected graph; finding nodes corresponding to all data table names in the data table set corresponding to the user role tag in the data table association undirected graph; and adjusting association path weights between the nodes to obtain an adjusted data table association undirected graph. In the technical solution, the data table association undirected graph is adjusted by using the user role tag and the data table set corresponding to the user role tag, so that the construction of the data table association undirected graph is more accurate. Figure 6 As shown in Figure 7 The step 15 can comprise the following steps.

[0215] Step 51, acquiring data table names involved in each operation statement on a database in a historical operation log, and counting the access times of each data table according to all operation statements in the historical operation log.

[0216] In this step, after the data table names involved in each operation statement on a database in the historical operation log are acquired, the access times of each data table involved in all operation statements are counted.

[0217] For example, data table 1 is accessed 23 times, and data table 2 is accessed 43 times.

[0218] Step 52, adding the access times of at least one data table in each data table clustering set to obtain an addition result as the access frequency of each data table clustering set.

[0219] In this step, in the above determined at least one data table cluster set, for each data table cluster set, the number of times each data table in the data table cluster set is accessed is summed to obtain the access frequency of the data table cluster set.

[0220] For example, the data table cluster set t1 includes: data table 1 is accessed 23 times; data table 2 is accessed 43 times, and the access frequency of the data table cluster set t1 is 23+43=66.

[0221] Step 53, according to the access frequency of each data table cluster set, all data table cluster sets are sorted in descending order of access frequency.

[0222] In this step, for each data table cluster set determined above, the access frequencies of all data table cluster sets are sorted in order of the size of the access frequency.

[0223] Example 1, there can be 5 data table cluster sets, the specific positions are: data table cluster set t1: 66 times, data table cluster set t2: 55 times, data table cluster set t3: 44 times, data table cluster set t4: 33 times, data table cluster set t5: 20 times.

[0224] Example 2, there can be 4 data table cluster sets, the specific positions are: data table cluster set t1: 66 times, data table cluster set t2: 55 times, data table cluster set t3: 44 times, data table cluster set t4: 33 times.

[0225] Step 54, if the number of all data table cluster sets is even, all data table cluster sets are combined in pairs according to the combination of the highest access frequency and the lowest access frequency, the combination of the second highest access frequency and the second lowest access frequency, forming at least one data table cluster.

[0226] In one possible implementation, if the number of all data table cluster sets is even, that is, it can be the case shown in the above example 2 (data table cluster set t1: 66 times, data table cluster set t2: 55 times, data table cluster set t3: 44 times, data table cluster set t4: 33 times).

[0227] Then the data table cluster set t1 and the data table cluster set t4 can be combined into a data table cluster u1; the data table cluster set t2 and the data table cluster set t3 can be combined into a data table cluster u2.

[0228] Step 55, if the number of all data table cluster sets is odd, the data table cluster set with the maximum access frequency is taken as a data table cluster alone, and the remaining data table cluster sets are combined in pairs according to the combination of the highest access frequency and the lowest access frequency, the combination of the second highest access frequency and the second lowest access frequency, to form at least one data table cluster.

[0229] In a possible implementation, if the number of all data table cluster sets is odd, that is, the case shown in Example 1 above (data table cluster set t1: 66 times, data table cluster set t2: 55 times, data table cluster set t3: 44 times, data table cluster set t4: 33 times, data table cluster set t5: 20 times) can occur.

[0230] Then, the data table cluster set t1 can be taken as a data table cluster u1 alone, the data table cluster set t2 and the data table cluster set t5 are combined into a data table cluster u2; and the data table cluster set t3 and the data table cluster set t4 are combined into a data table cluster u3.

[0231] The method for constructing a distributed storage system provided by the embodiment of the present application comprises the following steps: obtaining the data table name involved in each operation statement on a database in a historical operation log; according to all operation statements in the historical operation log, counting the access times of each data table; for each data table cluster set, adding the access times of at least one data table in the data table cluster set to obtain an addition result, and taking the addition result as the access frequency of each data table cluster set; according to the access frequencies of the data table cluster sets, sorting all data table cluster sets in descending order of access frequency; if the number of all data table cluster sets is even, combining all data table cluster sets in pairs according to the combination of the highest access frequency and the lowest access frequency, and the combination of the second highest access frequency and the second lowest access frequency, to form at least one data table cluster; if the number of all data table cluster sets is odd, taking the data table cluster set with the maximum access frequency as a data table cluster alone, and combining the remaining data table cluster sets in pairs according to the combination of the highest access frequency and the lowest access frequency, and the combination of the second highest access frequency and the second lowest access frequency, to form at least one data table cluster. In the technical solution, the data tables in the data table cluster set have correlation in the aspects of access operation and business, the efficiency of subsequent data access is improved, and the load balancing performance of the constructed distributed storage system is improved.

[0232] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0233] Figure 7 A structural schematic diagram of an embodiment of a construction apparatus of a distributed storage system provided by the present application is shown in FIG. 1. As shown in the figure, the construction apparatus of the distributed storage system comprises: ​

[0234] An acquisition module 61 is configured to acquire a historical operation log of user access to a database within a preset time;

[0235] A determination module 62 is configured to determine, according to the historical operation log, a data table associated undirected graph, a user role label, and a data table set corresponding to the user role label, wherein a node of the data table associated undirected graph is a data table, and an associated path between the nodes is an association degree between data tables;

[0236] An adjustment module 63 is configured to adjust, according to the data table set, the data table associated undirected graph to obtain an adjusted data table associated undirected graph;

[0237] A clustering module 64 is configured to cluster each data table in the adjusted data table associated undirected graph to form at least one data table clustering set;

[0238] A combination module 65 is configured to, for each data table clustering set, combine the data table clustering set according to an access frequency of the data table clustering set to generate at least one data table clustering cluster;

[0239] A generation module 66 is configured to construct at least one storage node according to the at least one data table clustering cluster to generate a load-balanced distributed storage system.

[0240] In a possible design of an embodiment of the present application, the determination module 62 determines, according to the historical operation log, the data table associated undirected graph, and is specifically configured to:

[0241] acquire a data table name involved in each operation statement on the database in the historical operation log;

[0242] for each operation statement, when the data table name contained in the operation statement is greater than 1, add 1 to an association degree between each data table contained in the operation statement;

[0243] when the data table name contained in the operation statement is 1, do not update the association degree between the data tables;

[0244] construct the data table associated undirected graph based on the association degrees between the data tables.

[0245] Optionally, the determination module 62 constructs the data table associated undirected graph based on the association degrees between the data tables, and is specifically configured to:

[0246] normalize the association degrees between the data tables to obtain association degree values between the data tables.​

[0247] Each data table is taken as a node in a data table association undirected graph, and an association degree value between each data table is taken as a weight of an association path between the data tables, so as to construct the data table association undirected graph.

[0248] In another possible design of the embodiments of the present application, the determining module 62 determines the user role label and the data table set corresponding to the user role label according to the historical operation log, and is specifically configured to:

[0249] obtain the data table name and the user ID involved in each operation statement on the database in the historical operation log;

[0250] obtain the user role label according to the user ID, and obtain the data table set corresponding to the user role label according to the data table name and the user ID corresponding to the user role label involved in each operation statement on the database.

[0251] Optionally, the determining module 62 obtains the user role label according to the user ID, and obtains the data table set corresponding to the user role label according to the data table name and the user ID corresponding to the user role label involved in each operation statement on the database, and is specifically configured to:

[0252] obtain the user role label corresponding to the user ID, and obtain the correspondence between the user ID and the user role label according to the information filled in by the user during registration;

[0253] extract all operation statements corresponding to the user ID from the historical operation log, and extract the data table name involved in all operation statements, to form the data table set corresponding to the user ID;

[0254] perform association degree calculation on each user ID belonging to the same user role label according to the data table set corresponding to the user ID;

[0255] perform clustering analysis on a plurality of user IDs belonging to the same user role label according to the association degree between each user ID belonging to the same user role label, to form at least one clustering set of at least one user ID belonging to the same user role label;

[0256] assign a secondary user role label to each clustering set based on the at least one clustering set of the at least one user ID belonging to the same user role label;

[0257] obtain the data table set corresponding to all user IDs in each secondary user role label, and take the data table involved in the data table set corresponding to all user IDs in the secondary user role label as the data table set corresponding to the secondary user role label;

[0258] The second-level user role label is taken as a user role label, and the data table set corresponding to the second-level user role label is taken as a data table set corresponding to the user role label.

[0259] Optionally, the determining module 62 performs correlation degree calculation on each user ID belonging to the same user role label according to the data table set corresponding to the user ID, and is specifically configured to:

[0260] For each two user IDs, the data table set corresponding to each user ID is obtained;

[0261] The coincidence degree of the data tables in the two data table sets corresponding to the two user IDs is calculated, and the coincidence degree is taken as the correlation degree of the two user IDs;

[0262] The calculation formula of the coincidence degree r is as follows:

[0263] r = x / 【0.5 * (s1 + s2)】

[0264] x is the number of the same data tables in the two data table sets, and s1 and s2 are the numbers of the data tables in the two data table sets, respectively.

[0265] In another possible design of the embodiment of the application, the adjusting module 63 is specifically configured to:

[0266] For each user role label, all data table names in the data table set corresponding to the user role label are obtained;

[0267] The average value of all correlation path weights in the data table correlation undirected graph is obtained;

[0268] All nodes corresponding to all data table names in the data table set corresponding to the user role label are found in the data table correlation undirected graph, and the correlation path weights between the nodes are adjusted, so as to obtain an adjusted data table correlation undirected graph.

[0269] Optionally, the adjusting module 63 finds all nodes corresponding to all data table names in the data table set corresponding to the user role label in the data table correlation undirected graph, and adjusts the correlation path weights between the nodes, and is specifically configured to:

[0270] A data table combination formed by each two data tables in the data table set corresponding to the user role label is obtained;

[0271] For a first data table and a second data table in each data table combination, when there is a correlation path between the first data table and the second data table in the data table correlation undirected graph, the correlation path weight is doubled;

[0272] When there is no association path between the first data table and the second data table in the data table association undirected graph, an association path between the first data table and the second data table is established, and an association path weight between the first data table and the second data table is set as an average of all association path weights.

[0273] In a possible design of the embodiment of the present application, the clustering module 64 is specifically configured to:

[0274] According to the path weights between the data table nodes in the adjusted data table association undirected graph, all the data tables in the adjusted data table association undirected graph are clustered to form at least one data table clustering set.

[0275] In another possible design of the embodiment of the present application, the generating module 66 is specifically configured to:

[0276] Obtain the data table name involved in each operation statement on the database in the historical operation log, and according to all the operation statements in the historical operation log, count the access times of each data table;

[0277] For each data table clustering set, add the access times of at least one data table in the data table clustering set to obtain an addition result, and the addition result is used as the access frequency of the data table clustering set;

[0278] According to the access frequencies of the data table clustering sets, all the data table clustering sets are sorted in descending order of the access frequencies;

[0279] If the number of all the data table clustering sets is even, all the data table clustering sets are combined in a manner that the highest access frequency and the lowest access frequency are combined, and the second highest access frequency and the second lowest access frequency are combined, to form at least one data table clustering cluster;

[0280] If the number of all the data table clustering sets is odd, the data table clustering set with the highest access frequency is separately used as a data table clustering cluster, and the remaining data table clustering sets are combined in a manner that the highest access frequency and the lowest access frequency are combined, and the second highest access frequency and the second lowest access frequency are combined, to form at least one data table clustering cluster.

[0281] The construction apparatus of the distributed storage system provided in the embodiment of the present application can be used to execute the construction method of the distributed storage system in any of the above embodiments, and has similar implementation principles and technical effects, which will not be described herein again.

[0282] It should be noted that the division of each module of the above apparatus is only a logical function division, and all or part of them can be integrated into one physical entity or physically separated when actually implemented. Moreover, all or part of these modules can be integrated together or independently implemented. The processing element described herein can be an integrated circuit having a signal processing capability. In the implementation process, each step of the above method or each module can be completed by the integrated logic circuit of the hardware in the processor element or the instruction in the form of software.

[0283] ​ The structure schematic diagram of the electronic device provided by the embodiment of the present application is shown in FIG. 1, which can include a processor 71, a memory 72, and computer program instructions stored in the memory 72 and executable on the processor 71, wherein the processor 71 executes the computer program instructions to implement the construction method of the distributed storage system provided by any of the above embodiments. ​

[0284] Optionally, the above various devices of the electronic device can be connected through a system bus.

[0285] The memory 72 can be a separate storage unit or a storage unit integrated in the processor 71. The number of the processor 71 is one or more.

[0286] It should be understood that the processor 71 can be a central processing unit (CPU), and can also be other general-purpose processors 71, digital signal processors 71 (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor 71 can be a microprocessor or can also be any conventional processor 71, etc. The steps of the method disclosed in the present application can be directly embodied as the execution of the hardware processor 71 or the execution of the combination of the hardware and software modules in the processor 71.

[0287] ​The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus or the like. The system bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or only one type of bus. The memory 72 can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk memory 72.

[0288] All or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a readable memory 72. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned memory 72 (storage medium) includes a read-only memory (ROM), a RAM, a flash memory 72, a hard disk, a solid state disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof.

[0289] The electronic device provided by the embodiments of the present application can be used to execute the construction method of the distributed storage system provided by any one of the above-mentioned method embodiments, and has similar implementation principles and technical effects, which will not be described here.

[0290] The embodiments of the present application provide a computer readable storage medium, and the computer readable storage medium stores computer instructions. When the computer instructions run on a computer, the computer executes the construction method of the distributed storage system.

[0291] The above-mentioned computer readable storage medium, the above-mentioned readable storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0292] Optionally, a readable storage medium is coupled to the processor, such that the processor is enabled to read information from, and write information to, the readable storage medium. Of course, the readable storage medium can be a part of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the readable storage medium can also be located as discrete components in a device.

[0293] The embodiment of the present application further provides a computer program product, which comprises a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and the at least one processor executes the computer program to implement the method for constructing the distributed storage system.

[0294] It should be understood that the present disclosure is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method of constructing a distributed storage system, characterized by, The method comprises the following steps: acquiring a historical operation log of a user accessing a database within a preset time; determining a data table association undirected graph, a user role label and a data table set corresponding to the user role label according to the historical operation log, wherein nodes of the data table association undirected graph are data tables, and an association path between nodes is an association degree between the data tables; adjusting a data table association degree of the data table association undirected graph according to the data table set to obtain an adjusted data table association undirected graph; clustering each data table in the adjusted data table association undirected graph to form at least one data table cluster set; combining the data table cluster set according to an access frequency of the data table cluster set to generate at least one data table cluster for each data table cluster set; constructing at least one storage node according to the at least one data table cluster to generate a load-balanced distributed storage system.

2. The method of claim 1, wherein, The method comprises the following steps: acquiring a data table name involved in each operation statement of the database in the historical operation log; for each operation statement, when the number of data table names contained in the operation statement is greater than 1, adding 1 to the association degree between each data table contained in the operation statement; when the number of data table names contained in the operation statement is 1, not updating the association degree between the data tables; constructing the data table association undirected graph based on the association degree between each data table.

3. The method of claim 2, wherein, The method comprises the following steps: normalizing the association degree between each data table to obtain an association degree value between each data table; constructing the data table association undirected graph by taking each data table as a node in the data table association undirected graph and taking the association degree value between each data table as a weight of an association path between the data tables.

4. The method of claim 1, wherein, The method comprises the following steps: acquiring a data table name involved in each operation statement of the database in the historical operation log, and a user ID; acquiring a user role label according to the user ID, and acquiring a data table set corresponding to the user role label according to the data table name and the user ID corresponding to the user role label involved in each operation statement of the database.

5. The method of claim 4, wherein, The method comprises the following steps: acquiring a user role label corresponding to the user ID, and the correspondence between the user ID and the user role label is obtained according to information filled in by a user during registration; extracting all operation statements corresponding to the user ID from the historical operation log, and extracting data table names involved in all operation statements to form a data table set corresponding to the user ID; According to the data table set corresponding to the user ID, the correlation degree of each user ID belonging to the same user role label is calculated; For the correlation degree between each user ID belonging to the same user role label, a clustering analysis is performed on the multiple user IDs belonging to the same user role label to form at least one clustering set of at least one user ID under the same user role label; Based on the at least one clustering set of at least one user ID under the same user role label, a secondary user role label is assigned to each clustering set; Obtain the data table set corresponding to all user IDs in each secondary user role label, and take the data tables involved in the data table set corresponding to all user IDs in the secondary user role label as the data table set corresponding to the secondary user role label; Each secondary user role label is taken as a user role label, and the data table set corresponding to the secondary user role label is taken as the data table set corresponding to the user role label.

6. The method of claim 5, wherein, According to the data table set corresponding to the user ID, the correlation degree of each user ID belonging to the same user role label is calculated, including: For each two user IDs, obtain the data table set corresponding to each user ID; Calculate the coincidence degree of the data tables in the two data table sets corresponding to the two user IDs, and take the coincidence degree as the correlation degree of the two user IDs; Wherein, the calculation formula of the coincidence degree r is: x is the number of the same data tables in the two data table sets, and s1 and s2 are the number of the respective data tables in the two data table sets.

7. The method of claim 1, wherein, According to the data table set, the data table association undirected graph is adjusted to obtain an adjusted data table association undirected graph, including: For each user role label, obtain all data table names in the data table set corresponding to the user role label; Obtain the average value of all association path weights in the data table association undirected graph; Find the nodes corresponding to all data table names in the data table set corresponding to the user role label in the data table association undirected graph, and adjust the association path weights between the nodes to obtain an adjusted data table association undirected graph.

8. The method of claim 7, wherein, The nodes corresponding to all data table names in the data table set corresponding to the user role label are found in the data table association undirected graph, and the association path weights between the nodes are adjusted, including: Obtain the data table combination formed by each two data tables in the data table set corresponding to the user role label; For the first data table and the second data table in each data table combination, when there is an association path between the first data table and the second data table in the data table association undirected graph, the association path weight is doubled; When there is no association path between the first data table and the second data table in the data table association undirected graph, an association path between the first data table and the second data table is established, and the association path weight between the first data table and the second data table is set to the average value of all association path weights.

9. The method of claim 1, wherein, The data tables in the adjusted data table association undirected graph are clustered to form at least one data table clustering set, including: According to the adjusted data table association undirected graph, the data table nodes and the path weight between the data table nodes are associated, all the data tables in the adjusted data table association undirected graph are clustered to form the at least one data table cluster set.

10. The method of claim 1, wherein, According to the access frequency of each data table cluster set, the data table cluster set is combined to generate at least one data table cluster, including: Obtaining the data table name involved in each operation statement of the database in the historical operation log, and according to all the operation statements in the historical operation log, the access frequency of each data table is counted; For each data table cluster set, the sum of the access frequency of at least one data table in the data table cluster set is added to obtain the addition result as the access frequency of each data table cluster set; According to the access frequency of each data table cluster set, all the data table cluster sets are sorted in descending order of access frequency; If the number of all data table cluster sets is even, all data table cluster sets are combined in pairs according to the combination of the highest access frequency and the lowest access frequency, and the combination of the second highest access frequency and the second lowest access frequency, to form the at least one data table cluster; If the number of all data table cluster sets is odd, the data table cluster set with the highest access frequency is separately taken as a data table cluster, and the remaining data table cluster sets are combined in pairs according to the combination of the highest access frequency and the lowest access frequency, and the combination of the second highest access frequency and the second lowest access frequency, to form the at least one data table cluster.

11. A construction apparatus of a distributed storage system, characterized by comprising: Including: An acquisition module is configured to acquire a historical operation log of a user accessing a database within a preset time; A determination module is configured to determine a data table association undirected graph, a user role label and a data table set corresponding to the user role label according to the historical operation log, wherein nodes of the data table association undirected graph are data tables, and an association path between nodes is an association degree between the data tables; An adjustment module is configured to adjust a data table association degree of the data table association undirected graph according to the data table set to obtain an adjusted data table association undirected graph; A clustering module is configured to cluster each data table in the adjusted data table association undirected graph to form at least one data table cluster set; A combination module is configured to combine, for each data table cluster set, the data table cluster set according to the access frequency of the data table cluster set to generate at least one data table cluster; A generation module is configured to construct at least one storage node according to the at least one data table cluster to generate a load-balanced distributed storage system.

12. An electronic device, comprising: Including: A processor and a memory connected in communication with the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the method of any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method of any one of claims 1-10.

14. A computer program product, characterised in that, A computer program is included, which, when executed by a processor, implements the method of any one of claims 1-10.

Images