Data processing method, device and database operating system
By obtaining the attribute information of data tables in the database, filtering and building table connections, the problem of inefficient connection reordering in the existing technology is solved, and the data processing efficiency of the database and the performance of the optimizer are improved.
Patent Information
- Application Number
- CN202010604984.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-29
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-06-29
AI Technical Summary
Existing database optimizers are inefficient when reordering connections, and usually only produce one or two new connection sequences, affecting database performance.
By obtaining the attribute information of the data table in the database, filter out the data table collection with association relationships, build table connections, and divide them into connection groups to generate expression information to support more optimization rules.
It quickly enumerates more expression information, improves the database data processing efficiency, and combines it with the optimizer to support more optimization rules.
Smart Images

Figure CN113297165B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device and database operating system. Background Art
[0002] With the development of technology, databases can not only store a large amount of data for a long time, but also manage data in an organized and shareable manner, achieving safe and stable data storage while accelerating the efficiency of data processing; in this process, the database optimizer will play a vital role, not only to find a reasonable execution plan within a reasonable time, but also to ensure that the query can return the correct results stably and quickly, and is the core component supporting database operation data; for the optimizer, connection reordering is relatively complex and consumes more rules, and connection reordering will directly affect the performance of the optimizer. In the prior art, the optimizer usually enumerates the connection order based on the commutative law and the associative law, and can only generate one or two new connection orders each time, which is inefficient. Therefore, how to quickly enumerate all new connection orders for connection reordering will directly affect the performance of the optimizer. Summary of the invention
[0003] In view of this, an embodiment of the present application provides a data processing method. The present application also relates to a data processing device, a database operating system, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.
[0004] According to a first aspect of an embodiment of the present application, there is provided a data processing method, including:
[0005] Get the attribute information of the data table in the database;
[0006] According to the attribute information, data tables with association relationships are screened out from the data tables to form a data table set;
[0007] Constructing table connections of the data tables in the data table set according to the association relationships between the data tables in the data table set;
[0008] The table connection is divided into at least one connection group, and expression information for performing data operations on the database is generated based on the connection group.
[0009] Optionally, the step of selecting data tables having an associated relationship from the data tables to form a data table set according to the attribute information includes:
[0010] Extracting primary key information from the attribute information;
[0011] The data tables having the same primary key information are screened out from the data tables to form the data table set.
[0012] Optionally, constructing a table connection of the data tables in the data table set according to the association relationship between the data tables in the data table set includes:
[0013] The data tables in the data table set are used as nodes, and the table connection is constructed based on the primary key information of the data tables in the data table set.
[0014] Optionally, dividing the table connection into at least one connection group includes:
[0015] Converting the table connection to obtain an intermediate table connection;
[0016] The intermediate table connection is split to obtain at least one connection group.
[0017] Optionally, the segmenting the intermediate table connection to obtain at least one connection group includes:
[0018] Determine the intermediate data table included in the intermediate table connection;
[0019] The intermediate table connections are segmented based on the association relationship between the intermediate data tables to obtain at least one of the connection groups.
[0020] Optionally, constructing a table connection of the data tables in the data table set according to the association relationship between the data tables in the data table set includes:
[0021] Determine the association relationship between the data tables in the data table set;
[0022] Constructing a connection network of data tables in the data table set according to the association relationship;
[0023] The connection tree obtained after converting the connection network is used as the table connection.
[0024] Optionally, generating expression information for performing data operations on the database based on the connection group includes:
[0025] Determine a first sub-connection and a second sub-connection included in the connection group;
[0026] Determine an association relationship between the first sub-connection and the second sub-connection by reading equivalent data of the connection group;
[0027] The expression information is generated according to the association relationship between the first sub-connection and the second sub-connection.
[0028] Optionally, after the step of generating expression information for performing data operations on the database based on the connection group is executed, the step further includes:
[0029] Obtaining an operation request submitted for a data table in the database;
[0030] Selecting target expression information in the metadata table of the expression information mapping according to the operation data included in the operation request;
[0031] The target expression information is input into a database optimizer corresponding to the database to obtain a target data table consisting of target data corresponding to the operation request.
[0032] Optionally, selecting target expression information in the metadata table of the expression information mapping according to the operation data included in the operation request includes:
[0033] Parsing the operation request to obtain the operation data;
[0034] Determining, according to the operation data, whether the number of data tables associated with the operation request is greater than a preset number threshold;
[0035] If yes, in the metadata table, selecting at least two sub-expression information corresponding to the target connection group associated with the operation request to form the target expression information;
[0036] If not, in the metadata table, the sub-expression information corresponding to the target connection group associated with the operation request is selected as the target expression information.
[0037] According to a second aspect of an embodiment of the present application, there is provided a data processing device, including:
[0038] An acquisition module is configured to acquire attribute information of a data table in a database;
[0039] A composition module is configured to filter out data tables with association relationships from the data tables according to the attribute information to form a data table set;
[0040] A construction module, configured to construct table connections of data tables in the data table set according to association relationships between data tables in the data table set;
[0041] The generating module is configured to divide the table connection into at least one connection group, and generate expression information for performing data operations on the database based on the connection group.
[0042] According to a third aspect of an embodiment of the present application, there is provided a database operating system, including:
[0043] Client, computing node, storage node;
[0044] The client is used to receive a query instruction input by a user;
[0045] The computing node is used to parse the query instruction to obtain expression information for operating on the database; determine the target connection group based on the expression information, and read the primary key information corresponding to the target connection group;
[0046] The storage node is used to query the data table corresponding to the primary key information in the database, and read the target data corresponding to the query instruction in the data table and return it to the client.
[0047] Optionally, determining the target connection group based on the expression information includes:
[0048] The expression information is input into a database optimizer corresponding to the database to obtain the target connection group.
[0049] Optionally, the expression information is created in the following manner:
[0050] Determine a first sub-connection and a second sub-connection included in the target connection group;
[0051] Determine an association relationship between the first sub-connection and the second sub-connection by reading equivalent data of the target connection group;
[0052] The expression information is generated according to the association relationship between the first sub-connection and the second sub-connection.
[0053] Optionally, the reading target data corresponding to the query instruction in the data table and returning it to the client includes:
[0054] Obtaining a data key value by parsing the expression information;
[0055] Read the data table based on the data key value, and determine the target data according to the reading result;
[0056] A target data table is formed according to the target data, and the target data table is returned to the client.
[0057] According to a fourth aspect of an embodiment of the present application, a computing device is provided, including:
[0058] Memory and processor;
[0059] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions:
[0060] Get the attribute information of the data table in the database;
[0061] According to the attribute information, data tables with association relationships are screened out from the data tables to form a data table set;
[0062] Constructing table connections of the data tables in the data table set according to the association relationships between the data tables in the data table set;
[0063] The table connection is divided into at least one connection group, and expression information for performing data operations on the database is generated based on the connection group.
[0064] According to a fifth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the data processing method are implemented.
[0065] The data processing method provided by the present application obtains attribute information of a data table in a database, screens out data tables with association relationships from the data table to form a data table set, constructs a table connection corresponding to the data table set according to the association relationships between the data tables in the data table set, then divides the table connection into at least one connection group, and finally generates expression information for performing data operations on the database. By constructing the table connection, the data tables with association relationships in the database are bound and the table connection is divided, so that more expression information can be quickly enumerated, thereby achieving better integration with the database optimizer, supporting more database optimization rules, and further improving the data processing efficiency of the database. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 is a flow chart of a data processing method provided by an embodiment of the present application;
[0067] Figure 2 This is a schematic diagram of a first table connection provided by an embodiment of the present application;
[0068] Figure 3 is a schematic diagram of a second table connection provided in an embodiment of the present application;
[0069] Figure 4 This is a schematic diagram of an intermediate table connection provided by an embodiment of the present application;
[0070] Figure 5 It is a processing flow chart of a data processing method applied to a query scenario provided by an embodiment of the present application;
[0071] Figure 6 is a structural schematic diagram of a data processing device provided by an embodiment of the present application;
[0072] Figure 7 It is a schematic diagram of the structure of a database operating system provided by an embodiment of the present application;
[0073] Figure 8 It is a structural block diagram of a computing device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0074] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.
[0075] The terms used in one or more embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present application. The singular forms of "a", "said" and "the" used in one or more embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.
[0076] It should be understood that, although the terms first, second, etc. may be used to describe various information in one or more embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".
[0077] In the present application, a data processing method is provided. The present application also relates to a data processing device, a database operating system, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.
[0078] Figure 1 A flowchart of a data processing method provided according to an embodiment of the present application is shown, which specifically includes the following steps:
[0079] Step S102: Acquire attribute information of a data table in a database.
[0080] In practical applications, before querying data from a database, it is necessary to enumerate all the connection sequences involved in the data tables in the database, and then select the final connection sequence according to the losses of each enumerated connection sequence as the query result output; however, in the enumeration process, it is usually implemented based on the commutative law and the associative law, and each call can only generate one or two new sequences, which is inefficient.
[0081] The data processing method provided by the present application, in order to improve the enumeration efficiency and thus improve the data processing efficiency of the database, obtains the attribute information of the data table in the database, screens out the data tables with association relationships in the data table to form a data table set, constructs the table connection corresponding to the data table set according to the association relationship between the data tables in the data table set, and then divides the table connection into at least one connection group, generates expression information for data operations on the database, and realizes that by constructing the table connection, the data tables with association relationships in the database are bound and the table connection is divided, so that more expression information can be quickly enumerated, so as to achieve better integration with the database optimizer, support more database optimization rules, and further improve the data processing efficiency of the database.
[0082] In practical applications, the database is a database that complies with the SQL (Structured Query Language) standard and can implement operations such as query, addition, update, deletion, summation and sorting; accordingly, the data table specifically refers to the storage format of data stored in the database, that is, a large amount of data in the database exists in the form of data tables, and different types of data correspond to different data tables. It should be noted that different types of data can be understood as different data types, different data dimensions or different data structures.
[0083] Based on this, the attribute information of the data table specifically refers to the description information corresponding to each data table in the database. The attribute information can be understood as information describing the attributes of the data table and the identification of the data table. For example, there are data tables P and O in the database, where "ID_P" is the primary key of data table P, and "ID_O" is the primary key of data table O. The P table records data related to "surname", "first name", "address" and "city", and the O table records data related to "command" and "P table primary key". At this time, it can be determined that the attribute information of data table P is {surname, first name, address, city, primary key}, and the attribute information of data table O is {command, P table primary key, primary key}, so that subsequent processing operations can be performed on the data table according to the attribute information to quickly enumerate more expression information.
[0084] Step S104: According to the attribute information, data tables having association relationships are selected from the data tables to form a data table set.
[0085] Specifically, based on the above-mentioned acquisition of attribute information of the data tables, further, data tables with association relationships are screened out from the acquired data tables according to the attribute information to form a data table set, so as to be used for subsequent construction of table connections of the data tables in the data table set.
[0086] In actual applications, in the process of screening the data tables that constitute the data table set from the data tables, since the database contains a large number of data tables and they are differentiated according to different data types, some data tables in the database have association relationships and some data tables do not have association relationships. In the process of operating the data in the database, it is often necessary to connect multiple data tables to obtain operation results that meet the operation requirements, and the operation of the data is based on expression information. Therefore, it is necessary to form data tables with association relationships into a data table set for subsequent generation of expression information to support operations on the data in the database.
[0087] Furthermore, in the process of forming the data table set, in order to support the accuracy of generating the expression information and improve the efficiency of enumerating the expression information, the data tables that meet the requirements of forming the data table set are screened based on the primary key information in the attribute information. In this embodiment, the specific implementation method is as follows:
[0088] Extracting primary key information from the attribute information;
[0089] The data tables having the same primary key information are screened out from the data tables to form the data table set.
[0090] Specifically, after acquiring the attribute information of each data table in the database, primary key information in the attribute information is extracted, the primary key information corresponds to each data table in the database, and then data tables with the same primary key information are selected to form the data table set.
[0091] In actual applications, since different data tables have different primary key information, in order to indicate the association relationship between data tables, the primary key information of the data table with the association relationship will also be recorded in its own data table, so that the association relationship between multiple data tables can be reflected; therefore, in this embodiment, data tables with the same primary key information are screened out, specifically referring to screening out data tables that record the primary key information of other data tables from a large number of data tables to form the data table set.
[0092] For example, in addition to data tables O and P, there is also a data table Q in the database, where data table P is shown in Table (1):
[0093] Table 1
[0094] ID_P LastName FirstName Address City 1 A AJ OS LC 2 B BG FS NC 3 C CT CS BC
[0095] Data table O is shown in table (2):
[0096] Table 2
[0097] ID_O OrderNo ID_P 1 77895 3 2 44678 2 3 22456 1
[0098] The data table Q is shown in Table (3):
[0099] Table 3
[0100]
[0101]
[0102] According to the attribute information of data table O, data table P and data table Q, the primary key information of each data table is extracted respectively, so as to determine that data table O and data table P have the same primary key information, that is, data table O records the primary key information of data table P. Therefore, data table O and data table P are selected to form a data table set for subsequent generation of database expression information, so as to realize rapid enumeration of all orders, thereby improving the optimization performance of the database optimizer.
[0103] In addition, in the process of screening out data tables with association relationships to form a data table set in the database, data tables with association relationships can also be determined by other methods, such as by querying keywords of each data table to determine data tables with association relationships, or querying unique identifiers of each data table to determine data tables with association relationships; the specific method of screening out data tables with association relationships to form a data table set can be set according to the actual application scenario, and this application does not make too many restrictions here.
[0104] Step S106: constructing table connections of the data tables in the data table set according to the association relationships between the data tables in the data table set.
[0105] Specifically, on the basis of the above-mentioned screening out of data tables with association relationships to form a data table set, further, a table connection corresponding to the data table set will be constructed according to the association relationships of the data tables in the data table set, wherein the table connection can be a network connection or a tree connection, thereby making it more convenient to enumerate and express information and support more optimization rules of the database.
[0106] Furthermore, constructing the table connection is the basis for supporting the subsequent split table connection and realizing the rapid enumeration of expression information. Therefore, in the process of constructing the table connection, it is necessary to use the data tables in the data table set as nodes and the associations between the data tables as edges to construct the table connection. In this embodiment, the specific implementation method is as follows:
[0107] The data tables in the data table set are used as nodes, and the table connection is constructed based on the primary key information of the data tables in the data table set.
[0108] Specifically, the association relationship between each data table in the data table set is determined based on the primary key information in the attribute information. Therefore, in the process of constructing the table connection, the primary key information between each data table in the data table set is used as the edge for constructing the table connection, and each data table is used as a node to construct the table connection.
[0109] In actual applications, in the process of constructing the table connection, in order to facilitate the subsequent segmentation of the table connection, the table connection can be constructed into a tree structure during the construction process, that is, the constructed table connection is a tree connection; for example, in the TPC-DS Q6 scenario, a table connection is constructed for a data table in a database, and the constructed table connection is as follows Figure 2 As shown, it exists in a tree structure. Figure 2 In the tree connection shown, each node corresponds to a data table in the data table set, and the connection line between each node represents the association relationship between adjacent data tables;
[0110] Based on this, according to Figure 2 From the content shown, it can be concluded that the data table corresponding to store sales (Store_sales) is associated with the data tables corresponding to customers (customer), items (item) and dates (date_dim), and the data table associated with the data table corresponding to customers (customer) is customer addresses (customer_addres), the data table associated with the data table corresponding to items (item) is aggregate 1 (aggregate1), and the data table associated with the data table corresponding to date (date_dim) is aggregate 2 (aggregate2). Subsequently, the tree connection can be split into 6 connection groups, thereby generating 6 expression information, which can improve the efficiency of query processing and other operations of the database in this scenario.
[0111] It should be noted that TPC-DS is a set of decision support system test benchmarks, mainly targeting the database corresponding to the retail industry. It is a decision support benchmark for decision support systems that includes multi-dimensional general application models, including queries and data maintenance. This benchmark is representative of the evaluation of the performance of the system under test (SUT) at the decision support system level; it can test large-scale data, answer actual business problems, execute queries with diverse or complex requirements (such as temporary queries, reports, iterative OLAP, data mining), and solve big data problems such as relational databases (RDBMS); thus, by constructing table connections in this scenario, a large amount of expression information corresponding to the database can be quickly and accurately enumerated, and efficient and diverse complex queries can be achieved.
[0112] In addition, in the process of constructing the table connection, since the data tables in the data table set may have cross-correlation relationships, the table connection constructed at this time may have a mesh structure, which may reduce the efficiency of the subsequent segmentation process. In this embodiment, when the constructed table connection is a connection network, it can be changed to a connection tree by conversion, thereby improving the efficiency of the subsequent segmentation process. The specific implementation method is as follows:
[0113] Determine the association relationship between the data tables in the data table set;
[0114] Constructing a connection network of data tables in the data table set according to the association relationship;
[0115] The connection tree obtained after converting the connection network is used as the table connection.
[0116] In practical applications, after determining the association relationship between each data table in the data table set, a connection network of the data tables in the data table set can be constructed based on the association relationship. The connection network can connect the data tables that have association relationships in the data set. At this time, if the connection network is segmented, multiple association relationships need to be segmented at the same time, which will greatly reduce the segmentation efficiency. The connection network can be converted to a connection tree, which is convenient for segmentation and can achieve comprehensive coverage of table connections, making subsequent enumeration more convenient.
[0117] See also Figure 3As shown, S1-S6 represent each data table in the data table set, and data table S1 has an association relationship with data tables S2, S3, S5 and S6, data table S2 has an association relationship with data tables S1 and S4, data table S3 has an association relationship with data tables S1 and S4, data table S4 has an association relationship with data tables S6, S5, S3 and S2, data table S5 has an association relationship with data tables S1 and S4, data table S6 has an association relationship with data tables S4 and S1, and the connection network constructed based on the association relationship between each data table is as follows: Figure 3 As shown, at this time, the connection network can be converted to obtain multiple connection trees, and the connection trees are used as table connections corresponding to the data table set, so as to make subsequent segmentation processing more convenient and improve processing efficiency.
[0118] In summary, in the process of constructing the table connection corresponding to the data table set based on the association relationship of the data tables in the data table set, in order to facilitate the subsequent segmentation processing and improve the efficiency of the segmentation processing, firstly, a connection network is constructed based on the association relationship and the data table, and then the connection network is converted to obtain a connection tree as the table connection, so that all connection relationships of the table connection can be expressed through the connection tree, and the efficiency of the subsequent segmentation processing can be effectively improved.
[0119] Step S108: dividing the table connection into at least one connection group, and generating expression information for performing data operations on the database based on the connection group.
[0120] Specifically, based on the above-mentioned construction of table connections of data tables in the data table set, the table connections are further segmented to obtain at least one connection group, so that expression information of the database can be generated based on the association relationship of sub-connections in the connection group.
[0121] In actual applications, in the process of splitting the table connection, since there are many data tables in the table connection and there are associations between the data tables, the expression connection is split to split into more connection groups, each connection group represents a connection relationship, so that it is possible to support the enumeration of more expression information. In this embodiment, the process of generating expression information based on any connection group is described. The process of splitting the table connection into multiple connection groups and then generating multiple expression information can refer to the content described in this embodiment, and will not be elaborated here.
[0122] In specific implementation, since the table connection is constructed based on the association relationship between multiple data tables and the data tables, after the table connection is split into at least one connection group, the at least one connection group will consist of two parts, namely a first sub-connection and a second sub-connection; for example, a table connection is constructed based on data table O and data table P, and the table connection is split at this time, and the split position is the connection relationship between data table O and data table P. The connection group obtained after the split is composed of a first sub-connection including data table O and a second sub-connection including data table P; that is, the first sub-connection and the second sub-connection represent two-part connections that each contain data tables with an association relationship after the split process.
[0123] Furthermore, in the process of segmenting the table connection, in order to increase the efficiency of the segmentation process, in this embodiment, the specific implementation method is as follows:
[0124] Converting the table connection to obtain an intermediate table connection;
[0125] The intermediate table connection is split to obtain at least one connection group.
[0126] Furthermore, when the intermediate table connection is split, the split is actually implemented based on the association relationship of the intermediate data tables in the intermediate table connection. In this embodiment, the specific implementation method is as follows:
[0127] Determine the intermediate data table included in the intermediate table connection;
[0128] The intermediate table connections are segmented based on the association relationship between the intermediate data tables to obtain at least one of the connection groups.
[0129] Specifically, the intermediate table connection refers to a connection structure obtained after converting the table connection. By converting the table connection, segmentation processing can be conveniently performed and all connection relationships expressed in the table connection can be covered;
[0130] Based on this, on the basis of obtaining the converted intermediate table connection, further, at this time, the intermediate data table in the intermediate table connection is determined, and then the intermediate table connection is segmented based on the association relationship between the intermediate data tables to obtain the connection group.
[0131] In practical applications, the table connection can be converted into multiple intermediate table connections. Since there are cross-relationships between the data tables, multiple intermediate table connections are required to reflect the complex connection relationships in the table connection, thereby realizing that all connection relationships of the table connection can be reflected through multiple intermediate table connections.
[0132] See also Figure 3As shown in the figure, the table connection constructed for the data tables S1-S6, in order to facilitate the segmentation process, the table connection will be converted, and the converted intermediate table connection is as follows Figure 4 (a) and Figure 4 As shown in (b) of FIG. 1 , in addition, the intermediate table connection obtained after the table connection is transformed also includes Figure 4 (a) and Figure 4 The intermediate table connections other than the intermediate table connection shown in (b) in FIG. 1 are only used in this embodiment because too many table connections are obtained after conversion. Figure 4 (a) and Figure 4 The intermediate table connection shown in (b) is described in detail. For other intermediate table connections, please refer to the corresponding description content, which will not be described in detail here.
[0133] Based on this, in determining Figure 4 After the intermediate table connection shown in (a) is completed, the distribution of data tables S1-S6 in the intermediate table connection is determined, and then the intermediate table connection is split. The split processing position is the association relationship between data table S2 and the data table corresponding to the previous node. At this time, the connection group obtained is composed of a sub-connection including data table S2 and a sub-connection including data tables S1 and S3-S6, which is used for the subsequent generation of expression information of the database to which data tables S1-S6 belong.
[0134] In summary, in the process of segmenting the table connection, in order to improve the segmentation efficiency and to fully cover the relationship between the data tables in the table connection, the table connection can be converted into an intermediate table connection and then segmented to obtain at least one connection group for subsequent generation of expression information of the database. Therefore, in the process of enumerating the expression information, a large number of connection groups can be obtained by segmentation, thereby enumerating a large amount of expression information, thereby improving the enumeration efficiency and speeding up the subsequent database data processing efficiency.
[0135] Furthermore, based on the above-mentioned segmentation process of the table connection to obtain at least one connection group, the association relationship between the sub-connections contained in the connection group is expressed as a new connection order in the database, and an expression information for performing data operations on the database can be generated based on the association relationship; wherein the expression information specifically refers to an expression string for performing data operations on the database, which can be applied to the optimizer of the database.
[0136] In practical applications, when the table connection is divided into multiple connection groups, the association relationships between the sub-connections contained in each connection group all express a new connection order in the database. Multiple connection groups correspond to multiple new connection orders. Based on multiple association relationships, multiple expression information for performing data operations on the database can be generated, thereby enumerating a large number of new connection orders for the database and generating a large amount of expression information, which can be used as a basis to enable the database to support more optimization rules.
[0137] In the specific implementation, in the process of generating the expression information, in order to improve the accuracy of the generated expression information and facilitate the full integration with the optimizer of the database during the application process, in this embodiment, the expression information can be generated in the following manner:
[0138] Determine a first sub-connection and a second sub-connection included in the connection group;
[0139] Determine an association relationship between the first sub-connection and the second sub-connection by reading equivalent data of the connection group;
[0140] The expression information is generated according to the association relationship between the first sub-connection and the second sub-connection.
[0141] In actual applications, in the process of splitting the table connection, it will be split into the first sub-connection and the second sub-connection, so the connection group is composed of the first sub-connection and the second sub-connection. At this time, by reading the equivalent data of the connection group, the association relationship between the first sub-connection and the second sub-connection can be determined; finally, according to the association relationship between the first sub-connection and the second sub-connection, the expression information of the database can be generated.
[0142] In specific implementation, the equivalent data specifically refers to the case where the connection conditions corresponding to the association relationship between the data table included in the first sub-connection and the data table included in the second sub-connection are equal, and it is represented as equivalent data. At this time, the association relationship between the first sub-connection and the second sub-connection can be determined by the equivalent data, so that expression information for the connection group can be generated to be recorded in the database, and then the data in the data table included in the first sub-connection and the data table included in the second sub-connection can be queried according to the expression information.
[0143] In addition, after generating the expression information corresponding to all connection groups obtained by the table connection segmentation processing, all expression information can be recorded in the metadata table of the database, so that during the application process, the expression information corresponding to the query request can be directly extracted, thereby obtaining a data table composed of data corresponding to the expression information.
[0144] Based on this, when there is an operation requirement for the data in the database, the metadata set recording the expression information is started, so as to select data that meets the operation requirement to form the target data. In this embodiment, the specific implementation method is as follows:
[0145] Obtaining an operation request submitted for a data table in the database;
[0146] Selecting target expression information in the metadata table of the expression information mapping according to the operation data included in the operation request;
[0147] The target expression information is input into a database optimizer corresponding to the database to obtain a target data table consisting of target data corresponding to the operation request.
[0148] Specifically, the operation request refers to a request to query, delete or add a data table in the database. Based on the operation information contained in the operation request, the target expression information corresponding to the operation request is selected in the metadata table mapped by the expression information, that is, the data table involved in the operation request is determined. At this time, the target expression information is input into the database optimizer corresponding to the database, and the data corresponding to the operation request can be filtered out from the data tables contained in the data table set to form the target data, thereby completing the operation processing of the data table.
[0149] In practical applications, the database optimizer may be a Cascades optimizer. The optimization process of the Cascades optimizer specifically refers to: first, converting the abstract syntax tree into an initial logical execution plan, that is, determining the operator tree represented by the logical plan; secondly, splitting the operator tree equivalently into table connections, thereby obtaining the initial input of the Cascades optimizer. At this time, the search process of the Cascades optimizer is divided into two stages. The first stage is exploration, in which the entire table connection is continuously traversed, all conversion rules are applied, new table connections are generated, and it is continuously iterated until no new table connection is generated; the second stage is implementation, in which the Cascades optimizer searches for a physical execution plan that meets the requirements for each table connection by applying corresponding implementation rules to the table connection; finally, after the second stage, the Cascades optimizer will generate a target physical execution plan, and the optimization process ends here. The target physical execution plan is handed over to subsequent modules for processing to obtain a target data table that meets the operation requirements.
[0150] Furthermore, in the process of selecting the target expression information in the metadata table, since different operation requests involve different numbers of data tables, different methods can be used to determine the target expression information. In this embodiment, the specific implementation method is as follows:
[0151] Parsing the operation request to obtain the operation data;
[0152] Determining, according to the operation data, whether the number of data tables associated with the operation request is greater than a preset number threshold;
[0153] If yes, in the metadata table, selecting at least two sub-expression information corresponding to the target connection group associated with the operation request to form the target expression information;
[0154] If not, in the metadata table, the sub-expression information corresponding to the target connection group associated with the operation request is selected as the target expression information.
[0155] Specifically, the operation request is first parsed to obtain the operation data, and then it is determined whether the number of data tables associated with the operation request is greater than a preset number threshold according to the operation data; if so, it indicates that the operation request involves too many data tables, so multiple data tables are required to obtain the target data table corresponding to the operation request, then at this time, at least two sub-expression information corresponding to the target connection group associated with the operation request can be selected in the metadata table to form the target expression information;
[0156] If not, indicating that the operation request involves fewer data tables, in the metadata table, the sub-expression information corresponding to the target connection group associated with the operation request is selected as the target expression information.
[0157] Continuing with the above example, when the operation request submitted for obtaining the data table in the database is to query the address of user A, the data tables involved in the database are determined to be data table O and data table P according to the query request, and the number of data tables is determined to be less than the preset number threshold, then the expression information corresponding to the connection group composed of data table O and data table P can be used as the target expression information, and the expression information can be input into the Cascades optimizer to obtain a data table containing user A's name, address (OS) and city (LC), and the data table can be used as the query result of the query request.
[0158] In summary, by recording all expression information in the metadata table of the database, it is possible to quickly and accurately obtain the target expression information corresponding to the operation request, thereby improving the query efficiency for the data in the database.
[0159] The data processing method provided by the present application obtains attribute information of a data table in a database, selects data tables with association relationships from the data table to form a data table set, constructs a table connection corresponding to the data table set according to the association relationships between the data tables in the data table set, and then divides the table connection into at least one connection group to generate expression information for data operations on the database. By constructing the table connection, the data tables with association relationships in the database are bound and the table connection is divided, so that more expression information can be quickly enumerated, thereby achieving better integration with the database optimizer, supporting more database optimization rules, and further improving the data processing efficiency of the database.
[0160] The following combination Figure 5 , taking the application of the data processing method provided by this application in a query scenario as an example, the data processing method is further described. Figure 5 A processing flow chart of a data processing method applied to a query scenario provided by an embodiment of the present application is shown, which specifically includes the following steps:
[0161] Step S502: Acquire attribute information of a data table in a database.
[0162] Specifically, in order to improve the enumeration of the connection order of data tables in the database, thereby realizing optimization support for the optimizer in the database, table connections will be constructed based on the data tables with associated relationships in the database, and the table connections will be split to realize the enumeration of the connection order, thereby generating all the expression information corresponding to the database, which is convenient for data query processing.
[0163] Step S504: extract primary key information from the attribute information.
[0164] Step S506, filtering out data tables with the same primary key information from the data tables to form a data table set.
[0165] Step S508: taking the data tables in the data table set as nodes and constructing table connections based on the primary key information of the data tables in the data table set.
[0166] Step S510: convert the table connection to obtain an intermediate table connection.
[0167] Step S512: determine the intermediate data table included in the intermediate table connection.
[0168] Step S514: split the intermediate table connections based on the association relationship between the intermediate data tables to obtain at least one connection group.
[0169] In practical applications, table joins can be converted into multiple intermediate table joins. Since there are cross-relationships between data tables, multiple intermediate table joins are required to reflect the complex connection relationships in the table joins, thereby realizing the full connection relationship of the table joins through multiple intermediate table joins.
[0170] See also Figure 3 As shown in the figure, the table connection constructed for the data tables S1-S6, in order to facilitate the segmentation process, the table connection will be converted, and the converted intermediate table connection is as follows Figure 4 (a) and Figure 4 As shown in (b) in the figure, based on this, when determining Figure 4 After the intermediate table connection shown in (a) is completed, the distribution of data tables S1-S6 in the intermediate table connection is determined, and then the intermediate table connection is split. The split processing position is the association relationship between data table S2 and the data table corresponding to the previous node. At this time, the connection group obtained is composed of a sub-connection including data table S2 and a sub-connection including data tables S1 and S3-S6, which is used for the subsequent generation of expression information of the database to which data tables S1-S6 belong.
[0171] Step S516: determine the first sub-connection and the second sub-connection included in the connection group.
[0172] Step S518: Determine the association relationship between the first sub-connection and the second sub-connection by reading equivalent data of the connection group.
[0173] Step S520: generating an expression string for performing data operations on a database corresponding to the data group according to the association relationship between the first sub-connection and the second sub-connection.
[0174] In actual applications, in the process of splitting the table connection, it will be split into the first sub-connection and the second sub-connection, so the connection group is composed of the first sub-connection and the second sub-connection. At this time, by reading the equivalent data of the connection group, the association relationship between the first sub-connection and the second sub-connection can be determined; finally, according to the association relationship between the first sub-connection and the second sub-connection, the expression string of the database can be generated.
[0175] In specific implementation, equivalent data specifically refers to the case where the connection conditions corresponding to the association relationship between the data table included in the first sub-connection and the data table included in the second sub-connection are equal, and it is represented as equivalent data. At this time, the association relationship between the first sub-connection and the second sub-connection can be determined by the equivalent data, so that an expression string for the connection group can be generated to be recorded in the database, and then the data in the data table included in the first sub-connection and the data table included in the second sub-connection can be queried according to the expression string.
[0176] In summary, by acquiring the attribute information of the data tables in the database, data tables with association relationships are screened out from the data tables to form a data table set, and according to the association relationships between the data tables in the data table set, a table connection corresponding to the data table set is constructed, and then the table connection is divided into at least one connection group, and expression information for data operations on the database is generated. By constructing the table connection, the data tables with association relationships in the database are bound and the table connection is divided, so that more expression information can be quickly enumerated, so as to achieve better integration with the database optimizer, support more database optimization rules, and further improve the data processing efficiency of the database.
[0177] Corresponding to the above method embodiment, the present application also provides a data processing device embodiment, Figure 6 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present application. Figure 6 As shown, the device comprises:
[0178] The acquisition module 602 is configured to acquire attribute information of a data table in a database;
[0179] A composition module 604 is configured to select data tables with association relationships from the data tables according to the attribute information to form a data table set;
[0180] A construction module 606 is configured to construct a table connection of the data tables in the data table set according to the association relationship between the data tables in the data table set;
[0181] The generating module 608 is configured to divide the table connection into at least one connection group, and generate expression information for performing data operations on the database based on the connection group.
[0182] In an optional embodiment, the component module 604 includes:
[0183] an extraction unit, configured to extract primary key information from the attribute information;
[0184] The screening unit is configured to screen out the data tables with the same primary key information from the data tables to form the data table set.
[0185] In an optional embodiment, the construction module 606 is further configured to:
[0186] The data tables in the data table set are used as nodes, and the table connection is constructed based on the primary key information of the data tables in the data table set.
[0187] In an optional embodiment, the generating module 608 includes:
[0188] a conversion unit, configured to convert the table connection to obtain an intermediate table connection;
[0189] The segmentation unit is configured to perform segmentation processing on the intermediate table connection to obtain at least one of the connection groups.
[0190] In an optional embodiment, the segmentation unit includes:
[0191] A determination subunit configured to determine an intermediate data table included in the intermediate table connection;
[0192] The segmentation subunit is configured to perform segmentation processing on the intermediate table connection based on the association relationship between the intermediate data tables to obtain at least one of the connection groups.
[0193] In an optional embodiment, the construction module 606 includes:
[0194] A correlation determination unit, configured to determine the correlation between the data tables in the data table set;
[0195] A connection network building unit is configured to build a connection network of data tables in the data table set according to the association relationship;
[0196] The connection network conversion unit is configured to use the connection tree obtained after the connection network is converted as the table connection.
[0197] In an optional embodiment, the generating module 608 includes:
[0198] A sub-connection determining unit, configured to determine a first sub-connection and a second sub-connection included in the connection group;
[0199] a reading equivalent data unit, configured to determine an association relationship between the first sub-connection and the second sub-connection by reading equivalent data of the connection group;
[0200] The expression information generating unit is configured to generate the expression information according to the association relationship between the first sub-connection and the second sub-connection.
[0201] In an optional embodiment, the data processing device further includes:
[0202] An operation request acquisition module is configured to acquire an operation request submitted for a data table in the database;
[0203] A selection module is configured to select target expression information in the metadata table mapped by the expression information according to the operation data included in the operation request;
[0204] The input module is configured to input the target expression information into a database optimizer corresponding to the database, and obtain a target data table composed of target data corresponding to the operation request.
[0205] In an optional embodiment, the selection module includes:
[0206] A parsing unit, configured to parse the operation request to obtain the operation data;
[0207] a judging unit configured to judge, according to the operation data, whether the number of data tables associated with the operation request is greater than a preset number threshold;
[0208] If yes, run a first selection unit, wherein the first selection unit is configured to select, in the metadata table, at least two sub-expression information corresponding to the target connection group associated with the operation request to form the target expression information;
[0209] If not, a second selection unit is executed, wherein the second selection unit is configured to select, in the metadata table, sub-expression information corresponding to the target connection group associated with the operation request as the target expression information.
[0210] The data processing device provided by the present application obtains attribute information of a data table in a database, selects data tables with association relationships from the data table to form a data table set, constructs a table connection corresponding to the data table set according to the association relationships between the data tables in the data table set, and then divides the table connection into at least one connection group, generates expression information for performing data operations on the database, and realizes binding of data tables with association relationships in the database by constructing the table connection and dividing the table connection, so that more expression information can be quickly enumerated, thereby achieving better integration with the database optimizer, supporting more database optimization rules, and further improving the data processing efficiency of the database.
[0211] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.
[0212] The following is an embodiment of a database operating system provided by this application:
[0213] Figure 7 A schematic diagram of the structure of a database operating system provided according to an embodiment of the present application is shown, wherein the database operating system 700 includes a client 702, a computing node 704, and a storage node 706;
[0214] The client 702 is used to receive a query instruction input by a user;
[0215] The computing node 704 is used to parse the query instruction to obtain expression information for operating the database; determine the target connection group based on the expression information, and read the primary key information corresponding to the target connection group;
[0216] The storage node 706 is used to query the data table corresponding to the primary key information in the database, and read the target data corresponding to the query instruction in the data table and return it to the client 702.
[0217] In actual applications, when querying data in a database, it is necessary to enumerate all the connection sequences involved in the data tables in the database, and then select the final connection sequence based on the losses of each enumerated connection sequence, and then select the target connection sequence corresponding to the query instruction from each connection sequence to read the data, and output it as the query result. However, in the enumeration process, it is usually implemented based on the commutative law and the associative law, and each call can only generate one or two new sequences, which is relatively inefficient, resulting in a slow return of data to the user.
[0218] The database operating system provided by the present application, in order to improve the enumeration efficiency and thus improve the processing efficiency of the database when reading and writing data, will parse the query instructions submitted by the user to obtain expression information, and then further process the expression information through the computing node to determine the target connection group and the primary key information corresponding to the target connection group, and finally query the data table according to the primary key information through the storage node in the database, and read the target data corresponding to the query instruction in the data table and return it to the client, so as to realize the method of determining the target connection group through the computing node, quickly determine the primary key information pointed to by the query instruction, and then combine with the storage node to quickly complete the reading of the target data, which not only improves the efficiency of reading and writing data in the database, but also improves the user experience.
[0219] In specific implementation, the database operating system specifically refers to an operating system that can perform data query according to the user's query instructions and feedback target data to the user. The database operating system includes a client 702, and the client 702 specifically represents a front-end interface for connecting to the user. Users with query needs can submit data query instructions by operating the controls integrated in the client.
[0220] At this time, the database operating system forwards the query instruction uploaded by the client to the computing node 704 for processing. The computing node 704 specifically refers to the node that parses the instruction and determines the direction of the query instruction. After receiving the query instruction, the computing node 704 will parse the query instruction to obtain the expression information corresponding to the query data, and then determine the target connection group according to the expression information, and read the primary key information corresponding to the target connection group, so that the specific location of the data to be queried can be determined according to the query instruction. Finally, the storage node 706 queries the data table in the database according to the primary key information, and reads the target data corresponding to the query instruction from the data table and returns it to the client 702. The storage node 706 specifically refers to the node that performs data reading and writing operations, which is mainly connected to the database. While completing data reading and writing operations, the metadata table of the database can be updated to achieve the purpose of monitoring the database.
[0221] In this embodiment, the computing node 706 is further configured as follows:
[0222] The expression information is input into a database optimizer corresponding to the database to obtain the target connection group.
[0223] Specifically, in the process of determining the target connection group, the computing node 706 requires data format conversion to query the database through expression information, and the database has enumerated a large number of connection groups. Therefore, in order to accurately output the target data that meets the query instructions, the target connection group can be determined through a database optimizer. The database optimizer can determine the user's query requirements based on the expression information, and determine the target connection group corresponding to the query requirements, and then complete the data query operation based on the target connection group.
[0224] Further, the expression information specifically refers to an expression string that can perform data operations on the database, which can be applied to the database optimizer to locate the target data. The expression information actually associates two data tables with an associated relationship, thereby determining the target data of the query instruction. In this embodiment, the expression information is created in the following way:
[0225] Determine a first sub-connection and a second sub-connection included in the target connection group;
[0226] Determine an association relationship between the first sub-connection and the second sub-connection by reading equivalent data of the target connection group;
[0227] The expression information is generated according to the association relationship between the first sub-connection and the second sub-connection.
[0228] In specific implementation, the target connection group is obtained by splitting the table connection. The process of table connection splitting can refer to the corresponding description content in the above embodiment. This embodiment will not be described in detail here. Based on this, a connection group consisting of a first sub-connection and a second sub-connection can be obtained at this time, and then the equivalent data of the connection group is read to determine the association relationship between the first sub-connection and the second sub-connection; finally, according to the association relationship between the first sub-connection and the second sub-connection, the expression information of the database can be generated.
[0229] Among them, the equivalent data specifically refers to the case where the connection conditions corresponding to the association relationship between the data table included in the first sub-connection and the data table included in the second sub-connection are equal, and it is represented as equivalent data. At this time, the association relationship between the first sub-connection and the second sub-connection can be determined by the equivalent data, so that expression information for the connection group can be generated to be recorded in the database, and subsequently the data in the data table included in the first sub-connection and the data table included in the second sub-connection can be queried according to the expression information.
[0230] After generating the expression information corresponding to all the connection groups obtained by the table connection segmentation processing, all the expression information can be recorded in the metadata table of the database, so that during the application process, the expression information corresponding to the query request can be directly extracted, thereby obtaining a data table composed of data corresponding to the expression information.
[0231] In this embodiment, the storage node 706 is further configured as:
[0232] Obtaining a data key value by parsing the expression information;
[0233] Read the data table based on the data key value, and determine the target data according to the reading result;
[0234] A target data table is formed according to the target data, and the target data table is returned to the client.
[0235] Specifically, after the storage node 706 queries the data table corresponding to the primary key information, it will need to extract the target data from the data table and return it to the user. Since the data stored in the data table may be relatively large, and the data stored in the data table are mostly stored in the same data table in the form of similar data, in order to accurately determine the data that the user needs to query, the data key value can be determined by expressing information, and the data table can be read by the key value to accurately determine the target data. Thereafter, the target data can be composed of a target data table and returned to the client. This not only returns data that meets the query requirements to the user, but also structures the target data for user use or viewing, further improving the user experience.
[0236] For example, when the query instruction obtained from user S is to query the address of user A, the computing node parses the query instruction and determines that the expression information is the expression information composed of data table O and data table P (data table O stores the attribute information of user A, and data table P stores the external information of user A). At this time, the target connection group can be determined, and the primary key information O_u of data table O and the primary key information P_u of data table P can be determined by reading the target connection group. At this time, the storage node of the database operating system can determine data table O and data table P according to the primary key information O_u and P_u, and query data table O and data table P to determine the data of user A's name, address and city. Finally, the data of user A's name, address and city are combined into a new data table and returned to the client, so that the relevant information of user A can be displayed to user S.
[0237] The database operating system provided by the present application, in order to improve the enumeration efficiency and thus improve the processing efficiency of the database when reading and writing data, will parse the query instructions submitted by the user to obtain expression information, and then further process the expression information through the computing node to determine the target connection group and the primary key information corresponding to the target connection group, and finally query the data table according to the primary key information through the storage node in the database, and read the target data corresponding to the query instruction in the data table and return it to the client, so as to realize the method of determining the target connection group through the computing node, quickly determine the primary key information pointed to by the query instruction, and then combine with the storage node to quickly complete the reading of the target data, which not only improves the efficiency of reading and writing data in the database, but also improves the user experience.
[0238] Figure 8 The structure block diagram of a computing device 800 provided according to an embodiment of the present application is shown. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and the database 850 is used to store data.
[0239] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.
[0240] In one embodiment of the present application, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.
[0241] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 800 may also be a mobile or stationary server.
[0242] The processor 820 is used to execute the following computer executable instructions:
[0243] Get the attribute information of the data table in the database;
[0244] According to the attribute information, data tables with association relationships are screened out from the data tables to form a data table set;
[0245] Constructing table connections of the data tables in the data table set according to the association relationships between the data tables in the data table set;
[0246] The table connection is divided into at least one connection group, and expression information for performing data operations on the database is generated based on the connection group.
[0247] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data processing method.
[0248] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which are used when executed by a processor to:
[0249] Get the attribute information of the data table in the database;
[0250] According to the attribute information, data tables with association relationships are screened out from the data tables to form a data table set;
[0251] Constructing table connections of the data tables in the data table set according to the association relationships between the data tables in the data table set;
[0252] The table connection is divided into at least one connection group, and expression information for performing data operations on the database is generated based on the connection group.
[0253] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above data processing method.
[0254] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0255] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0256] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0257] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0258] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the present application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can understand and use the present application well. The present application is only limited by the claims and their full scope and equivalents.
Claims
1. A data processing method, include: Get the attribute information of the data table in the database; According to the attribute information, data tables with association relationships are screened out from the data tables to form a data table set; Constructing table connections of the data tables in the data table set according to the association relationships between the data tables in the data table set; Converting the table connection to obtain an intermediate table connection; Determine the intermediate data table included in the intermediate table connection; segment the intermediate table connection based on the association relationship between the intermediate data tables to obtain at least one connection group, and generate expression information for performing data operations on the database based on the connection group, wherein the intermediate table connection is a connection structure obtained after converting the table connection.
2. The data processing method according to claim 1, wherein according to the attribute information, data tables with association relationships are selected from the data tables to form a data table set, include: Extracting primary key information from the attribute information; The data tables having the same primary key information are screened out from the data tables to form the data table set.
3. The data processing method according to claim 2, wherein the table connection of the data tables in the data table set is constructed according to the association relationship between the data tables in the data table set, include: The data tables in the data table set are used as nodes, and the table connection is constructed based on the primary key information of the data tables in the data table set.
4. The data processing method according to claim 1, wherein the table connection of the data tables in the data table set is constructed according to the association relationship between the data tables in the data table set, include: Determine the association relationship between the data tables in the data table set; Constructing a connection network of data tables in the data table set according to the association relationship; The connection tree obtained after converting the connection network is used as the table connection.
5. The data processing method according to claim 1, wherein the expression information for performing data operations on the database is generated based on the connection group. include: Determine a first sub-connection and a second sub-connection included in the connection group; Determine an association relationship between the first sub-connection and the second sub-connection by reading equivalent data of the connection group; The expression information is generated according to the association relationship between the first sub-connection and the second sub-connection.
6. The data processing method according to claim 1, after the step of generating expression information for performing data operations on the database based on the connection group is executed, include: Obtaining an operation request submitted for a data table in the database; Selecting target expression information in the metadata table of the expression information mapping according to the operation data included in the operation request; The target expression information is input into a database optimizer corresponding to the database to obtain a target data table consisting of target data corresponding to the operation request.
7. The data processing method according to claim 6, wherein the target expression information is selected in the metadata table of the expression information mapping according to the operation data included in the operation request, include: Parsing the operation request to obtain the operation data; Determining, according to the operation data, whether the number of data tables associated with the operation request is greater than a preset number threshold; If yes, in the metadata table, selecting at least two sub-expression information corresponding to the target connection group associated with the operation request to form the target expression information; If not, in the metadata table, the sub-expression information corresponding to the target connection group associated with the operation request is selected as the target expression information.
8. A data processing device, include: An acquisition module is configured to acquire attribute information of a data table in a database; A composition module is configured to filter out data tables with association relationships from the data tables according to the attribute information to form a data table set; A construction module, configured to construct table connections of data tables in the data table set according to association relationships between data tables in the data table set; A generating module is configured to convert the table connection to obtain an intermediate table connection; Determine the intermediate data table included in the intermediate table connection; The intermediate table connection is segmented based on the association relationship between the intermediate data tables to obtain at least one connection group, and expression information for performing data operations on the database is generated based on the connection group, wherein the intermediate table connection is a connection structure obtained after converting the table connection.
9. A database operating system, include: Client, computing node, storage node; The client is used to receive a query instruction input by a user; The computing node is used to parse the query instruction to obtain expression information for operating on the database; wherein the expression information is obtained by the method according to any one of claims 1 to 7; determine the target connection group based on the expression information, and read the primary key information corresponding to the target connection group; The storage node is used to query the data table corresponding to the primary key information in the database, and read the target data corresponding to the query instruction in the data table and return it to the client.
10. The database operating system according to claim 9, wherein the target connection group is determined based on the expression information. include: The expression information is input into a database optimizer corresponding to the database to obtain the target connection group.
11. The database operating system according to claim 9, wherein the expression information is created in the following manner: Determine a first sub-connection and a second sub-connection included in the target connection group; Determine an association relationship between the first sub-connection and the second sub-connection by reading equivalent data of the target connection group; The expression information is generated according to the association relationship between the first sub-connection and the second sub-connection.
12. The database operating system according to claim 9, wherein the target data corresponding to the query instruction is read from the data table and returned to the client. include: Obtaining a data key value by parsing the expression information; Read the data table based on the data key value, and determine the target data according to the reading result; A target data table is formed according to the target data, and the target data table is returned to the client.
13. A computing device, include: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions: Get the attribute information of the data table in the database; According to the attribute information, data tables with association relationships are screened out from the data tables to form a data table set; Constructing table connections of the data tables in the data table set according to the association relationships between the data tables in the data table set; Converting the table connection to obtain an intermediate table connection; Determine the intermediate data table included in the intermediate table connection; segment the intermediate table connection based on the association relationship between the intermediate data tables to obtain at least one connection group, and generate expression information for performing data operations on the database based on the connection group, wherein the intermediate table connection is a connection structure obtained after converting the table connection.
14. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 7.
