Database table processing method and device, database system and related equipment

Abstract

The embodiment of the invention provides a database table processing method and device, a database system and related equipment. According to the method, the database operation statement for adding the logic field in the logic table is received, the corresponding physical table structure information is automatically analyzed and generated, and then the database operation statement for newly establishing the physical table is established and executed, so that automatic mapping from logic table change to physical table establishment is realized. According to the scheme, the complexity of an underlying physical storage structure is shielded, a user can complete field expansion only by writing a database operation statement (such as a logic DDL statement) for a logic table without manually writing a database operation statement (such as a physical DDL statement) for a physical table, the field expansion process is simplified, and the field expansion efficiency is improved.

Description

Technical Field

[0001] This specification relates to the field of database technology, and in particular to a database table processing method, apparatus, database system and related equipment. Background Technology

[0002] As service demands continue to evolve and expand, the ability to expand database table fields has become a key element in supporting system flexibility and maintainability.

[0003] In traditional database field extension schemes, whether using the "extended field" or "extended table" model, physical DDL (Data Definition Language) statements must be manually written to directly manipulate the underlying physical storage structure. This approach suffers from problems such as operational complexity, error susceptibility, and high maintenance costs. Summary of the Invention

[0004] This specification provides a database table processing method, apparatus, database system, and related equipment, which provide a new field expansion scheme.

[0005] This specification provides a database table processing method, comprising: obtaining a first database operation statement, the first database operation statement being used to instruct the addition of a first logical field to a first logical table; determining the table structure information of a first physical table to be created based on the parsing result of the first database operation statement, the table structure information of the first physical table including the field name of the first physical field corresponding to the first logical field; generating a second database operation statement based on the table structure information of the first physical table; and executing the second database operation statement to create a first physical table including the first physical field for the first logical table in the database.

[0006] This specification also provides a database table processing method, comprising: obtaining a first query statement, the first query statement being used to instruct the execution of a query operation on a target logical table; determining the table name of the target logical table and the field name of the logical field to be queried in the target logical table based on the parsing result of the first query statement; searching in a mapping relationship table for the table name of a target physical table and the field name of the physical field to be queried, which have a mapping relationship with the table name of the target logical table and the field name of the logical field to be queried; generating a second query statement based on the table name of the target physical table and the field name of the physical field to be queried; and executing the second query statement to query the data of the physical field to be queried in the target physical table.

[0007] This specification also provides a database system, including: an interaction layer, a logical table abstraction layer, and a physical storage layer; the interaction layer is used to obtain a first database operation statement and send it to the logical table abstraction layer, the first database operation statement being used to instruct the addition of a first logical field in a first logical table; the logical table abstraction layer is used to determine the table structure information of the first physical table to be created based on the parsing result of the first database operation statement, the table structure information of the first physical table including the field name of the first physical field corresponding to the first logical field; generate a second database operation statement based on the table structure information of the first physical table; execute the second database operation statement to create a first physical table including the first physical field in the database for the first logical table; and the physical storage layer is used to persistently store the data of the first physical table.

[0008] This specification also provides an electronic device, including: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute one or more computer instructions for: performing steps in a database table processing method.

[0009] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in a database table processing method.

[0010] This specification also provides a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the steps in the database table processing method.

[0011] In this embodiment, by receiving database operation statements that add logical fields to a logical table, the system automatically parses and generates corresponding physical table structure information, and then creates and executes database operation statements for creating a new physical table. This achieves automated mapping from logical table changes to physical table creation. This solution shields the complexity of the underlying physical storage structure. Users only need to write database operation statements for logical tables (such as logical DDL statements), without manually writing database operation statements for physical tables (such as physical DDL statements), to complete field expansion, simplifying the field expansion process and improving field expansion efficiency. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the structure of an exemplary database system provided in the embodiments of this specification; Figure 2This is a flowchart of a database table processing method provided in the embodiments of this specification; Figure 3 This is a flowchart of another database table processing method provided in the embodiments of this specification; Figure 4 This is a flowchart of another database table processing method provided in the embodiments of this specification; Figure 5 This is a flowchart of another database table processing method provided in the embodiments of this specification; Figure 6 This is a schematic diagram of the structure of a database table processing device provided in the embodiments of this specification; Figure 7 This is a schematic diagram of the structure of an exemplary electronic device provided in the embodiments of this specification. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0014] It should be noted that, in the cases involving user information in the embodiments of this specification, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this specification are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0015] The various models (including but not limited to language models or large models) mentioned in this specification comply with relevant laws and standards.

[0016] The following is a description of some of the terms used in this instruction manual: Database tables are generally conceptually divided into logical tables and physical tables. Logical tables primarily describe the logical structure of data, shielding it from the complexity of underlying storage, and serve as the data interface for direct manipulation by users and applications. Logical tables can be mapped to physical tables, which are the database tables that actually store data and have physical storage space.

[0017] A logical table is a logically abstract data structure in a database system, oriented towards users or applications. It defines the table's name, fields (columns), data types, primary keys, constraints, etc., shielding the complexity of the underlying storage and providing a unified and concise data access interface for applications. Logical tables are the objects that developers directly manipulate when writing various database operation statements, such as SQL statements.

[0018] A physical table is the actual physical entity in which data is stored within the underlying database instance. It is the concrete implementation of a logical table, carrying the actual data records and directly affecting data storage efficiency and access performance.

[0019] A field, also often called a column or attribute, defines a specific attribute or characteristic of data. Here, fields in a logical table are referred to as logical fields, and fields in a physical table are referred to as physical fields.

[0020] Database operation statements are instructions used to perform definition, manipulation, query, control, or transaction management operations on data or database objects in a database. They are typically written in SQL (Structured Query Language). Database operation statements include, but are not limited to: Data Definition Language (DDL) statements, Data Manipulation Language (DML) statements, Data Query Language (DQL) statements, Data Control Language (DCL) statements, and Transaction Control Language (TCL) statements, etc.

[0021] The following introduces several existing database table field expansion schemes: Extended field solution: This involves manually writing DDL statements such as ALTER TABLE ADD COLUMN to directly modify the structure of an existing physical table, thereby adding a new field. However, this solution requires users to have a deep understanding of the physical storage layout (such as sharding rules and table naming conventions), making it technically challenging. When performing DDL operations on large-scale tables at the terabyte (TB) level, the database typically needs to apply a table-level exclusive lock, causing read and write operations to be blocked. The locking time for the physical table can last for several minutes or even hours, leading to service interruptions or severe performance degradation. Furthermore, when a logical table is hosted by multiple physical tables (such as in a sharded database scenario), global field extension cannot be achieved through a single DDL statement.

[0022] Extended Table Solution: To avoid frequent changes to the main table structure, a "main table + extended table" architecture is adopted. Developers create an independent physical table (extended table) to store newly added fields, which are linked to the main table via foreign keys. Queries require manually writing JOIN statements to merge data from the main and extended tables. The disadvantages of this solution are: users must manually design the extended table structure and manually write DDL statements such as CREATE TABLED to create the physical table, resulting in high maintenance overhead; manually writing subsequent JOIN statements leads to high query complexity, verbose SQL statements, and a high risk of errors; developers need to understand the physical distribution of fields, which is not transparent to developers and increases their cognitive burden.

[0023] Existing field extension solutions require users to have a deep understanding of the physical storage structure, which is a high-barrier-to-entry process and heavily reliant on manual intervention; users who manually write physical DDL statements are prone to errors and are difficult to maintain.

[0024] In some embodiments of this specification, a solution is provided that automatically parses and generates corresponding physical table structure information by receiving database operation statements that add logical fields to a logical table, and then creates and executes database operation statements for creating a new physical table, thus achieving automated mapping from logical table changes to physical table creation. This solution shields the complexity of the underlying physical storage structure. Users only need to write database operation statements for logical tables (such as logical DDL statements), without having to manually write database operation statements for physical tables (such as physical DDL statements) to complete field expansion, simplifying the field expansion process and improving field expansion efficiency.

[0025] The technical solutions provided by the various embodiments of this specification are described in detail below with reference to the accompanying drawings. These specific embodiments can be combined with each other, and for the same or similar concepts or processes, further details will not be repeated in some embodiments.

[0026] Figure 1 This is a schematic diagram of the structure of an exemplary database system provided in the embodiments of this specification. See also... Figure 1 The database system may include: an interaction layer, a logical table abstraction layer, and a physical storage layer.

[0027] The interaction layer is the software layer in the database system responsible for exchanging information with users or external applications. It primarily receives database operation requests from users or external applications, translates these requests into database operation statements through a unified interface, and returns the execution results of these statements to the user or external application. This unified interface is implemented based on one or more standard protocol interfaces, ensuring that different users or applications can access the database system in a standardized manner.

[0028] The logical table abstraction layer is a software layer that manages logical tables. It mainly establishes a mapping relationship between logical tables and physical tables, and transforms database operation statements for logical tables into database operation statements that can be executed in the physical storage layer, thereby achieving decoupling between logical tables and physical tables.

[0029] Optionally, the logical table abstraction layer may include a query optimizer, storage manager, metadata service, and transaction manager, etc.

[0030] The query optimizer primarily provides query optimization services, responsible for converting database operation statements targeting logical tables into database operation statements adapted to physical tables, and converting database operation statements targeting physical tables into physical execution plans.

[0031] Metadata service is used to manage mapping tables to support the conversion of operations on logical tables to operations on physical tables. The mapping tables include the mapping relationship between the table structure information of logical tables and the table structure information of at least one existing physical table.

[0032] The storage manager is primarily used for storing and managing data in physical tables. It is the core module in a database system responsible for organizing and accessing physical data. Through data page management, buffer pool mechanisms, I / O scheduling, index structure maintenance, and space allocation strategies, it achieves efficient storage, fast retrieval, and reliable persistence of physical table data.

[0033] The transaction manager is responsible for ensuring the atomicity, consistency, isolation, and durability of transactions, which are the core components for ensuring ACID properties.

[0034] The physical storage layer is the underlying support module in the database system responsible for the physical storage and access of data. It is mainly used for persistent storage of data in physical tables.

[0035] Optionally, the database system may also include a monitoring layer, which can collect and analyze various status data of the database system. For example, the monitoring layer can monitor the transaction logs generated by the transaction manager in real time, and analyze the transaction logs to obtain the commit status, rollback status, and abnormal operation information of transactions in real time. Furthermore, the monitoring layer is also used to collect runtime performance metrics of the database system, such as, but not limited to, queries per second (QPS), query execution time, or query success rate.

[0036] The following is a brief introduction to the working principle of a database system during the field expansion phase: During the field expansion phase, firstly, the interaction layer receives requests from users or applications to add logical fields to a logical table through a unified interface. Secondly, the interaction layer transforms these requests into logical field addition statements for the logical table through the unified interface. Next, the logical table abstraction layer transforms these logical field addition statements into physical field addition statements or physical table creation statements for existing physical tables. The physical table creation statement requests the creation of a new physical table, which includes the new physical field corresponding to the logical field to be added. Finally, the logical table abstraction layer executes the physical field addition statement or the physical table creation statement. This results in either the addition of the new physical field corresponding to the logical field to be added to an existing physical table stored in the physical storage layer, or the addition of a new physical table to the physical storage layer, which includes the new physical field corresponding to the logical field to be added.

[0037] The field extension solution provided in this manual allows adding new fields to be done simply by obtaining the logical table-oriented DDL statement (such as ALTER LOGICAL TABLE ... ADD COLUMN), which automatically completes the changes to the underlying physical storage structure (including the addition of physical fields or the creation of new physical tables). Developers do not need to manually write physical DDL statements such as ALTER TABLE and CREATE TABLE, simplifying the field extension process and improving development efficiency.

[0038] For example, when expanding fields using the field expansion scheme provided in this manual, developers only need the following logical DDL statement for adding a new field: "ALTER LOGICAL TABLE customers ADD COLUMN satisfaction_score INT". This logical DDL statement indicates that a new integer logical field named satisfaction_score (satisfaction score) will be added to the logical table customers. The database system automatically converts the logical DDL statement for adding a new field into a physical DDL statement for adding a new field, which is "ALTER TABLE customers_loyalty_ext ADD COLUMN satisfaction_score INT". This physical DDL statement indicates that a new integer physical field named satisfaction_score (satisfaction score) will be added to the physical table customers_loyalty_ext.

[0039] In traditional extended table solutions, developers must directly manipulate the physical table, executing a physical DDL statement like: "ALTER TABLE customers_loyalty_ext ADD COLUMN satisfaction_score INT". Furthermore, all SQL queries involving this field must be modified simultaneously, explicitly adding JOIN clauses and field references, making the process cumbersome and error-prone.

[0040] The following is a brief introduction to the working principle of a database system during the query phase: During the data query phase, see Figure 1 As shown in section 1.0, firstly, the interaction layer receives query requests for logical tables from users or applications through a unified interface; then, see... Figure 1 As shown in Figure 1.1, the logical table abstraction layer receives SQL query statements for logical tables sent by the unified interface of the interaction layer; then, see... Figure 1 As shown in section 1.2, the logical table abstraction layer obtains the mapping relationship between logical tables and physical tables from the metadata service, rewrites the SQL query statements for logical tables to obtain the SQL query statements for physical tables; then, see... Figure 1 As shown in Figure 1.3, the logical table abstraction layer can obtain relevant monitoring data from the monitoring layer, and perform query optimization on the SQL query statements of the physical tables based on the monitoring data to obtain the physical execution plan. Next, see... Figure 1 As shown in Figures 1.4, 1.5, and 1.6, the storage manager and transaction manager work together to ensure that query operations adhere to the ACID properties of transactions. The storage manager executes the physical execution plan to perform query operations on physical tables in the physical management layer. For example, querying data in a physical field of physical table 1, physical table 2, or physical table n, where n is a positive integer. The transaction manager provides transaction support during its execution, managing the start, commit, and rollback of transactions.

[0041] The data query solution provided in this manual allows users to directly write query statements for logical tables without needing to be aware of the underlying physical table's storage structure. Through a query rewriting mechanism, logical queries are automatically converted into access operations to the underlying physical tables before execution, without requiring modification of existing query logic. This achieves complete decoupling between logical tables and the physical storage layer, significantly reducing the length of query statements and simplifying the complexity of data queries.

[0042] For example, a user-input query statement that directly targets a logical table might look like this: SELECT customer_id customer_name email loyalty_points satisfaction_score total_purchase_amount FROM customers WHERE loyalty_points > 1000 AND (satisfaction_score < 3 OR satisfaction_score IS NULL) ORDER BY total_purchase_amount DESC; The above query statement is based on a unified logical table called "customers". Users do not need to be aware of the underlying physical table structure or explicitly specify JOIN relationships. Internally, the system automatically identifies the physical table corresponding to each field (such as customers_main, customers_loyalty_ext, and customers_purchase_ext) through metadata management and query rewriting mechanisms, and generates multi-table join query statements for the physical tables. The whole process is transparent to the user, who does not need to care about data distribution, table join conditions, or physical field storage locations.

[0043] In traditional extended table solutions, data is distributed across multiple physical tables, requiring explicit specification of table join logic during queries. For example, the query to retrieve complete customer information would look like this: SELECT cm.customer_id cm.customer_name, cc.email cl.loyalty_points, cl.satisfaction_score cp.total_purchase_amount FROM customers_main cm LEFT JOIN customers_contact_ext cc ON cm.customer_id = cc.customer_id LEFT JOIN customers_loyalty_ext cl ON cm.customer_id = cl.customer_id LEFT JOIN customers_purchase_ext cp ON cm.customer_id = cp.customer_id WHERE cl.loyalty_points > 1000 AND (cl.satisfaction_score < 3 ORcl.satisfaction_score IS NULL) ORDER BY cp.total_purchase_amount DESC; Traditional extended table solutions require manually writing multiple LEFT JOIN and other join query statements during the data query phase, managing table aliases (cm, cc, cl, cp) and join conditions. The SQL query statements are verbose and error-prone, resulting in high complexity. If a new field (such as satisfaction_score) is added, all query statements involving customer data need to be modified, leading to poor scalability and high maintenance costs.

[0044] Figure 1 The database system shown is merely an example, and this description does not limit the specific system architecture of the database system.

[0045] Figure 2 This is a flowchart of a database table processing method provided in the embodiments of this specification, such as... Figure 2 As shown, the method may include the following steps: 202. Obtain the first database operation statement, which is used to instruct the addition of the first logical field to the first logical table.

[0046] Specifically, the first database operation statement submitted by the application or user can be obtained through a unified interface. The first database operation statement can be regarded as a logical DDL statement that adds a new logical field to the logical table. For example, the first database operation statement is: "ALTER LOGICAL TABLE customers ADD COLUMN satisfaction_scoreINT". This logical DDL statement means that an integer logical field named satisfaction_score is added to the logical table customers.

[0047] 204. Based on the parsing results of the first database operation statement, determine the table structure information of the first physical table to be created. The table structure information of the first physical table includes the field name of the first physical field corresponding to the first logical field.

[0048] Specifically, the first database operation statement is parsed to obtain the corresponding AST (Abstract Syntax Tree). By traversing and analyzing this AST, the parsing results related to the first logical table are extracted. The parsing results include, but are not limited to, the following: operation type (ADD COLUMN), logical table name (customers), logical field name (satisfaction_score), data type (INT), default value, constraints, and other additional attributes.

[0049] Based on the parsing results of the first database operation statement, the table structure information of the first physical table to be created is determined. This table structure information guides the creation of the underlying physical tables, ensuring that changes to logical tables are correctly mapped to physical tables. The table structure information of the first physical table includes, but is not limited to: physical table name, primary key field, field name, field type, and field length of the first physical field.

[0050] The physical table name is automatically generated according to a preset naming convention, such as customers_loyalty_ext; Among them, the primary key field (e.g., customer_id) serves as a unique identifier associated with the main physical table (e.g., customers_main) corresponding to the logical table, ensuring data consistency; The field name of the first physical field can be the same as the first logical field (e.g., satisfaction_score), or it can be mapped according to a unified naming convention (e.g., adding a prefix, suffix, or using conversion rules such as camelCase to underscore). Field type: Inherited from the data type of the first logical field. Field length: For variable-length types (such as strings), the length limit defined in the logical definition is retained.

[0051] 206. Generate the second database operation statement based on the table structure information of the first physical table.

[0052] 208. Execute the second database operation statement to create a first physical table in the database that includes the first physical field for the first logical table.

[0053] Specifically, the second database operation statement can be viewed as a physical DDL statement that adds a new physical field to a physical table. For example, the second database operation statement is: "CREATE TABLE customers_experience_ext(customer_idBIGINT PRIMARY KEY, satisfaction_score INT)". This physical DDL statement creates the first physical table, customers_experience_ext, which serves as the actual storage medium for the logical field satisfaction_score. Here, customer_id acts as the primary key, used for efficient association with the main table (i.e., the main physical table) customers_main or other extended tables; satisfaction_score is the corresponding first physical field, and its name can be consistent with the first logical field.

[0054] After generating the second database operation statement, the second database operation statement is also executed. In this way, a corresponding first physical table is created in the underlying database for the first logical table (such as customers). This first physical table is used to carry the newly added first physical field (such as satisfaction_score) and serves as the persistent storage unit for the extended data of the first logical table.

[0055] In some optional embodiments, the above method further includes: adding a mapping relationship between the table structure information of the first logical table and the table structure information of the first physical table to the mapping relationship table to support subsequent query rewriting and metadata traceability.

[0056] Specifically, a mapping table can be maintained in the metadata service. Through the mapping relationships in the mapping table, queries targeting logical tables can be automatically rewritten into queries involving at least one physical table during query execution. For example, a single-table query on the logical table "customers" can be automatically converted into a JOIN query on multiple physical tables such as "customers_main" and "customers_experience_ext".

[0057] Optionally, the table structure information of the first logical table may include, but is not limited to, the table name of the first logical table, the field name of the first logical field, the field data type, and the primary key definition. The table structure information of the first physical table may include, but is not limited to, the table name of the first physical table, the field name of the first physical field, the field data type, the primary key, the foreign key, and the join key.

[0058] The field extension solution provided in this manual automatically parses and generates the corresponding physical table structure information by receiving database operation statements that add logical fields to a logical table. It then creates and executes the database operation statements used to create the new physical table, achieving automated mapping from logical table changes to physical table creation. This solution shields the user from the complexity of the underlying physical storage structure. Users only need to write database operation statements for logical tables (such as logical DDL statements), eliminating the need to manually write database operation statements for physical tables (such as physical DDL statements) to complete field extensions. This simplifies the field extension process and improves efficiency.

[0059] Figure 3 This is a flowchart of another database table processing method provided in the embodiments of this specification, such as... Figure 3 As shown, the method may include the following steps: 302. Obtain the first database operation statement, which is used to instruct the addition of the first logical field to the first logical table.

[0060] 304. Determine whether at least one second physical table related to the first logical field supports locking. At least one second physical table is at least a part of all existing physical tables corresponding to the first logical table in the database. Execute step 306 or step 312.

[0061] Specifically, after the first logical table has been pre-created, the database already stores the physical table corresponding to the first logical table (i.e., the existing physical table). There are no restrictions on how the at least one second physical table related to the first logical field is identified. For example, it can be selected based on the semantic similarity between the semantic features of the first logical field and the semantic features of the table names of existing physical tables, choosing the one with higher semantic similarity. Another example is determining the second physical table to which the first logical field belongs based on field classification rules, where the field classification rules define the mapping relationship between logical fields and physical tables. Yet another example is identifying related physical tables based on join key consistency. All physical tables using the same join key (such as customer_id) as a primary key or foreign key are considered data carriers belonging to the same logical table. When a new logical field is added, these existing physical tables sharing the same join key are identified as "second physical tables related to the first logical field."

[0062] After identifying at least one second physical table associated with the first logical field, determine whether that second physical table supports locking. For example, if the database instance or storage engine to which the second physical table belongs supports locking, then the second physical table supports locking. Otherwise, the second physical table does not support locking. Alternatively, the attributes of the second physical table can indicate whether locking is supported. Another example is determining whether the second physical table supports locking based on the user permission information of the database user currently performing the operation. Of course, there are no restrictions on the method used to determine whether the second physical table supports locking.

[0063] 306. If at least one second physical table does not support locking, the table structure information of the first physical table to be created is determined based on the parsing result of the first database operation statement. The table structure information of the first physical table includes the field name of the first physical field corresponding to the first logical field.

[0064] 308. Generate the second database operation statement based on the table structure information of the first physical table.

[0065] 310. Execute the second database operation statement to create a first physical table in the database that includes the first physical field for the first logical table.

[0066] Specifically, if at least one second physical table does not support locking, a new physical table can be created to expand the fields, and steps 306 to 308 can be executed.

[0067] 312. If at least one third physical table that supports locking exists in at least one second physical table, then a fourth physical table is selected from at least one third physical table.

[0068] Specifically, if at least one second physical table contains at least one third physical table that supports locking, the table structure of the existing physical table can be modified to expand the fields, and steps 312 to 316 can be executed.

[0069] In practical applications, at least a portion of the third physical tables can be directly selected as the fourth physical table from at least one third physical table. Optionally, the selection of the fourth physical table from at least one third physical table can be achieved by: obtaining model input parameters, wherein the model input parameters include at least one of the following: the data sparsity and / or historical access data of at least one third physical table, the simultaneous access probability and / or semantic similarity between the first logical field and at least one third physical table; and using a machine learning model to select the fourth physical table from at least one third physical table based on the model input parameters.

[0070] The machine learning model, after being trained, can accurately predict the physical table suitable for adding new physical fields. The machine learning model can be a supervised learning model, such as a random forest model or a neural network model.

[0071] The data sparsity of the third physical table can characterize the proportion of non-null fields in the third physical table. The higher the data sparsity, the more suitable the third physical table is as a physical table for adding new fields; the lower the data sparsity, the less suitable the third physical table is as a physical table for adding new fields.

[0072] Among them, the historical access data of the third physical table (such as historical access frequency or historical access count) can represent the access frequency of the third physical table. Prioritize avoiding mapping new fields to physical tables with high access frequency, and instead select physical tables with lower access frequency and lighter load to add new fields.

[0073] The simultaneous access probability between the first logical field and the third physical table represents the probability that the first logical field and the physical fields in the third physical table will be accessed by the same query statement. The higher the probability, the more suitable the third physical table is as the physical table for which new fields need to be added; the lower the probability, the less suitable the third physical table is as the physical table for which new fields need to be added.

[0074] Specifically, the model input parameters are fed into the machine learning model, which scores at least one third physical table, obtaining a score for each third physical table. The third physical table whose score meets the requirements is selected as the fourth physical table. For example, the third physical table with the highest score is selected as the fourth physical table, or the third physical table with a score exceeding a preset value is selected as the fourth physical table; there are no restrictions on this.

[0075] For example, machine learning models can be scored using formula (1): P=w1ⅹf+w2ⅹJ(A,B)(1) Where f is the historical access frequency of the third physical table, P is the score of the third physical table, w1 and w2 are weights, and the sum of w1 and w2 is 1. J(A, B) is the Jaccard coefficient, where A is the keyword set of the field names of the first logical field, and B is the keyword set of the field names of all physical fields in the third physical table. The keyword set is obtained by word segmentation and keyword extraction of the field names. J(A, B) can measure the probability of common access between the first logical field and the third physical table.

[0076] Specifically, the semantic similarity between the first logical field and the third physical table can be calculated based on the vector form of the field names in the first logical field and the vector form of the set of physical fields in the third physical table. The vectorized representation is generated through a word embedding model, and the similarity can be calculated using methods such as cosine similarity and maximum matching degree.

[0077] 314. Based on the table structure change information of the fourth physical table, generate the third database operation statement, wherein the table structure change information of the fourth physical table includes the field name of the second physical field corresponding to the first logical field.

[0078] Optionally, the table structure change information of the fourth physical table may include, but is not limited to: change type (such as "add field" or "modify field"), field name of the second physical field in the fourth physical table that the first logical field maps to, and table name of the fourth physical table (such as customers_experience_ext).

[0079] The third database operation statement is, for example: ALTER TABLE customers_experience_ext ADDCOLUMN satisfaction_score INT.

[0080] 316. Execute the third database operation statement to perform a table structure change operation on the fourth physical table in the database. The changed fourth physical table includes the second physical field.

[0081] Specifically, the table structure change operations include: adding a field (ADD COLUMN) or modifying a field (MODIFY COLUMN or ALTER COLUMN). That is, field expansion can be achieved by modifying existing physical fields in the fourth physical table, or by adding new physical fields to the fourth physical table.

[0082] In practical applications, the fourth physical table can be locked or not before executing the third database operation statement. After the third database operation statement is completed, the locked fourth physical table can be released.

[0083] In some optional embodiments, the method further includes updating the mapping relationship between the table structure information of the first logical table and the table structure information of the fourth physical table in the mapping relationship table. The updated mapping relationship includes the field name of the first logical field and the field name of its corresponding second physical field. Specifically, timely updating the mapping relationship in the mapping relationship table enables the automatic rewriting of logical table-oriented query statements (i.e., logical queries) into underlying query statements (i.e., physical queries) involving one or more physical tables during the subsequent query execution phase.

[0084] The field extension schemes provided in this manual support both creating new physical tables and modifying the structure of existing physical tables, thus enriching the field extension methods. Regardless of the method, both schemes automatically translate database operation statements (e.g., logical DDL statements) targeting logical tables into database operation statements (e.g., physical DDL statements) targeting physical tables. This achieves automated mapping from logical table changes to physical table creation or modification. This scheme shields users from the complexity of the underlying physical storage structure, eliminating the need for manual writing of database operation statements for physical tables, simplifying the field extension process and improving efficiency.

[0085] Figure 4 This is a flowchart of another database table processing method provided in the embodiments of this specification, such as... Figure 4 As shown, the method may include the following steps: 402. Obtain the fourth database operation statement, which is used to instruct the creation of the first logical table.

[0086] Specifically, the fourth database operation statement submitted by the application or user can be obtained through a unified interface. The fourth database operation statement can be regarded as a logical DDL statement for creating a logical table.

[0087] For example, if an e-commerce platform needs to manage customer information, including basic information, contact information, loyalty information, purchase history, etc., it needs to create a logical table for customer information from the user's perspective. Users do not need to care about the underlying physical table structure; they only need to write the following logical DDL statement to create the logical table: CREATE LOGICAL TABLE customers( customer_id INT PRIMARY KEY, customer_name VARCHAR(255), email VARCHAR(255), phone VARCHAR(50), address VARCHAR(500), loyalty_points INT, membership_level VARCHAR(50), preferred_contact_method VARCHAR(50), last_purchase_date DATE, total_purchase_amount DECIMAL(10, 2), purchase_frequency VARCHAR(20), signup_channel VARCHAR(100), campaign_source VARCHAR(100), customer_segment VARCHAR(50), created_date TIMESTAMP ); In traditional extended table solutions, to achieve dynamic field expansion, users must manually write multiple physical DDL statements to create the main table and each extended table, and define their relationships. This relies heavily on the user's understanding of the underlying physical storage structure, resulting in high operational complexity and a high risk of errors.

[0088] For example, the physical DDL statement for creating the master table corresponding to customer basic information is as follows: CREATE TABLE customers_main ( customer_id INT PRIMARY KEY, customer_name VARCHAR(255), created_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP); The physical DDL statement for creating the extended table 1 corresponding to the contact information is as follows: CREATE TABLE customers_contact_ext ( id INT PRIMARY KEY AUTO_INCREMENT, customer_id INT, email VARCHAR(255), phone VARCHAR(50), address VARCHAR(500), FOREIGN KEY (customer_id) REFERENCES customers_main(customer_id)); The physical DDL statement for creating Extended Table 2 corresponding to loyalty information is as follows: CREATE TABLE customers_loyalty_ext ( id INT PRIMARY KEY AUTO_INCREMENT, customer_id INT, loyalty_points INT DEFAULT 0, membership_level VARCHAR(50), preferred_contact_method VARCHAR(50), FOREIGN KEY (customer_id) REFERENCES customers_main(customer_id)); 404. Based on the parsing results of the fourth database operation statement, determine the table structure information of at least one fifth physical table to be created.

[0089] Specifically, the fourth database operation statement is parsed to obtain the corresponding AST. By traversing and analyzing this AST, the parsing results related to the first logical table are extracted. These parsing results include, but are not limited to: the table name of the first logical table (customers), the field names of the logical fields in the first logical table, primary key information (customer_id), and field category information. Field category information includes, for example: basic information: customer_name, created_date; contact information: email, phone, address; membership attributes: loyalty_points, membership_level; purchase behavior: last_purchase_date, total_purchase_amount; marketing tags: signup_channel, customer_segment, etc.

[0090] In practical applications, the table structure information of the fifth physical table to be created is determined based on the parsing results of the fourth database operation statement. For example, fields with similar semantics can be aggregated into the same physical table based on their semantic information; frequently accessed fields (hot data) and infrequently accessed fields (cold data) can be stored in different physical tables; fields that may be frequently changed can be stored independently to avoid the risk of table locking caused by frequent DDL operations on the main table structure.

[0091] For example, the fifth physical tables are the main table (customers_main), the contact information extension table (customers_contact_ext), the membership system extension table (customers_loyalty_ext), the purchase behavior extension table (customers_purchase_ext), the marketing tag extension table (customers_marketing_ext), and so on.

[0092] The main table's structure information includes: physical table name (customers_main), physical field names (customer_id, customer_name, created_date), primary key (customer_id), and join key (customer_id); the contact information extension table's structure information includes: physical table name (customers_contact_ext), physical field names (customer_id, email, phone, address), foreign key (customer_id), and join key (customer_id); the purchase behavior extension table's structure information includes: physical table name (customers_purchasing_database ... The table structure information for the marketing tag extension table is as follows: physical table name (customers_marketing_ext), physical field names (customer_id, last_purchase_date, total_purchase_amount, purchase_frequency), foreign key (customer_id), and join key (customer_id).

[0093] 406. Based on the table structure information of at least one fifth physical table, generate at least one fifth database operation statement.

[0094] 408. Execute at least one fifth database operation statement to create at least one fifth physical table in the database for the first logical table.

[0095] Specifically, the fifth database operation statement can be regarded as a physical DDL statement for creating physical tables. Below are a few exemplary physical DDL statements.

[0096] For example, the physical DDL statement for creating the main table is as follows: CREATE TABLE customers_main ( customer_id BIGINT PRIMARY KEY, customer_name VARCHAR(255), created_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); For example, the physical DDL statement for creating the contact information extended table is as follows: CREATE TABLE customers_contact_ext ( id BIGINT AUTO_INCREMENT PRIMARY KEY, customer_id BIGINT NOT NULL, email VARCHAR(255), phone VARCHAR(50), address VARCHAR(500), FOREIGN KEY (customer_id) REFERENCES customers_main(customer_id) ); After generating the fifth database operation statement, the fifth database operation statement is also executed. In this way, at least one corresponding fifth physical table is created in the underlying database for the first logical table.

[0097] In some optional embodiments, after executing at least one fifth database operation statement, the following can be further performed: the table structure information of the fifth physical table can be used as the table structure information of an existing physical table of the first logical table; a mapping relationship between the table structure information of the first logical table and the table structure information of at least one existing physical table can be added to the mapping relationship table; wherein, the table structure information of the first logical table includes at least one of the following: the table name of the first logical table, the field name of the logical field, and the join key; the table structure information of the existing physical table includes at least one of the following: the table name of the existing physical table, the field name of the physical field, and the join key. Of course, the table structure information can also maintain more metadata, such as join type, shard key, etc. Join types include, but are not limited to: INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL OUTER JOIN, etc.

[0098] Specifically, a mapping table can be maintained in the metadata service. Through the mapping relationships in the mapping table, queries that target logical tables can be automatically rewritten into queries that involve at least one physical table during query execution, thus supporting more intelligent query rewriting.

[0099] This manual describes a database table creation method that automatically parses and generates corresponding physical table structure information by receiving database operation statements for creating logical tables. It then creates and executes the database operation statements used to create the new physical tables, achieving automated mapping from logical table creation to physical table creation. This solution shields users from the complexity of the underlying physical storage structure. Users do not need to concern themselves with the underlying physical table structure; they only need to write database operation statements for logical tables (such as logical DDL statements), eliminating the need to manually write database operation statements for physical tables (such as physical DDL statements) to complete the underlying physical table creation task. This simplifies the table creation process and improves table creation efficiency.

[0100] Figure 5 This is a flowchart of another database table processing method provided in the embodiments of this specification, such as... Figure 6 As shown, the method may include the following steps: 502. Obtain the first query statement, which is used to instruct the execution of a query operation on the target logical table.

[0101] Specifically, the first query statement submitted by the application or user can be obtained through a unified interface. This first query statement can be viewed as a logical DML statement that queries the logical fields in a logical table. For example, the first query statement might be: "SELECT customer_id, customer_name, email, loyalty_points, satisfaction_score, total_purchase_amount FROM customers WHERE loyalty_points > 1000 AND (satisfaction_score < 3 OR satisfaction_score IS NULL) ORDER BY total_purchase_amount DESC". Users do not need to be aware of the underlying physical storage structure, write JOIN queries, or know which physical tables the fields are distributed in; they only need to access the fields in the logical table using standard SQL.

[0102] In this embodiment, the target logical table can be a logical table obtained by changing or creating the table structure using the aforementioned method embodiments, and there are no restrictions on this.

[0103] 504. Based on the parsing results of the first query statement, determine the table name of the target logical table and the field name of the logical field to be queried in the target logical table.

[0104] Specifically, the first query statement can be subjected to syntax analysis, semantic analysis, etc., to obtain the parsing results of the first query statement. The parsing results of the first query statement include, but are not limited to: the table name of the target logical table, the field name of the logical field to be queried, the filtering conditions, the sorting field, the grouping field or the aggregate function, etc.

[0105] 506. Based on the table name of the target logical table and the field name of the logical field to be queried, find the table name of the target physical table and the field name of the physical field to be queried that have a mapping relationship with the table name of the target logical table and the field name of the logical field to be queried in the mapping relationship table.

[0106] Specifically, a mapping table can be maintained in the metadata service. This table records the mapping relationship between the table structure information of logical tables and the table structure information of existing physical tables. Through the mapping relationships in the table, one can find the table name of the target physical table corresponding to the target logical table, and the field name of the physical field corresponding to the logical field to be queried. Of course, more metadata can also be found, such as join keys, join types, etc.

[0107] 508. Generate a second query statement based on the table name of the target physical table and the field name of the physical field to be queried.

[0108] Specifically, the query is rewritten based on the table name of the target physical table and the field name of the physical field to be queried, resulting in a second query statement. This second query statement can be viewed as a physical DML statement that queries the physical fields in the physical table. Because the parsing results of the first query statement differ, various types of second query statements can be generated. These types of second query statements include, but are not limited to: simple query statements, conditional query statements, sorting query statements, grouping query statements, or aggregation query statements, etc.

[0109] Optionally, the target physical table can be one, and the second query statement can be a single-table query statement.

[0110] Optionally, if there are multiple target physical tables, the second query statement is generated based on the table names of the target physical tables and the field names of the physical fields to be queried. The implementation method is as follows: find the join key of multiple target physical tables in the mapping table; generate a second query statement of multi-table query type based on the table names of multiple target physical tables, the field names of the physical fields to be queried, and the join key of multiple target physical tables.

[0111] In practical applications, a second query statement of multi-table query type can be generated based on the table names of multiple target physical tables, the field names of the physical fields to be queried, and the join keys of multiple target physical tables.

[0112] In practical applications, a second query statement of multi-table query type can be generated directly based on the table names of multiple target physical tables, the field names of the physical fields to be queried, and the join keys of multiple target physical tables. Alternatively, an initial second query statement of multi-table query type can be generated based on the table names of multiple target physical tables, the field names of the physical fields to be queried, and the join keys of multiple target physical tables. By performing predicate condition pushdown or join order optimization on the initial second query statement, the final second query statement is obtained, thereby reducing the cost of subsequent query calculations and significantly improving query performance. The goal of join order optimization is to rearrange the order of the JOIN tables while maintaining the same query semantics, enabling the generation of a more efficient execution plan and improving query performance.

[0113] Optionally, one possible implementation for generating a second query statement of the multi-table query type based on the table names of multiple target physical tables, the field names of the physical fields to be queried, and the join keys of the multiple target physical tables is as follows: An initial second query statement of the multi-table query type is generated based on the table names of multiple target physical tables, the field names of the physical fields to be queried, and the join keys of the multiple target physical tables; the join order optimization agent is used to optimize the join order of the initial second query statement of the multi-table query type to obtain the second query statement of the multi-table query type, wherein the join order optimization agent is obtained through reinforcement learning.

[0114] Specifically, a join order optimization agent is constructed using reinforcement learning methods. This agent can learn complex query patterns, predict better join orders, and improve query performance.

[0115] Optionally, the join order optimization agent is obtained as follows: Multiple sample join query statements are obtained, and the following operations are repeated until a join order optimization agent that meets the requirements is obtained: For any sample join query statement, the current state and action space of the sample join query statement are obtained. The current state includes the join conditions between the two database tables involved in the sample join query statement, and the action space includes multiple decision actions, each used to update the join order between the database tables in the sample join query statement; the join order optimization agent, based on the current state of the sample join query statement, determines the target decision action from the action space and executes the target decision action to update the sample join query statement; based on the query execution result of the updated sample join query statement, the reward value of the target decision action is determined; and the join order optimization agent is optimized based on the reward value of the target decision action.

[0116] Specifically, sample join queries are the join queries that participate in model training. The more sample join queries there are, the stronger the join order optimization agent's join order optimization capability will be. A current state is constructed for each sample join query, serving as input to the join order optimization agent. This current state can characterize the features of the sample join query, including but not limited to: the join conditions between two database tables (e.g., m.customer_id = c.customer_id), the set of database tables to be queried by the sample join query, and the current join order (i.e., the table order in the current JOIN clause). The action space defines all the decision actions that the join order optimization agent can execute in the current state. All possible table join orders of the sample join queries constitute the various decision actions in the action space. The join order optimization agent can predict a better join order from the action space based on the current state of the sample join query, and the decision action corresponding to the better join order is used as the target decision action.

[0117] After determining the target action, the target action is executed to update the join order between database tables in the previous sample join query statement, thereby updating the sample join query statement. Executing the updated sample join query statement yields the query execution result, which may include, but is not limited to, query execution time or query accuracy. Based on the query execution result of the updated sample join query statement, the reward value for the target action is determined; and the agent is optimized by adjusting the join order based on the reward value of the target action.

[0118] For example, the reward function is as follows: R = -t + α × acc (2) In formula (2), R is the reward value, t is the query execution time, acc is the query accuracy, and α is the weight coefficient.

[0119] 510. Execute the second query statement to retrieve the data of the physical field to be queried in the target physical table.

[0120] Specifically, by rewriting the query to obtain a second query statement in the form of a physical DML statement, the second query statement can be executed to retrieve the data of the physical field to be queried in the target physical table.

[0121] This specification implements a database table query method that automatically parses and obtains the corresponding physical table structure information by receiving query statements for logical tables. Then, through a query rewriting mechanism, it generates and executes query statements for querying physical tables, achieving automated mapping from logical table queries to physical table queries. This solution shields the complexity of the underlying physical storage structure. Users do not need to concern themselves with the underlying physical table structure; they only need to write database operation statements (such as logical DML statements) for logical tables, eliminating the need to manually write database operation statements (such as physical DML statements) for physical tables to complete the underlying physical table query task. This simplifies the database table query process and improves query performance.

[0122] Figure 6 This is a schematic diagram of the structure of a database table processing device provided in an embodiment of this specification. See also... Figure 6 The device may include: The acquisition module 61 is used to acquire the first database operation statement, which is used to instruct the addition of the first logical field in the first logical table. The parsing module 62 is used to determine the table structure information of the first physical table to be created based on the parsing result of the first database operation statement. The table structure information of the first physical table includes the field name of the first physical field corresponding to the first logical field. The generation module 63 is used to generate the second database operation statement based on the table structure information of the first physical table; Execution module 64 is used to execute a second database operation statement to create a first physical table in the database that includes a first physical field for the first logical table.

[0123] Optionally, when the parsing module 62 determines the table structure information of the first physical table to be created based on the parsing result of the first database operation statement, it is used to: determine whether at least one second physical table related to the first logical field supports locking, and at least one second physical table is at least a part of all existing physical tables corresponding to the first logical table in the database; if at least one second physical table does not support locking, then the table structure information of the first physical table to be created is determined based on the parsing result of the first database operation statement.

[0124] Optionally, the parsing module 62 is further configured to: select a fourth physical table from at least one third physical table if at least one second physical table contains at least one third physical table that supports locking; The generation module 63 is also used to generate a third database operation statement based on the table structure change information of the fourth physical table, wherein the table structure change information of the fourth physical table includes the field name of the second physical field corresponding to the first logical field. The execution module 64 is also used to execute third database operation statements to perform a table structure change operation on a fourth physical table in the database. The changed fourth physical table includes the second physical field.

[0125] Optionally, the operation types for table structure change operations include: adding fields or modifying fields.

[0126] Optionally, when parsing module 62 selects a fourth physical table from at least one third physical table, it is used to: obtain model input parameters, wherein the model input parameters include at least one of the following: the data sparsity and / or historical access data of at least one third physical table, the simultaneous access probability and / or semantic similarity between the first logical field and at least one third physical table; and select a fourth physical table from at least one third physical table using a machine learning model based on the model input parameters.

[0127] Optionally, the execution module 64 is further configured to add a mapping relationship between the table structure information of the first logical table and the table structure information of the first physical table in the mapping relationship table; and / or update the mapping relationship between the table structure information of the first logical table and the table structure information of the fourth physical table in the mapping relationship table, wherein the updated mapping relationship includes the field name of the first logical field and the field name of the corresponding second physical field.

[0128] Optionally, module 61 is also used to obtain a fourth database operation statement, which is used to instruct the creation of the first logical table; The parsing module 62 is also used to determine the table structure information of at least one fifth physical table to be created based on the parsing result of the fourth database operation statement; The generation module 63 is also used to generate at least one fifth database operation statement based on the table structure information of at least one fifth physical table; The execution module 64 is also configured to execute at least one fifth database operation statement to create at least one fifth physical table in the database for the first logical table.

[0129] Optionally, the execution module 64 is further configured to use the table structure information of the fifth physical table as the table structure information of the existing physical table of the first logical table; add a mapping relationship between the table structure information of the first logical table and the table structure information of at least one existing physical table in the mapping relationship table; wherein the table structure information of the first logical table includes at least one of the following: the table name of the first logical table, the field name of the logical field, and the join key; the table structure information of the existing physical table includes at least one of the following: the table name of the existing physical table, the field name of the physical field, and the join key.

[0130] Optionally, the acquisition module 61 is also used to acquire a first query statement, which is used to instruct the execution of a query operation on the target logical table; The parsing module 62 is also used to determine the table name of the target logical table and the field name of the logical field to be queried in the target logical table based on the parsing result of the first query statement; and to search in the mapping relationship table for the table name of the target physical table and the field name of the physical field to be queried that have a mapping relationship with the table name of the target logical table and the field name of the logical field to be queried. The generation module 63 is also used to generate a second query statement based on the table name of the target physical table and the field name of the physical field to be queried; Execution module 64 is also used to execute a second query statement to retrieve data of the physical field to be queried in the target physical table.

[0131] Optionally, if there are multiple target physical tables, the generation module 63, when generating the second query statement based on the table name of the target physical table and the field name of the physical field to be queried, is used to find the join key of multiple target physical tables in the mapping relationship table; and generates a second query statement of multi-table query type based on the table name of multiple target physical tables, the field name of the physical field to be queried, and the join key of multiple target physical tables.

[0132] Optionally, when generating a second query statement of a multi-table query type based on the table names of multiple target physical tables, the field names of the physical fields to be queried, and the join keys of multiple target physical tables, the generation module 63 is used to generate an initial second query statement of a multi-table query type based on the table names of multiple target physical tables, the field names of the physical fields to be queried, and the join keys of multiple target physical tables; and to optimize the join order of the initial second query statement of the multi-table query type using a join order optimization agent, which is obtained through reinforcement learning.

[0133] Optionally, the join order optimization agent is obtained as follows: Multiple sample join query statements are obtained, and the following operations are repeated until a join order optimization agent that meets the requirements is obtained: For any sample join query statement, the current state and action space of the sample join query statement are obtained. The current state includes the join conditions between the two database tables involved in the sample join query statement, and the action space includes multiple decision actions, each used to update the join order between the database tables in the sample join query statement; the join order optimization agent, based on the current state of the sample join query statement, determines the target decision action from the action space and executes the target decision action to update the sample join query statement; based on the query execution result of the updated sample join query statement, the reward value of the target decision action is determined; and the join order optimization agent is optimized based on the reward value of the target decision action.

[0134] For a detailed description of the database table processing device, please refer to the foregoing embodiments, which will not be repeated here.

[0135] This specification also provides a database system, including: an interaction layer, a logical table abstraction layer, and a physical storage layer; The interaction layer is used to obtain the first database operation statement and send it to the logical table management layer. The first database operation statement is used to instruct the addition of the first logical field in the first logical table. The logical table abstraction layer is used to determine the table structure information of the first physical table to be created based on the parsing result of the first database operation statement. The table structure information of the first physical table includes the field name of the first physical field corresponding to the first logical field. Based on the table structure information of the first physical table, a second database operation statement is generated. The second database operation statement is executed to create a new first physical table in the database that includes the first physical field for the first logical table. The physical storage layer is used for persistent storage of data in the first physical table.

[0136] Optionally, the logical table abstraction layer includes: a query optimizer, a storage manager, a metadata service, and a transaction manager; the query optimizer provides query optimization services; the metadata service manages mapping tables, which include the mapping relationship between the table structure information of logical tables and the table structure information of at least one existing physical table; the storage manager manages the storage of data in physical tables; and the transaction manager manages transactions for database operations related to logical tables and physical tables respectively.

[0137] Optionally, the interaction layer is also used to obtain a fourth database operation statement and send it to the logical table management layer. The fourth database operation statement is used to instruct the creation of the first logical table. The logical table abstraction layer is also used to determine the table structure information of at least one fifth physical table to be created based on the parsing results of the fourth database operation statement; generate at least one fifth database operation statement based on the table structure information of at least one fifth physical table; and execute at least one fifth database operation statement to create at least one fifth physical table for the first logical table in the database.

[0138] Optionally, the interaction layer is also used to obtain the first query statement and send it to the logical table management layer. The first query statement is used to instruct the execution of a query operation on the target logical table. The logical table abstraction layer is also used to determine the table name of the target logical table and the field name of the logical field to be queried in the target logical table based on the parsing result of the first query statement; based on the table name of the target logical table and the field name of the logical field to be queried, it searches in the mapping relationship table for the table name of the target physical table and the field name of the physical field to be queried that have a mapping relationship with the table name of the target logical table and the field name of the logical field to be queried; based on the table name of the target physical table and the field name of the physical field to be queried, it generates a second query statement; and executes the second query statement to query the data of the physical field to be queried in the target physical table.

[0139] For a detailed introduction to the database system, please refer to the aforementioned embodiments, which will not be repeated here.

[0140] Figure 7 This specification illustrates a schematic diagram of an electronic device provided in an exemplary embodiment, which is applicable to the database table processing method provided in the foregoing embodiments. For example... Figure 7As shown, the electronic device 700 mainly consists of a communication interface 702, a user interface 704, a processor 706, and a memory 708. These components are interconnected and communicate with each other through a system bus, network, or other connection mechanism 410. The communication interface 702 enables the device 700 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 702 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 702 can also be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi (Wireless Fidelity), Bluetooth, Global Positioning System (GPS), or wide-area wireless interface such as WiMAX (Wireless Maximum) or LTE (Long Term Evolution). Of course, the communication interface 702 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 702 may also include multiple physical communication interfaces, such as a Wi-Fi interface, a Bluetooth interface, and a wide-area wireless interface.

[0141] User interface 704 includes receiving user input and providing output to the user. Therefore, user interface 704 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), LED (Light Emitting Diode), display using DLP (Digital Light Processing) technology, printer, and other known or future similar devices. User interface 704 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other known or future similar devices. In some embodiments, user interface 704 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, electronic device 700 may support remote access from other devices via communication interface 702 or another physical interface (not shown). User interface 704 can be configured to receive user input, the position and movement of which can be indicated by an indicator or cursor described herein. User interface 704 can also be configured as a display device for rendering or displaying text fragments.

[0142] Processor 706 may include one or more general-purpose processors and / or special-purpose processors. Memory 708 may include one or more volatile and / or non-volatile memory components and may be integrated wholly or partially with processor 706. Memory 708 may include removable and non-removable components.

[0143] The processor 706 is capable of executing program instructions 718 (e.g., compiled or uncompiled program logic and / or machine code) stored in memory 708 to perform the various functions described herein.

[0144] Memory 708 may contain non-transitory computer-readable media, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Memory 708 stores program instructions that, when executed by device 700, enable device 700 to perform any of the methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Processor 706 executing program instructions 718 may cause processor 706 to use data 712.

[0145] For example, program instructions 718 may include an operating system 722 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 700 and one or more applications 720 (e.g., a browser, social application, or game application). Similarly, data 712 may include operating system data 716 and application data 714. Operating system data 716 is primarily accessible to the operating system 722, while application data 714 is primarily accessible to one or more applications 720. Application data 714 may reside in a file system visible or hidden from the user of device 700.

[0146] Application 720 can communicate with operating system 722 through one or more application programming interfaces (APIs). These APIs help application 720 read and / or write application data 714, transmit or receive information via communication interface 702, receive or display information on user interface 704, etc.

[0147] In some terminology, application 720 may be simply referred to as "app". Furthermore, application 720 can be downloaded to device 700 through one or more online app stores or app markets. However, applications can also be installed on device 700 in other ways, such as through a web browser or a physical interface on electronic device 700 (e.g., a USB port).

[0148] Accordingly, embodiments of this specification also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium.

[0149] Accordingly, embodiments of this specification also provide a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above-described method embodiments. It should be understood that each step or combination of steps in the above-described method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above-described method embodiments.

[0150] In the embodiments of this specification, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the access relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following associated objects have an "or" relationship. Furthermore, in the embodiments of this specification, "first," "second," "third," etc., are only used to distinguish the content of different objects and have no other special meaning.

[0151] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 104 can be device A; or the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0152] Additionally, it should be noted that the terms "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0153] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving data sent by B, or it can be understood as A indirectly receiving data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending data directly to A, or it can be understood as B indirectly sending data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0154] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, product, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes that element.

[0155] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0156] The terminology used in the embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one. “A plurality” generally includes at least two, but does not exclude the inclusion of at least one.

[0157] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0158] The above are merely embodiments of this specification and are not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A database table processing method, characterized in that, include: Obtain a first database operation statement, which is used to instruct the addition of a first logical field to a first logical table; Based on the parsing result of the first database operation statement, the table structure information of the first physical table to be created is determined. The table structure information of the first physical table includes the field name of the first physical field corresponding to the first logical field. Based on the table structure information of the first physical table, generate the second database operation statement; Execute the second database operation statement to create a first physical table in the database that includes the first physical field for the first logical table.

2. The method according to claim 1, characterized in that, Based on the parsing result of the first database operation statement, the table structure information of the first physical table to be created is determined, including: Determine whether at least one second physical table related to the first logical field supports locking, wherein the at least one second physical table is at least a portion of all existing physical tables corresponding to the first logical table in the database; If none of the at least one second physical table supports locking, the table structure information of the first physical table to be created is determined based on the parsing result of the first database operation statement.

3. The method according to claim 2, characterized in that, Also includes: If at least one third physical table that supports locking exists in at least one second physical table, then a fourth physical table is selected from the at least one third physical table; Based on the table structure change information of the fourth physical table, a third database operation statement is generated, wherein the table structure change information of the fourth physical table includes the field name of the second physical field corresponding to the first logical field; The third database operation statement is executed to perform a table structure change operation on the fourth physical table in the database. The changed fourth physical table includes the second physical field.

4. The method according to claim 3, characterized in that, The types of table structure change operations include: adding fields or modifying fields.

5. The method according to claim 3, characterized in that, Selecting a fourth physical table from at least one third physical table includes: Obtain model input parameters, wherein the model input parameters include at least one of the following: the data sparsity and / or historical access data of the at least one third physical table, the simultaneous access probability and / or semantic similarity between the first logical field and the at least one third physical table; A fourth physical table is selected from the at least one third physical table using a machine learning model based on the model's input parameters.

6. The method according to claim 3, characterized in that, Also includes: Add a mapping relationship between the table structure information of the first logical table and the table structure information of the first physical table to the mapping relationship table; And / or, The mapping relationship between the table structure information of the first logical table and the table structure information of the fourth physical table is updated in the mapping relationship table. The updated mapping relationship includes the field name of the first logical field and the field name of the corresponding second physical field.

7. The method according to claim 1, characterized in that, Before obtaining the first database operation statement, it also includes: Obtain the fourth database operation statement, which is used to instruct the creation of the first logical table; Based on the parsing results of the fourth database operation statement, determine the table structure information of at least one fifth physical table to be created; Based on the table structure information of the at least one fifth physical table, generate at least one fifth database operation statement; Execute the at least one fifth database operation statement to create the at least one fifth physical table for the first logical table in the database.

8. The method according to claim 7, characterized in that, After executing the at least one fifth database operation statement, the process further includes: The table structure information of the fifth physical table is used as the table structure information of the existing physical table of the first logical table; Add a mapping relationship between the table structure information of the first logical table and the table structure information of at least one existing physical table to the mapping relationship table; The table structure information of the first logical table includes at least one of the following: the table name of the first logical table, the field name of the logical field, and the join key; the table structure information of the existing physical table includes at least one of the following: the table name of the existing physical table, the field name of the physical field, and the join key.

9. A database table processing method, characterized in that, include: Obtain the first query statement, which is used to instruct the execution of a query operation on the target logical table; The target logical table is obtained by modifying or creating the table structure using the method described in any one of claims 1 to 8; Based on the parsing result of the first query statement, determine the table name of the target logical table and the field name of the logical field to be queried in the target logical table; Based on the table name of the target logical table and the field name of the logical field to be queried, search in the mapping relationship table for the table name of the target physical table and the field name of the physical field to be queried that have a mapping relationship with the table name of the target logical table and the field name of the logical field to be queried; A second query statement is generated based on the table name of the target physical table and the field name of the physical field to be queried; Execute the second query statement to retrieve the data of the physical field to be queried in the target physical table.

10. The method according to claim 9, characterized in that, If there are multiple target physical tables, a second query statement is generated based on the table names of the target physical tables and the field names of the physical fields to be queried, including: Search the mapping table for the join key of multiple target physical tables; Based on the table names of the multiple target physical tables, the field names of the physical fields to be queried, and the join keys of the multiple target physical tables, a second query statement of the multi-table query type is generated.

11. The method according to claim 10, characterized in that, Based on the table names of the multiple target physical tables, the field names of the physical fields to be queried, and the join keys of the multiple target physical tables, a second query statement of multi-table query type is generated, including: Based on the table names of the multiple target physical tables, the field names of the physical fields to be queried, and the join keys of the multiple target physical tables, an initial second query statement of the multi-table query type is generated; The join order optimization agent is used to optimize the join order of the initial second query statement of the multi-table query type, so as to obtain the second query statement of the multi-table query type. The join order optimization agent is obtained by reinforcement learning.

12. The method according to claim 11, characterized in that, The method for obtaining the connection order optimization agent is as follows: Obtain multiple sample join query statements and repeat the following operations until the join order optimization agent that meets the requirements is obtained: For any sample join query statement, obtain the current state and action space of the sample join query statement. The current state includes the join conditions between the two database tables involved in the sample join query statement. The action space includes multiple decision actions, each of which is used to update the join order between the database tables in the sample join query statement. The connection order optimization agent determines a target decision action from the action space based on the current state of the sample connection query statement, and executes the target decision action to update the sample connection query statement. Based on the query execution results of the updated sample join query statement, the reward value of the target decision action is determined; The connection sequence is optimized based on the reward value of the target decision action.

13. A database system, characterized in that, include: Interaction layer, logical table abstraction layer, physical storage layer; The interaction layer is used to obtain a first database operation statement and send it to the logical table management layer. The first database operation statement is used to instruct the addition of a first logical field in the first logical table. The logical table abstraction layer is used to determine the table structure information of the first physical table to be created based on the parsing result of the first database operation statement. The table structure information of the first physical table includes the field name of the first physical field corresponding to the first logical field. Based on the table structure information of the first physical table, a second database operation statement is generated. The second database operation statement is executed to create a first physical table in the database that includes the first physical field for the first logical table. The physical storage layer is used for persistent storage of the data in the first physical table.

14. The system according to claim 13, characterized in that, The logical table abstraction layer includes: query optimizer, storage manager, metadata service, and transaction manager; The query optimizer is used to provide query optimization services; The metadata service is used to manage a mapping relationship table, which includes the mapping relationship between the table structure information of logical tables and the table structure information of at least one existing physical table. The storage manager is used for storage management of data in physical tables; The transaction manager is used to manage transactions for database operations related to logical tables and physical tables respectively.

15. An electronic device, characterized in that, include: A memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions for: performing the steps of the method according to any one of claims 1-12.

16. A computer-readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it is able to perform the steps of the method described in any one of claims 1-12.

17. A computer program product, characterized in that, include: A computer program / instruction that, when executed by a processor, enables the implementation of the steps in the method according to any one of claims 1-12.

Images