Data processing method, electronic equipment and storage medium

Abstract

The invention provides a data processing method, electronic equipment and a computer readable storage medium, and relates to the technical field of databases. According to the method and the device, when the DDL operation is carried out on the table in the database, the table is not locked, and at the moment, the DML operation can be carried out on the table. In the DDL operation process of the table, when DML operation is carried out on the table, reconstruction processing can be carried out on data to be subjected to DML operation in the table firstly, so that the data to be subjected to DML operation meets the structure of the table subjected to DDL operation, and then DML operation is carried out on the data to be subjected to DML operation in the table. Therefore, the consistency of the data in the table can be ensured, and the DML operation can be prevented from being blocked by the DDL operation.

Description

Technical Field

[0001] The present application relates to the field of database technology, and in particular to a data processing method, an electronic device, and a computer-readable storage medium. Background Art

[0002] As the business develops, a large amount of data in the application system needs to be stored in the database. The data in the database can be stored in the form of tables. Users can query the data in the database through the data query language (DQL) provided by the database management system (DBMS), and can also perform DDL operations on the tables in the database through the data definition language (DDL) provided by the DBMS system, such as modifying the structure of the table. They can also perform DML operations on the tables in the database through the data manipulation language (DML) provided by the DBMS system, such as modifying the data in the table.

[0003] In the prior art, in order to ensure data consistency, when a table in a database is operated by DDL, the table will be locked. At this time, the table can no longer be operated by DML, resulting in DDL operations blocking DML operations, thereby reducing data processing efficiency. Therefore, how to prevent DDL operations from blocking DML operations has become a technical problem to be solved. Summary of the invention

[0004] Some embodiments of the present application provide a data processing method, an electronic device, a computer-readable storage medium, and a computer program product. The technical solution of the present application can prevent a DDL operation from blocking a DML operation.

[0005] In a first aspect, the present application provides a data processing method for an electronic device, the method comprising: during a first change operation on a first table, receiving a first instruction, wherein the first instruction is used to instruct to modify first data in the first table to second data; determining that the first change operation has not yet adjusted the format of the first data in the first table to: meet the format requirements of the first table after the first change operation; adjusting the format of the first data in the first table to meet the format requirements of the first table after the first change operation; and modifying the first data in the first table to second data.

[0006] The first change operation may be the DDL operation mentioned in this application, and the first instruction may be the DML instruction mentioned in this application. The first table may be the table A mentioned in this application, such as a student information table.

[0007] The data processing method of the embodiment of the present application does not lock the table when performing DDL operation on the table in the database, and DML operation can be performed on the table at this time. In the process of performing DDL operation on the table, when performing DML operation on the table, the data to be operated by DML in the table can be reconstructed first, so that the data to be operated by DML meets the structure of the table after the DDL operation, that is, the format of the data to be operated by DML meets the format requirements of the table after the DDL operation, and then the DML operation is performed on the data to be operated by DML in the table. In this way, the consistency of the data in the table can be guaranteed, and the DDL operation can be prevented from blocking the DML operation.

[0008] In some embodiments, determining that the first change operation has not yet adjusted the format of the first data in the first table to meet the format requirements of the first table after the first change operation includes: corresponding to determining that the format of the first data does not match the format parameters of the first table after the first change operation, determining that the first change operation has not yet adjusted the format of the first data to meet the format requirements of the first table after the first change operation.

[0009] The format parameters may be the metadata mentioned in this application, which includes database name, database character set, table name, table size, number of rows in the table, table character set, table fields, table indexes, table description, field type, field precision, field description, etc.

[0010] In some embodiments, adjusting the format of the first data in the first table to meet the format requirements of the first table after the first change operation includes: calling a data reconstruction thread to adjust the format of the first data so that the format of the first data matches the format parameters of the first table after the first change operation.

[0011] During the DDL operation on table A in the database, the data reconstruction thread may be called to adjust the format of the data in the first table A so that the format of the data matches the metadata of table A after the DDL operation.

[0012] For example, the first row of the student information table contains column names, which are "name", "age", and "class", and the corresponding metadata is "name, age, class". The corresponding student information in the fourth row of the student information table is "Wang Wu, 13, Class 9". After the metadata "name, age, class" is updated to "name, age, grade, class" through DDL operation, a column "grade" will be added between the second and third columns of the student information table. At this point, the fourth row of data in the student information table, "Wang Wu, 13, Class 9", needs to be modified to "Wang Wu, 13, NULL, Class 9" to make the data in the fourth row match the modified metadata.

[0013] In some embodiments, calling a data reconstruction thread to adjust the format of the first data includes: calling the data reconstruction thread to parse the second instruction used to indicate the first change operation to determine the format parameters of the first table after the first change operation; calling the data reconstruction thread to adjust the format of the first data based on the format parameters of the first table after the first change operation.

[0014] The second instruction may be a DML instruction.

[0015] After the data in the table is operated by DDL, the structure of the table changes. If the DML operation is performed directly on the data in the table, the modified data will not match the structure of the table, that is, the metadata of the table. Before the electronic device performs a DML operation on the data in the table, it calls the data reconstruction thread to adjust the format of the data to be operated by DML (i.e., the first data), so that the format of the data to be operated by DML matches the structure of the table, thereby avoiding inconsistent table data after the DML operation is performed on the data to be operated by DML.

[0016] In some embodiments, the first instruction includes a data manipulation language DML instruction, and the second instruction includes a data definition language DDL instruction.

[0017] In some embodiments, the method further includes: performing a first change operation on the first table to obtain a second table, and saving the second table; receiving a first request sent by a first application of the first version, the first request being used to request binding with the second table; and binding the first application with the second table.

[0018] The first application may be a student information management application. The first table may be a table before the DDL operation, and the second table may be a table after the DDL operation.

[0019] For example, when Table A of version V1.1 (as an example of the first table) is updated to Table A of version V1.2 (as an example of the second table) after DDL operation, when the APP of version V1.2 sends the first request to the database, the database can bind Table A of version V1.2 to the APP of version V1.2, so that the APP of version V1.2 can use Table A of version V1.2. Thus, different versions of applications can use different versions of tables, and the old version of the application can continue to use the old version of the table without using the latest version of the table, and there is no need to adapt to the latest version of the table. This ensures business continuity and reduces workload.

[0020] In some embodiments, the method further includes: binding the second application to the first form; receiving a third request sent by the second application, the third request being used to instruct switching the second application to be bound to the second form; unbinding the second application from the first form; and binding the second application to the second form.

[0021] The second application can be the application before the first application is updated. For example, the first application is the V1.2 version APP, and the second application is the V1.1 version APP. When the V1.1 version APP is bound to the V1.1 version Table A, after the V1.1 version Table A is updated to the V1.2 version Table A, the V1.1 version APP can send a third request to the database. Request the database to unbind the V1.1 version APP from the V1.1 version Table A, and then bind the V1.1 version APP to the V1.2 version Table A, so that the V1.1 version APP can use the V1.2 version Table A, that is, can use the updated Table A. In this way, the application program can switch between different tables for use, which has strong flexibility.

[0022] In some embodiments, the method further includes: in response to determining that the formats of all data in the first table meet the format requirements of the first table and the first table is not bound to an application, deleting the first table.

[0023] When it is determined that the metadata update of the first table is completed, the data in the table is reconstructed, and the first table is not bound to any application, it means that the first table is no longer used. At this time, the first table can be deleted to save storage space.

[0024] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device; and a processor, when the processor executes the instructions in the memory, the electronic device can execute the method described in any embodiment of the first aspect of the present application. The beneficial effects that can be achieved in the second aspect can refer to the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here.

[0025] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed on a computer, the computer can execute the method described in any embodiment of the first aspect. The beneficial effects that can be achieved in the third aspect can refer to the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here.

[0026] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed, causes a computer to execute the method described in any embodiment of the first aspect. The beneficial effects that can be achieved in the fourth aspect can refer to the beneficial effects of the method provided in any embodiment of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 An exemplary application scenario of the present application is shown;

[0028] Figure 2A A diagram showing the principles of DDL operations provided in some embodiments;

[0029] Figure 2B A schematic diagram of a DML operation provided in some embodiments;

[0030] Figure 2C A diagram showing the principle of DML operation provided in an embodiment of the present application;

[0031] Figure 3 An example flowchart of a data processing method provided in an embodiment of the present application;

[0032] Figure 4A An example diagram of table update provided in an embodiment of the present application;

[0033] Figure 4B An example diagram of the application of metadata multi-version management provided in the embodiment of the present application;

[0034] Figure 5 An example diagram of the DDL operation process provided in the embodiment of the present application;

[0035] Figure 6 An example of a data reconstruction process provided in the embodiment of the present application Figure 1 ;

[0036] Figure 7 An example diagram of the DML operation process provided in the embodiment of the present application;

[0037] Figure 8 An example diagram of the DQL operation process provided in the embodiment of the present application;

[0038] Fig. 9 An example diagram of the metadata cleaning process provided in the embodiment of the present application;

[0039] Fig.10 Example of the system architecture of the database provided in the embodiment of the present application Figure 1 ;

[0040] Fig.11 An example diagram of the online DDL process provided in the embodiment of the present application;

[0041] Fig.12 Figure 2 is an example of the system architecture of the database provided in the embodiment of the present application;

[0042] Fig.13 Figure 2 is an example of a data reconstruction process provided in an embodiment of the present application;

[0043] Fig.14 A block diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0044] The embodiment of the present application is used to provide a data processing method. The method provided by the present application supports concurrent execution of DDL operations and DML operations, and avoids DDL operations blocking DML operations.

[0045] In order to facilitate understanding of the technical solution of the present application, some of the terms involved in the present application are first explained below.

[0046] Database metadata: Metadata (dictionary) is some data belonging to the database itself, including database name, database character set, table name, table size, number of rows in the table, table character set, table fields, table indexes, table description, field type, field precision, field description, etc. This article will refer to "database metadata" as "metadata".

[0047] DDL operations are operations to define, modify, and adjust the database schema of a database. Schema is a structural description of a database. In a relational database, the schema defines tables, fields in tables, relationships between tables and fields, table indexes, table column names, etc. In this application, DDL operations can be understood as modifications to metadata.

[0048] DML operations are operations such as adding, deleting, and modifying data in a table.

[0049] DQL operations are queries on data in the database.

[0050] Figure 1 An exemplary application scenario of the present application is shown. Figure 1As shown, when a certain business (for example, e-commerce business, student information management business, office business) has a lot of data, the data of the business can be stored in the database of the database server (referring to the server where the database is deployed), and then computers, mobile phones and other terminal devices can obtain these data by accessing the database of the database server. However, as the business develops, it may be necessary to modify the data stored in the database. At this time, the user can send corresponding data modification instructions to the database server through the terminal device connected to the database server, such as DDL instructions, DML instructions, etc. After the database receives the data modification instruction, it can call the DBMS system used to manage the database to modify the data in the database.

[0051] In this application, the terminal device can be a laptop, desktop computer, tablet computer, mobile phone and other electronic devices, without limitation. The database can be a structured query language (SQL), a non-relational database (NoSQL), Oracle, MySQL, openGauss and other types of databases. In addition to being deployed in a server, the database of this application can also be deployed in a cloud platform, edge devices (e.g., edge servers, routers, switches, IoT devices, etc.), mobile devices (e.g., laptops, tablet computers, mobile phones, etc.) and other electronic devices, without limitation.

[0052] In some embodiments, when user A wants to change table A in the database, such as modifying the structure of table A, user A can send a corresponding DDL instruction to the database server through a terminal device such as a computer to instruct the database to perform a DDL operation on table A to modify the structure of table A. In order to ensure data consistency, after performing a DDL operation on table A, the database also needs to reconstruct the data in table A (such as user data) so that the data in table A meets the format requirements after the change of table A. In addition, before the data in table A is reconstructed, the database will lock table A to prevent other users from modifying the data in table A, resulting in the data in table A not meeting the format requirements after the change of table A. When table A is locked, when the database receives a DML instruction, the DML instruction will be shelved. Only after the structural modification of table A is completed and all the data in table A is reconstructed, the DML instruction will be executed to modify the data in the reconstructed table A. In other words, DDL instructions and DML instructions cannot be executed concurrently, and DDL operations will block DML operations. In this way, the modification efficiency of the table is reduced.

[0053] For example, Figure 2AAs shown, when user A wants to add a column (whose column name is "grade") before the third column of the student information table (as an example of table A), the corresponding DDL instruction can be sent to the database through the terminal device to instruct the database to perform the corresponding DDL operation, such as modifying the metadata {name, age, class} of the student information table to metadata {name, age, grade, class}. In this way, a column named "grade" can be added before the third column of the student information table. Then, the database can call the data reconstruction thread to reconstruct the data in the student information table so that the reconstructed data matches the modified structure of the student information table. For example, the data "Zhang San, 13, Class 7" in the first row can be reconstructed into data "Zhang San, 13, NULL, Class 7", where NULL means empty. During the data reconstruction of the DDL operation, the student information table needs to be locked to prevent other users from modifying the data in the student information table, so as to ensure data consistency. If the student information table is not locked during the data reconstruction of the DDL operation, data anomalies may occur when other users modify the data in the student information table.

[0054] For example, Figure 2B As shown in the figure, during the DDL operation on the student information table, if the student information table is not locked, when the data "Wang Wu, 13, Class 9" in the third row of the student information table is modified to "Wang Wu, 13, Class 7", the "Class 7" in the data will appear in the "Grade" column, resulting in the data not corresponding to the column name, that is, the data is inconsistent. Therefore, during the DDL operation on the student information table, the student information table needs to be locked. However, locking the student information table will cause the DDL operation to block the DML operation, thereby reducing the data processing efficiency.

[0055] In order to solve the above technical problems, the present application provides a data processing method. In an embodiment of the present application, in the process of performing a DDL operation (for example, changing the structure of a table) on a table in the database based on a DDL instruction, when the database receives a DML instruction, the database will determine whether the data in the table meets the format requirements of the table after the DDL operation based on the DDL instruction. When the database determines that the data does not meet the format requirements of the table after the DDL operation, it will call a data reconstruction thread to reconstruct the data so that the data meets the corresponding format requirements, and then perform a DML operation (such as modification) on the data. In this way, the consistency of the data in the table can be guaranteed without locking the table. In addition, the database does not need to wait until the DDL operation is completed before executing the DML operation, thereby avoiding the DDL operation from blocking the DML operation and improving the data processing efficiency.

[0056] For example, Figure 2CAs shown, during the process of the database performing DDL operations on the student information table, the student information table is not locked. In this case, the database can perform DML operations on the student information table. For example, if the third row of data in the student information table has not yet been reconstructed, and the user wants to change the class of the student "Wang Wu" in the third row from "Class 9" to "Class 7", the database can first call the data reconstruction thread to reconstruct the data "Wang Wu, 13, Class 9" in the third row into "Wang Wu, 13, NULL, Class 9", and then perform DML operations on the reconstructed data "Wang Wu, 13, NULL, Class 9" to change the reconstructed data "Wang Wu, 13, NULL, Class 9" into "Wang Wu, 13, NULL, Class 7". In this way, both the mismatch between the data and the table structure and blocking can be avoided.

[0057] The specific embodiments of the present application are described below.

[0058] Figure 3 This is a flowchart of a data processing method provided in an embodiment of the present application. Figure 3 As shown, the method comprises the following steps:

[0059] S101: The database receives a DDL instruction for table A.

[0060] When a user wants to make changes to Table A in the database, such as changing the structure of Table A, a DDL instruction corresponding to Table A can be sent to the database through a terminal device connected to the database to instruct the server to perform a DDL operation on Table A.

[0061] For example, Figure 2A As shown, the student information table includes four rows and three columns. The first row is the column names, including "name", "age", and "class", and the corresponding metadata includes "name, age, class". The second to fourth rows are student information, where the second row includes "Zhang San", "13", and "Class 7", the third row includes "Li Si", "12", and "Class 8", and the fourth row includes "Wang Wu", "13", and "Class 9". When the user wants to insert a column named "grade" before the third column "class" of the student information table in the database (as an example of Table A), the corresponding DDL instruction can be sent to the database through a terminal device connected to the database to instruct the database to insert a column in front of the third column of the student information table, and its column name is "grade".

[0062] S102: The database performs a DDL operation on table A in the database.

[0063] After receiving the DDL instruction for table A, the database performs a DDL operation on table A in the database, such as changing the structure of table A by modifying the metadata of table A.

[0064] In order to ensure the data consistency of Table A, after the database completes the structural change of Table A, it can reconstruct the data in Table A so that the data in Table A matches the modified structure of Table A, that is, the format of the data in Table A meets the format requirements of Table A. After the database completes the reconstruction of all the data in Table A, the entire DDL operation process ends.

[0065] For example, Figure 2A As shown, after receiving the DDL instruction for the student information table, the database can modify the metadata "name, age, class" of the student information table to "name, age, grade, class", so that a column named "grade" can be inserted before the third column of the student information table. After the database modifies the structure of the student information table, it can call the data reconstruction thread to reconstruct the data in the student information table so that the data in the student information table matches the structure of the student information table, that is, the format of the data in the student information table meets the format requirements of the student information table.

[0066] It is understandable that each time the database performs a DDL operation on Table A, Table A will be updated. In some cases, after Table A is updated, if the application (application, APP) using Table A (such as a student information management application) does not perform corresponding adaptation based on the updated Table A, and still parses the updated Table A in the same way as before the update, parsing anomalies may occur, resulting in anomalies in the data read by the APP, and further causing anomalies in the corresponding business.

[0067] For example, Figure 4A As shown in the figure, after the database performs DDL operations on the student information table, a column "grade" is inserted between the second column "age" and the third column "class" of the original student information table, so that the student information table is updated from three columns to four columns. At this time, if the student information management application parses the updated student information table according to the parsing method before the update, it can only read the first three columns of the student information table, but cannot read the data in the fourth column, and cannot query the student's class information, which causes abnormalities in the student information management application business.

[0068] Based on this, every time the database performs a DDL operation on Table A, the APP using Table A needs to be adapted accordingly. However, during the adaptation period, the APP cannot use Table A, resulting in interruption of the APP's business, poor business continuity, and the problem of difficult smooth evolution of the business. In addition, the versions of the APP (such as the student information management application) installed on the terminal devices of different users may be different. For example, the terminal device of user A has the V1.1 version of the APP installed, while the terminal device of user B has the V1.2 version of the APP installed. Among them, the V1.2 version of the APP needs to use the updated Table A (such as the updated student information table), while the V1.1 version of the APP only needs to use the table A before the update. However, since Table A has been updated, the V1.1 version of the APP can only use the updated Table A, and the V1.1 version of the APP needs to be adapted for the updated Table A to avoid the V1.1 version of the APP parsing anomalies for the updated Table A. In this way, a certain amount of workload is added.

[0069] In order to solve the problem of difficult smooth evolution of business, in some embodiments, each time the database performs a DDL operation on Table A in the database, it can immediately create and store a metadata snapshot (Snapshot) of Table A (referring to a static view of the metadata at a certain point in time) to obtain the corresponding version of metadata. Among them, each version of metadata corresponds to a version of the table. This article refers to the version of the metadata snapshot as a "metadata version (schema version, SV)". In this way, after the database performs multiple DDL operations on Table A, it can obtain Table A corresponding to multiple SVs. In this case, the old version of the APP can use the old version of Table A without having to adapt to the new version of Table A, thereby ensuring business continuity.

[0070] For example, Figure 4BAs shown, when the database adds column c2 to Table A of version V0.0, Table A of version V1.1 can be obtained. When the database adds column c3 to Table A of version V1.1, Table A of version V1.2 can be obtained. When the database adds column c2 to Table A of version V1.2, Table A of version V1.3 can be obtained. Among them, Table A of version V1.1, Table A of version V1.2 and Table A of version V1.3 can all be stored in the database to be provided to different versions of APP. In this way, even if Table A is updated, it does not affect the use of APP. For example, the APP of version V1.1 can be bound to the SV with the version name V1.1, which is equivalent to establishing a user link on the SV with the version name V1.1, which corresponds to the APP of version V1.1, so that the APP of version V1.1 can use the Table A of version V1.1. Similarly, the APP of version V1.2 can be bound to the SV with the version name V1.2, so that the APP of version V1.2 can use the Table A of version V1.2. And the V1.3 version of the APP can be bound to the SV with the version name of V1.3, so that the V1.3 version of the APP can use the V1.3 version of Table A.

[0071] exist Figure 4B In the example, when Table A is updated from V1.2 to V1.3, the V1.2 APP can still use the V1.2 Table A, and does not need to be additionally adapted to the V1.3 Table A, thereby avoiding interruptions to the V1.2 APP’s business, ensuring business continuity and reducing workload. Moreover, when the user updates the APP from V1.2 to V1.3, since the V1.3 APP has been adapted to the V1.3 Table A, it can directly use the V1.3 Table A, allowing the APP’s business to migrate (evolve) smoothly.

[0072] S103: The database receives a DML instruction for table A.

[0073] When the database performs DDL operations on table A, table A will not be locked. At this time, the user can send DML instructions for table A to the database through the terminal device to add, delete, modify, and other operations on the data in table A.

[0074] For example, Figure 2C As shown, when the user wants to change the class of student "Wang Wu" from "Class 9" to "Class 7", the corresponding DML instruction can be sent to the database through the terminal device to instruct the database to change the class of student "Wang Wu" in the student information table from "Class 9" to "Class 7".

[0075] S104, the database determines whether the data to be operated by the DML has been reconstructed. If not, execute step S105. If yes, execute step S106.

[0076] After receiving the DML instruction, the database can determine whether the data to be operated by DML in table A has been reconstructed. If not, the data to be operated by DML needs to be reconstructed first, and then the DML operation can be performed on the data. If so, the data to be operated by DML does not need to be reconstructed, and the DML operation can be performed directly on the data.

[0077] S105: The database calls a data reconstruction thread to reconstruct the data to be subjected to the DML operation.

[0078] When the database determines that the data to be operated by DML has not been reconstructed, it can call the data reconstruction thread to determine the modified structure of table A based on the DDL instruction, that is, the modified metadata. Then, based on the modified structure, the data to be operated by DML is reconstructed so that the data matches the structure of table A, that is, the format of the data meets the format requirements of table A.

[0079] For example, Figure 2C As shown, when the third row of data in the student information table (as an example of data to be operated by DML) has not been reconstructed, the database can call the data reconstruction thread to reconstruct the third row of data in the student table "Wang Wu, 13, Class 9" into "Wang Wu, 13, NULL, Class 9" according to the modified metadata "name, age, grade, class" of the student information table.

[0080] It should be noted that the data reconstruction of DML operations and the data reconstruction of DDL operations can be executed concurrently, and the data that has been reconstructed will not be reconstructed again. Since the data reconstruction of DDL operations reconstructs the data of the entire table, while DML operations reconstruct part of the data in the table, the data reconstruction of DML operations can speed up the data reconstruction of DDL operations, thereby speeding up the execution of DDL operations.

[0081] S106: The database performs a DML operation on the data to be operated on.

[0082] After the data to be subjected to the DML operation is reconstructed, the data to be subjected to the DML operation meets the format requirements of Table A. At this time, the database can execute the DML instruction and perform the corresponding DML operation on the data.

[0083] For example, Figure 2CAs shown, after the third row of data "Wang Wu, 13, Class 9" in the student information table is reconstructed into "Wang Wu, 13, NULL, Class 9", the database can execute the DML instruction to modify the reconstructed data "Wang Wu, 13, NULL, Class 9" into "Wang Wu, 13, NULL, Class 7". In the embodiment of the present application, the database can perform DDL operations and DML operations on the tables in the database at the same time without blocking. This is because the embodiment of the present application decouples metadata updates and data reconstruction, and the database can not only reconstruct the data in the table when executing the DDL operation, but also reconstruct the data in the table when executing the DML operation. In this way, when the database executes the DML operation, it does not need to wait for the DDL operation to be completed, that is, it does not need the data reconstruction of the entire table to be completed, and it can directly reconstruct the data to be operated by DML in the table, so that the data to be operated by DML matches the structure of the table. Then perform the DML operation on the data to be operated by DML, and the situation of data inconsistency can be avoided. Based on this, the DDL operation and the DML operation can be executed concurrently, thereby improving the data processing efficiency.

[0084] The specific process of DDL operation is introduced below.

[0085] Figure 5 This is a flowchart of a DDL operation provided in an embodiment of the present application.

[0086] like Figure 5 As shown, the DDL operation includes the following steps:

[0087] S201: Parsing DDL instructions.

[0088] When a user wants to modify the structure of a table in the database (such as a student information table), the user can send a corresponding DDL instruction to the database through a terminal device. After receiving the DDL instruction, the database can call the DBMS system to parse the DDL instruction and obtain the corresponding parsing result.

[0089] S202: Update metadata.

[0090] After the database parses the DDL instructions, it can update the metadata of the table based on the parsing results.

[0091] For example, if the parsing result of the DDL instruction updates the metadata "name, age, class" of the student information table to "name, age, grade, class", the database can modify the metadata "name, age, class" of the student information table to "name, age, grade, class".

[0092] S203: Write to the log.

[0093] In an embodiment of the present application, a log generated before the metadata update is completed may be written into a database to determine information such as the metadata update process and update time based on the log.

[0094] S204: Create and store a metadata snapshot.

[0095] After updating the metadata of a table, the database can create and store a metadata snapshot of the table to obtain the corresponding SV, so that different versions of APP can use tables of different SVs, thereby realizing multi-version management of database metadata to ensure business continuity.

[0096] S205: Cache the updated metadata.

[0097] After updating the metadata of the table, the database can cache the metadata in a buffer.

[0098] S206: Start an asynchronous thread to reconstruct the data.

[0099] After the database updates the metadata of the table, the structure of the table changes. At this time, the database needs to start an asynchronous thread (ie, a data reconstruction thread) to reconstruct the data in the table so that the data in the table matches the updated structure of the table.

[0100] In an embodiment of the present application, the database can update the metadata of the table according to the DDL instruction, and after the update is completed, save the metadata snapshot of the table, thereby realizing multi-version management of the database metadata and ensuring business continuity.

[0101] It can be understood that after the database performs DDL operations on the tables in the database, the structure of the tables is changed. In order to avoid the mismatch between the data in the table and the structure of the table, the data in the table needs to be reconstructed so that the data in the table meets the structural requirements of the table. The following is an introduction to the process of data reconstruction.

[0102] Figure 6 This is an example diagram of the data reconstruction process provided in an embodiment of the present application.

[0103] like Figure 6 As shown, the process includes the following steps:

[0104] S301: Start a data reconstruction thread to scan the table page by page.

[0105] The tables in the database can be divided into multiple pages for storage. When executing DDL operations, the database can call the data reconstruction thread to scan a table (such as Table A) in the database in units of pages.

[0106] S302: Get Page version.

[0107] Page version refers to the version of the data in each page of the table.

[0108] In some embodiments, each time the data of each page of the table is updated, the data version (Page version) information of the page can be stored. In this case, the database can obtain the data version information of each page of the table to determine the Page version of the page of the table. The data version information may include information such as the version number and the version name.

[0109] In other embodiments, each time the data in the table is updated, the database generates a corresponding log to record the data update information, such as update time, update result, etc. In this case, the update time of the data in each page of the table can be determined by querying the log, and then the Page version of the table on that page can be determined according to the update time.

[0110] S303: Create or obtain a reconstruction task.

[0111] In some embodiments, after obtaining the Page version of a certain page table, the database can determine the latest update time of the Page version of the page table by querying the log corresponding to the page table, and then determine whether the Page version is the latest version based on the latest update time. If so, it means that the data in the page table has been reconstructed and no reconstruction is required. If not, it means that the data in the page table has not been reconstructed. At this time, the database can call the DBMS system to create a reconstruction task for subsequent execution based on the data difference between the current Page version (source Page version) and the latest Page version (destination Page version). In some cases, if the DBMS system has pre-created a corresponding reconstruction task, the database can directly obtain the reconstruction task for subsequent execution without having to recreate the reconstruction task.

[0112] S304: Reconstruct the Tuple data in the Page.

[0113] Tuple is a collection of ordered objects, such as a row of data in a table.

[0114] In some embodiments, the database may call a data reconstruction thread to perform a reconstruction task, and reconstruct each row of data in each page of the table in row order.

[0115] For example, Figure 2AAs shown, when the database reconstructs the data in the student information table, each row of data in the student information table can be reconstructed starting from the first row. Taking the first row of data as an example, the database can reconstruct the first row of data (data of the current Page version) "Zhang San, 13, Class 7" into "Zhang San, 13, NULL, Class 7" (data of the latest Page version).

[0116] In other embodiments, other methods may be used to reconstruct the data in the table. For example, each column of data in the table may be reconstructed starting from the first column of the table in units of columns.

[0117] S305: Update the Page version.

[0118] After the database updates the data in a page table, update the Page version corresponding to the page table.

[0119] S306: Complete Page reconstruction.

[0120] After the database completes updating the Page version of a page of a table, the data of the page of the table is reconstructed, that is, the Page reconstruction is completed. In the embodiment of the present application, the database can reconstruct each page of the table in this way so that all the data in the table matches the structure of the table, and after the reconstruction of all the data in the table is completed, the reconstruction task is ended.

[0121] In an embodiment of the present application, when the structure of a table changes, the database can reconstruct the data in the table so that the data in the table matches the changed structure of the table to avoid data inconsistency.

[0122] In the above Figure 5 and Figure 6 In the embodiment described, the database does not lock the table during the process of performing DDL operations on the table and reconstructing the data in the table. In this case, the database can perform other operations on the table, such as DML operations, DQL operations, etc.

[0123] The following is an explanation of the DML operation process.

[0124] Figure 7 This is a flowchart of a DML operation provided in an embodiment of the present application.

[0125] like Figure 7 As shown, the process of DML operation includes the following steps:

[0126] S401: Parse DML instructions.

[0127] When the database performs a DDL operation on a table in the database and the table is not locked, when a user wants to modify a page of data in the table, the user can send a corresponding DML instruction to the database through a terminal device. After receiving the DML instruction, the database can call the DBMS system to parse the DML instruction and obtain the corresponding parsing result.

[0128] S402: Determine the target Page.

[0129] After parsing the DML instruction, the database can determine the target Page to be modified based on the parsing result.

[0130] S403: Reconstruct the target Page.

[0131] After determining the target Page, the database can call the data reconstruction thread to execute the DML instruction to reconstruct the data in the target Page, for example, row by row or column by column.

[0132] For example, Figure 2C As shown, the database can reconstruct the third row of data "Wang Wu, 13, Class 9" in the student information table into "Wang Wu, 13, NULL, Class 9".

[0133] S404: Execute DML operation to update data.

[0134] After the reconstruction of the data in the target Page is completed, the database can perform DML operations on the data in the target Page.

[0135] For example, Figure 2C As shown, after the database reconstructs the third row of data "Wang Wu, 13, Class 9" in the student information table into "Wang Wu, 13, NULL, Class 9", it can perform DML operations on the reconstructed data "Wang Wu, 13, NULL, Class 9" to modify the reconstructed data "Wang Wu, 13, NULL, Class 9" into "Wang Wu, 13, NULL, Class 7".

[0136] S405: Complete the DML operation.

[0137] After the database completes the modification of the data in the target Page, the DML operation is completed.

[0138] In the embodiment of the present application, the database first reconstructs a page of data in a table, and then performs a DML operation on the data on the page, that is, performs a DML operation on a page basis. In other embodiments, a DML operation may also be performed on a row, column, or single data basis.

[0139] For example, the database may first reconstruct a row of data in a table, and then perform a DML operation on the row of data.

[0140] For another example, the database may first reconstruct a column of data in a table, and then perform a DML operation on the column of data.

[0141] For another example, the database may first reconstruct certain data in a table and then perform a DML operation on the data.

[0142] In the embodiment of the present application, the database will reconstruct the data in the table before performing DML operations on the data in the table, so that the data matches the structure of the table. Therefore, when the database performs DML operations on the data in the table, even if the structure of the table changes, it will not cause the abnormal phenomenon of inconsistency in the data in the table. In addition, the data in the table also needs to be reconstructed during the DDL operation. Reconstructing the data in the table before the DML operation can speed up the execution of the DDL operation.

[0143] It can be understood that by adding a data reconstruction operation before the DML operation, the table is no longer locked during the DDL operation, and the DDL operation can be prevented from blocking the DML operation, thereby improving the execution efficiency of the DML operation.

[0144] Since the database does not lock the table when performing DDL operations on the table in the database, in this case, the database can not only perform DML operations on the table, but also DQL operations on the table. The following introduces the process of DQL operations.

[0145] Figure 8 This is a flowchart of a DQL operation provided in accordance with an embodiment of the present application.

[0146] like Figure 8 As shown, the process of DQL operation includes the following steps:

[0147] S501: Parse the DQL instruction.

[0148] When a user wants to query data in a database, he can send a DQL instruction to the database through a terminal device. After receiving the DQL instruction, the database can call the DBMS system to parse the DQL instruction to obtain the corresponding parsing result. The parsing result may include the target data to be queried, the page information of the page where the target data is located (for example, page header, page number, etc.), the version of the table to which the target data belongs, and other information.

[0149] S502: Get the request version.

[0150] In the embodiment of the present application, each time the database modifies the metadata of a table in the database, the latest metadata snapshot is saved to obtain the latest version of the metadata. In this way, after multiple modifications are made to the metadata of the table, multiple versions of metadata, that is, multiple versions of the table, are obtained.

[0151] In some embodiments, when the terminal device is connected to the database, the APP on the terminal device (e.g., the student information management application) can be bound to the specified SV. In this case, the specified SV can be used as the request version. In other embodiments, when the terminal device is connected to the database, if the APP on the terminal device (e.g., the student information management application) is not bound to the specified SV, the latest SV can be used as the request version.

[0152] S503: Determine the target page.

[0153] After the database determines the table to which the target data to be queried belongs, it can determine the page where the target data to be queried is located from multiple pages of the target table as the target page according to the page information of the page where the target data is located in the parsing result of the DQL instruction.

[0154] S504: Query target data of the target page.

[0155] After the database determines the target page, it queries the target data from the target page according to the parsing result of the DQL instruction.

[0156] S505: Return target data.

[0157] After the database finds the target data, it returns the target data to the user's terminal device. In this way, the user can find the target data through the terminal device.

[0158] In the embodiment of the present application, since the DQL operation only queries data from the table and does not modify the data in the table, and the table is not locked, the DDL operation will not block the DQL operation, and the DQL operation can be executed normally.

[0159] Since the database saves a metadata snapshot every time it modifies the metadata of a table, it will store many versions of metadata. In order to save storage space, the stored metadata needs to be cleaned up. The following is an introduction to the metadata cleaning process.

[0160] Fig. 9 This is an example diagram of the metadata cleaning process provided in an embodiment of the present application.

[0161] like Fig. 9 As shown, the metadata cleanup process includes the following steps:

[0162] S601: Metadata version deletion.

[0163] This application adopts a multi-version metadata management solution. Compared with traditional databases, the metadata cleaning method must be adjusted accordingly. The metadata cleaning is changed to version-driven, that is, the metadata cleaning process can only be started when it is confirmed that the metadata corresponding to the version will no longer be used.

[0164] In some embodiments, when the metadata satisfies the following two conditions at the same time, it can be determined that the metadata will no longer be used.

[0165] Condition 1: There is no corresponding version of user data, that is, the data in the table has been reconstructed after the metadata is updated.

[0166] Condition 2: There is no user link for the corresponding version, that is, the table of the SV is not bound to the APP.

[0167] When the metadata satisfies both the above conditions 1 and 2, it means that the metadata version will no longer be used and can be deleted.

[0168] After determining that the metadata is no longer used based on the above conditions 1 and 2, the SV corresponding to the metadata can be deleted. In the database management system (i.e., DBMS system), each transaction will have a unique transaction commit sequence number (CSN), which is usually represented as an increasing integer or other unique identifier, used to determine the commit order of the transaction. After each metadata modification transaction is executed, the CSN can be submitted to the transaction manager once to determine the metadata modification transaction order based on the CSN, thereby determining the order of the SV. It can be understood that one SV corresponds to one CSN.

[0169] S602: Obtain metadata version CSN.

[0170] In some embodiments, after determining that the SV corresponding to the metadata that is no longer in use is determined, the CSN corresponding to the SV may be obtained from the transaction manager.

[0171] S603: truncate the metadata log with CSN as the foreground point.

[0172] The metadata log is a log that records metadata.

[0173] In some embodiments, SVs that meet the above conditions one and two can be determined, and then the SV with the largest CSN can be determined among these SVs. The CSN of this SV is used as the Forzen point (truncation point), and then the metadata log is truncated according to the Forzen point, such as deleting the metadata log before the Forzen point.

[0174] S604: Metadata cleanup is completed.

[0175] When the metadata log before the Forzen point is deleted, the corresponding metadata is deleted to complete the metadata cleanup.

[0176] The embodiment of the present application can clean up the metadata that is no longer used to release more storage space and reduce the pressure of data storage. It can be understood that when the SV is cleaned up, the corresponding table is also cleaned up.

[0177] Fig.10 This is an example diagram of the DBMS system architecture of the database provided in the embodiment of the present application. The database in the present application can be a database of the type of SQL, NoSQL, Oracle, MySQL, openGauss, etc., and the following description will be made using the SQL database as an example.

[0178] like Fig.10 As shown, the architecture includes advanced command line tools, process management (Process Manager) module, client connection management (Client Communications Manager) module, relational query processing (Relational Query Processor) module, transactional storage management (Transactional Storage Manager) module, shared components and utilities (Shared Components and Utilities) module.

[0179] Among them, advanced command line tools include pt-online-schema-change (a tool developed by Percona) and pt-show-grants. pt-online-schema-change is used to modify tables online without locking them. pt-show-grants is used to print out database authorization related information in a standardized manner.

[0180] The client connection management module is responsible for handling client connections, maintaining connection status, and related operations when transmitting data between the client and the database server.

[0181] The process management module includes the dispatch and scheduling module and the admission control module. The call module is used to manage and coordinate system resources in order to effectively execute and allocate tasks, processes or requests. The admission control module is used to manage and control the entry of new requests, tasks or resources into the system.

[0182] The relational query processing module includes the SQL parsing and authorization module, the query rewriting module, the query optimization module, the plan execution module, and the DDL execution module.

[0183] Among them, the SQL parsing and authorization module is responsible for parsing the database query language, verifying user permissions, and deciding whether to allow query operations. The query rewrite module is used to optimize and rewrite queries submitted by users to improve query performance, reduce execution costs or achieve other optimization goals. The query optimization module is responsible for analyzing and optimizing query statements submitted by users to generate the optimal execution plan to achieve the best query performance. The plan execution module is responsible for actually executing database query operations according to the execution plan generated by the query optimizer, and returning the query results to the user or application. The DDL execution module is used to handle the database structure definition, metadata management and various utility operations, and is responsible for managing the creation, modification, and deletion of database objects, as well as performing various tasks related to database structure and management.

[0184] The transactional storage management module includes the access method module, the buffer manager module, the lock manager module, and the log manager module. Among them, the access method module is used to manage and process the storage and access methods of data in the database. The buffer manager module is used to store metadata, logs and other data. The lock manager module is used to manage concurrent access to the database data to ensure isolation and consistency between transactions. The log manager module is used to record database operations and changes to achieve data persistence, recovery and consistency, and is responsible for recording transaction operation logs so that data recovery and rollback operations can be performed in the event of system failure or crash.

[0185] The shared components and tool modules include the Catalog Manager module, the Memory Manager module, the Administration, Monitoring & Utilities module, and the Batch Utilities module.

[0186] The catalog management module is used to manage the metadata information of the database, including the definition and attributes of database objects, tables, columns, indexes, views, users, permissions, etc. The memory management module is responsible for managing the allocation, use and release of system memory to support the execution of database operations and queries. The management, monitoring and utility module is used to manage, monitor and execute various database management tasks and utility operations. The batch tool module is used to execute batch processing tasks and utility operations.

[0187] For the above Fig.10 For the database shown, you can use the advanced command line tool pt-online-schema-change in the database to perform online DDL operations on the table. In the process of using pt-online-schema-change to perform online DDL operations on the table, DML operations are allowed on the data table. Online DDL means that the database management system supports the definition, modification, and adjustment of the schema at runtime without stopping the database service or restarting it. In other words, structural changes can be made to tables, indexes, columns, etc. while the database is active. The database schema is a structural description of the database. In a relational database, the database schema defines the tables, fields in the tables, and the relationships between tables and fields.

[0188] The following describes the process of performing online DDL operations using pt-online-schema-change.

[0189] Fig.11 The following is a flow chart showing how to perform online DDL operations for pt-online-schema-change. Fig.11 As shown, the process includes the following steps:

[0190] S701: Receive an online DDL instruction.

[0191] When a user wants to perform an online DDL operation on a table in a database, he can send an online DDL instruction to the database.

[0192] S702: Create a replication table XXX_new.

[0193] After receiving the online DDL instruction, the database can call pt-online-schema-change to copy the structure of the table named "XXX" to the replicated table named "XXX_new".

[0194] S703: Execute DDL operations on XXX_new.

[0195] After creating the replicated table XXX_new, the database can call pt-online-schema-change to perform DDL operations on the replicated table XXX_new, such as adding fields, changing field types, etc.

[0196] S704: Create Update, Insert, and Delete triggers.

[0197] The database can call pt-online-schema-change to create Update triggers, Insert triggers, and Delete triggers in the original table.

[0198] S705: Start data replication and log synchronization.

[0199] After creating Update triggers, Insert triggers, and Delete triggers in the original table, you can use Update triggers, Insert triggers, and Delete triggers to copy the data in the original table to the copied table.

[0200] In order to ensure data security, the database is usually deployed in a master-slave database mode to achieve the purpose of data backup. In order to ensure data synchronization between the master and slave databases, when the database calls pt-online-schema-change to perform DDL operations on the table, the log of the master database (master) can be configured, and then the log of the master database can be synchronized to the slave database (slave) to copy the changes in the table to the slave database, so that the slave database can adjust the table in its own database according to the synchronized log to ensure consistency between the master and slave.

[0201] S706: Complete data replication and log synchronization.

[0202] After all the data in the original table is copied to the copy table, data replication is completed. After the logs of the primary database server are synchronized to the secondary database server, log synchronization is completed.

[0203] S707: Replace XXX with XXX_new.

[0204] After completing data replication and log synchronization, the database can call pt-online-schema-change to replace the original table with the replicated table.

[0205] S708: Delete XXX.

[0206] The database calls pt-online-schema-change to delete the original table.

[0207] S709: Complete the online DDL.

[0208] After the database calls pt-online-schema-change to complete the DDL operation on the replicated table, the online DDL is completed.

[0209] Since pt-online-schema-change creates a duplicate table (secondary table) for the original table (primary table) and adds Update triggers, Insert triggers, and Delete triggers to the original table, these triggers can be used to synchronize data changes in the table. When performing DDL operations on the table, DML operations can be performed on the table, supporting concurrent execution of DDL operations and DML operations to avoid blocking. However, creating a duplicate table will double the storage space and increase the pressure on data storage. And if the data in the table is large, it will take a long time for the data synchronization between the primary and secondary tables. Therefore, the pt-online-schema-change solution is difficult to apply to scenarios with increased data volume, and has the defects of large storage space consumption and long data synchronization time.

[0210] In some embodiments, the Instant algorithm can also be used for online DDL operations. This algorithm only updates metadata and does not immediately reconstruct data. The DDL instruction execution speed is very fast and does not block DML operations and DQL operations. However, the Instant algorithm only supports a small number of DDL type operations such as adding columns, and the Instant algorithm will not be supported if the number of reserved columns is insufficient. Therefore, the Instant algorithm has certain limitations.

[0211] In the above Fig.10 As shown in the database system, Fig.12 As shown, the technical solution of the present application improves the client connection management module, SQL parsing and authorization module, directory management module, DDL execution module and log management module in the database system.

[0212] For example, the client connection management module adds the capability of version connection management on the basis of existing capabilities. The SQL parsing and authorization module adds the capability of SV management command line parsing on the basis of existing functions. The directory management module adds the capability of multi-version metadata management on the basis of existing capabilities, such as multi-version metadata caching capability, version metadata query capability, and version metadata update capability. The log management module adds the capability of metadata history version management and garbage collection on the basis of existing capabilities.

[0213] Combine the following Fig.12 The data system architecture example diagram shown introduces the DDL operation process of this application.

[0214] Fig.13 A flowchart of a DDL operation process provided in an embodiment of the present application.

[0215] like Fig.13 , the DDL operation includes the following steps:

[0216] S801: The SQL parsing and authorization module parses the DDL instruction.

[0217] When a user wants to modify the structure of table A in the database, the user can send the corresponding DDL instruction to the database through the terminal device. After receiving the DDL instruction, the database can call the SQL parsing and authorization module in the DBMS system to parse the DDL instruction and obtain the corresponding parsing result.

[0218] S802: The DDL execution module executes a metadata update operation.

[0219] After the SQL parsing and authorization module parses the DDL instructions, the DDL execution module executes the DDL operation and updates the metadata.

[0220] For example, if the parsing result of the DDL instruction updates the metadata "name, age, class" of the student information table to "name, age, grade, class", the database can modify the metadata "name, age, class" of the student information table to "name, age, grade, class".

[0221] S803: The buffer management module updates metadata.

[0222] After the DDL execution module performs the metadata update operation, the buffer management module stores the updated metadata in the buffer.

[0223] S804: The log management module writes the log.

[0224] After the buffer management module stores the updated metadata in the buffer, the log management module can generate a corresponding log based on the metadata update process, update time and other information, and write the log into the database. The log can be used to determine the metadata update process and update time.

[0225] S805: The buffer management module creates and stores a metadata snapshot.

[0226] The buffer management module may create and store a metadata snapshot of the table.

[0227] S806: The directory management module updates the metadata cache.

[0228] The directory management module stores the updated metadata in the buffer.

[0229] S807: The process management module starts an asynchronous thread to complete data reconstruction.

[0230] Since the structure of the table changes after the database updates the metadata of the table, the database can call the process management module to start the starting thread (ie, the data reconstruction thread) to reconstruct the data in the table so that the data in the table matches the updated structure of the table.

[0231] In the embodiment of the present application, since the DDL execution module does not lock the table when executing the DDL operation, the DML operation can be performed on the table. In this case, when the database receives the DML operation instruction, the process management module can first call the data reconstruction thread to reconstruct the data to be operated by the DML operation, so that the data to be operated by the DML operation meets the structural changes of the table, and then the DML operation is performed on the data to be operated by the DML operation. In this way, neither data inconsistency nor blocking will occur.

[0232] It can be understood that the data processing method of the present application adds a data reconstruction operation before the DML operation, solves the blocking problem, and does not create a duplicate table. Compared with the pt-online-schema-change solution, it does not add additional storage space and can be applied to scenarios with large data volumes. Moreover, compared with the limitations of the Instant algorithm, the data processing method of the present application can be applied to various DDL operations and is not limited by the reserved space, so it has good applicability.

[0233] This application provides a metadata multi-version management solution that is fully compatible with traditional database usage. At the same time, in order to enable the database to provide multiple versions of user data services based on the same data, this application provides users with a complete set of interfaces to manage metadata versions, and users can decide whether to use these interfaces based on their own needs.

[0234] The following is an introduction to the interface provided by this application.

[0235] (1) Create SV

[0236] After the database administrator (DBA) calls the database to perform a specific DDL operation, the database will generate a corresponding metadata snapshot and store it in the metadata snapshot system table (pg_catalog_snapshot) as an internal SV. After each DDL operation, the database can call the SV creation interface provided by this application to create the corresponding SV.

[0237] The following uses SQL as an example to illustrate the interface for creating SV.

[0238] The SQL syntax is shown in Table 1:

[0239] Table 1

[0240]

[0241] The interface parameters are shown in Table 2:

[0242] Table 2

[0243]

[0244] As shown in Table 2, in this interface, the parameter SCHEMAVERSION is the SV name, which can be expressed as version_name. The parameter SCHEMAVERSION is unique in the same CDB or ADB and cannot be repeated. Its data type can be varchar type, and the data length is 1-64 bytes. The parameter COMMENTS is an optional parameter used to save version description information and distinguish SVs. Its data type is varchar type, and the data length is 0-256 bytes.

[0245] The SQL examples are shown in Table 3:

[0246] Table 3

[0247]

[0248] For example, Figure 4B As shown, the version name of the initial SV is V0.0, corresponding to the initial table A of version V0.0. After the database performs DDL operations on the initial table A of version V0.0, such as adding column c2 to table A of version V0.0, the SV creation interface can be called to create an SV with a version name of V1.1, and the SV can be bound to the APP of version V1.1, which is equivalent to establishing a link of the APP of version V1.1 on the SV with a version name of V1.1, so that the APP of version V1.1 can use the table A of version V1.1. Similarly, when the data server performs DDL operations on the table A of version V1.1, such as adding column c3, the SV creation interface can be called to create an SV with a version name of V1.2 and the SV can be bound to the APP of version V1.2, so that the APP of version V1.2 can use the table A of version V1.2. When the data server performs DDL operations on the V0.0 version of the table, such as adding column c2, it can call the SV creation interface to create an SV with the version name V1.3 and bind the SV to the V1.3 version of the APP, so that the V1.3 version of the APP can use the V1.3 version of table A.

[0249] (2) Query SV

[0250] When the database receives an SV query request, it can call the SV query interface provided by this application to query the information of the specified SV.

[0251] The SV query interface is explained using SQL as an example.

[0252] The SQL syntax is shown in Table 4:

[0253] Table 4

[0254]

[0255] An example of the returned result is shown in Table 5:

[0256] Table 5

[0257] schema version_name create_time comments schema1 V1.1 2021-09-24 09:15:42.099642+08 For App V1.1 schema1 V1.2 2021-09-25 09:15:42.099642+08 For App V1.2

[0258] The returned result parameters are shown in Table 6:

[0259] Table 6

[0260]

[0261] As shown in Table 6, in this interface, the parameter schema (database mode) is the current schema name, such as schema1 in Table 5. The parameter version_name is the SV name, such as V1.1 and V1.2 in Table 5. The parameter create_time is the timestamp corresponding to the SV, such as 2021-09-24 09:15:42.099642+08 and 2021-09-25 09:15:42.099642+08 in Table 5. The parameter create_time is automatically generated by the DBMS system and can be displayed in Unix timestamp format. The parameter comments is the version description information, which is used to distinguish SVs, such as For App V1.1 and For App V1.2 in Table 5.

[0262] For example, Figure 4B As shown, when the database receives an SV query request, the SV query interface can be called to query the SV information shown in Table 5 above, such as the schema name, SV name, the timestamp corresponding to the SV, and the version description information (i.e., the APP information bound to the SV).

[0263] (3) Delete the specified SV

[0264] When the database receives an SV deletion instruction, it can call the SV deletion interface provided by the present application to delete the specified SV to delete the table corresponding to the SV.

[0265] The following uses SQL as an example to illustrate the SV deletion interface.

[0266] The SQL syntax is shown in Table 7:

[0267] Table 7

[0268]

[0269] The interface parameters are shown in Table 8:

[0270] Table 8

[0271]

[0272] As shown in Table 8, in this interface, the parameter SCHEMAVERSION indicates the name of the SV to be deleted, its data type is varchar type, and the data length is 1-64 bytes.

[0273] For example, Figure 4B As shown, when the database receives a deletion instruction, the SV deletion interface can be called to delete the specified SV. For example, the SV with the version name V0.0 can be deleted to delete the V0.0 version of Table A.

[0274] (4) Query the definition of the table in SV

[0275] When the database receives a table definition query instruction, it can call the table definition query interface provided by the present application to query the definition of the specified table.

[0276] The table definition query interface is explained using SQL as an example.

[0277] The SQL syntax is shown in Table 9:

[0278] Table 9

[0279]

[0280] The interface parameters are shown in Table 10:

[0281] Table 10

[0282]

[0283] As shown in Table 10, the parameter TABLE indicates the table name to be queried. The parameter SCHEMAVERSION indicates the SV name corresponding to the table name to be queried, and its data type is varchar type, and the data length is 1-64 bytes.

[0284] An example of the returned result is shown in Table 11:

[0285] Table 11

[0286] column type modifiers Storage Description id integer PRIMARY KEY not null plain name character varying(50) extended extended gender character varying(20) extended

[0287] The returned result parameters are shown in Table 12:

[0288] Table 12

[0289] parameter illustrate column Table field name type Field Type modifiers Field Modifiers Storage Field storage method Description Field Description

[0290] As shown in Table 12, the column parameter indicates the field name, such as id, name, and gender in Table 11. The type parameter indicates the field type, such as integer, character varying(50), and character varying(20) in Table 11. Character varying(50) indicates that a string of up to 50 characters can be stored, and character varying(20) indicates that a string of up to 20 characters can be stored. The modifiers parameter indicates field modifiers, such as PRIMARY KEY not null and extended in Table 11. PRIMARY KEY not null indicates a primary key non-null constraint. Extended indicates that out-of-row storage is allowed, but compression is allowed. The Storage parameter indicates the field storage method, such as plain and extended in Table 11. Plain indicates avoiding compression and out-of-row storage. Description indicates the field description information.

[0291] For example, Figure 4B As shown, when the database receives a table definition query instruction sent by the V1.3 version APP, the table definition query interface can be called to query the table definition information corresponding to the SV with the version name V1.3. For example, the query results shown in Table 11 can be queried.

[0292] (5) Query all metadata information in a specified SV

[0293] When the database receives a metadata information query instruction, it can call the metadata information query interface provided by this application to query all metadata information in the specified SV.

[0294] The metadata information query interface is explained using SQL as an example.

[0295] The SQL syntax is shown in Table 13:

[0296] Table 13

[0297]

[0298] The interface parameters are shown in Table 14:

[0299] Table 14

[0300] parameter illustrate SCHEMA The name of the schema to query SCHEMAVERSION The name of the SV to be queried

[0301] As shown in Table 14, in this interface, the parameter SCHEMA indicates the name of the schema to be queried.

[0302] SCHEMAVERSION indicates the name of the data version to be queried.

[0303] Examples of returned results are shown in Table 15 List of table, Table 16 List of index, Table 17 List of view, Table 18 List of functions, and Table 19 List of sequence:

[0304] Table 15 List of table

[0305] Name Type Owner Storage table1 table gauss {orientation=row,compression=no} table2 table gauss {orientation=row,compression=no} table3 table gauss {orientation=row,compression=no}

[0306] In Table 15, the parameter Name indicates the table name, for example, table1, table2, table3. The parameter Type indicates the data type. When Type is table, it indicates that the data type is table. The parameter Owner indicates the data source. When Owner is gauss (Gaussian database), it indicates that the table comes from gauss. The parameter Storage indicates the storage mode. For example, when Storage is {orientation=row, compression=no}, it indicates that the table is stored in row mode and is not compressed.

[0307] Table 16 List of index (index table)

[0308] Name Type Owner Table Storage index1 index gauss t1 index2 index gauss t2 index3 index gauss t3

[0309] In Table 16, the parameter Name indicates the index name, such as index1, index2, and index3. The parameter Type indicates the data type. When Type is index, it indicates that the data type is index. The parameter Owner indicates the data source. When Owner is gauss, it indicates that the table comes from gauss. The parameter Table indicates the table name, such as t1, t2, and t3. The parameter Storage indicates the storage method.

[0310] Table 17 List of view (view table)

[0311] Name Type Owner Storage view1 view gauss view2 view gauss view3 view gauss

[0312] In Table 17, the parameter Name indicates the view name, such as view1, view2, and view3. The parameter Type indicates the data type. When Type is view, it indicates that the data type is view. The parameter Owner indicates the data source. When Owner is gauss, it indicates that the table comes from gauss. The parameter Storage indicates the storage method.

[0313] Table 18 List of functions

[0314] Name Result data type Argument data types Type Fenced mode fun1 text id character varying normal f fun2 integer normal f fun 3 text normal f

[0315] In Table 18, the parameter Name indicates the function name to be queried, that is, the function name, such as fun1, fun2, fun3. The parameter Result data type indicates the data result type. When Type is text, it indicates that the data type is text. When Type is integer, it indicates that the data type is integer. The parameter Argument data types indicates the parameter data type. For example, id character varying indicates that the ID character varies. The parameter Type indicates the function mode. When Type is normal, it indicates the normal function mode. The parameter Fencedmode indicates the fence mode. When Fencedmode is f, it indicates that the corresponding function is not enabled.

[0316] Table 19 List of sequence

[0317] Name Type Owner Storage seq1 sequence gauss seq2 sequence gauss seq3 sequence gauss

[0318] In Table 19, the parameter Name indicates the sequence name, for example, seq1, seq2, seq3. The parameter Type indicates the data type. When Type is sequence, it indicates that the data type is sequence. The parameter Owner indicates the data source. When Owner is gauss, it indicates that the table comes from gauss. The parameter Storage indicates the storage method.

[0319] For example, Figure 4B As shown, when the database receives a metadata information query instruction sent by the V1.3 version APP, the metadata information query interface can be called to query all metadata object information under the SV with the version name V1.3, such as tables, indexes, views, stored procedures (function parameters), sequences, etc.

[0320] (6) Query the user links established on the specified SV

[0321] When the database receives a user link query instruction, it can call the user link query interface provided by the present application to query the user link information established on the specified SV.

[0322] The user link query interface is explained using SQL as an example.

[0323] The DBA needs to manage all user links. By calling this interface, the DBA obtains all user links established on the specified SV and then manages the user links.

[0324] Querying the user link displays the query results of the system view pg_stat_activity. Each record represents a working thread. The system view pg_stat_activity displays some information about the active threads of the current session, such as the status and query of the current session.

[0325] An example of viewing user link information in the system view pg_stat_activity is shown in Table 20:

[0326] Table 20

[0327]

[0328] In Table 20, the parameter datid indicates the data ID, such as 15140. The parameter datname indicates the data name, such as postgres. The parameter schema indicates the database schema name, such as scehma1. The parameter shemaversion indicates the SV name, such as V1.1. The pid indicates the process control symbol, such as 14035566254125, 14035566254126. The parameter Owner indicates the data source. When Owner is gauss, it indicates that the table comes from gauss. The parameter backend_start indicates the timestamp of the process start time, that is, the time when the user link is established on the specified SV, for example, 2021-09-24 09:15:42.099642+08, 2021-09-25 09:15:42.099642+08.

[0329] The SQL syntax is shown in Table 21:

[0330] Table 21

[0331]

[0332] When the APP requests to establish a user link, the request contains schema information and can also specify the corresponding SV information. The client and server will save the corresponding link information. To query the link list, you can modify the definition of the system view pg_stat_activity based on the existing implementation of openGauss and add the schema field and schema_version field to the definition.

[0333] The interface parameters are shown in Table 22:

[0334] Table 22

[0335] parameter illustrate SCHEMA The database schema name corresponding to the user link SCHEMAVERSION SV name corresponding to the user link

[0336] Return result: The query result of the current system view pg_stat_activity is displayed.

[0337] As shown in Table 22, in this interface, the parameter SCHEMA indicates the database schema name corresponding to the user link. The parameter SCHEMAVERSION indicates the SV name corresponding to the user link.

[0338] For example, Figure 4B As shown, when the server receives a user link query request sent by the V1.1 version APP, it can call the user link query interface to query the user link information on the SV with the version name V1.1, such as the information shown in Table 20.

[0339] (7) Switch SV

[0340] When the database receives the SV switching instruction, it can call the SV switching interface provided by this application to switch the SV bound to the APP.

[0341] The SV switching interface is described using SQL as an example.

[0342] The SQL syntax is shown in Table 23:

[0343] Table 23

[0344]

[0345] The interface parameters are shown in Table 24:

[0346] Table 24

[0347] parameter illustrate SCHEMAVERSION Target SV Name

[0348] As shown in Table 24, in this interface, the parameter SCHEMAVERSION indicates the target SV name.

[0349] The returned results are shown in Table 25:

[0350] Table 25

[0351] Return result scene Return information Execution Success SUCCESS Execution failed Cause of failure

[0352] As shown in Table 25, when the returned result is successful, that is, the SV switching fails, the returned information is "SUCCESS". When the returned result is failed, that is, the SV switching succeeds, the returned information includes the failure reason.

[0353] When the APP is connected to the database, if the APP has not been bound to the SV, the database can call the SV switching interface to bind the APP to the specified SV, so that the APP can use the table corresponding to the specified SV. Figure 4B As shown, when the V1.2 version of the APP is connected to the database for the first time, the database can call the SV switching interface to bind the V1.2 version of the APP to the SV with the version name V1.2, so that the V1.2 version of the APP can use the V1.2 version of Table A. At this time, when the database receives the SV switching instruction sent by the V1.2 version of the APP, it can call the SV switching interface to switch the SV bound to the APP, such as unbinding the APP from the SV with the version name V1.2, and then binding the APP to the SV with the version name V1.3, so that the APP can use the V1.3 version of Table A. In this way, the database can switch the SV bound to the APP by calling the SV switching interface, which improves the flexibility and continuity of the business.

[0354] The following is an introduction to metadata management.

[0355] (1) Metadata version SV

[0356] After the metadata is updated, a new metadata snapshot will be automatically formed, which is consistent with the concept of version in multi-version concurrency control (MVCC). Metadata snapshots are used to mark versions in this application. The main advantage is that snapshot-based metadata queries can be quickly implemented with the help of the existing MVCC algorithm in the in-situ update storage engine (Ustore) without the need for repeated development. During system operation, the corresponding metadata snapshot can be recorded and stored in the system table after each DDL operation is executed. In the actual processing process, since the metadata affected by some DDL instructions does not need to be managed in multiple versions, such as indexes, constraints and other related DDL operations, not all DDL operations will record the corresponding metadata snapshots.

[0357] The Ustore storage engine, also known as the In-place Update storage engine, is a new storage mode added to the openGauss kernel. The Ustore storage engine stores the latest version of "valid data" and the historical version of "junk data" separately. The latest version of "valid data" is stored on the data page, and a separate Undo space is opened up to uniformly manage the historical version of "junk data". Therefore, the data space will not expand due to frequent updates, and the centralized recovery of "junk data" is more efficient.

[0358] (2) User link version management

[0359] In this application, both user request parsing and request execution require the support of SV, and the SV corresponding to the user request is determined by the user link. Therefore, after using the metadata multi-version feature, when the user link is established, the connection manager needs to be responsible for managing the corresponding relationship between the user link and the SV.

[0360] If the user specifies an SV when initiating a database connection request through the APP, the APP will be bound to the specified SV so that the APP can use the table corresponding to the specified SV. During the connection, if the SV switching interface is not called to switch the SV, the bound SV will remain unchanged. If the SV is not specified when initiating a database connection request, the user request corresponding to the database connection will be processed in real time according to the latest metadata, which can be directly obtained by querying the system table. In this way, it can provide user business version management capabilities while being fully compatible with traditional database operating habits, providing support for the smooth evolution of user business.

[0361] It can be understood that DDL operations are always performed based on the latest metadata. The target metadata can be obtained directly by querying the system table without considering version issues.

[0362] (3) Historical metadata management

[0363] Different databases and storage engines can manage historical data in different ways. In the append-update storage engine (Astore), historical version data is mixed with the latest data and stored in the Page. A special garbage collector scans the Page in the table, identifies the version of the tuple in the Page, and cleans up the historical version tuple that will no longer be used to free up storage space. In the Ustore storage engine, only the latest version of data is saved in the Page, and historical version data is centrally stored in the Undo Log (a mechanism for recording historical version information of transaction operations). When cleaning garbage (metadata), the Frozen point can be determined, and then the log can be truncated according to the Frozen point to retain the corresponding log (such as the latest log) and complete the corresponding garbage cleaning.

[0364] In this application, the historical version of metadata can be used for a long time. For Astore, the garbage collector can skip the scan of the system table during operation. The cleaning of historical version data is triggered by the SV deletion operation. In Ustore, since the historical version of metadata and the historical version of user data are mixed and stored in the Undo Block (the basic unit for storing transaction historical version data), and the truncation algorithm used in the existing log cleaning algorithm, in order to maintain the metadata of the historical version, a large number of Undo logs (logical units for storing transaction historical version data) will not be cleaned for a long time, occupying a large amount of storage space. Therefore, for Ustore, it is necessary to store the Undo Record corresponding to the system table separately, and its cleaning is also triggered by the SV deletion operation.

[0365] (4) Metadata Query

[0366] When the metadata multi-version feature is not enabled, the database processes requests corresponding to the latest metadata, and the latest metadata can be obtained by directly querying the system table. The specific implementation adopts MVCC technology. When using the metadata multi-version feature, the user connects to the database through the specified SV. The DML instructions and DQL instructions sent to the database through the connection are processed according to the metadata corresponding to the SV. The corresponding metadata is obtained through the metadata database snapshot query specified in the SV.

[0367] The following is an introduction to user data (data stored in the table) management.

[0368] In traditional databases, since multiple versions of metadata are not supported, when reconstructing user data, user data can only be reconstructed in units of tables. As a result, during the reconstruction of user data versions, user requests cannot accurately perceive the version status of user data, and data consistency can only be ensured by locking or switching the master copy. Locking user data can ensure data consistency during the evolution process, but it will block other user requests in the database system. The master copy switching method will not block user requests but will consume a lot of storage space. In addition, for update-intensive applications, master copy synchronization is difficult to implement and takes a long time.

[0369] In this application, asynchronous reconstruction of user data is completed by user data version identification and data version conversion. This solution can make DDL operations and DML operations non-blocking, realize online DDL, and enable the database to provide database services for requests for different versions of metadata based on the same user data.

[0370] In traditional databases, user data storage mainly solves the problems of data storage format, free space management, storage space recycling, etc. This application mainly explains the user data version problem introduced by multiple versions of metadata, mainly including user data version identification, user data version conversion and user data reconstruction.

[0371] (1) User data version identification

[0372] In this application, in order to implement online DDL, the entire table is not locked during the user data reconstruction process, and the user data version update and metadata update are performed asynchronously. When users operate based on different SVs, multiple different versions of user data will coexist in one table in the database. Therefore, in order to correctly parse user data, a solution must be provided to accurately identify the user data version.

[0373] Considering that the storage engine manages transaction information related to user data, the transaction information can be used to confirm the timing of data generation, and then the data version can be identified with the help of the transaction information, so there is no need to adjust the storage structure of user data. Taking Ustore as an example, when identifying the user data version, the transaction information corresponding to the tuple can be obtained first, and then the SV corresponding to the tuple can be determined by the transaction information.

[0374] (2) User data version conversion

[0375] When performing DQL or DML operations on user data, if the user data version is inconsistent with the target data version, the user data needs to be converted. There are two types of user data version conversion: version evolution and version rollback. Version evolution occurs when the data version is lower than the user's requested version; conversely, when the data version is higher than the requested version, a version rollback is required.

[0376] User data in the database always evolves from a low version to a high version. Therefore, the ReorgTask (reorganization task) corresponding to the table can be managed in the form of a linked list. When performing subsequent version conversions, the specific content of the version conversion task can be quickly determined through tblOid (table object identifier), srcVersion (source version of the data) and destVersion (destination version of the data). In specific implementation, the ReorgTask can be stored in the form of a linked list in the corresponding Table version in the relation cache (RelCache) to achieve global sharing. When querying a reconstruction task, if the corresponding task does not exist in the cache, call the reconstruction task creation interface to create and cache the corresponding task.

[0377] In some embodiments, when performing version conversion processing of UHeapTupleData (a data used to represent UHeap tuples), toDeleteColList (list of columns to be deleted) and toAddColList (list of columns to be added) can be generated according to the metadata differences of different versions. Then the current IsNull field of the original tuple is obtained, and it is determined whether the IsNull field is in toDeleteColList. If so, it means that the IsNull field will be deleted and the processing of the field will be skipped. If not, the corresponding bit in the original IsNull is copied to the new IsNull. If the current field is not empty, the original field value is copied.

[0378] In a database (such as a cloud-native database), newly added fields can only be stored after existing fields. For newly added fields, the corresponding bitmap value does not need to be stored, and the newly added field value is the default value, and there is no need to fill it (except when the user explicitly assigns the default value through Insert or Update). Therefore, during reconstruction, the fields in toAddColList are not processed.

[0379] For user data version rollback, you only need to swap the contents in toDeleteColList and toAddColList to implement user data version rollback.

[0380] (3) Transaction execution (DML / DQL)

[0381] When executing a transaction, the first thing to do is to confirm the request version and the user data version. If the two versions are inconsistent, version conversion is required and then processed according to the traditional database transaction execution process.

[0382] Since DML operations will change the transaction information corresponding to the data, changes in transaction information will cause changes in the version information corresponding to the data. Therefore, a strong constraint must be added to the management of user data, that is, when executing DML operations to update data, the updated user data version must be the latest SV in the current database to ensure that the user data corresponds to the real version. In other words, when a transaction updates user data, regardless of whether the SV corresponding to the request is the latest version, the user data must be versioned and stored in the structure corresponding to the latest SV to ensure that the SV corresponding to the data can be identified through the transaction information corresponding to the data. Since queries do not change the transaction information of user data, there is no need to version evolve the user data during the query process.

[0383] The following is an introduction to user data reconstruction.

[0384] In this application, when DDL updates the user data structure, it is necessary to perform version evolution on the user data. In the specific implementation process, it is reflected in the reconstruction of user data. Taking the addition and deletion of data columns as an example, the fields corresponding to the target columns in the user data will be added and deleted during the reconstruction process. The data reconstruction thread is started, and the thread completes the corresponding reconstruction task.

[0385] The data reconstruction thread will obtain the Pages in the user table one by one through the full table scan (SeqScan), and reconstruct the tuples in the Page one by one. In order to record the reconstruction progress, it is necessary to add a field in the page header (PageHeader) to record the CSN number corresponding to the SV when the last reconstruction was completed in the current page, so as to facilitate the quick determination of the starting page after the reconstruction failure is recovered. When reconstructing the data in the table, you can directly obtain the specified page through the cache interface provided by the buffer pool, and then reconstruct the user data in the page and make necessary updates to the corresponding index.

[0386] User data reconstruction in the database is the process of converting user data from a lower version to a higher version and storing it, which is equivalent to modifying the data in the page.

[0387] When the database reconstructs a tuple, it will clean up the deleted columns in the original tuple, and the newly added columns will not occupy additional storage space. Therefore, the entire reconstruction will only release storage space and will not occupy additional space.

[0388] When reconstructing tuples in a page of a table, you can obtain the CSN corresponding to the table and obtain the data reconstruction task from the cache based on the CSN. If there is no data reconstruction task, build the data reconstruction task by querying the system table and cache it. Then, according to the data reconstruction task, perform version conversion on the tuples in the page, store the reconstructed tuple next to the previous tuple in the page, and update the corresponding RowPtr (row pointer). The corresponding index does not need to be updated. After completing the reconstruction of all tuples in the page, update the reconstruction field in the page header, and update the corresponding free space size in the free space table (FSM, freespace map).

[0389] In the Ustore storage engine, deleted tuples are only marked as deleted. To actually clean up the corresponding tuples, we must wait until all transactions with a CSN smaller than the CSN of the deleted tuple are completed. The specific reason is to ensure that long transactions can still obtain the tuple through old snapshots. Therefore, when reconstructing the data in a page, there may be user data in the page that has been deleted but not cleaned up. For this part of user data, it can be skipped directly during reconstruction, and the original tuple can be retained. The specific reason is that the record value in the page will no longer be used. When querying the historical value of the tuple in an old transaction, it can be traced back through the version chain of the time series database slot (TD slot).

[0390] Fig.14 1 shows a schematic diagram of the structure of an electronic device 100. The electronic device 100 may be the database server mentioned in this application. Fig.14 As shown, the electronic device 100 includes: one or more processors 1032, a communication interface 1033, a memory 1031, a bus system 1034, and one or more programs. The communication interface 1033, the processor 1032, and the memory 1031 are interconnected via the bus system 1034; the bus system 1034 can be a peripheral component interconnection standard bus or an extended industrial standard structure bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The one or more programs are stored in the memory 1031, and the one or more programs include instructions, which, when executed by the electronic device 100, enable the electronic device 100 to execute Figure 3 , Figure 5 , Figure 6 , Figure 7 , Figure 8 , Fig. 9 , Fig.11 , Fig.13 The related method in any of the accompanying drawings.

[0391] The present application also provides a computer storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by the electronic device 100, cause the electronic device 100 to execute Figure 3 , Figure 5 , Figure 6 , Figure 7 , Figure 8 , Fig. 9 , Fig.11 , Fig.13 The related method in any of the accompanying drawings.

[0392] The present application also provides a computer program product including instructions, which, when executed on the electronic device 100, enables the electronic device 100 to execute Figure 3 , Figure 5 , Figure 6 , Figure 7 , Figure 8 , Fig. 9 , Fig.11 , Fig.13 The related method in any of the accompanying drawings.

[0393] Among them, the electronic device, computer storage medium or computer program product provided in this application is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0394] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0395] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0396] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0397] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0398] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0399] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0400] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing a computer program instruction on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a database server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, database server or data center. The computer-readable storage medium may be any available medium that a computer can access or may contain one or more data storage devices such as database servers and data centers that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium, or a semiconductor medium (eg, a solid state disk (SSD)).

[0401] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, applied to an electronic device, characterized in that: The method comprises: During a first change operation on a first table, a first instruction is received, wherein the first instruction is used to instruct to modify first data in the first table to second data; determining that the first change operation has not yet adjusted the format of the first data in the first table to: meet the format requirement of the first table after the first change operation; adjusting the format of the first data in the first table to meet the format requirement of the first table after the first change operation; The first data in the first table is modified into the second data.

2. The method according to claim 1, characterized in that The determining that the first change operation has not yet adjusted the format of the first data in the first table to meet the format requirement of the first table after the first change operation includes: Corresponding to determining that the format of the first data does not match the format parameters of the first table after the first change operation, it is determined that the first change operation has not adjusted the format of the first data to meet the format requirement of the first table after the first change operation.

3. The method according to claim 2, characterized in that The adjusting the format of the first data in the first table to meet the format requirement of the first table after the first change operation includes: The data reconstruction thread is called to adjust the format of the first data so that the format of the first data matches the format parameters of the first table after the first change operation.

4. The method according to claim 3, characterized in that The calling data reconstruction thread to adjust the format of the first data includes: calling the data reconstruction thread to parse a second instruction indicating the first change operation to determine a format parameter of the first table after the first change operation; The data reconstruction thread is called to adjust the format of the first data based on the format parameters of the first table after the first change operation.

5. The method according to claim 4, characterized in that The first instruction includes a data manipulation language DML instruction, and the second instruction includes a data definition language DDL instruction.

6. The method according to claim 1, characterized in that The method further comprises: Performing the first change operation on the first table to obtain a second table, and saving the second table; receiving a first request sent by a first application, where the first request is used to request binding with the second table; Bind the first application to the second table.

7. The method according to claim 6, characterized in that The method further comprises: The second application is bound to the first table; receiving a third request sent by the second application, wherein the third request is used to instruct to switch the second application to be bound to the second table; Unbinding the second application from the first table; Bind the second application to the second table.

8. The method according to claim 7, characterized in that The method further comprises: Corresponding to determining that the formats of all data in the first table meet the format requirement of the first table and the first table is not bound to an application, the first table is deleted.

9. An electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device; A processor, when the processor executes the instructions in the memory, can enable the electronic device to execute the data processing method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the data processing method according to any one of claims 1 to 8.

Images