Method for realizing copy table writing based on multimode database

Abstract

The invention discloses a copy table writing method based on a multi-mode database, and relates to the field of multi-mode database optimization. Comprising the following steps: 1, compiling and optimizing a multi-mode copy table to generate a multi-mode copy table expression; 2, generating a multi-mode copy table write-in operator and a multi-mode copy table write-in operator executor, and executing the write-in of the copy table; the copy table writing efficiency of the multi-mode scene can be further improved, the read-write speed of the multi-mode database is improved, the multi-mode database system is optimized, and elastic expansion and cost control of the multi-mode database can be achieved.

Description

Technical Field

[0001] The invention discloses a method for writing a copy table based on a multi-mode database, and relates to the field of multi-mode database optimization. Background Art

[0002] In the operation of multi-mode databases, there is often a need to copy the data of one table to another table. For example, to manage the electricity consumption data of a community, the daily data is stored in the real-time monitoring table, and the data of the day is regularly transferred to the historical electricity consumption table at 0:00 every day. Due to the different data storage methods of different data models in the multi-mode database, the copy table writing involving multi-mode cannot be used directly, so new expressions and execution operators need to be added, and the Insert part and the Select part need to be compiled and executed separately. During the execution process, data conversion and verification are performed, which easily leads to problems such as syntax incompatibility. Summary of the invention

[0003] In view of the problems of the prior art, the present invention provides a method for writing a replicated table based on a multi-mode database, thereby solving the syntax compatibility problem of writing replicated tables in multi-mode scenarios.

[0004] The specific scheme proposed by the present invention is:

[0005] The present invention provides a method for writing a replication table based on a multi-mode database, comprising:

[0006] Step 1: Compile and optimize the multi-mode replication table to generate the multi-mode replication table expression:

[0007] Step 11: Perform syntax check, execute buildInsert to compile, fill in Insert content, and execute buildSelect to build the Select part.

[0008] Step 12: Determine whether the data models of the source table and the target table can be converted to each other. If yes, compile the Select part. Otherwise, an error is reported indicating that the data types do not match.

[0009] Step 13: Determine whether the number of columns written to the multi-mode replication table is consistent with the number of columns queried. If yes, continue to build the multi-mode replication table expression and fill the result returned by the Select part compilation into the multi-mode replication table expression.

[0010] Step 2: Generate a multi-mode replication table write operator and a multi-mode replication table write operator executor, and execute the write of the replication table:

[0011] Step 21: Fill the data of the multi-mode replication table expression into the data structure of the multi-mode replication table write operator.

[0012] Generate a multi-mode replication table write operator executor according to the data structure of the multi-mode replication table write operator.

[0013] Step 22: Read data and execute the source data acquirer in the source table execution engine to obtain the source table integrated data.

[0014] Step 23: According to the data model of the source table, convert the data into the data type supported by the target table.

[0015] Step 24: Build the target table Insert plan.

[0016] Step 25: Execute the Insert plan and write data to the target table.

[0017] Step 26: Return the execution result.

[0018] Furthermore, in step 1 of the method for writing a copy table based on a multi-mode database, a pre-step 10 is also performed: creating a source table table1 and writing data normally, creating a target table table2, and executing a copy table statement: insert in to table2 select * from table1 statement.

[0019] Furthermore, in step 12 of the method for writing a replicated table based on a multi-mode database, it is first determined whether the multi-mode replicated table is a multi-mode replicated table scenario, otherwise the same-mode replicated table writing process is performed, and if so, it is further determined whether the data models of the source table and the target table can be converted to each other.

[0020] Furthermore, step 23 of the method for writing a copy table based on a multi-mode database also verifies whether the data type of the source table is correct according to the write column order. If it is not correct, an error is returned. If the verification passes, step 24 is performed.

[0021] The present invention also provides a writing device for implementing a replication table based on a multi-mode database, comprising a compilation optimization module and an operator executor module, wherein the operator executor module comprises a data reading module, a data conversion module and a data writing module.

[0022] The compilation optimization module performs compilation optimization of the multi-mode replication table and generates the multi-mode replication table expression:

[0023] Step 11: Perform syntax check, execute buildInsert to compile, fill in Insert content, and execute buildSelect to build the Select part.

[0024] Step 12: Determine whether the data models of the source table and the target table can be converted to each other. If yes, compile the Select part. Otherwise, an error is reported indicating that the data types do not match.

[0025] Step 13: Determine whether the number of columns written to the multi-mode replication table is consistent with the number of columns queried. If yes, continue to build the multi-mode replication table expression and fill the result returned by the Select part compilation into the multi-mode replication table expression.

[0026] The operator executor module generates the multi-mode replication table write operator and the multi-mode replication table write operator executor, and executes the write of the replication table:

[0027] Step 21: The operator executor module fills the data of the multi-mode replication table expression into the multi-mode replication table and writes it into the operator's data structure.

[0028] Generate a multi-mode replication table write operator executor according to the data structure of the multi-mode replication table write operator.

[0029] Step 22: The data reading module reads data and executes the source data acquirer in the source table execution engine to obtain the source table integration data.

[0030] Step 23: According to the data model of the source table, the data conversion module converts the data into the data type supported by the target table.

[0031] Step 24: The data writing module builds the target table Insert plan.

[0032] Step 25: The data writing module executes the Insert plan and writes the data to the target table.

[0033] Step 26: The data writing module returns the execution result.

[0034] Furthermore, the compilation optimization module of the writing device for implementing a copy table based on a multi-mode database also executes the pre-step 10: creating a source table table1 and writing data normally, creating a target table table2, and executing a copy table statement: insert into table2 select*from table1 statement.

[0035] Furthermore, the compilation optimization module of the writing device for implementing a replicated table based on a multi-mode database first determines in step 12 whether the multi-mode replicated table is a multi-mode replicated table scenario, otherwise, the same-mode replicated table writing process is performed, and if so, it continues to determine whether the data models of the source table and the target table can be converted to each other.

[0036] Furthermore, the data conversion module of the writing device for implementing a copy table based on a multi-mode database also verifies whether the source table data type is correct according to the write column order in step 23. If it is not correct, an error is returned. If the verification passes, step 24 is performed.

[0037] The benefits of the present invention are:

[0038] The present invention solves the syntax compatibility problem of copy table writing in multi-mode scenarios, can further improve the copy table writing efficiency in multi-mode scenarios, improve the reading and writing speed of multi-mode databases, optimize the multi-mode database system, and can achieve elastic expansion and cost control of multi-mode databases. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the SQL compilation optimization process of writing multi-mode replication tables in the method of the present invention.

[0040] Figure 2 It is a schematic diagram of the framework of the multi-mode replication table operator executor.

[0041] Figure 3 It is a flowchart of the multi-mode replication table writing to the multi-mode replication table operator executor. DETAILED DESCRIPTION

[0042] Multi-mode database: A database that supports multiple data models, including document model, key-value storage, graph model, and time series data model;

[0043] Source table: the table from which data is obtained in the copy table write syntax;

[0044] Target table: the table to which data is written in the copy table write syntax;

[0045] CrossInsertSelectExpr: Adds a new expression for writing to the replicated table in multi-mode scenarios, which is used in compilation.

[0046] CrossInsertSelectNode: Added a new logical operator for writing to a replication table in multi-mode scenarios;

[0047] CrossInsertSelecter: Multi-mode replication table operator executor, used to actually execute the multi-mode replication table operator.

[0048] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0049] Example 1

[0050] The present invention provides a method for writing a replication table based on a multi-mode database, comprising:

[0051] Step 1: Compile and optimize the multi-mode replication table to generate the multi-mode replication table expression:

[0052] Step 10: Create the source table table1 and write data normally, create the target table table2, and execute the copy table statement: insert into table2 select * from table1 statement,

[0053] Step 11: Perform syntax check, execute buildInsert to compile, fill in Insert content, and execute buildSelect to build the Select part.

[0054] Step 12: First determine whether the multi-mode replication table is a multi-mode replication table scenario. Otherwise, perform the same-mode replication table write process. If yes, continue to determine whether the data models of the source table and the target table can be converted to each other. If yes, compile the Select part. Otherwise, an error is reported indicating that the data type does not match.

[0055] Step 13: Determine whether the number of columns written to the multi-mode replication table is consistent with the number of columns queried. If so, continue to build the multi-mode replication table expression and fill the results returned by the Select part compilation into the multi-mode replication table expression.

[0056] For example, the syntax remains unchanged:

[0057] insert into table_name[(field_name_list)]select_stmt

[0058] Add the CrossInsertSelectExpr expression. The expression structure is as follows:

[0059] type CrossInsertSelectExpr struct{

[0060] Input RelExpr / / Sub-expression (Select part compiled)

[0061] STableType int32 / / Source table type

[0062] TTableID uint64 / / target table ID

[0063] TTableType int32 / / target table type

[0064] Cols[]int32 / / Write column ID list

[0065] ColIdxs[]int32 / / Write column ID index list

[0066] }

[0067] Step 2: Generate a multi-mode replication table write operator and a multi-mode replication table write operator executor, and execute the write of the replication table:

[0068] Step 21: Fill the data of the multi-mode replication table expression into the data structure of the multi-mode replication table write operator. For example:

[0069] The operator CrossInsertSelectNode has the following structure:

[0070]

[0071] Generate a multi-mode replication table write operator executor according to the data structure of the multi-mode replication table write operator.

[0072] For example, the operator executor CrossInsertSelecter has the following executor structure:

[0073]

[0074] Step 22: Read data and execute the source data acquirer in the source table execution engine to obtain the source table integrated data.

[0075] Step 23: According to the data model of the source table, convert the data into the data type supported by the target table, and verify whether the source table data type is correct according to the write column order. If it is not correct, return an error. If the verification passes, proceed to step 24.

[0076] Step 24: Build the target table Insert plan.

[0077] Step 25: Execute the Insert plan and write data to the target table.

[0078] Step 26: Return the execution result.

[0079] Example 2

[0080] The present invention also provides a writing device for implementing a replication table based on a multi-mode database, comprising a compilation optimization module and an operator executor module, wherein the operator executor module comprises a data reading module, a data conversion module and a data writing module.

[0081] The compilation optimization module performs compilation optimization of the multi-mode replication table and generates the multi-mode replication table expression:

[0082] Step 11: Perform syntax check, execute buildInsert to compile, fill in Insert content, and execute buildSelect to build the Select part.

[0083] Step 12: Determine whether the data models of the source table and the target table can be converted to each other. If yes, compile the Select part. Otherwise, an error is reported indicating that the data types do not match.

[0084] Step 13: Determine whether the number of columns written to the multi-mode replication table is consistent with the number of columns queried. If yes, continue to build the multi-mode replication table expression and fill the result returned by the Select part compilation into the multi-mode replication table expression.

[0085] The operator executor module generates the multi-mode replication table write operator and the multi-mode replication table write operator executor, and executes the write of the replication table:

[0086] Step 21: The operator executor module fills the data of the multi-mode replication table expression into the multi-mode replication table and writes it into the operator's data structure.

[0087] Generate a multi-mode replication table write operator executor according to the data structure of the multi-mode replication table write operator.

[0088] Step 22: The data reading module reads data and executes the source data acquirer in the source table execution engine to obtain the source table integration data.

[0089] Step 23: According to the data model of the source table, the data conversion module converts the data into the data type supported by the target table.

[0090] Step 24: The data writing module builds the target table Insert plan.

[0091] Step 25: The data writing module executes the Insert plan and writes the data to the target table.

[0092] Step 26: The data writing module returns the execution result.

[0093] As the information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention, the specific contents can be found in the description of the embodiment of the method of the present invention and will not be repeated here.

[0094] Similarly, the device of the present invention solves the syntax compatibility problem of replica table writing in multi-mode scenarios, can further improve the efficiency of replica table writing in multi-mode scenarios, improve the reading and writing speed of multi-mode databases, optimize the multi-mode database system, and achieve elastic expansion and cost control of multi-mode databases.

[0095] It should be noted that not all steps and modules in the above-mentioned processes and device structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0096] The above-described embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or changes made by those skilled in the art based on the present invention are within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A method for writing a replicated table based on a multi-mode database, characterized in that include: Step 1: Compile and optimize the multi-mode replication table to generate the multi-mode replication table expression: Step 11: Perform syntax check, execute buildInsert to compile, fill in Insert content, and execute buildSelect to build the Select part. Step 12: Determine whether the data models of the source table and the target table can be converted to each other. If yes, compile the Select part. Otherwise, an error is reported indicating that the data types do not match. Step 13: Determine whether the number of columns written to the multi-mode replication table is consistent with the number of columns queried. If so, continue to build the multi-mode replication table expression and fill the results returned by the Select part of the compilation into the multi-mode replication table expression. Step 2: Generate a multi-mode replication table write operator and a multi-mode replication table write operator executor, and execute the write of the replication table: Step 21: Fill the data of the multi-mode replication table expression into the data structure of the multi-mode replication table write operator. Generate a multi-mode replication table write operator executor according to the data structure of the multi-mode replication table write operator. Step 22: Read data and execute the source data acquirer in the source table execution engine to obtain the source table integrated data. Step 23: According to the data model of the source table, convert the data into the data type supported by the target table. Step 24: Build the target table Insert plan. Step 25: Execute the Insert plan and write data to the target table. Step 26: Return the execution result.

2. According to the method for writing a copy table based on a multi-mode database according to claim 1, it is characterized by: In step 1, the prerequisite step 10 is also performed: create the source table table1 and write data normally, create the target table table2, and execute the copy table statement: insert into table2 select * from table1 statement.

3. The method for writing a duplicate table based on a multi-mode database according to claim 1 is characterized in that In step 12, it is first determined whether the multi-mode replication table is a multi-mode replication table scenario, otherwise the same-mode replication table writing process is performed, and if so, it is continued to determine whether the data models of the source table and the target table can be converted to each other.

4. The method for writing a duplicate table based on a multi-mode database according to claim 1 is characterized in that Step 23 also verifies whether the source table data type is correct according to the written column order. If it is not correct, an error message is returned. If the verification passes, proceed to step 24.

5. A writing device for implementing a replication table based on a multi-mode database, characterized in that It includes a compilation optimization module and an operator executor module. The operator executor module includes a data reading module, a data conversion module and a data writing module. The compilation optimization module performs compilation optimization of the multi-mode replication table and generates the multi-mode replication table expression: Step 11: Perform syntax check, execute buildInsert to compile, fill in Insert content, and execute buildSelect to build the Select part. Step 12: Determine whether the data models of the source table and the target table can be converted to each other. If yes, compile the Select part. Otherwise, an error is reported indicating that the data types do not match. Step 13: Determine whether the number of columns written to the multi-mode replication table is consistent with the number of columns queried. If so, continue to build the multi-mode replication table expression and fill the results returned by the Select part of the compilation into the multi-mode replication table expression. The operator executor module generates the multi-mode replication table write operator and the multi-mode replication table write operator executor, and executes the write of the replication table: Step 21: The operator executor module fills the data of the multi-mode replication table expression into the multi-mode replication table and writes it into the operator's data structure. Generate a multi-mode replication table write operator executor according to the data structure of the multi-mode replication table write operator. Step 22: The data reading module reads data and executes the source data acquirer in the source table execution engine to obtain the source table integration data. Step 23: According to the data model of the source table, the data conversion module converts the data into the data type supported by the target table. Step 24: The data writing module builds the target table Insert plan. Step 25: The data writing module executes the Insert plan and writes the data to the target table. Step 26: The data writing module returns the execution result.

6. The device for writing a copy table based on a multi-mode database according to claim 5, characterized in that The compilation optimization module also executes the pre-step 10: creates the source table table1 and writes data normally, creates the target table table2, and executes the copy table statement: insert into table2 select*from table1 statement.

7. The device for writing a copy table based on a multi-mode database according to claim 5, characterized in that The compilation optimization module first determines in step 12 whether the multi-mode replication table is a multi-mode replication table scenario, otherwise the same-mode replication table writing process is performed, and if so, it continues to determine whether the data models of the source table and the target table can be converted to each other.

8. The device for writing a copy table based on a multi-mode database according to claim 5, characterized in that In step 23, the data conversion module also checks whether the data type of the source table is correct according to the write column order. If it is not correct, an error message is returned. If the check passes, step 24 is performed.

Images