A method for writing replicated tables based on a multi-model database

Abstract

The application discloses a kind of based on multi-mode database and realizes the write method of copy table, it is related to the field of multi-mode database optimization;Including: step 1: the compilation optimization of multi-mode copy table is carried out, and multi-mode copy table expression is generated;Step 2: generate multi-mode copy table write operator and multi-mode copy table write operator executor, and execute the write of copy table;The application can further improve the copy table write efficiency of multi-mode scene, improve the read-write speed of multi-mode database, optimize multi-mode database system, can realize the elastic extension and cost control of multi-mode database.

Description

Technical Field

[0001] This invention discloses a method for writing replicated tables based on a multi-model database, which relates to the field of multi-model database optimization. Background Technology

[0002] In multi-model database operations, there is often a need to copy data from one table to another. For example, managing electricity consumption data for a residential community involves storing daily data in a real-time monitoring table, and then transferring that day's data to a historical electricity consumption table at midnight each day. Because different data models in a multi-model database use different data storage methods, copying tables cannot be directly used. Therefore, new expressions and execution operators need to be added, and the Insert and Select parts need to be compiled and executed separately. Data conversion and validation operations are performed during execution, which can easily lead to syntax incompatibility issues. Summary of the Invention

[0003] This invention addresses the problems of existing technologies by providing a method for writing replicated tables based on a multi-model database, thus resolving the syntax compatibility issues of writing replicated tables in multi-model scenarios.

[0004] The specific solution proposed in this invention is as follows:

[0005] This invention provides a method for writing replicated tables based on a multi-model database, comprising:

[0006] Step 1: Perform compilation optimization for the multi-modal replication table to generate the multi-modal replication table expression:

[0007] Step 11: Perform syntax validation, execute buildInsert to compile, populate the Insert content, and execute buildSelect to build the Select section.

[0008] Step 12: Determine if the data models of the source table and the target table can be converted to each other. If yes, compile the Select section; otherwise, report an error indicating a data type mismatch.

[0009] Step 13: Determine if the number of columns written to the multi-model replication table matches the number of columns queried. If they match, continue constructing the multi-model replication table expression, filling the multi-model replication table expression with the results returned by the compiled Select part.

[0010] Step 2: Generate the multi-modal copy table write operator and the multi-modal copy table write operator executor, and execute the write of the copy table:

[0011] Step 21: Fill the data of the multi-modal replication table expression into the data structure of the multi-modal replication table write operator.

[0012] Generate a multi-mode copy table write operator executor based on the data structure of the multi-mode copy table write operator.

[0013] Step 22: Perform data reading. Execute the source data retriever in the source table execution engine to obtain the integrated data from the source table.

[0014] Step 23: Based on the data model of the source table, convert the data into a data type that the target table can support.

[0015] Step 24: Construct the target table Insert plan.

[0016] Step 25: Execute the Insert plan to write data to the target table.

[0017] Step 26: Return the execution result.

[0018] Furthermore, in step 1 of the method for writing a copy table based on a multi-model database, a preliminary step 10 is performed: creating a source table table1 and writing data normally, creating a target table table2, and executing the copy table statement: insert to table2 select * from table1.

[0019] Furthermore, in step 12 of the method for writing a replicated table based on a multi-model database, it is first determined whether the multi-model replicated table is a multi-model replicated table scenario. Otherwise, the same-model replicated table writing process is performed. If it is, it is further determined whether the data models of the source table and the target table can be converted to each other.

[0020] Furthermore, step 23 of the method for writing a replicated table based on a multi-model database also verifies whether the data type of the source table is correct according to the order of the columns being written. If it is incorrect, an error is returned; if the verification passes, step 24 is performed.

[0021] This invention also provides a writing device for a replicated table based on a multi-model database, including a compilation optimization module and an operator executor module. The operator executor module includes a data reading module, a data conversion module, and a data writing module.

[0022] The compilation optimization module performs compilation optimization for multi-modal replication tables, generating multi-modal replication table expressions:

[0023] Step 11: Perform syntax validation, execute buildInsert to compile, populate the Insert content, and execute buildSelect to build the Select section.

[0024] Step 12: Determine if the data models of the source table and the target table can be converted to each other. If yes, compile the Select section; otherwise, report an error indicating a data type mismatch.

[0025] Step 13: Determine if the number of columns written to the multi-model replication table matches the number of columns queried. If they match, continue constructing the multi-model replication table expression, filling the multi-model replication table expression with the results returned by the compiled Select part.

[0026] The operator executor module generates a multi-modal copy table write operator and a multi-modal copy table write operator executor, and then executes the write of the copy table:

[0027] Step 21: The operator executor module fills the data of the multimodal copy table expression into the data structure of the multimodal copy table write operator.

[0028] Generate a multi-mode copy table write operator executor based on the data structure of the multi-mode copy table write operator.

[0029] Step 22: The data reading module reads data by executing the source data retriever in the source table execution engine to obtain the integrated data from the source table.

[0030] Step 23: Based on the data model of the source table, the data transformation module converts the data into a data type that the target table can support.

[0031] Step 24: The data writing module constructs the target table Insert plan.

[0032] Step 25: The data writing module executes the Insert plan, writing data to the target table.

[0033] Step 26: The data writing module returns the execution result.

[0034] Furthermore, the compilation optimization module of the writing device for replicating tables based on a multi-model database also performs the pre-step 10: creating the source table table1 and writing data normally, creating the target table table2, and executing the table replication statement: insert into table2 select * from table1 statement.

[0035] Furthermore, in step 12, the compilation optimization module of the write device for implementing a copy table based on a multi-model database first determines whether the multi-model copy table is a multi-model copy table scenario; otherwise, it performs the same-model copy table write process; if it is, it continues to determine whether the data models of the source table and the target table can be converted to each other.

[0036] Furthermore, in step 23, the data conversion module of the writing device for replicating tables based on a multi-model database also verifies whether the data type of the source table is correct according to the order of the writing columns. If it is incorrect, an error is returned; if the verification passes, step 24 is performed.

[0037] The advantages of this invention are:

[0038] This invention solves the syntax compatibility problem of copy table writing in multi-model scenarios, which can further improve the efficiency of copy table writing in multi-model scenarios, improve the read and write speed of multi-model databases, optimize multi-model database systems, and enable elastic expansion and cost control of multi-model databases. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the SQL compilation optimization process for multi-mode table copying in the method of this invention.

[0040] Figure 2 This is a schematic diagram of the framework of the multi-mode copy table operator executor.

[0041] Figure 3 This is a flowchart illustrating the process of writing multi-mode copy table to multi-mode copy table operator executor. Detailed Implementation

[0042] Multi-model database: A database that supports multiple data models, including document model, key-value store, graph model and time series data model, etc.

[0043] Source table: The table whose data is retrieved in the copy table syntax;

[0044] Target table: The table whose data is written in the copy table syntax;

[0045] CrossInsertSelectExpr: Adds a copy table write expression for multi-modal scenarios, for use during compilation;

[0046] CrossInsertSelectNode: Adds a logical operator for copying table entries in multi-mode scenarios;

[0047] CrossInsertSelecter: A multi-mode copy table operator executor used to actually execute multi-mode copy table operators.

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0049] Example 1

[0050] This invention provides a method for writing replicated tables based on a multi-model database, comprising:

[0051] Step 1: Perform compilation optimization for the multi-modal replication table to generate the multi-modal replication table expression:

[0052] Step 10: Create the source table table1 and write data to it normally. Create the target table table2 and execute the table copy statement: insert into table2 select * from table1.

[0053] Step 11: Perform syntax validation, execute buildInsert to compile, populate the Insert content, and execute buildSelect to build the Select section.

[0054] Step 12: First, determine if the multi-model replication table is a multi-model replication scenario. Otherwise, proceed with the same-model replication table write process. If it is, continue to determine if the data models of the source table and the target table can be converted to each other. If yes, compile the Select part; otherwise, report an error indicating a data type mismatch.

[0055] Step 13: Determine if the number of columns written to the multi-model replication table is consistent with the number of columns queried. If so, continue to construct the multi-model replication table expression and fill the multi-model replication table expression with the results returned by the compilation of the Select part.

[0056] For example, if the syntax remains unchanged, it is still:

[0057] insert into table_name[(field_name_list)]select_stmt

[0058] A new CrossInsertSelectExpr expression has been added, with the following structure:

[0059] type CrossInsertSelectExpr struct{

[0060] Input RelExpr / / Subexpression (Select part compiled)

[0061] STableType int32 / / Source table type

[0062] TTableID uint64 / / Target table ID

[0063] TTableType int32 / / Target table type

[0064] Cols[]int32 / / Write the list of column IDs

[0065] ColIdxs[]int32 / / Write the column ID index list

[0066] }

[0067] Step 2: Generate the multi-modal copy table write operator and the multi-modal copy table write operator executor, and execute the write of the copy table:

[0068] Step 21: Fill the data of the multi-modal copy table expression into the data structure of the multi-modal copy table write operator. For example:

[0069] The operator CrossInsertSelectNode has the following structure:

[0070]

[0071] Generate a multi-mode copy table write operator executor based on the data structure of the multi-mode copy table write operator.

[0072] For example, the operator executor CrossInsertSelecter has the following structure:

[0073]

[0074] Step 22: Perform data reading. Execute the source data retriever in the source table execution engine to obtain the integrated data from the source table.

[0075] Step 23: Based on the data model of the source table, convert the data into a data type that the target table can support. Verify the data type of the source table according to the order in which the columns are written. If incorrect, return an error; otherwise, proceed to step 24.

[0076] Step 24: Construct the target table Insert plan.

[0077] Step 25: Execute the Insert plan to write data to the target table.

[0078] Step 26: Return the execution result.

[0079] Example 2

[0080] This invention also provides a writing device for a replicated table based on a multi-model database, including a compilation optimization module and an operator executor module. The operator executor module includes a data reading module, a data conversion module, and a data writing module.

[0081] The compilation optimization module performs compilation optimization for multi-modal replication tables, generating multi-modal replication table expressions:

[0082] Step 11: Perform syntax validation, execute buildInsert to compile, populate the Insert content, and execute buildSelect to build the Select section.

[0083] Step 12: Determine if the data models of the source table and the target table can be converted to each other. If yes, compile the Select section; otherwise, report an error indicating a data type mismatch.

[0084] Step 13: Determine if the number of columns written to the multi-model replication table matches the number of columns queried. If they match, continue constructing the multi-model replication table expression, filling the multi-model replication table expression with the results returned by the compiled Select part.

[0085] The operator executor module generates a multi-modal copy table write operator and a multi-modal copy table write operator executor, and then executes the write of the copy table:

[0086] Step 21: The operator executor module fills the data of the multimodal copy table expression into the data structure of the multimodal copy table write operator.

[0087] Generate a multi-mode copy table write operator executor based on the data structure of the multi-mode copy table write operator.

[0088] Step 22: The data reading module reads data by executing the source data retriever in the source table execution engine to obtain the integrated data from the source table.

[0089] Step 23: Based on the data model of the source table, the data transformation module converts the data into a data type that the target table can support.

[0090] Step 24: The data writing module constructs the target table Insert plan.

[0091] Step 25: The data writing module executes the Insert plan, writing data to the target table.

[0092] Step 26: The data writing module returns the execution result.

[0093] The information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description in the method embodiment of the present invention, and will not be repeated here.

[0094] Similarly, the device of the present invention solves the syntax compatibility problem of copy table writing in multi-mode scenarios, which can further improve the copy table writing efficiency in multi-mode scenarios, improve the read and write speed of multi-mode databases, optimize multi-mode database systems, and realize the elastic expansion and cost control of multi-mode databases.

[0095] It should be noted that not all steps and modules in the above processes and device structures are mandatory; some steps or modules can be omitted as needed. The execution order of each step is not fixed and can be adjusted as required. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0096] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A method for writing replicated tables based on a multi-model database, characterized by: Comprise: Step 1: perform the compilation optimization of the multi-mode copy table, generate the multi-mode copy table expression: Step 11: perform syntax checking, execute buildInsert for compilation, fill in the Insert content, execute buildSelect to build the Select part, Step 12: determine whether the data models of the source table and the target table can be converted to each other, if yes, compile the Select part, otherwise, report an error indicating that the data types do not match, Step 13: determine whether the number of multi-mode copy table write columns and the number of query columns are consistent, if yes, continue to build the multi-mode copy table expression, and fill the result returned by the Select part compilation into the multi-mode copy table expression, Step 2: generate the multi-mode copy table write operator and the multi-mode copy table write operator executor, and execute the write of the copy table: Step 21: fill the data of the multi-mode copy table expression into the data structure of the multi-mode copy table write operator, Generate the multi-mode copy table write operator executor according to the data structure of the multi-mode copy table write operator, Step 22: perform data reading, execute the source data getter in the source table execution engine to obtain the integrated data of the source table, Step 23: according to the data model of the source table, convert the data into a data type supported by the target table, Step 24: build the target table Insert plan, Step 25: execute the Insert plan to write data to the target table, Step 26: return the execution result.

2. The method for writing a replicated table based on a multi-mode database according to claim 1, characterized in that In step 1, the pre-step 10 is also performed: create the source table table1 and normally write data, create the target table table2, and execute the copy table statement: insert into table2 select*from table1 statement.

3. The method for writing a replicated table based on a multi-mode database according to claim 1, characterized in that In step 12, first determine whether the multi-mode copy table is a multi-mode copy table scenario, otherwise perform the same-mode copy table write process, if yes, continue to determine whether the data models of the source table and the target table can be converted to each other.

4. The method for writing a replicated table based on a multi-mode database according to claim 1, characterized in that In step 23, the source table data type is also checked according to the write column order, if incorrect, an error is returned, and if the verification is passed, step 24 is performed.

5. A write device for replicating tables based on a multi-model database, characterized in that: Comprise a compilation optimization module and an operator executor module, the operator executor module comprises a data reading module, a data conversion module and a data writing module, The compilation optimization module performs the compilation optimization of the multi-mode copy table, and generates the multi-mode copy table expression: Step 11: perform syntax checking, execute buildInsert for compilation, fill in the Insert content, execute buildSelect to build the Select part, Step 12: determine whether the data models of the source table and the target table can be converted to each other, if yes, compile the Select part, otherwise, report an error indicating that the data types do not match, Step 13: determine whether the number of multi-mode copy table write columns and the number of query columns are consistent, if yes, continue to build the multi-mode copy table expression, and fill the result returned by the Select part compilation into the multi-mode copy table expression, The operator executor module generates the multi-mode copy table write operator and the multi-mode copy table write operator executor, and executes the write of the copy table: Step 21: The operator executor module fills the data of the multi-mode copy table table expression into the data structure of the multi-mode copy table write operator, According to the data structure of the multi-mode copy table write operator, a multi-mode copy table write operator executor is generated, Step 22: The data reading module reads data, executes the source data getter in the source table execution engine, and obtains the integrated data of the source table, Step 23: According to the data model of the source table, the data conversion module converts the data into a data type that can be supported by the target table, Step 24: The data writing module constructs the target table Insert plan, Step 25: The data writing module executes the Insert plan and writes data to the target table, Step 26: The data writing module returns the execution result.

6. The write apparatus for a replicated table based on a multi-model database according to claim 5, wherein The compilation optimization module also performs the pre-step 10: creates the source table table1 and normally writes data, creates the target table table2, and executes the copy table statement: insert into table2 select*from table1 statement.

7. The write apparatus for a replicated table based on a multi-model database according to claim 5, wherein The compilation optimization module first judges whether the multi-mode copy table is a multi-mode copy table scenario in step 12, otherwise the same-mode copy table write process is performed, and if yes, it continues to judge whether the data models of the source table and the target table can be converted to each other.

8. The write apparatus for a replicated table based on a multi-model database according to claim 5, wherein The data conversion module also checks whether the source table data type is correct according to the write column order in step 23, returns an error if not correct, and proceeds to step 24 if the check is passed.

Images