Methods, apparatuses, electronic devices and computer-readable media for synchronizing wide tables

Abstract

This disclosure relates to a method, apparatus, electronic device, and computer-readable medium for synchronizing wide tables, belonging to the field of computer technology. The method includes: responding to a synchronization operation instruction for a wide table; obtaining a target single table and an operation type corresponding to the synchronization operation instruction; obtaining the association tree structure of the wide table and determining the target node corresponding to the target single table in the association tree structure, wherein each node in the association tree structure includes a cache object and multiple association condition objects, and each node corresponds to a single table; updating the cache object in the target node according to the synchronization operation instruction; and processing the target single table according to the operation type and the association condition objects of each node in the association tree structure. This disclosure, by performing wide table association calculations in the cache, can ensure real-time synchronization efficiency of wide tables under conditions of limited storage resources or network transmission constraints.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a method for synchronizing wide tables, a device for synchronizing wide tables, an electronic device, and a computer-readable medium. Background Technology

[0002] Among existing mainstream technologies, such as Apache Flink (a distributed streaming data processing engine), real-time incremental synchronization of multiple single tables into a wide table can be achieved through SQL (Structured Query Language) and API (Application Programming Interface).

[0003] However, when performing join calculations on multiple single tables, since the data storage of checkpoints may reach GB or even TB levels, synchronization cannot be performed if there are no storage media resources or insufficient storage space.

[0004] Therefore, there is an urgent need in the field for a method to synchronize wide tables, which can achieve real-time synchronization of wide tables under the condition of limited storage resources.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this disclosure is to provide a method, apparatus, electronic device, and computer-readable medium for synchronizing wide tables, thereby enabling real-time synchronization of wide tables, at least to a certain extent, under conditions of limited storage resources.

[0007] According to a first aspect of this disclosure, a method for synchronizing wide tables is provided, comprising:

[0008] In response to a synchronization operation command for a wide table, obtain the target single table and operation type corresponding to the synchronization operation command;

[0009] Obtain the association tree structure of the wide table and determine the target node corresponding to the target single table in the association tree structure, wherein each node in the association tree structure includes a cache object and multiple association condition objects, and each node corresponds to a single table;

[0010] The cached object in the target node is updated according to the synchronization operation instruction, and the target single table is processed according to the operation type and the association condition objects of each node in the association tree structure.

[0011] In one exemplary embodiment of this disclosure, the operation type includes an insertion operation, and the processing of the target single table according to the operation type and the association condition objects of each node in the association tree structure includes:

[0012] Based on the single table type in the target node and the association condition objects of each node in the association tree structure, determine the target retrieval node corresponding to the target node;

[0013] Determine whether the single table corresponding to the target retrieval node contains the corresponding single table data;

[0014] If included, the key table in the wide table is obtained, and the key data in the key table is determined to be complete based on the association condition objects of each node in the association tree structure.

[0015] If the key data in the key table is complete, then the single-table data from the target single table will be inserted into the wide table.

[0016] In one exemplary embodiment of this disclosure, the operation type includes a deletion operation, and the processing of the target single table according to the operation type and the association condition objects of each node in the association tree structure includes:

[0017] Obtain the key table in the wide table, and determine whether the key data in the key table is complete based on the association condition objects of each node in the association tree structure;

[0018] If the key data in the key table is complete, then the deletion operation is performed according to the single table type in the target node.

[0019] In one exemplary embodiment of this disclosure, performing the deletion operation based on the single table type in the target node includes:

[0020] If the single table type in the target node is a critical table, then delete the wide table.

[0021] In one exemplary embodiment of this disclosure, performing the deletion operation based on the single table type in the target node includes:

[0022] If the single table type in the target node is a regular table, then delete the single table data corresponding to the target node and update the data in the wide table.

[0023] In one exemplary embodiment of this disclosure, the method further includes:

[0024] Remove the cached objects and associated condition objects from the cache in the target node.

[0025] In one exemplary embodiment of this disclosure, the operation type includes an update operation, and the processing of the target single table according to the operation type and the association condition objects of each node in the association tree structure includes:

[0026] Determine whether the target field in the target table corresponding to the update operation is a key field;

[0027] If the target field is not the key field, then delete the original field data in the target field and insert the updated field data;

[0028] If the target field is the key field, then the updated field data is inserted into the cache object corresponding to the target single table.

[0029] According to a second aspect of this disclosure, a synchronization processing apparatus for wide tables is provided, comprising:

[0030] The operation instruction acquisition module is used to respond to the synchronization operation instruction for the wide table and acquire the target single table and operation type corresponding to the synchronization operation instruction.

[0031] The wide table structure acquisition module is used to acquire the association tree structure of the wide table and determine the target node corresponding to the target single table in the association tree structure. Each node in the association tree structure includes a cache object and multiple association condition objects, and each node corresponds to a single table.

[0032] The target single table processing module is used to update the cached object in the target node according to the synchronization operation instruction, and to process the target single table according to the operation type and the association condition objects of each node in the association tree structure.

[0033] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform a synchronous processing method for a wide table as described in any one of the preceding claims by executing the executable instructions.

[0034] According to a fourth aspect of this disclosure, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the synchronization processing method for wide tables described in any one of the preceding claims.

[0035] The exemplary embodiments disclosed herein can have the following beneficial effects:

[0036] In the wide table synchronization processing method of this exemplary embodiment, the target single table and operation type corresponding to the synchronization operation instruction, as well as the association tree structure of the wide table, are obtained. Then, the association tree structure is scanned according to the synchronization operation instruction, and the cached data is updated, thereby performing the association calculation of the wide table in the cache. The wide table synchronization processing method in this exemplary embodiment has two advantages: First, since the association calculations between tables are all performed in the cache, no additional storage medium is required, and wide table synchronization processing can be performed even under conditions of insufficient storage medium resources or insufficient storage space. Second, each node can run on the same host, eliminating network transmission problems. Therefore, it can solve the problem of synchronization job failure or delay caused by network limitations, ensuring the real-time synchronization efficiency of the wide table and achieving a latency of less than a second for incremental real-time synchronization of the wide table.

[0037] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0039] Figure 1 A flowchart illustrating a method for synchronizing wide tables according to an exemplary embodiment of this disclosure is shown.

[0040] Figure 2 A schematic diagram of a wide table association tree structure according to a specific embodiment of the present disclosure is shown;

[0041] Figure 3 A flowchart illustrating a wide table data insertion operation according to an exemplary embodiment of this disclosure is shown;

[0042] Figure 4 A flowchart illustrating the insertion of data into table A according to a specific embodiment of this disclosure is shown.

[0043] Figure 5 A flowchart illustrating the insertion of data into table B according to a specific embodiment of this disclosure is shown.

[0044] Figure 6 A flowchart illustrating the insertion of data into table C according to a specific embodiment of this disclosure is shown;

[0045] Figure 7A flowchart illustrating the insertion of data into table D according to a specific embodiment of the present disclosure is shown.

[0046] Figure 8 A flowchart illustrating a wide table data deletion operation according to an exemplary embodiment of this disclosure is shown;

[0047] Figure 9 A flowchart illustrating the deletion of key table data according to a specific embodiment of this disclosure is shown.

[0048] Figure 10 A flowchart illustrating the deletion of data in a regular table according to a specific embodiment of the present disclosure is shown.

[0049] Figure 11 A flowchart illustrating a wide table data update operation according to an exemplary embodiment of this disclosure is shown.

[0050] Figure 12 A flowchart illustrating the process of updating wide table data according to a specific embodiment of this disclosure is shown;

[0051] Figure 13 A block diagram of a synchronization processing apparatus for a wide table according to an exemplary embodiment of the present disclosure is shown;

[0052] Figure 14 A schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure is shown. Detailed Implementation

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0054] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0055] In mainstream technologies, such as Apache Flink (a distributed streaming engine), real-time incremental synchronization of multiple single tables into a wide table can be achieved through SQL (Structured Query Language) and APIs (Application Programming Interfaces). Taking Flink as an example, its core relies on a checkpoint mechanism to achieve streaming processing of the existing state. Checkpoints generate global snapshots using the Global Snapshot (ABS) algorithm and store them in a distributed file system or RocksDB (storage engine). After each checkpoint completes, it generates an SSTable (Sorted String Table, the underlying data storage format of Bigtable). Subsequent checkpoints merge these previously generated SSTables into a new SSTable, and finally, after all checkpoints are completed, the final wide table is output.

[0056] However, when performing joins on multiple single tables, most mainstream technologies store intermediate computation results in storage media. For example, Flink uses RocksDB as its default state backend, recording a RocksDB snapshot at each checkpoint and persisting it to the file system. The state is stored in TaskManager (cluster Slave nodes), and checkpoints are stored in JobManager (cluster Master nodes). In practice, this may present the following two problems:

[0057] 1. Data storage at checkpoints may reach GB or even TB levels. If there are no storage media resources or insufficient storage space, it may be impossible to use existing technologies for synchronization.

[0058] 2. Since checkpoints are stored in the jobManager, and each jobManager may run on different hosts, the files generated after a checkpoint may be subject to network limitations during the transfer between nodes, causing synchronization jobs to fail or be delayed.

[0059] To address the aforementioned issues, this example implementation first provides a method for synchronizing wide tables. (See reference...) Figure 1 As shown, the above-mentioned method for synchronizing wide tables may include the following steps:

[0060] Step S110. Respond to the synchronization operation command for the wide table and obtain the target single table and operation type corresponding to the synchronization operation command.

[0061] Step S120. Obtain the association tree structure of the wide table and determine the target node corresponding to the target single table in the association tree structure.

[0062] Step S130. Update the cached object in the target node according to the synchronization operation instruction, and process the target single table according to the operation type and the association condition objects of each node in the association tree structure.

[0063] In the wide table synchronization processing method of this exemplary embodiment, the target single table and operation type corresponding to the synchronization operation instruction, as well as the association tree structure of the wide table, are obtained. Then, the association tree structure is scanned according to the synchronization operation instruction, and the cached data is updated, thereby performing the association calculation of the wide table in the cache. The wide table synchronization processing method in this exemplary embodiment has two advantages: First, since the association calculations between tables are all performed in the cache, no additional storage medium is required, and wide table synchronization processing can be performed even under conditions of insufficient storage medium resources or insufficient storage space. Second, each node can run on the same host, eliminating network transmission problems. Therefore, it can solve the problem of synchronization job failure or delay caused by network limitations, ensuring the real-time synchronization efficiency of the wide table and achieving a latency of less than a second for incremental real-time synchronization of the wide table.

[0064] Below, in conjunction with Figures 2 to 12 The steps described above in this example implementation will be explained in more detail.

[0065] In step S110, in response to a synchronization operation instruction for a wide table, the target single table and operation type corresponding to the synchronization operation instruction are obtained.

[0066] A wide table refers to a database table that links together metrics, dimensions, and attributes related to a business theme. By using wide tables, the problem of linking multiple tables in multidimensional analysis can be solved, and the speed of data query and the convenience of analysis operations can be improved.

[0067] In this example implementation, the wide table can be calculated from the relationships between multiple single tables. The calculation of the wide table is a process of continuously updating the cache. The operation types for the wide table can include insertion operations, deletion operations, and update operations.

[0068] In step S120, the association tree structure of the wide table is obtained, and the target node corresponding to the target single table in the association tree structure is determined.

[0069] In this example implementation, each node in the association tree structure corresponds to a single table. Each node includes a cache object (Buffer) and multiple association condition objects (index). The cache object satisfies the LRU (Least Recently Used, a cache eviction algorithm) key-value structure, where the key is the primary key value of the record and the value is the record. Each association condition object corresponds to an association condition.

[0070] Figure 2 This diagram illustrates a wide table association tree structure according to a specific embodiment of the present disclosure. The wide table contains four single tables: A, B, C, and D. Each single table contains multiple fields, such as a1, a2, b1, b2, etc. Assume tables A and B are key tables, with key fields a2 and b2 respectively. Table A is associated with tables B and C. The association condition between tables A and B is a.a1 = b.b1, and the association condition between tables A and C is a.a2 = c.c2. Table C is associated with table D, and the association condition between tables C and D is c.c1 = d.d1. Here, key fields are sets of fields that can uniquely identify a single piece of data in the wide table, similar to a composite index in MySQL (Relational Database Management System). Key tables are the single tables containing the key fields.

[0071] In step S130, the cached object in the target node is updated according to the synchronization operation instruction, and the target single table is processed according to the operation type and the associated condition objects of each node in the associated tree structure.

[0072] In this example implementation, the cached object in the target node is first updated according to the synchronization operation instruction, and then the target single table is synchronized according to different operation types and the association condition objects of each node in the association tree structure.

[0073] In this example implementation, such as Figure 3 As shown, when the operation type is insert, the target single table is processed according to the operation type and the association condition objects of each node in the association tree structure. Specifically, this may include the following steps:

[0074] Step S310. Determine the target retrieval node corresponding to the target node based on the single table type in the target node and the association condition objects of each node in the association tree structure.

[0075] In this example implementation, the single table type in the target node can include key tables and ordinary tables. The target retrieval node corresponding to the target node can include other nodes in the association tree structure that are related to the target node as well as key nodes. Therefore, one or more target retrieval nodes can be determined based on the single table type of the target single table itself and the association condition objects of each node.

[0076] Step S320. Determine whether the single table corresponding to the target retrieval node contains the corresponding single table data.

[0077] After identifying the target retrieval node, scan the data in the single table corresponding to each target retrieval node to determine whether the single table corresponding to each target retrieval node contains the corresponding single table data, that is, to determine whether the keys in the cache have corresponding associated fields.

[0078] Step S330. If included, obtain the key table in the wide table, and determine whether the key data in the key table is complete based on the association condition objects of each node in the association tree structure.

[0079] If each target retrieval node's corresponding table contains the corresponding table data, then the key data in the key table is checked for completeness based on the association condition object. If no data is found in the single table corresponding to the target retrieval node, the scan is terminated and the process ends.

[0080] Step S340. If the key data in the key table is complete, insert the single-table data from the target single table into the wide table.

[0081] If the key data in the key table is complete, then insert the data from the target single table into the wide table. If the key data is incomplete, then no operation is performed.

[0082] Figures 4 to 7 To provide a synchronization processing method for a wide table corresponding to an insertion operation according to a specific embodiment of this disclosure, in order to Figure 2 The example of each node in the association tree structure of the wide table shown is an illustration of the steps described above in this example implementation.

[0083] like Figure 4 The diagram shown is a complete flowchart of inserting data into table A in a specific embodiment of this disclosure. The specific steps of the flowchart are as follows:

[0084] Step S410. Data from table A is received; update the cached object in table A.

[0085] Step S420. Scan the associated tree structure of the wide table.

[0086] Step S430. Determine if table B contains data.

[0087] If the index-kv is retrieved based on the associated fields of tables A and B, and no data is found in table B (i.e., no match is found), then proceed to step S440; if data is found in table B, then proceed to step S450.

[0088] Step S440. Terminate the scan.

[0089] Step S450. Determine if the key data is complete.

[0090] Determine whether the key data in the wide table is complete based on the index. If it is incomplete, proceed to step S460; if it is complete, proceed to step S470.

[0091] Step S460. No operation is performed.

[0092] Step S470. Insert a wide table.

[0093] If all the key data is complete, insert the data from table A into the wide table.

[0094] like Figure 5 The diagram shown is a complete flowchart of inserting data into table B in a specific embodiment of this disclosure. The specific steps of the flowchart are as follows:

[0095] Step S510. Data from table B is obtained; update the cached object in table B.

[0096] Step S520. Scan the associated tree structure of the wide table.

[0097] Step S530. Determine if there is data in table A.

[0098] If the index-kv is retrieved based on the associated fields of tables A and B, and no data is found in table A (i.e., no match is found), then proceed to step S540; if the data is found in table A, then proceed to step S550.

[0099] Step S540. Terminate the scan.

[0100] Step S550. Determine if table C contains data.

[0101] Scan table C and determine if there is any data. If no data is found in table C, proceed to step S560; if data is found in table C, proceed to step S57.

[0102] Step S560. Terminate the scan.

[0103] Step S570. Determine if the key data is complete.

[0104] Determine whether the key data in the wide table is complete based on the index. If it is incomplete, proceed to step S580; if it is complete, proceed to step S590.

[0105] Step S580. No operation is performed.

[0106] Step S590. Insert a wide table.

[0107] If all the key data is complete, insert the data from table B into the wide table.

[0108] like Figure 6 The diagram shown is a complete flowchart of inserting data into table C in a specific embodiment of this disclosure. The specific steps of the flowchart are as follows:

[0109] Step S610. Data from table C is obtained; update the cached object in table C.

[0110] Step S620. Scan the associated tree structure of the wide table.

[0111] Step S630. Update the cached data of tables A, B, and D.

[0112] Step S640. Insert a wide table.

[0113] Populate the wide table with the calculated data from the cache.

[0114] like Figure 7 The diagram shown is a complete flowchart of inserting data into table D in a specific embodiment of this disclosure. The specific steps of the flowchart are as follows:

[0115] Step S710. Data from table D is obtained; update the cached object in table D.

[0116] Step S720. Scan the associated tree structure of the wide table.

[0117] Step S730. Determine if table C contains data.

[0118] If no data is found in table C, i.e. no match is found, proceed to step S740; if data is found in table C, proceed to step S750.

[0119] Step S740. Terminate the scan.

[0120] Step S750. Determine if the key data is complete.

[0121] Determine whether the key data in the wide table is complete based on the index. If it is incomplete, proceed to step S760; if it is complete, proceed to step S770.

[0122] Step S760. No operation is performed.

[0123] Step S770. Insert a wide table.

[0124] If all the key data is complete, insert the data from table D into the wide table.

[0125] In this example implementation, such as Figure 8 As shown, when the operation type is a delete operation, the target single table is processed according to the operation type and the association condition objects of each node in the association tree structure. Specifically, this may include the following steps:

[0126] Step S810. Obtain the key table in the wide table, and determine whether the key data in the key table is complete based on the association condition objects of each node in the association tree structure.

[0127] At the same time, the cached objects and associated condition objects in the target node are removed from the cache.

[0128] Step S820. If the key data in the key table is complete, perform the deletion operation according to the single table type in the target node.

[0129] Specifically, if the single table in the target node is a critical table, then the wide table is deleted. If the single table in the target node is a regular table, then the corresponding single table data in the target node is deleted, and the data in the wide table is updated. If the critical data in the critical table is incomplete, no operation is performed.

[0130] Figure 9 and Figure 10 To provide a synchronization processing method for the wide table corresponding to a deletion operation according to a specific embodiment of this disclosure, in order to Figure 2 The example of each node in the association tree structure of the wide table shown is an illustration of the steps described above in this example implementation.

[0131] like Figure 9 The diagram shown is a complete flowchart of deleting key table data in a specific embodiment of this disclosure. The specific steps of the flowchart are as follows:

[0132] Step S910. When a deletion event occurs in table A or table B, update the corresponding single-table cache object.

[0133] Step S920. Scan the associated tree structure of the wide table.

[0134] If the index-kv is retrieved based on the association field of the AB table, and a match is found, the cached data is removed from the corresponding value and also removed from the buffer. If no match is found, no operation is performed.

[0135] Step S930. Determine if the key data is complete.

[0136] If incomplete, proceed to step S940; if complete, proceed to step S950.

[0137] Step S940. No operation is performed.

[0138] Step S950. Delete the wide table.

[0139] like Figure 10 The diagram shown is a complete flowchart of deleting data from a regular table in a specific embodiment of this disclosure. The specific steps of the flowchart are as follows:

[0140] Step S1010. When the C or D table deletion event occurs, update the corresponding single-table cache object.

[0141] Step S1020. Scan the associated tree structure of the wide table.

[0142] If the index-kv is retrieved based on the associated field of the CD table, and a match is found, the cached data is removed from the corresponding value and also removed from the buffer. If no match is found, no operation is performed.

[0143] Step S1030. Determine if the key data is complete.

[0144] If incomplete, proceed to step S1040; if complete, proceed to step S1050.

[0145] Step S1040. No operation is performed.

[0146] Step S1050. Delete table C or table D and its corresponding child node data.

[0147] Step S1060. Update the wide table data.

[0148] In this example implementation, such as Figure 11 As shown, when the operation type is an update operation, the target single table is processed according to the operation type and the association condition objects of each node in the association tree structure. Specifically, this may include the following steps:

[0149] Step S1110. Determine whether the target field in the target table corresponding to the update operation is a key field.

[0150] When an update event occurs, different processing procedures need to be selected depending on whether the updated data is a critical field.

[0151] Step S1120. If the target field is not a key field, delete the original field data in the target field and insert the updated field data.

[0152] If the target field is not a key field, delete the original field data before the update, and then insert the updated data.

[0153] Step S1130. If the target field is a key field, then insert the updated field data into the cache object corresponding to the target single table.

[0154] If the target field is a critical field, only the Buffer data is updated, and the updated field data is inserted into the cache object corresponding to the target single table.

[0155] Figure 12 To provide a synchronization processing method for the wide table corresponding to the update operation in a specific embodiment of this disclosure, Figure 2 The example of each node in the association tree structure of the wide table is used to illustrate the above steps in this example implementation. The specific steps of the flowchart are as follows:

[0156] Step S1210. Obtain the target field to be updated.

[0157] Step S1220. Determine if it is a key field.

[0158] If yes, proceed to step S1230; otherwise, proceed to step S1240.

[0159] Step S1230. Update only the data in the cached object.

[0160] Step S1240. Delete the original field data before the update.

[0161] Step S1250. Insert the updated field data.

[0162] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0163] Furthermore, this disclosure also provides a synchronization processing apparatus for wide tables. (See reference...) Figure 13 As shown, the synchronization processing device for the wide table may include an operation instruction acquisition module 1310, a wide table structure acquisition module 1320, and a target single table processing module 1330. Wherein:

[0164] The operation instruction acquisition module 1310 can be used to respond to synchronization operation instructions for wide tables and acquire the target single table and operation type corresponding to the synchronization operation instruction.

[0165] The wide table structure acquisition module 1320 can be used to acquire the association tree structure of the wide table and determine the target node corresponding to the target single table in the association tree structure. Each node in the association tree structure includes a cache object and multiple association condition objects, and each node corresponds to a single table.

[0166] The target single table processing module 1330 can be used to update the cached objects in the target node according to the synchronization operation instructions, and process the target single table according to the operation type and the association condition objects of each node in the association tree structure.

[0167] In some exemplary embodiments of this disclosure, the target single-table processing module 1330 may include a target retrieval node determination unit, a target retrieval data judgment unit, a key data judgment unit, and a single-table data insertion unit. Wherein:

[0168] The target retrieval node determination unit can be used to determine the target retrieval node corresponding to the target node based on the single table type in the target node and the association condition objects of each node in the association tree structure.

[0169] The target retrieval data judgment unit can be used to determine whether the single table corresponding to the target retrieval node contains the corresponding single table data.

[0170] The key data judgment unit can be used to retrieve the key table in the wide table if it is included, and determine whether the key data in the key table is complete based on the association condition objects of each node in the association tree structure.

[0171] The single-table data insertion unit can be used to insert single-table data from the target single table into a wide table if the key data in the key table is complete.

[0172] In some exemplary embodiments of this disclosure, the target single-table processing module 1330 may further include a key data judgment unit and a deletion operation execution unit. Wherein:

[0173] The key data judgment unit can be used to obtain the key tables in the wide table and, based on the association condition objects of each node in the association tree structure, determine whether the key data in the key tables is complete.

[0174] The delete operation execution unit can be used to perform a delete operation based on the single table type in the target node if the key data in the key table is complete.

[0175] In some exemplary embodiments of this disclosure, the deletion operation execution unit may include a key table deletion unit, which can be used to delete a wide table if the single table type in the target node is a key table.

[0176] In some exemplary embodiments of this disclosure, the deletion operation execution unit may further include a regular table deletion unit, which can be used to delete the data in the regular table corresponding to the target node and update the data in the wide table if the single table in the target node is of the regular table type.

[0177] In some exemplary embodiments of this disclosure, the target single-table processing module 1330 may further include a cache object removal unit, which can be used to remove cache objects and associated condition objects in the target node from the cache.

[0178] In some exemplary embodiments of this disclosure, the target single-table processing module 1330 may further include a key field determination unit, a general field update unit, and a key field update unit. Wherein:

[0179] The key field determination unit can be used to determine whether the target field in the target table corresponding to the update operation is a key field;

[0180] The regular field update unit can be used to delete the original field data in the target field and insert the updated field data if the target field is not a key field;

[0181] The key field update unit can be used to insert the updated field data into the cache object corresponding to the target single table if the target field is a key field.

[0182] The specific details of each module / unit in the above-mentioned wide table synchronization processing device have been described in detail in the corresponding method embodiment section, and will not be repeated here.

[0183] Figure 14 A schematic diagram of a computer system suitable for implementing embodiments of the present invention is shown.

[0184] It should be noted that, Figure 14 The computer system 1400 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0185] like Figure 14 As shown, the computer system 1400 includes a central processing unit (CPU) 1401, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1402 or programs loaded from storage section 1408 into random access memory (RAM) 1403. The RAM 1403 also stores various programs and data required for system operation. The CPU 1401, ROM 1402, and RAM 1403 are interconnected via a bus 1404. An input / output (I / O) interface 1405 is also connected to the bus 1404.

[0186] The following components are connected to I / O interface 1405: an input section 1406 including a keyboard, mouse, etc.; an output section 1407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1408 including a hard disk, etc.; and a communication section 1409 including a network interface card such as a LAN card, modem, etc. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to I / O interface 1405 as needed. Removable media 1411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1410 as needed so that computer programs read from them can be installed into storage section 1408 as needed.

[0187] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1409, and / or installed from removable medium 1411. When the computer program is executed by central processing unit (CPU) 1401, it performs various functions defined in the system of this application.

[0188] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0189] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0190] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0191] It should be noted that although several modules for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0192] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0193] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for synchronizing wide tables, characterized in that, include: In response to a synchronization operation command for a wide table, obtain the target single table and operation type corresponding to the synchronization operation command; Obtain the association tree structure of the wide table and determine the target node corresponding to the target single table in the association tree structure. Each node in the association tree structure includes a cache object and multiple association condition objects. Each node corresponds to a single table. The association condition object is an object that has an association relationship with the node. Each association condition object corresponds to an association condition. The cached object in the target node is updated according to the synchronization operation instruction, and the target single table is processed according to the operation type and the association condition objects of each node in the association tree structure.

2. The synchronization processing method for wide tables according to claim 1, characterized in that, The operation type includes an insertion operation. The processing of the target single table based on the operation type and the association condition objects of each node in the association tree structure includes: Based on the single table type in the target node and the association condition objects of each node in the association tree structure, determine the target retrieval node corresponding to the target node; Determine whether the single table corresponding to the target retrieval node contains the corresponding single table data; If included, the key table in the wide table is obtained, and the key data in the key table is determined to be complete based on the association condition objects of each node in the association tree structure. If the key data in the key table is complete, then the single-table data from the target single table will be inserted into the wide table.

3. The synchronization processing method for wide tables according to claim 1, characterized in that, The operation type includes a deletion operation. The process of processing the target single table based on the operation type and the association condition objects of each node in the association tree structure includes: Obtain the key table in the wide table, and determine whether the key data in the key table is complete based on the association condition objects of each node in the association tree structure; If the key data in the key table is complete, then the deletion operation is performed according to the single table type in the target node.

4. The synchronization processing method for wide tables according to claim 3, characterized in that, The step of performing the deletion operation based on the single table type in the target node includes: If the single table type in the target node is a critical table, then delete the wide table.

5. The synchronization processing method for wide tables according to claim 3, characterized in that, The step of performing the deletion operation based on the single table type in the target node includes: If the single table type in the target node is a regular table, then delete the single table data corresponding to the target node and update the data in the wide table.

6. The synchronization processing method for wide tables according to claim 3, characterized in that, The method further includes: Remove the cached objects and associated condition objects from the cache in the target node.

7. The synchronization processing method for wide tables according to claim 1, characterized in that, The operation type includes an update operation. The process of processing the target single table based on the operation type and the association condition objects of each node in the association tree structure includes: Determine whether the target field in the target table corresponding to the update operation is a key field; If the target field is not the key field, then delete the original field data in the target field and insert the updated field data; If the target field is the key field, then the updated field data is inserted into the cache object corresponding to the target single table.

8. A synchronization processing device for wide tables, characterized in that, include: The operation instruction acquisition module is used to respond to the synchronization operation instruction for the wide table and acquire the target single table and operation type corresponding to the synchronization operation instruction. The wide table structure acquisition module is used to acquire the association tree structure of the wide table and determine the target node corresponding to the target single table in the association tree structure. Each node in the association tree structure includes a cache object and multiple association condition objects. Each node corresponds to a single table. The association condition object is an object that has an association relationship with the node. Each association condition object corresponds to an association condition. The target single table processing module is used to update the cached object in the target node according to the synchronization operation instruction, and to process the target single table according to the operation type and the association condition objects of each node in the association tree structure.

9. An electronic device, characterized in that, include: processor; as well as A memory for storing one or more programs that, when executed by the processor, cause the processor to implement the synchronous processing method for wide tables as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the synchronization processing method for wide tables as described in any one of claims 1 to 7.

Images