A method and system for real-time data processing of multiple table associations in an Oracle database
By building a model tree and generating metadata information, generating a target table result set and sharing it in real time, the problems of low efficiency and high resource consumption of multi-table association query in the existing technology are solved, and the effect of real-time data processing of multi-table association in oracle database is achieved.
Patent Information
- Application Number
- CN202410894151.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-04
- Publication Date
- 2025-04-25
- Estimated Expiration
- 2044-07-04
AI Technical Summary
When conducting database multi-table association query, the prior art needs to create triggers in the source database, resulting in strong invasiveness, low query efficiency and high resource consumption, making it difficult to realize real-time data processing.
A real-time data processing method for multi-table association of oracle database is proposed. By building a model tree and generating metadata information, the target table result set is generated, and sent to the target end through the TCP/IP protocol, real-time sharing of the multi-table association query result set is realized without establishing a trigger in the database.
Real-time sharing of multi-table associated query result sets is realized, which reduces invasiveness to the source database, improves query efficiency, saves resource usage, and can obtain data that users care about in real time from complex data.
Smart Images

Figure CN118820241B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of database technology, and in particular to a method and system for real-time data processing of multi-table association in an Oracle database. Background Art
[0002] Currently, the capture and synchronization is performed by creating triggers in the source database, which requires operations on the customer's source database and is invasive. Many customers are not allowed to perform any operations in the production environment. Therefore, it has certain limitations. For the method of transferring the intermediate files analyzed from the source to the target database, a lot of data needs to be synchronized, and the resource usage is large. It is difficult to obtain the data that users want to obtain in real time from the complex data. It may also require customers to directly query multiple tables in the system, and the direct query efficiency is relatively slow and consumes more resources of the source database. Summary of the invention
[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.
[0004] To this end, the present invention proposes a real-time data processing method for multi-table association of Oracle database, which does not need to establish triggers in the database, does not affect the original performance of the database, does not require additional permissions, and has no intrusiveness to the database. Therefore, data extraction can be effectively realized.
[0005] Another object of the present invention is to provide another Oracle database multi-table association real-time data processing method.
[0006] The third object of the present invention is to provide an Oracle database multi-table association real-time data processing system.
[0007] The fourth object of the present invention is to provide another Oracle database multi-table association real-time data processing system.
[0008] To achieve the above object, the present invention proposes a method for real-time data processing of multiple table associations in an Oracle database, which is applied to a source end. The method comprises:
[0009] Build a model tree based on the configuration relationships parsed from the original table and target table of the SQL statement that associates multiple tables in the Oracle database;
[0010] Creating metadata information according to the model tree;
[0011] According to the determined structure of the model tree, a corresponding target table result set is generated through row pseudo-column identification information of the original table, metadata information, and the correspondence between the fields in the original table and the target table;
[0012] The converted target table result set is sent to the corresponding receiving program of the target end through the TCP / IP protocol to complete the real-time sharing of the multi-table association query result set.
[0013] The Oracle database multi-table association real-time data processing method of the embodiment of the present invention may also have the following additional technical features:
[0014] In one embodiment of the present invention, a model tree is constructed based on the configuration relationship parsed from the original table and the target table of the SQL statement of multi-table association in the Oracle database, including:
[0015] Parse the original table and target table configuration items of the SQL statement configured in the Oracle database configuration file to obtain the complete fields and data types of the original table and the fields of the target table;
[0016] Parse the data types and the correspondence between the fields in the original table and the target table, and build a model tree based on the parsed correspondence.
[0017] In one embodiment of the present invention, before determining the structure of the model tree, the method further includes:
[0018] Obtain the DML operation information of the original table by analyzing the redo log, and store the DML operation information in an intermediate file;
[0019] The intermediate file is read.
[0020] In one embodiment of the present invention, a corresponding target table result set is generated through row pseudo-column identification information of the original table, metadata information, and a correspondence between fields in the original table and the target table, including:
[0021] The insert operation of the original table is judged: if the value of the field of the original table meets the condition of the associated table in the configuration file, it affects the result set of the target table, and the conversion of the original table data change column to the corresponding target table data change column is completed according to the model tree, and the first intermediate file result set is assembled according to the metadata information and sent to the target end;
[0022] The update operation of the original table is judged: if the updated column of the original table is not in the field of the target table, it is judged that it does not affect the result set of the target table and is not sent to the target end; if the updated column of the original table is in the field of the target table, it is judged according to the model tree that it does not affect the result set of the target table and is not sent to the target end; if the updated column of the original table is in the field of the target table, it is judged according to the model tree that it affects the result set of the target table, and the conversion of the original table data change column to the corresponding target table data change column is completed according to the model tree, and assembled into a second intermediate file result set according to the metadata information and sent to the target end;
[0023] Determine the deletion operation of the original table: delete one of the records in the original table, determine it as affecting the target table result set according to the model tree, complete the conversion of the original table data change column to the corresponding target table data change column according to the model tree, and assemble it into a third intermediate file result set based on the metadata information and send it to the target end.
[0024] To achieve the above object, the present invention proposes, on the other hand, a method for real-time data processing of multi-table association in an Oracle database, which is applied to a target end and comprises:
[0025] Receiving the first intermediate file result set, the second intermediate file result set and the third intermediate file result set through the TCP / IP protocol;
[0026] According to the format requirements of the target system, the interface of the target system is used to sequentially load the incremental files corresponding to the received first intermediate file result set, the second intermediate file result set and the third intermediate file result set to complete the real-time sharing of the multi-table association query result set.
[0027] To achieve the above object, the present invention further provides an Oracle database multi-table association real-time data processing system, comprising:
[0028] The model parsing module is used to build a model tree based on the configuration relationship parsed from the original table and target table of the SQL statement of multi-table association in the Oracle database;
[0029] A metadata module, used for creating metadata information according to the model tree;
[0030] An incremental conversion module is used to generate a corresponding target table result set according to the determined model tree structure and through the row pseudo-column identification information of the original table, metadata information, and the correspondence between the fields in the original table and the target table;
[0031] The incremental sending module is used to send the converted target table result set to the corresponding receiving program of the target end through the TCP / IP protocol to complete the real-time sharing of the multi-table association query result set.
[0032] To achieve the above object, the present invention further provides an Oracle database multi-table association real-time data processing system, comprising:
[0033] An incremental receiving module, used for receiving a first intermediate file result set, a second intermediate file result set and a third intermediate file result set through a TCP / IP protocol;
[0034] The data loading module is used to sequentially load the incremental files corresponding to the received first intermediate file result set, second intermediate file result set and third intermediate file result set according to the format requirements of the target system using the interface of the target system to complete the real-time sharing of multi-table association query result sets.
[0035] The Oracle database multi-table association real-time data processing method and system of the embodiment of the present invention constructs a model for complex SQL associated with multiple tables. The constructed model includes all original tables involved in the associated query, which are defined as ctab. The result table presented by the result set of the final query is defined as stab. The corresponding relationship constructed by the model includes the corresponding relationship between each column element of ctab and each column element of stab, including directly referenced columns and column elements converted using complex functions; the model construction includes the corresponding relationship between the unique element of ctab and multiple elements of stab, and then the real-time change of the final stab data is obtained by real-time conversion according to the model tree through the real-time change of ctab row data, thereby completing the real-time data sharing of the final multi-table association result set.
[0036] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0038] Figure 1 is a flow chart of a method for processing real-time data of multiple table associations in an Oracle database according to an embodiment of the present invention;
[0039] Figure 2 is a flowchart of another method for processing real-time data of multiple table associations in an Oracle database according to an embodiment of the present invention;
[0040] Figure 3 This is a structural diagram of an Oracle database multi-table association real-time data processing system according to an embodiment of the present invention;
[0041] Figure 4 It is a structural diagram of another Oracle database multi-table association real-time data processing system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0042] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0043] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0044] The following describes an Oracle database multi-table association real-time data processing method and system according to an embodiment of the present invention with reference to the accompanying drawings.
[0045] Figure 1 is a flow chart of a method for processing real-time data in an Oracle database with multiple table associations according to an embodiment of the present invention. Figure 1 As shown, applied to the source end, the method includes:
[0046] S1, build a model tree based on the configuration relationship parsed from the original table and target table of the SQL statement of multi-table association in the Oracle database.
[0047] Specifically, select one or more original tables, define the target table, add the rowid (pseudo column used to uniquely mark rows in the table) field related to the original table, and replace the business logic configuration association condition SQL (Structured Query Language) statement.
[0048] It can be understood that the present invention mainly parses the sql (structured query statement configuring associated conditions) statements, ctab (original table) and stab (target table) configuration items configured in the configuration file, and parses out the complete fields of the original table, data types and the fields of the target table, data types and the correspondence between the fields in the original table and the target table.
[0049] Furthermore, a model tree is constructed based on the parsed correspondence. The fields in the target table need to be additionally added with one or more rowid fields (pseudo columns used to uniquely mark rows in a table) of the original table as unique identifiers. And a unique index needs to be added to the rowid field (pseudo columns used to uniquely mark rows in a table) of the original table in the target table.
[0050] S2, creating metadata information based on the model tree.
[0051] It is understandable that in Oracle database, you can use Oracle Data Modeler to build metadata models. This tool supports multiple modeling methods, such as ER (entity relationship), OO (object-oriented), UML (unified modeling language), etc. When building metadata models, you need to consider the hierarchical structure, attributes and relationships of the data, as well as the special properties of database objects, such as storage type, index status, permissions, etc.
[0052] Before establishing a metadata model, it is necessary to define metadata specifications, that is, to specify the standards, naming rules, data types, formats, etc. of metadata. This helps to ensure the consistency and maintainability of metadata. Metadata specifications can be customized according to actual needs. Generally speaking, the following contents need to be included: classification and organization of metadata; naming rules and namespaces of metadata; data types and formats of metadata; storage methods and management rules of metadata; visualization and display methods of metadata.
[0053] The metadata warehouse is a database that centrally stores metadata information. It contains the definitions, attributes, structures, and relationships of various metadata. In Oracle databases, you can use Oracle Metadata Manager to create and manage metadata warehouses. When creating a metadata warehouse, you need to pay attention to the following points:
[0054] Select appropriate database engines and storage devices; define metadata extraction, conversion, and loading rules; establish metadata indexing and query functions; and ensure the security and confidentiality of metadata.
[0055] Furthermore, after the metadata model and metadata warehouse are established, you can start integrating and using metadata. Oracle Database provides a variety of tools and interfaces, such as Oracle Enterprise Manager and Oracle Metadata API, which can easily obtain, analyze, manage and apply metadata information. When integrating and using metadata, you also need to pay attention to the following points: combine metadata with business and application systems; use metadata for data analysis and intelligent decision-making; strengthen metadata quality and availability management; and establish a complete metadata management process and mechanism.
[0056] In summary, metadata modeling is an important part of database management, and Oracle database has rich experience and technical solutions in metadata modeling practice. Creating metadata information models based on the model tree, defining metadata specifications, establishing metadata warehouses, and integrating and utilizing metadata can help database managers better understand data, plan and manage data resources and application systems, improve data quality and reliability, reduce workload and costs, and maintain data consistency and stability.
[0057] S3, according to the determined structure of the model tree, and through the row pseudo-column identification information of the original table, metadata information, and the correspondence between the fields in the original table and the target table, a corresponding target table result set is generated.
[0058] It can be understood that the present invention obtains the DML (database operation language, used to operate records in the database table, such as adding, deleting, modifying, and checking) operations of the original table by analyzing the redo log, and stores them in the intermediate file.
[0059] Furthermore, by reading the intermediate file, according to the established model tree structure, the corresponding target table result set is generated through the rowid (pseudo column used to uniquely mark rows in the table) information of the original table, metadata information, the correspondence between the original table fields and the target table fields.
[0060] For example, the insert operation of the original table is judged as follows:
[0061] If the value of the field meets the conditions of the associated table in the configuration file and affects the target table result set, the conversion of the original table data change column to the corresponding target table data change column will be completed according to the model tree. And it will be assembled into the first intermediate file result set according to the metadata information and sent to the target end. If the value of the field does not meet the conditions of the associated table in the configuration file, it will not affect the target table result set and will not be sent to the target end.
[0062] Exemplarily, the update operation of the original table is judged as follows:
[0063] The updated columns of the original table are not in the fields of the target table, so they are directly judged as not affecting the result set of the target table and are not sent to the target end;
[0064] The updated columns of the original table are in the fields of the target table. According to the model tree, they are judged not to affect the result set and are not sent to the target end.
[0065] The updated columns of the original table are in the fields of the target table. According to the model tree, they are determined to affect the result set. The data change columns of the original table are converted to the corresponding data change columns of the target table according to the model tree. The second intermediate file result set is assembled according to the metadata information and sent to the target end.
[0066] For example, the deletion operation of the original table will be judged as follows:
[0067] If one of the records in the original table is deleted, the model tree determines that it affects the result set, and the data change column of the original table is converted to the corresponding data change column of the target table according to the model tree. The third intermediate file result set is assembled according to the metadata information and sent to the target end.
[0068] S4, sending the converted target table result set to the corresponding receiving program on the target end through the TCP / IP protocol to complete the real-time sharing of the multi-table association query result set.
[0069] Specifically, the present invention sends the converted result set to the corresponding receiving program of the target end through the TCP / IP (Internet-based network protocol) protocol. If the reception is not successful, a retransmission mechanism will be triggered, and a new conversion process will not be performed until the target end receives it successfully. Thus, the real-time sharing of the query result set is completed at the target end.
[0070] According to the Oracle database multi-table association real-time data processing method of the embodiment of the present invention, the intermediate file analyzed by the source end can be directly copied to the target end database, or copied to the intermediate database, or copied to the message middleware and then extracted to different target end databases or big data components through the consumer program. The present invention can reduce the data that needs to be synchronized, save resource usage, and can also obtain the data that users care about in real time from the complex data. The present invention greatly reduces the time for users to query and convert the data they care about, and improves efficiency. At the same time, there is no need to create additional objects in the source database to assist synchronization, which is not invasive to the source database and will not cause additional burden to the user's system.
[0071] In order to implement the above embodiment, Figure 2 As shown, this embodiment also provides another Oracle database multi-table association real-time data processing method, which is applied to the target end and includes:
[0072] S101, receiving a first intermediate file result set, a second intermediate file result set, and a third intermediate file result set through a TCP / IP protocol;
[0073] S102, according to the format requirements of the target system, using the interface of the target system, sequentially load the incremental files corresponding to the received first intermediate file result set, the second intermediate file result set and the third intermediate file result set, so as to complete the real-time sharing of the multi-table association query result set.
[0074] Specifically, based on the first intermediate file result set, the second intermediate file result set and the third intermediate file result set data output by the source end, the target end server stores the received data stream on the target end server disk through the TCP / IP (Internet-based network protocol) protocol.
[0075] Furthermore, according to the format requirements of the target system, the interface of the target system is used to sequentially load the incremental files corresponding to the received first intermediate file result set, second intermediate file result set and third intermediate file result set to complete real-time sharing of multi-table association query result sets.
[0076] According to the Oracle database multi-table association real-time data processing method of the embodiment of the present invention, the client does not need to directly query the multi-table association in the system. The direct query efficiency is relatively slow and consumes more source database resources. The data collected at the target end can be directly analyzed and used by the business end. The present invention greatly reduces the time for users to query and convert the data they are concerned about, and improves efficiency. At the same time, there is no need to create additional objects in the source database to assist synchronization, which is non-invasive to the source database and will not cause additional burden to the user's system.
[0077] In order to implement the above embodiment, Figure 3 As shown, this embodiment also provides another Oracle database multi-table association real-time data processing system 10, including:
[0078] The model parsing module 100 is used to construct a model tree according to the configuration relationship parsed from the original table and the target table of the SQL statement of multi-table association in the Oracle database;
[0079] A metadata module 200, used to create metadata information according to the model tree;
[0080] The incremental conversion module 300 is used to generate a corresponding target table result set according to the determined structure of the model tree and through the row pseudo-column identification information of the original table, metadata information, and the correspondence between the fields in the original table and the target table;
[0081] The incremental sending module 400 is used to send the converted target table result set to the corresponding receiving program of the target end through the TCP / IP protocol to complete the real-time sharing of the multi-table association query result set.
[0082] Furthermore, the above-mentioned model parsing module 100 is also used for:
[0083] Parse the original table and target table configuration items of the SQL statement configured in the Oracle database configuration file to obtain the complete fields and data types of the original table and the fields of the target table;
[0084] Parse the data types and the correspondence between the fields in the original table and the target table, and build a model tree based on the parsed correspondence.
[0085] Furthermore, before the incremental conversion module 300, the incremental analysis module is also included, which is used to:
[0086] Obtain the DML operation information of the original table by analyzing the redo log, and store the DML operation information in an intermediate file;
[0087] Read the intermediate file.
[0088] Furthermore, the incremental conversion module 300 is also used for:
[0089] The insert operation of the original table is judged: if the value of the field of the original table meets the condition of the associated table in the configuration file, it affects the result set of the target table, and the conversion of the original table data change column to the corresponding target table data change column is completed according to the model tree, and the first intermediate file result set is assembled according to the metadata information and sent to the target end;
[0090] The update operation of the original table is judged: if the updated column of the original table is not in the field of the target table, it is judged that it does not affect the result set of the target table and is not sent to the target end; if the updated column of the original table is in the field of the target table, it is judged according to the model tree that it does not affect the result set of the target table and is not sent to the target end; if the updated column of the original table is in the field of the target table, it is judged according to the model tree that it affects the result set of the target table, and the conversion of the original table data change column to the corresponding target table data change column is completed according to the model tree, and assembled into a second intermediate file result set according to the metadata information and sent to the target end;
[0091] Determine the deletion operation of the original table: delete one of the records in the original table, determine it as affecting the target table result set according to the model tree, complete the conversion of the original table data change column to the corresponding target table data change column according to the model tree, and assemble it into a third intermediate file result set based on the metadata information and send it to the target end.
[0092] According to the oracle database multi-table association real-time data processing system of the embodiment of the present invention, the intermediate file analyzed by the source end can be directly copied to the target end database, or copied to the intermediate database, or copied to the message middleware and then extracted to different target end databases or big data components through the consumer program. The present invention can reduce the data that needs to be synchronized, save resource usage, and can also obtain the data that users care about in real time from the complex data. The present invention greatly reduces the time for users to query and convert the data they care about, and improves efficiency. At the same time, there is no need to create additional objects in the source database to assist synchronization, which is not invasive to the source database and will not cause additional burden to the user's system.
[0093] In order to implement the above embodiment, Figure 4 As shown, this embodiment also provides another processing system 20 for Caché database logical replication to heterogeneous data, including:
[0094] The incremental receiving module 30 is used to receive the first intermediate file result set, the second intermediate file result set and the third intermediate file result set through the TCP / IP protocol;
[0095] The data loading module 40 is used to sequentially load the incremental files corresponding to the received first intermediate file result set, second intermediate file result set and third intermediate file result set according to the format requirements of the target system using the interface of the target system to complete the real-time sharing of multi-table association query result sets.
[0096] According to the Oracle database multi-table association real-time data processing system of the embodiment of the present invention, the client does not need to directly query the multi-table association in the system. The direct query efficiency is relatively slow and consumes more resources of the source database. The data collected at the target end can be directly analyzed and used by the business end. The present invention greatly reduces the time for users to query and convert the data they are concerned about, and improves efficiency. At the same time, there is no need to create additional objects in the source database to assist synchronization, which is non-invasive to the source database and will not cause additional burden to the user's system.
[0097] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0098] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
Claims
1. A method for real-time data processing of multiple table associations in an Oracle database, characterized in that: Applied to the source end, the method includes: Build a model tree based on the configuration relationships parsed from the original table and target table of the SQL statement that associates multiple tables in the Oracle database; Creating metadata information according to the model tree; According to the determined structure of the model tree, a corresponding target table result set is generated through row pseudo-column identification information of the original table, metadata information, and the correspondence between the fields in the original table and the target table; The converted target table result set is sent to the corresponding receiving program of the target end through the TCP / IP protocol to complete the real-time sharing of the multi-table association query result set; The model tree is constructed based on the configuration relationships parsed from the original table and target table of the SQL statement for multi-table association in the Oracle database, including: Parse the original table and target table configuration items of the SQL statement configured in the Oracle database configuration file to obtain the complete fields and data types of the original table and the fields of the target table; Before determining the structure of the model tree, the method further includes: Obtain the DML operation information of the original table by analyzing the redo log, and store the DML operation information in an intermediate file; Reading the intermediate file; Parse the data types and the correspondence between the fields in the original table and the target table, and build a model tree based on the parsed correspondence; The corresponding target table result set is generated through the row pseudo-column identification information, metadata information of the original table, and the correspondence between the fields in the original table and the target table, including: The insert operation of the original table is judged: if the value of the field of the original table meets the condition of the associated table in the configuration file, it affects the result set of the target table, and the conversion of the original table data change column to the corresponding target table data change column is completed according to the model tree, and the first intermediate file result set is assembled according to the metadata information and sent to the target end; The update operation of the original table is judged: if the updated column of the original table is not in the field of the target table, it is judged that it does not affect the result set of the target table and is not sent to the target end; if the updated column of the original table is in the field of the target table, it is judged according to the model tree that it does not affect the result set of the target table and is not sent to the target end; if the updated column of the original table is in the field of the target table, it is judged according to the model tree that it affects the result set of the target table, and the conversion of the original table data change column to the corresponding target table data change column is completed according to the model tree, and assembled into a second intermediate file result set according to the metadata information and sent to the target end; Determine the deletion operation of the original table: delete one of the records in the original table, determine it as affecting the target table result set according to the model tree, complete the conversion of the original table data change column to the corresponding target table data change column according to the model tree, and assemble it into a third intermediate file result set based on the metadata information and send it to the target end.
2. An Oracle database multi-table association real-time data processing system, characterized in that: include: The model parsing module is used to build a model tree based on the configuration relationship parsed from the original table and the target table of the SQL statement of multi-table association in the Oracle database; A metadata module, used for creating metadata information according to the model tree; Incremental analysis module for: Obtain the DML operation information of the original table by analyzing the redo log, and store the DML operation information in an intermediate file; Reading the intermediate file; An incremental conversion module is used to generate a corresponding target table result set according to the determined model tree structure and through the row pseudo-column identification information of the original table, metadata information, and the correspondence between the fields in the original table and the target table; An incremental sending module is used to send the converted target table result set to a corresponding receiving program at the target end through the TCP / IP protocol to complete the real-time sharing of multi-table association query result sets; The model parsing module is also used to: Parse the original table and target table configuration items of the SQL statement configured in the Oracle database configuration file to obtain the complete fields and data types of the original table and the fields of the target table; Parse the data types and the correspondence between the fields in the original table and the target table, and build a model tree based on the parsed correspondence; The incremental conversion module is also used to: The insert operation of the original table is judged: if the value of the field of the original table meets the condition of the associated table in the configuration file, it affects the result set of the target table, and the conversion of the original table data change column to the corresponding target table data change column is completed according to the model tree, and the first intermediate file result set is assembled according to the metadata information and sent to the target end; The update operation of the original table is judged: if the updated column of the original table is not in the field of the target table, it is judged that it does not affect the result set of the target table and is not sent to the target end; If the updated column of the original table is in the field of the target table, it is determined according to the model tree that it does not affect the result set of the target table and is not sent to the target end; if the updated column of the original table is in the field of the target table, it is determined according to the model tree that it affects the result set of the target table, and the conversion of the original table data change column to the corresponding target table data change column is completed according to the model tree, and assembled into a second intermediate file result set according to the metadata information and sent to the target end; Determine the deletion operation of the original table: delete one of the records in the original table, determine it as affecting the target table result set according to the model tree, complete the conversion of the original table data change column to the corresponding target table data change column according to the model tree, and assemble it into a third intermediate file result set based on the metadata information and send it to the target end.
Citation Information
Patent Citations
Data origin collection method of relational databases
CN105912595A
Data synchronization method and system
CN113282599A