Applying changes in target database system

Abstract

In a method for dynamically selecting an application algorithm for each change in a target database system, the processor provides at least two application algorithms for applying the changes to tables in the target database system. For each of the at least two application algorithms, the processor determines the performance behavior of each algorithm for the size of the change applied to the table by the corresponding application algorithm. The processor receives a data change request to apply the changes to the table. The processor determines the size of the change to the table. Based on the performance behavior of each application algorithm, the processor selects one of the at least two application algorithms that provides the best performance for that size. The processor uses the selected application algorithm that provides the best performance for that size to apply the change to the table.

Description

Background Technology

[0001] This invention relates to the field of database systems, and more specifically, to a method for dynamically selecting an application algorithm for applying changes in a target database system.

[0002] Replication is the process of maintaining a defined dataset across more than one location. It can involve copying specified changes from a source location to a target location and synchronizing the data between the two locations. The source and target can reside on the same machine or on logical servers on different machines within a distributed network. Several methods exist for moving data from one system to another. However, these methods may require further improvement. Summary of the Invention

[0003] Various aspects of embodiments of the present invention disclose a method, computer program product, and computer system for dynamically selecting a computer implementation of an application algorithm to be used to modify an application applied to a target database system. Embodiments of the present invention may be freely combined with each other if they are not mutually exclusive.

[0004] The processor provides multiple application algorithms for applying changes to a target database system. For each of the provided application algorithms, the processor determines the performance behavior of the application algorithm for different sizes of changes applied to tables in the target database system. The processor receives a data change request to apply changes to a table. The processor determines the size of the requested change to the table. The processor uses the determined performance behavior to select an application algorithm that provides the best performance for the determined size. The processor uses the selected application algorithm to apply the requested changes to the table. Attached Figure Description

[0005] Figure 1 A data processing system according to an embodiment of the present invention is shown.

[0006] Figure 2 This is a flowchart of a method for applying changes to a target database system according to an embodiment of the present invention.

[0007] Figure 3 This is a flowchart of a method for applying changes to a target database system according to an embodiment of the present invention.

[0008] Figure 4 This is a flowchart of a method for applying changes to a target database system according to an embodiment of the present invention.

[0009] Figure 5 This is a flowchart of a method for applying changes to a target database system according to an embodiment of the present invention.

[0010] Figure 6A This is a flowchart of a method for determining the performance behavior of an application algorithm according to an embodiment of the present invention.

[0011] Figure 6B This is a flowchart of a method for determining the performance behavior of an application algorithm according to an embodiment of the present invention.

[0012] Figure 6C The curves illustrate the performance behavior of the application algorithm according to an embodiment of the present invention.

[0013] Figure 7 This refers to a computerized system suitable for implementing one or more method steps according to embodiments of the present invention. Detailed Implementation

[0014] The description of various embodiments of the present invention is presented for illustrative purposes and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and alterations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements to technologies found in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0015] The target database system can be part of a data analysis system. The data analysis system includes both a target database system and a source database system. The data analysis system can be, for example, a data warehousing system or a master data management system. This data analysis system can implement data warehousing, master data management, or another technology using the source and target database systems, wherein the target database system includes a target database configured to receive / include copies of the contents of the corresponding source database of the source database system. The source database system can be connected to the target database system via a connection. This connection can be, for example, a TCP / IP connection or another connection enabling data communication between the source and target database systems. The source database system can be, for example, a transaction engine, and the target database system can be an analysis engine. For example, the source database system can be an online transaction processing (OLTP) system, and the target database system can be an online analytical processing (OLAP) system. The source database system can include a source dataset, and the target database system can include a target dataset. The source dataset can be part of the source database, and the target dataset can be part of the target database. The source and target datasets can be stored in the same or different formats. The formats can differ in terms of encryption, compression, row-oriented storage versus column-oriented storage, etc. For example, the source dataset can be stored in a row-oriented format, while the target dataset can be stored in a column-oriented format; that is, the target dataset can be stored by column rather than by row. The contents of the source dataset can be changed through one or more database transactions.

[0016] A data analysis system can be configured to replicate changes occurring in source tables of a source database system to a target database system, such that the changes can be applied to target tables in the target database system corresponding to the source tables. Applying changes can, for example, include inserting one or more records, updating one or more records, and / or deleting one or more records in one or more tables of the target database system. A data record or table record is a collection of related data items, such as a specific user's name, date of birth, and category. A record represents an entity, where an entity is a user, object, or concept whose information is stored in the record. For this purpose, multiple application algorithms (which may also be referred to as update strategies) can be provided, each specifying a sequence of replication operations to be performed to apply changes to the target database system. For example, application algorithms can include incremental load-based algorithms and batch load-based algorithms. For example, an incremental load-based algorithm may require each record change in the log record to be applied individually in the target database system. Incremental load-based algorithms are particularly advantageous for small datasets because large chunks can be expensive. A batch load-based application algorithm may, for example, require the recorded changes in the log record to be segmented into batches. Those batches can then be applied to the target database system via a batch load interface. Batch-based algorithms can be advantageous for large datasets. However, setting up batch loading can be too expensive and should not be spent on small blocks consisting of only a few rows.

[0017] Therefore, depending on the changes to be applied to the target database system, the applied algorithm can have different performance characteristics. However, accurately and systematically selecting the appropriate algorithm may not be a straightforward action. This invention addresses this problem by providing an optimal and efficient method for dynamically selecting the optimal applied algorithm for each change. This method can be time-efficient because it saves the additional time required to apply the algorithm inappropriately otherwise. This method can be optimal because these decisions can be based on accurate and up-to-date performance behavior.

[0018] The performance behavior of each application algorithm can be represented by a data structure, for example. This data structure can include data points p1, p2…p N (N≥2), where each data point p i (i = 1, ..., N) includes changing the size s i Value and m (m≥1) performance parameter values The number N can reflect the number of changes. In one example, the number N can be time-based; for instance, it could be the number of changes caused by all operations within the last two hours. For example, data point p1 could be defined as... Data point p2 can be defined as And so on. For example, if the number of performance parameters is 1, i.e., m = 1, the data structure can be a two-dimensional structure. Data points in a two-dimensional structure can define a two-dimensional space, which can be represented, for example, by a curve with two axes, one associated with the magnitude of the change and the other with the value of the performance parameter. For example, if the number of performance parameters is 2 (i.e., m = 2), the data structure can be a three-dimensional structure. Data points in a three-dimensional structure can define a three-dimensional space, which can be represented, for example, by a cube with three axes, one associated with the magnitude of the change, and the other two associated with the values ​​of the performance parameters. Thus, the data points of the data structure can define an (m+1)-dimensional space. When a new request to apply changes in the target database system is received, the requested change size s... t For example, it can be used to find distinct nearest points in (m+1)-dimensional space and select those nearest points of size s. t A single, associated application algorithm provides optimal performance. Different nearest points mean that each nearest point is associated with a different application algorithm. The values ​​of performance parameters can depend on, for example, the number of columns, the data type of the columns, hardware configuration (such as CPU speed), type of memory, size of CPU cache, etc.

[0019] According to one embodiment, the method further includes evaluating the performance of the selected application algorithm by applying the requested changes, and updating the performance behavior of the selected application algorithm using the evaluated performance and the determined size. For example, this could be done for a requested size s. t Changes are made to evaluate m performance parameters. This can lead to new points in the data structures associated with the selected application algorithm. This embodiment can be advantageous because it dynamically updates the performance behavior of the selected application algorithm. This enables a self-tuning system.

[0020] According to one embodiment, the method further includes the steps of repeatedly determining the size for each received data change request for the table, selecting an application algorithm, and applying the change. This enables the table's performance behavior to be updated dynamically.

[0021] According to one embodiment, the method further includes: performing the performance behavior determination step on each additional table of the target database system, thereby causing each table of the target database system to be associated with a corresponding performance behavior; and repeating the steps of determining the size, selecting the application algorithm, and applying the change for each received data change request using the performance behavior associated with the specific table of the target database system. This embodiment can be advantageous because it makes the performance behavior of the application algorithm also dependent on the table. This can be particularly advantageous because the same application algorithm may be the most efficient algorithm for a given size of change to a given table with a few columns, but may be less efficient for the same size of change to another table with more columns, or those columns may have different data types.

[0022] According to one embodiment, determining the performance behavior of an application algorithm includes: executing the application algorithm to apply data changes to a table a predefined number N times, each applied data change having a size; evaluating at least one performance parameter indicative of the performance of the execution of the application algorithm for each of the data changes; and providing a data structure of N data points representing the performance behavior, where each data point indicates the evaluated performance parameter and the associated size of the data change. This embodiment can be advantageous because, during the initial setup of the data analysis system, no measurements may be available. This embodiment can provide initial execution of the application algorithm to provide initial measurements, which can be refined later. In one example, the initial execution of the application algorithm can be a virtual execution with dummy data, for example, 1000 rows can be inserted into the table and the execution time can be measured. These 1000 row changes can be rolled back without being committed. In another example, the execution of the application algorithm can be triggered by N received change requests. The application algorithm can be selected randomly from a predetermined algorithm or via a loop or some other scheme. This is done until enough measurement points of N points are available.

[0023] According to one embodiment, the number N is less than the configurable maximum number of executions (e.g., 3 times). The method also includes constructing new data points in the data structure using interpolation of N data points. For example, two (i.e., N=2) INSERTs can be performed using each applied algorithm for inserting multiple rows. The number of rows in the two INSERTs can be different and have greater diversity, e.g., 100 rows and 700 rows (e.g., but not 100 rows and 101 rows). Multiple performance parameters can be evaluated for each of the two INSERTs. This defines a first baseline, e.g., in the form of a linear curve, on which extrapolation can then be applied. Splines can be used to approximate the curve or for simple linear interpolation.

[0024] According to one embodiment, the performance parameter is either execution time or memory usage.

[0025] According to one embodiment, the change in size includes the number of records to be inserted, the number of records to be deleted, and / or the number of records to be updated.

[0026] According to one embodiment, selecting an application algorithm involves determining whether the performance of two application algorithms is similar for a given size, and if they are similar, selecting either of the two application algorithms. The two application algorithms may have similar performance at a point where their performance behaviors intersect. These intersections may have a very important characteristic: they are uninterrupted in terms of performance. The performance of processing "n" rows with either algorithm A or algorithm B is the same. This means that which algorithm is chosen is irrelevant. Slight deviations from the intersection will not produce a significant "jump" in the performance curve. For example, if algorithm A is better for fewer than 1000 rows and algorithm B is better for more than 1000 rows, then choosing B for 990 rows is still very close to choosing algorithm A. This can have extremely helpful implications for a smooth transition when switching from algorithm A to algorithm B.

[0027] According to one embodiment, the application algorithm is either an application based on batch loading of records or an application based on loading individual records.

[0028] According to one embodiment, the method further includes a source table associated with the table in a providing source database system, wherein the source database system and the target database system are configured to synchronize data between each other; and wherein a data change request is received in response to the data change being applied to the source table, thereby replicating the data change.

[0029] The embodiments of the present invention can be implemented in various forms, and exemplary implementation details will be discussed below with reference to the accompanying drawings.

[0030] Figure 1 This is a block diagram of a data processing system (or data analysis system) 100 according to an embodiment of the present invention. According to an embodiment of the present invention, the data processing system 100 can be configured to perform data synchronization between a source database system 101 and a target database system 103 using a data synchronization system 102. The source database system 101 may be, for example, an online transaction processing (OLTP) system. The target database system 103 may be, for example, an online analytical processing (OLAP) system. Communication between the source database system 101 and the target database system 103 may be performed, for example, via a TCP / IP communication layer.

[0031] The source database system 101 includes one or more source tables 105 of the source database 106 and a transaction recovery log 107. Entries or log records in the transaction recovery log 107 describe changes made to rows or records in the source table 105 at the source database system 101. Figure 1 Example content of log record 130 is shown. Log record 130 may include timestamps, LRSNs, and attribute changes. More specifically, log records in transaction recovery log 107 may, for example, contain information defining (1) the table being modified, (2) the values ​​of the key columns in the rows being modified, (3) the old and new values ​​of all columns in the modified rows, and (4) the transaction (unit of work) that caused the change. By definition, an insertion is a new data record and therefore does not have an old value. For a deletion change, by definition, there is no new data record, only the old data record. Therefore, the transaction log record for a row used for insertion may contain only the new column values, while the transaction log record for a row used for deletion may contain only the old column values. The transaction log record for an updated row may contain both the new and old values ​​of all row columns. The order of log records in the transaction recovery log may reflect the order of the transaction's change operations, and the order of transaction commit records may reflect the order in which the transactions were completed. The type of row operation in the transaction log record may be, for example, deletion, insertion, or update.

[0032] Data synchronization system 102 includes log reader 104. Although shown as part of data synchronization system 102, in another example, log reader 104 may be part of source database system 101. Log reader 104 reads log records from transaction recovery log 107 and provides them to change record classifier 120. Change record classifier 120 classifies log records based on changes (e.g., to determine the size of the change), such that algorithm selection module 121 selects one of application algorithms 108.1 to 108.3 based on the classification of log records, and performs an update based on the selected application algorithm. Application algorithm selection and change application using the application algorithm may be performed periodically (e.g., hourly), or automatically when a predefined amount of log records (e.g., 1000 log records) is stored in transaction recovery log 107, such that the amount of log records can be used to define new change requests. In each iteration, only newly added log records relative to previously provided log records may be processed. The selected application algorithm may include, for example, a batch-based update strategy or one or more incremental update strategies, corresponding to batch-based algorithm 108.3 and incremental algorithms 108.1 and 108.2, respectively. Synchronization can be performed differently for the batch-based update strategy and the incremental update strategy.

[0033] Log reader 104 can be configured to perform log shipping of transaction recovery log 107 to target database system 103 based on an incremental update algorithm selected by selection module 121. Shipping can be performed, for example, by sending a stream of log records formed by the log records of transaction recovery log 107. The log stream being shipped can be associated with, for example, a stream ID. The stream of log records can be, for example, a stream of merged log records. This enables efficient processing of log records at target database system 103. Target database system 103 may include multiple algorithms 108.1 and 108.2, each associated with a corresponding incremental update strategy. Figure 1 Only three algorithms are shown for illustrative purposes, but the system is not limited to these types or numbers of algorithms. The target database system 103 also includes one or more target table copies 113. The target database system includes a log streaming interface for receiving log streams from the source database system 101. Each of algorithms 108.1 and 108.2 can be configured to receive a stream of log records via the log streaming interface. Each of algorithms 108.1 and 108.2 can buffer the received log records and merge changes into batches to improve efficiency when applying modifications to table copies 113 of the target database 114, for example, via a batch load interface.

[0034] In another example, the batch load (snapshot update) algorithm 108.3 can be performed between the source database system 101 and the target database system 103 based on a selected batch load-based update strategy. The load can be an entire table of data or a set of partitions of a table at a given point in time, and is performed directly from the source database system 101 to the target database system 103. The data on the target database system 103 can reflect the state of the source database system at the time the load is performed.

[0035] Although shown as separate components, in another example, the data synchronization system 102 may be part of either the source database system 101 or the target database system 103. In one example, the source database system 101 and the target database system 103 may be on the same system or different systems in a distributed network.

[0036] Figure 2 This is an embodiment of the invention for sending data to table T of the target database system. g A flowchart illustrating the method of applying the changes. For illustrative purposes, in... Figure 2 The method described in [the document] can be used in [the following context] Figure 1 The system shown in the figure is implemented, but is not limited to this implementation. Figure 2 The method can be performed, for example, by the data synchronization system 102. Figure 2 The method could, for example, enable the source table T in the source database system to be...s (corresponding to T) g The changes made in the source database system are applied to the target database system, and thus synchronization between the source and target database systems is possible.

[0037] In step 201, multiple application algorithms can be provided to apply changes to the target database system. For example, several application algorithms App1, ... App... r Where r ≥ 2. For example, the application algorithm can include a single-record application algorithm and a batch loading application algorithm. A single-record application algorithm can apply each change recorded in each log record individually. Each algorithm can apply changes in different ways, thus effectively leading to other application algorithms. For example, a single-record application algorithm can update only table T. g The data that has changed since the last update, or the entire table T that can be updated. g Whenever in the corresponding source table T s When a change occurs. This allows for two different algorithms to be applied to a single record.

[0038] In step 203, the application algorithm App1, ...App can be determined. r The performance behavior of each applied algorithm in Table T. Performance behavior indicates how well the applied algorithm performs based on the performance of the applied algorithm on Table T. g The size of the change varies. Therefore, each application algorithm (App1, ..., App) can be used. r Apply N changes to table T g Each of the N changes can have its own size. For example, the size could be the size of the table T to be inserted. g The number of records in and / or the records to be retrieved from table T g The number of records deleted. For changes made to each application, this can be done for application App1, ...App... r Each application in the evaluation process uses performance parameters. This yields r data structures (e.g., discrete point curves curv1, ..., curv). r Each of the r data structures includes N change sizes s1...s1 representing the individual measurements associated with m performance parameters. N N data points For example, if the number of performance parameters is 1, then each data point can be a pair of values ​​that includes the magnitude of the change and the corresponding performance parameter value. Performance parameters could be, for example, execution time and memory usage.

[0039] For example, step 203 can be performed as part of a preprocessing step, such as using table T during the runtime of the data analysis system. gPreviously, this was performed offline. In another example, step 203 can be performed during the runtime of the data analysis system. The number of changes, N, can be chosen to be as small as possible, for example, N=2, because step 203 can be resource-intensive, as it evaluates each application algorithm for each change.

[0040] However, if the number of changes N is insufficient to estimate the performance of the requested changes, the resulting data structure curv1, ..., curv can be further expanded by adding additional points. r This addition of additional points can be performed using, for example, interpolation. Interpolation can be linear or spline interpolation. Therefore, the data structures curv1, ..., curv r Each data structure in the list can have N+x data points. In another example, the data structures curv1, ..., curv r These can be processed to model their behavior using mathematical functions. Therefore, step 203 can yield r data structures or representations, each with discrete points, for applying the algorithm App1, ..., App2. r The performance behavior is given by r mathematical functions.

[0041] A data change request can be received in step 205 to apply the changes to the table. For example, in response to detecting a change in the source table, the data change request can be sent to a data synchronization system, for example, Figure 1 Data synchronization system 102.

[0042] In response to receiving a data change request, the size of the requested change can be determined in step 207. The definition of size can be the same as that used to determine performance behavior. For example, the size s of the requested data change. t This could be the number of records to be inserted and / or the number of records to be extracted from table T. g The number of records deleted.

[0043] In step 209, the determined size s t It can be used to select the size s that has been determined. t Compared to other application algorithms, this provides the best performance. Therefore, it can be determined for a given size s. t Estimation of application algorithm App1, ...App r The performance of each applied algorithm can be assessed using the data structures curv1, ...curv determined in step 203. r Alternatively, a mathematical model can be used to perform this estimation. For example, when using a data structure with discrete points, curv1, ...curv r In this case, a specific size s can be identified for each data structure. tThe closest points. This yields r closest points. The performance parameter values ​​of these closest points can be compared, and the optimal closest point can be selected. The selected closest point can be associated with one of the application algorithms. This application algorithm can be the one selected in step 209.

[0044] When using a mathematical model, the determined size s t It can be used as an independent variable or input to each mathematical model in order to estimate the magnitude s t The associated performance. This yields a set of values ​​for r performance parameters. These can be compared to identify the best performance and associated application algorithms.

[0045] In step 211, the requested changes can be applied to table T using the selected application algorithm. g .

[0046] Figure 3 This is an embodiment of the invention for sending data to table T of the target database system. g A flowchart illustrating the method of applying the changes. For illustrative purposes, in... Figure 3 The method described in [the document] can be used in [the following context] Figure 1 The system shown is implemented, but is not limited to, this implementation. For example, Figure 3 This method can be executed by the data synchronization system 102. For example, Figure 3 This method allows for the retrieval of source table T from the source database system. s (corresponding to T) g The changes made in the source database system are applied to the target database system, and thus enable synchronization between the source and target database systems.

[0047] Steps 301 to 311 are Figure 2 Steps 201 to 211. Furthermore... Figure 3 The method also includes step 313, which updates the performance behavior determined in step 303 for the selected application algorithm. This can be done, for example, by linking the points associated with the received changes. It is added to the data structure representing the performance behavior of the selected application algorithm for execution. Additionally, it can be targeted at the table T to be applied. g Repeat steps 305 to 313 for each data change.

[0048] Figure 4 This is a flowchart of a method for applying changes to multiple tables in a target database system according to an embodiment of the present invention. For illustrative purposes, in Figure 4 The method described in [the document] can be used in [the following context] Figure 1 The system shown is implemented, but is not limited to, this implementation. For example, Figure 4The method can be executed by the data synchronization system 102. Figure 4 This method can, for example, enable changes made in multiple source tables (corresponding to multiple target tables) of the source database system to be applied to the target database system, and thus enable synchronization between the source and target database systems.

[0049] Multiple application algorithms (e.g., as described in step 201) can be provided in step 401 for applying changes to the target database system.

[0050] In step 403, and for each of the multiple tables, the applicable algorithm App1, ... App can be determined. r The performance behavior of each applied algorithm in the dataset. For example, step 403 may include: performing [action] on each of the multiple tables. Figure 2 Step 203. This may be advantageous because it makes the performance behavior of the applied algorithm also dependent on the table.

[0051] In step 405, a data change request may be received to apply one or more changes to one or more tables, respectively. These changes may have different sizes because they are applied to different tables.

[0052] For each of the one or more tables, steps 407 to 411 can be applied using the table's performance behavior. Steps 407 to 411 are... Figure 2 Steps 207 to 211.

[0053] Figure 5 This is a flowchart of a method for applying changes to multiple tables in a target database system according to an embodiment of the present invention. For illustrative purposes, in Figure 5 The method described in [the document] can be used in [the following context] Figure 1 The system shown is implemented, but is not limited to, this implementation. For example, Figure 5 This method can be executed by the data synchronization system 102. For example, Figure 5 This method allows changes made in multiple source tables (corresponding to multiple target tables) of the source database system to be applied to the target database system, and thus enables synchronization between the source and target database systems.

[0054] Steps 501 to 511 are Figure 4 Steps 401 to 411. Furthermore... Figure 5 The method also includes step 513, which updates the performance behavior determined for the selected application algorithm in step 503. Furthermore, steps 505 through 513 can be repeated for each change to be applied to one or more of the multiple tables.

[0055] Figure 6AThis is a flowchart of a method for determining the performance behavior of an application algorithm according to an embodiment of the present invention.

[0056] The algorithm can be executed a predefined number N times in step 601 to apply data changes to the table, each of which has a size.

[0057] For each data change, at least one performance parameter indicative of the performance of the applied algorithm can be evaluated in step 603. For example, the execution time of the applied algorithm can be measured for each of the N changes.

[0058] In step 605, a data structure with N data points can be provided. The data structure represents the performance behavior of the applied algorithm. Each data point indicates the performance parameter being evaluated and the associated magnitude of the data change. Figure 6C An example of N=2 measurement data points for two application algorithms, App1 and App2, is shown.

[0059] Figure 6B This is a flowchart of a method for determining the performance behavior of an application algorithm according to an embodiment of the present invention.

[0060] Figure 6B The method includes steps 601 to 605 and step 607 as described above. In step 607, additional points can be used to expand the provided data structure representing the performance behavior of the applied algorithm. This can be performed, for example, using interpolation between these data points. For example, this represents... Figure 6C In this process, linear interpolation is performed between two data points for each applied algorithm. Additional points can be any point along the dashed line (interpolation).

[0061] Figure 7 This refers to a general-purpose computerized system 700 suitable for implementing at least a portion of the method steps according to embodiments of the present invention.

[0062] It should be understood that the methods described herein are at least partially non-interactive and automated by computerized systems such as servers or embedded systems. However, in exemplary embodiments, the methods described herein can be implemented in (partially) interactive systems. These methods can be further implemented in software 712, 722 (including firmware 722), hardware (processor) 705, or combinations thereof. In exemplary embodiments, the methods described herein are implemented as executable programs in software and executed by a dedicated or general-purpose digital computer (such as a personal computer, workstation, minicomputer, or mainframe). Thus, the most general system 700 includes a general-purpose computer 701.

[0063] In an exemplary implementation, regarding the hardware architecture, such as Figure 7As shown, computer 701 includes processor 705, memory (main memory) 710 coupled to memory controller 715, and one or more input and / or output (I / O) devices (or peripherals) 10, 745 communicatively coupled via local input / output controller 735. Input / output controller 735 may be, but is not limited to, one or more buses or other wired or wireless connections as known in the art. Input / output controller 735 may have additional elements omitted for simplicity, such as controllers, buffers (cache memory), drivers, repeaters, and receivers, to enable communication. Furthermore, the local interface may include address, control, and / or data connections to enable appropriate communication between the aforementioned components. As described herein, I / O devices 10, 745 may generally include any general-purpose cryptographic card or smart card known in the art.

[0064] Processor 705 is a hardware device for executing software (particularly software stored in memory 710). Processor 705 may be any custom or commercial processor among a number of processors associated with computer 701, a central processing unit (CPU), a coprocessor, a semiconductor-based microprocessor (in the form of a microchip or chipset), or any device typically used for executing software instructions.

[0065] Memory 710 may include any or a combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, etc.)) and non-volatile memory elements (e.g., ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM)). Note that memory 710 may have a distributed architecture, in which different components are located far apart from each other but can be accessed by processor 705.

[0066] The software in memory 710 may include one or more separate programs, each program including an ordered list of executable instructions for implementing logical functions (particularly those involved in embodiments of the invention). Figure 7 In the example, the software in memory 710 includes instructions 712, such as instructions for managing a database (like a database management system).

[0067] The software in memory 710 should also typically include a suitable operating system (OS) 711. OS 711 essentially controls the execution of other computer programs, such as possible software 712 for implementing the methods described herein.

[0068] The method described herein can be in the form of a source program 712, an executable program 712 (object code), a script, or any other entity including a set of instructions 712 to be executed. When it is a source program, it needs to be translated by a compiler, assembler, interpreter, etc., which may or may not be included in memory 710, in order to operate appropriately in conjunction with OS 711. Furthermore, the method can be written in an object-oriented programming language with data and method classes, or in a procedural programming language with routines, subroutines, and / or functions.

[0069] In an exemplary embodiment, a conventional keyboard 750 and mouse 755 may be coupled to an input / output controller 735. Other output devices (such as I / O devices 745) may include input devices, such as, but not limited to, printers, scanners, microphones, etc. Finally, I / O devices 10, 745 may also include devices for transmitting both input and output, such as, but not limited to, network interface cards (NICs) or modulators / demodulators (for accessing other files, devices, systems, or networks), radio frequency (RF) or other transceivers, telephone interfaces, bridges, routers, etc. I / O devices 10, 745 may be any general-purpose cryptographic cards or smart cards known in the art. System 700 may further include a display controller 725 coupled to a display 730. In an exemplary embodiment, system 700 may also include a network interface for coupling to a network 765. Network 765 may be an IP-based network for communication between computer 701 and any external servers, clients, etc., via a broadband connection. Network 765 transmits and receives data between computer 701 and external system 30, which may be used to perform some or all of the steps of the methods discussed herein. In an exemplary embodiment, network 765 may be a managed IP network managed by a service provider. Network 765 may be implemented wirelessly, for example using wireless protocols and technologies such as WiFi, WiMax, etc. Network 765 may also be a packet-switched network, such as a local area network, wide area network, metropolitan area network, Internet network, or other similar type of network environment. Network 765 may be a fixed wireless network, wireless local area network (LAN), wireless wide area network (WWAN), personal area network (PAN), virtual private network (VPN), intranet, or other suitable network system, and includes devices for receiving and transmitting signals.

[0070] If the computer 701 is a PC, workstation, intelligent device, etc., the software in the memory 710 may also include a Basic Input / Output System (BIOS) 722. The BIOS is a collection of basic software routines that initialize and test the hardware at startup, boot the OS 711, and support data transfer between hardware devices. The BIOS is stored in ROM so that it can be executed when the computer 701 is activated.

[0071] When computer 701 is running, processor 705 is configured to execute software 712 stored in memory 710 to transfer data to and from memory 710, and to control the operation of computer 701 in general according to the software. The methods and OS 711 described herein, in whole or in part (but usually the latter), are read by processor 705, may be buffered within processor 705, and then executed.

[0072] When the systems and methods described herein are implemented in software 712, such as Figure 7 As shown, the method can be stored on any computer-readable medium (e.g., memory 720) for use by or in conjunction with any computer-related system or method. Memory 720 may include disk storage, such as HDD storage.

[0073] The procedures described herein are identified based on their implementation in specific embodiments of the invention. However, it should be understood that any particular procedural terminology used herein is for convenience only, and therefore the invention should not be limited to use only in any particular application identified and / or implied by such terminology.

[0074] The present invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the present invention.

[0075] Computer-readable storage media can be tangible means for retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0076] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0077] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may be personalized to execute computer-readable program instructions by utilizing state information from the computer-readable program instructions in order to perform aspects of this invention.

[0078] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0079] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0080] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other device perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions indicated in the blocks may occur in a different order than shown in the figures. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

Claims

1. A computer-implemented method, comprising: At least two application algorithms are provided by one or more processors, the at least two application algorithms being used to apply changes to tables in a target database system; The one or more processors determine the performance behavior of each of the at least two application algorithms for each application algorithm, for a change in size applied to the table by the corresponding application algorithm, wherein the performance behavior is defined as a data structure including data points, wherein each data point includes a change in size value and at least one performance parameter, wherein the change in size value is a value reflecting the size of the requested change to the table, and wherein the at least one performance parameter is at least one of execution time and memory usage. The one or more processors receive a data change request to apply changes to the table; The size of the change to the table is determined by one or more processors; The one or more processors select, based on the performance behavior of each application algorithm, one of the at least two application algorithms that provides the best performance for the size; The changes are applied to the table by the one or more processors using the selected application algorithm that provides the best performance for the size; The performance of the selected application algorithm is evaluated by the one or more processors by applying the changes to the table based on the performance behavior. as well as The performance behavior of the selected application algorithm is updated by the one or more processors by adding new data points to the data structure, the new data points indicating the performance of the selected application algorithm as measured by the at least one performance parameter and the magnitude of the change to the table.

2. The computer-implemented method according to claim 1, further comprising: The determination of the size, the selection, and the application are repeated by the one or more processors for each received data change request for the table; as well as The performance behavior of the table is dynamically updated by the one or more processors.

3. The computer-implemented method according to claim 1, further comprising: The one or more processors determine the corresponding performance behavior for each additional table in the target database system, thereby causing each table in the target database system to be associated with the corresponding performance behavior; as well as The one or more processors use the performance behavior associated with a specific table in the target database system to repeat the determination of the size, the selection, and the application for each received data change request for the specific table.

4. The computer-implemented method of claim 1, wherein determining the performance behavior of each application algorithm comprises: The one or more processors execute the corresponding application algorithm a predefined number N times to apply data changes to the table, wherein each applied data change has a size; The one or more processors evaluate at least one performance parameter that indicates the performance of the execution of the application algorithm for each of the data changes; as well as The data structure, consisting of N data points representing the performance behavior, is provided by the one or more processors, wherein each data point indicates the associated magnitude of the at least one performance parameter and the data change.

5. The computer-implemented method according to claim 4, further comprising: The predefined number N is less than the configurable maximum number of executions; as well as The one or more processors construct new data points in the data structure by interpolating the N data points.

6. The computer-implemented method of claim 1, wherein the changed size includes at least one of the following: the number of records to be inserted, the number of records to be deleted, and the number of records to be updated.

7. The computer-implemented method according to claim 1, wherein selecting the application algorithm comprises: The one or more processors determine whether the performance of the two application algorithms is similar for the given size; as well as In response to determining that the performance of the two application algorithms is similar for the size, the one or more processors select either of the two application algorithms.

8. The computer-implemented method of claim 1, wherein the selected application algorithm is selected from the group consisting of applications based on batch loading of records and applications based on individual record loading.

9. The computer-implemented method according to claim 1, further comprising: The source table associated with the table in the source database system is provided by the one or more processors, wherein the source database system and the target database system are configured to synchronize data between each other; and The data change request is received in response to the data change being applied to the source table, thereby replicating the data change.

10. A computer program product, comprising: Program instructions, the program instructions including: Program instructions for providing at least two application algorithms for applying changes to tables in a target database system; Program instructions for determining, for each of the at least two application algorithms, the performance behavior of each application algorithm applied to the table by the corresponding application algorithm for a change in size, wherein the performance behavior is defined as a data structure comprising data points, wherein each data point comprises a change in size value and at least one performance parameter, wherein the change in size value is a value reflecting the size of the requested change to the table, and wherein the at least one performance parameter is at least one of execution time and memory usage. Used to receive program instructions for applying changes to the data change request of the table; Program instructions for determining the magnitude of the change to the table; Program instructions for selecting, based on the performance behavior of each application algorithm, one of the at least two application algorithms that provides the best performance for the size; Program instructions for applying the changes to the table using the selected application algorithm that provides the best performance for the size; Program instructions for evaluating the performance of the selected application algorithm by applying the changes to the table based on the performance behavior; and Program instructions for updating the performance behavior of the selected application algorithm by adding new data points to the data structure, the new data points indicating the performance of the selected application algorithm as measured by the at least one performance parameter and the magnitude of the change to the table.

11. The computer program product according to claim 10, further comprising: Program instructions for repeating the program instructions for determining the size, the program instructions for selecting, and the program instructions for applying for each received data change request of the table; as well as Program instructions for dynamically updating the performance behavior of the table.

12. The computer program product according to claim 10, further comprising: Program instructions for determining the corresponding performance behavior for each additional table in the target database system, thereby causing each table in the target database system to be associated with the corresponding performance behavior; as well as For each received data change request for the target database system, the program instructions for determining the size, the program instructions for selection, and the program instructions for application are repeated to determine the size, select, and apply the performance behavior associated with a specific table in the target database system.

13. The computer program product of claim 10, wherein the program instructions for determining the performance behavior of each application algorithm comprise: Program instructions for executing the corresponding application algorithm a predefined number N times to apply data changes to the table respectively, wherein each applied data change has a size; Program instructions for evaluating at least one performance parameter, which instructs the performance of the execution of the application algorithm, for each of the data changes; as well as Program instructions for providing the data structure of N data points representing the performance behavior, wherein each data point indicates the associated magnitude of the at least one performance parameter and the data change.

14. A computer system, comprising: One or more computer processors; One or more computer-readable storage media; Program instructions, commonly stored on the one or more computer-readable storage media, are intended to be executed by at least one of the one or more computer processors, and the stored program instructions include: Program instructions for providing at least two application algorithms for applying changes to tables in a target database system; Program instructions for determining, for each of the at least two application algorithms, the performance behavior of each application algorithm applied to the table by the corresponding application algorithm for a change in size, wherein the performance behavior is defined as a data structure comprising data points, wherein each data point comprises a change in size value and at least one performance parameter, wherein the change in size value is a value reflecting the size of the requested change to the table, and wherein the at least one performance parameter is at least one of execution time and memory usage. Used to receive program instructions for applying changes to the data change request of the table; Program instructions for determining the magnitude of the change to the table; Program instructions for selecting, based on the performance behavior of each application algorithm, one of the at least two application algorithms that provides the best performance for the size; Program instructions for applying the changes to the table using the selected application algorithm that provides the best performance for the size; Program instructions for evaluating the performance of the selected application algorithm by applying the changes to the table based on the performance behavior; and Program instructions for updating the performance behavior of the selected application algorithm by adding new data points to the data structure, the new data points indicating the performance of the selected application algorithm as measured by the at least one performance parameter and the magnitude of the change to the table.

15. The computer system of claim 14, further comprising: Program instructions for repeating the program instructions for determining the size, the program instructions for selecting, and the program instructions for applying for each received data change request of the table; as well as Program instructions for dynamically updating the performance behavior of the table.

16. The computer system of claim 14, further comprising: Program instructions for determining the corresponding performance behavior for each additional table in the target database system, thereby causing each table in the target database system to be associated with the corresponding performance behavior; as well as For each received data change request for the target database system, the program instructions for determining the size, the program instructions for selection, and the program instructions for application are repeated to determine the size, select, and apply the performance behavior associated with a specific table in the target database system.

Images