Database load unit for replication log playback

Abstract

To provide a database load unit for replication log replay.SOLUTION: A process for using different load units comprises: loading a database object into a primary in-memory store; capturing it in a replay log according to a predetermined format and transmitting it to a secondary database system; checking, by the secondary database system, the value of a log replay configuration parameter in response to reception of the replay log; if the configuration parameter is a first value, replaying the replay log to load the corresponding database objects into a secondary in-memory store according to a first format; if the configuration parameter is a second value, replaying the log and loading the object according to the second format; and if the configuration parameter is a third value, replaying the log and loading the object in the same format that was used by the primary database system.SELECTED DRAWING: Figure 8

Description

[[Technical Field]]

[0001] The present disclosure generally relates to database processing. [[Background Art]]

[0002] A database is an organized collection of data. A database typically organizes data to correspond to a logical arrangement of the data. This facilitates operations on the data, for example, searching for values in the database, adding data to the database, sorting data in the database, or summarizing related data in the database. A Database Management System (DBMS) mediates interactions between the database, users, and applications to organize, create, update, capture, analyze, and otherwise manage data in the database.

[0003] To efficiently process queries, a database is typically configured to perform in-memory operations on data. In an in-memory database, data required for query execution and response is loaded into memory, and the query is executed against the in-memory data. However, many applications have large-scale data stores, and due to memory limitations, it may be difficult to load all data required by these applications into memory. The amount of data processed by database systems continues to increase at a faster pace than the evolution of memory devices to store more data. [[Summary of the Invention]] [[Means for Solving the Problems]]

[0004] In some implementations, the primary database system loads database objects into the primary in-memory store according to a given format determined by the primary database system. The primary database system captures the loading of database objects into a replay log according to the given format. The primary database system sends the replay log to the secondary database system. In response to receiving the replay log, the secondary database system checks the value of the log replay configuration parameter. If the log replay configuration parameter is of the first value, the secondary database system replays the log and loads the corresponding database objects into the secondary in-memory store according to the first format, which may or may not be different from a given format used by the primary database system. If the log replay configuration parameter is of the second value, the secondary database system replays the first log and loads the database objects into the secondary in-memory store according to the second format, which may or may not be different from a given format used by the primary database system. If the log replay configuration parameter is set to the third value, the secondary database system replays the first log and loads at least one database object into the secondary in-memory store in the same format (i.e., given format) as it was used by the primary database system.

[0005] Non-temporary computer program products (i.e., physically embodied computer program products) will also be described. When executed by one or more data processors of one or more computing systems, a non-temporary computer program product stores instructions causing at least one data processor to perform the operations described herein. Similarly, computer systems that may include one or more data processors and memory coupled to one or more data processors will also be described. The memory may temporarily or permanently store instructions causing at least one processor to perform one or more of the operations described herein. Furthermore, the methods can be implemented by one or more data processors within a single computing system or distributed between two or more computing systems. Such computing systems can exchange data and / or commands, or other instructions, etc., via one or more connections, including connections via a network (e.g., the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, etc.) and direct connections between one or more of multiple computing systems.

[0006] Details of one or more variations of the subject matter described herein will be illustrated in the accompanying drawings and the following description. Other features and advantages of the subject matter described herein will become apparent from the description and drawings, as well as from the claims.

[0007] The accompanying drawings incorporated herein, and which constitute part thereof, illustrate specific aspects of the subject matter disclosed herein and, together with the description, are useful in illustrating some of the principles associated with the disclosed implementations. [Brief explanation of the drawing]

[0008] [Figure 1]This figure shows an example of a database system according to several exemplary implementations of the subject matter of the present invention. [Figure 2] This figure shows another example of the system, based on several exemplary implementations of the subject matter of the present invention. [Figure 3] This figure shows an example of a system for implementing load unit determination, based on several exemplary implementations of this subject. [Figure 4] This figure shows another example of a system for implementing load unit determination, with several exemplary implementation forms of this subject. [Figure 5] This figure shows an example of a system with a primary and secondary database, representing several exemplary implementations of this subject. [Figure 6] This figure shows another example of a system with a primary and secondary database, illustrating several exemplary implementations of this subject. [Figure 7] This figure shows examples of processes for dynamically determining the load unit for loading database objects and / or database object subcomponents into memory, based on several exemplary implementations of this subject. [Figure 8] This figure shows an example of the process for using different load units when loading database objects into memory in a primary and secondary database system, based on several exemplary implementations of this subject. [Figure 9] This figure shows an example of the process for executing a failover load script in a secondary database system, based on several exemplary implementations of this subject. [Figure 10A] This figure shows examples of the system in several exemplary implementation forms of this subject. [Figure 10B] This figure shows another example of the system, illustrating several exemplary implementations of this subject. [Figure 11]This figure shows an example of the process for determining the load unit when loading database objects into memory in a secondary database system, based on several exemplary implementations of this subject. [Figure 12] This figure shows an example of a process for adjusting the buffer cache size in a secondary database system, based on at least log replay configuration parameters, using several exemplary implementations of this subject. [Modes for carrying out the invention]

[0009] Data structures are typically created and ingested in memory, but once ingested, they may be persisted in persistent storage. Once in persistent storage, data structures can be removed from memory when no longer needed. Then, if the data structure is needed again in memory in the future, it can be reconstructed from the information persisted in persistent storage. Loading a data structure refers to reconstructing the data structure in memory from the information persisted in persistent storage. The representation of a data structure in persistent storage may not match its representation in memory, but the information stored in persistent storage is sufficient to completely reconstruct the data structure in memory.

[0010] In a database, a database object is a data structure used to store or retrieve data. A common type of database object is a table. Other types of database objects include columns, indexes, stored procedures, sequences, and views. A database may store each database object as a set of substructures that form and represent the database object as a whole. Note that the terms “substructure” and “subcomponent” may be used interchangeably in this specification. In the case of a column, substructures may include dictionaries, data vectors, and indexes. A dictionary may associate each unique data value with a corresponding value identifier (ID). Value IDs may be numbered sequentially. A data vector may contain value IDs that map to the actual values ​​of the database object. An index may contain a sequential list of each unique value ID and one or more positions in the data vector containing the value IDs.

[0011] When loading database objects from a persistent location to an in-memory location, the database can use several different formats to load database objects into memory. For example, one format is called column-loadable format. In column-loadable format, when a column is queried, the entire column is loaded into memory. Loading the entire column into memory means loading all of the column's subcomponents (e.g., data, dictionary, and index) into memory. Another type of format that can be adopted is called page-loadable format. In page-loadable format, only the page containing the portion of the column relevant to the query is loaded into memory. In other embodiments, other types of formats may be adopted.

[0012] When loading data into memory, load units can be used to specify the format in which the data is loaded and the granularity of the data loaded from the persistence store. In other words, a "load unit" defines the granularity of the data loaded from the persistence store to the in-memory store. Data description language (DDL) statements can change the load unit of various database objects, such as tables, partitions or sets of partitions of tables, columns or sets of columns of tables, or composite indexes defined on tables. For example, different types of load units that can be specified include page-loadable, column-loadable, and default-loadable. In the default-loadable format, database objects recursively inherit the load unit of their parent.

[0013] The format or load unit specifies how columns are loaded into memory, but the format does not necessarily determine how columns are persisted. In some database deployments, load units for column-loadable and page-loadable formats may use different persistence based on the load unit definition. For example, column-loadable columns may be persisted by serializing the column's data, dictionary, and index into a single page chain. For page-loadable columns, these columns may be persisted using a paging scheme so that pages containing the necessary column portions for queries are easily identifiable and can be brought into memory for access. For page-loadable columns, a separate page chain is persisted for each of the column's data, dictionary, and index subcomponents.

[0014] In some embodiments, when loading database objects from a persistence store location to an in-memory location, the database may decide whether to load the database objects in a column-loadable or page-loadable format, depending on the current workload, corresponding attributes, and / or one or more other operating conditions. In some cases, it is preferable to store database objects in a column-loadable format, while in other cases, it is preferable to store them in a page-loadable format. The mechanisms and data layouts for storing database objects in a column-loadable format differ from the configurations for storing database objects in a page-loadable format. Therefore, switching between the two configurations may require a complete rewrite of the data persistence. This rewrite consumes considerable memory and processing resources. Thus, switching to an in-memory format can be expensive in terms of the memory and processing resources used for the necessary rewrite. Therefore, there is a need for improved techniques that do not require rewriting the persistence store for the converted load unit.

[0015] For example, if a given database object has a column-loadable load unit and the processing logic decides not to convert the load unit, the given database object is loaded into the in-memory store in a column-loadable format. When loading a column into the in-memory store in a column-loadable format, the processing logic loads the entire column into the in-memory store. If a given database object has a page-loadable load unit and the processing logic decides not to convert the load unit, the given database object is loaded into the in-memory store in a page-loadable format. In the page-loadable format, the processing logic loads only the relevant pages into the in-memory store.

[0016] By way of example, infrequently used tables are loaded into memory in a page-loadable format, even if such tables have column-loadable load units. For these infrequently used tables, only pages relevant to the query being executed are loaded into memory. In some embodiments, more frequently used tables are loaded into memory in a column-loadable format, even if such tables have page-loadable load units. For these more frequently used tables, entire columns are loaded into memory. In some embodiments, when storing from an in-memory store to a persistent store, processing logic stores database objects in a single "unified persistent format" regardless of their in-memory formats.

[0017] The unified persistent format allows each database object to be loaded into memory from the same persistence in either a column-loadable format or a page-loadable format. The unified persistent format also enables load unit conversion for a given database object in the persistent store without rewriting the given database object. The unified persistent format includes separate composite page chains for the data, dictionary, and index subcomponents of a given database object. A composite page chain may have one or more sub-page chains.

[0018] Another type of persistence format is the serial persistence format, which sequentially stores subcomponents together like a single page chain. The serial persistence format may be suitable for scenarios where data components are relatively small and can be efficiently stored together in a single chain. The serial persistence format is often used for smaller datasets and has advantages such as compact storage and fast access to datasets. On the other hand, the unified persistence format is more suitable for larger and more complex datasets where the sizes of subcomponents may vary greatly. The unified persistence format allows each subcomponent to be stored optimally and independently of each other, taking into account the specific size and characteristics of each subcomponent. Unified persistence provides advantages including improved performance and the capability to load persistence data in a column-loadable or page-loadable format, which is not available when the dataset is stored in the serial persistence format.

[0019] Modern database systems can provide data replication services that mirror data to improve performance. Data replication involves copying data from a database on one server (i.e., a primary database) to another database on another server or client (i.e., a secondary database). In some implementations, the secondary database is configured to be an exact copy in terms of processing resources, memory capacity, storage capacity, and the like. However, while this configuration simplifies management of the primary and secondary databases, it may result in a configuration that is costly to maintain.

[0020] Therefore, some organizations may implement a secondary database, which is a scaled-down version of the primary database, as a non-limiting example, to reduce costs. In such cases, data replication during failures or downtime and switching from the primary to the secondary database become more complex. For example, if the secondary database has half the memory capacity of the primary database, the load units used by the primary database to load database objects may not be optimal when applied to the secondary database. To elaborate further, if the primary database uses a column-loadable format for all or most database objects loaded into the in-memory store, these database objects are fully loaded into the in-memory store. This technique may not be feasible in the secondary database due to the smaller capacity of the in-memory store. Therefore, to account for differences in the physical configuration of the primary database compared to the secondary database, the secondary database may use different load units for the same database objects loaded by the primary database. As an example, the secondary database may use a page-loadable format for database objects that are loaded in a column-loadable format in the primary database. As mentioned earlier, the page-loadable format loads only the pages relevant to the query into the in-memory store, reducing memory usage.

[0021] Referring now to Figure 1, a system diagram is shown illustrating an example of a database system 100 according to several exemplary embodiments. In Figure 1, the database system 100 may include one or more client devices 110, a database execution engine 130, and one or more databases 140. Note that the terms “database execution engine,” “processor,” and “processing logic” may be used interchangeably in this specification. It is shown that the database 140 includes tables 150 representing any number and type of database objects stored by the database 140.

[0022] One or more client devices 110, a database execution engine 130, and one or more databases 140 may be connected communicatively via a network 120. One or more databases 140 may include various relational databases, such as in-memory databases, column-based databases, and row-based databases. One or more client devices 110 may include processor-based devices, such as mobile devices, wearable devices, personal computers, workstations, and Internet of Things (IoT) appliances. The network 120 may be a wired network and / or a wireless network, such as a public land mobile network (PLMN), a local area network (LAN), a virtual local area network (VLAN), a wide area network (WAN), and the internet.

[0023] Referring next to Figure 2, a diagram is shown illustrating an example of a system 200 that corresponds to an implementation of the subject matter of the present invention. The system 200 includes applications 240A-N that interact with an in-memory database 210. Applications 240A-N represent any number and type of applications that run within the system 200. For example, applications 240A-N may include analytical services, transaction processing services, reporting services, dashboard services, etc. Applications 240A-N interact with the in-memory database 210 using structured queries or other expressions. The system 200 also includes one or more platforms 245. In some embodiments, a given platform 245 may be a data warehouse service that exposes online analytical processing (OLAP) services to users or applications. A database management service 260 provides management services and access to the configuration of the in-memory database 210. A user interface (UI) 270 mediates the interaction between the database management service 260 and the user or application, for example, to configure the in-memory database 210 or to set user preferences for sorting behavior.

[0024] Within the in-memory database 210, the data management service 250 manages transactions with the main storage 230 and the delta storage 235. The data management service 250 can provide compute and planning engines, modeling services, real-time replication services, data integration services, and / or other services. The main storage 230 supports high-speed read access to the data in the in-memory database 210 in the datastore 225. Read operations access both the main storage 230 and the delta storage 235, where the delta storage 235 contains any recently modified data not yet incorporated into the main storage 230. Data in the main storage 230 can be backed up to persistent storage 280. Persistent storage 280 may be disk storage or other suitable type of memory device. Modified data in the delta storage 235 is also backed up to persistent storage 280 so that changes are retained even if events such as database failures or downtime occur. The buffer cache 233 refers to a portion of memory that provides temporary storage for data loaded from persistent storage 280. For example, if a query or database operation requires a column or a portion of a database table, the corresponding page needed to respond to the query can be identified and loaded into buffer cache 233.

[0025] In some embodiments, persistent storage 280 stores database objects in a single, unified persistence format, regardless of the object load unit. The unified persistence format will be described in more detail throughout the remainder of this disclosure. It should be understood that the exemplary architecture of system 200 is merely to illustrate what may be adopted in some embodiments. In other embodiments, system 200 may be structured in other suitable ways using the arrangement of other components.

[0026] Referring to Figure 3, a block diagram of an example system 300 for implementing load unit determination is shown. When a table is persisted to the persistence store 325 using a unified persistence format, each column of the table has a separate page chain for each subcomponent of data, dictionary, and index. These are shown as the data page chain 330, the dictionary page chain 340, and the index page chain 350. When loading a portion of the table from the persistence store 325 into memory, a load unit determination 320 is made dynamically regarding the intended structure of the table in the in-memory store. If the load unit determination 320 uses the page load attribute 315 (i.e., a page loadable load unit), the table is loaded into memory using a page loadable format. The page loadable format has separate page chains for the data, dictionary, and index portions of the table. Having separate page chains for the data, dictionary, and index portions of the table allows only the relevant pages, rather than entire columns, to be loaded into memory. If the load unit determination uses the in-memory load attribute 310, the table is loaded into memory using a column loadable load unit, also known as the in-memory load format or column loadable format.

[0027] When table columns are loaded into memory using a column-loadable format, the entire data is read from persistence store 325, and index vectors, dictionaries, and inverse index vectors are created as in-memory structures. Once the data is read from persistence store 325 and the in-memory structures are created, the corresponding pages from persistence store 325 are no longer accessed. In-memory index vectors allow direct access to specific offsets within the in-memory store.

[0028] When table columns are loaded into memory using a page-loadable format, only the relevant data is read from persistence store 325 and loaded into memory. For example, in a page-loadable format, the processor first calculates which page an offset is on, then loads that page, and then reads the data from that page. Within that page, additional calculations from the header may be required to reach a specific offset for a particular row.

[0029] Referring now to Figure 4, another example of system 400 for implementing load unit determination is shown. As illustrated, system 400 includes a persistence store 410 that stores data in a unified persistence format. For a table stored in a unified persistence format, each component of each column (i.e., attribute) of the table has a separate page chain. For example, data components are stored as data page chain 420, dictionary components as dictionary page chain 430, and index components as index page chain 440.

[0030] If a given portion of a table needs to be loaded from the persistence store 410 into the in-memory store, a load unit decision 450 is made to determine in what form the given portion will be stored in memory. For example, the decision on the form in which a given portion is loaded into memory is based on at least an attribute associated with the table portion. For example, the attribute may be a page load attribute 460, or the attribute may be an in-memory load attribute 470. As another example, the decision on the form in which a given portion is loaded into memory is also based on real-time operating conditions (e.g., memory usage, how often the table is accessed). If the real-time operating conditions indicate that a load unit conversion should not be performed based on memory usage, access frequency, and other factors, the load unit decision 450 may revert to a default for the attribute associated with the table. For queries that reference a column with a page load attribute 460, if no load unit conversion is implemented, the data is loaded into memory in a page loadable format. For queries that reference a column with an in-memory load attribute 470, if no load unit conversion is implemented, the data is loaded into memory in a column loadable format.

[0031] If load unit conversion is implemented, columns with page load attribute 460 are loaded into the in-memory store in a column-loadable format. Alternatively, if a column has in-memory load attribute 470 and load unit conversion is implemented, the column is loaded into the in-memory store in a page-loadable format. For example, if memory usage is below a threshold, load unit conversion may be implemented by converting page-loadable load units to column-loadable load units. Alternatively, or additionally, if memory usage exceeds a threshold, load unit conversion may be implemented by converting column-loadable load units to page-loadable load units. Alternatively, or additionally, if the frequency of column access exceeds a threshold, load unit conversion may be implemented by converting page-loadable load units to column-loadable load units. Alternatively, or additionally, if the frequency of column access is below a threshold, load unit conversion may be implemented by converting column-loadable load units to page-loadable load units. Other conditions for performing load unit conversion are also possible and contemplated.

[0032] When a table is loaded into the in-memory store, if changes are made to the table in a page-loadable format, the delta component 480 is merged with the unchanged data and stored again in the persistence store 410 in a unified persistence format. Similarly, if changes are made to a table in a column-loadable format within the in-memory store, the delta component 485 is merged with the unchanged data and stored again in the persistence store 410 in a unified persistence format. In other words, in this example, regardless of which load unit is used to load the table into memory, the changed data is merged again and stored in the persistence store 410 in a unified persistence format.

[0033] Next, referring to Figure 5, a system 500 is shown with a primary database 510 and a secondary database 520. As shown in the figure, data is replicated from the primary database 510 to the secondary database 520. A redo log may also be sent from the primary database 510 to the secondary database 520. The redo log contains corrections made in the primary database 510, and the redo log is sent to the secondary database 520 so that the same corrections can be applied to the secondary database 520. The secondary database 520 is prepared for use in the event of a failure of the primary database 510. If the primary database 510 fails, the system 500 can switch to using the secondary database 520 instead of the primary database 510, and columns may be used in page format in the secondary database 520 before they are fully loaded into memory.

[0034] In the primary database 510, columns are loaded according to an in-memory format, with the entire column loaded into memory. This is indicated by the loading of the column's data, dictionary, and all subcomponents of the index into the in-memory store. Because the unified persistence format allows for different load units, the secondary database 520 loads columns into memory in a page-loadable format using a page structure. In the secondary database 520, the load unit can be determined on a per-column basis. For example, in the case of the secondary database 520, if a given column is not fully loaded into memory, it points to the corresponding page chain in the persistence store. During access, the relevant data is loaded into memory page by page. For example, when a query is executed, only a portion of the data required for the query is loaded into memory from the persistence store.

[0035] In some embodiments, the system 500 may include various types of configuration parameters. For example, a first configuration parameter may be called the “load unit parameter during log replay” or “log replay configuration parameter,” and may have one of three values: default, page loadable, or column loadable. If the log replay configuration parameter is set to default, tables on the secondary database 520 will match the format of tables on the primary database 510. If the log replay configuration parameter is set to page loadable, a particular table will be page loadable in the secondary database 520, even if that table is loaded in a column loadable format in the primary database 510. Similarly, if the log replay configuration parameter is set to column loadable, a given table will be column loadable in the secondary database 520, even if that table is loaded in a page loadable format in the primary database 510.

[0036] For example, another configuration parameter, called the post-failover load unit parameter, can have one of two values: continue as log replay or reload and reset to primary. If the post-failover load unit parameter is set to continue as log replay, the load unit continues in the secondary database 520 in the same load format used in the primary database 510. If the post-failover load unit parameter is set to reload and reset to primary, the load unit in the secondary database 520 is switched to be the same as that of the primary database 510. This explicitly executes the steps of an unload or reload script to identify the loaded columns, and then unloads and reloads them again in the defined load unit. For example, this script is integrated with the failover script. As another example, this script becomes active when the post-failover load unit configuration parameter is set.

[0037] Next, referring to Figure 6, another example of system 600 is shown, which has a primary database 610 and a secondary database 620. System 600 has the opposite configuration compared to system 500, in that the primary database 610 loads columns into memory in a page-loadable format, while the secondary database 620 loads columns into memory in a column-loadable format. This is the flexibility made possible by a unified persistence format, which allows the primary database 610 and the secondary database 620 to load data from the same persistence in either a page-loadable or column-loadable format without rewriting the persistence.

[0038] For example, the configuration of system 600 may be adopted immediately after primary database 610 fails and secondary database 620 takes over as the new primary database. In this example, a recovery operation is performed on primary database 610 to set it up as the new secondary database. The recovery operation may include, in a non-exclusive example, running previously replicated redo logs.

[0039] Next, referring to Figure 7, the process for dynamically determining the load unit for loading database objects and / or database object subcomponents into memory is shown. At the start of Method 700, the operation to begin loading a column into memory is shown (block 705). For example, when a query targeting a column is received, the loading of the column into memory may begin. The database then determines whether the column needs to be paged (condition block 710). If the column needs to be paged (condition block 710, "yes" leg), a paged data structure is created and the relevant pages of the column are loaded into memory (block 715). If the column does not need to be paged (condition block 710, "no" leg), the entire column is loaded into memory and no paged data structure is created (block 720). After blocks 715 and 720, Method 700 may terminate.

[0040] Referring next to Figure 8, the process for using different load units when loading database objects into memory in the primary and secondary database systems is illustrated. At the start of Method 800, at least one database object is loaded into the primary in-memory store in the primary database system according to a first form (block 805). As an example, the first form is a column-loadable form. The term “primary in-memory store” refers to the in-memory store in the primary database system. The primary database system refers to the main or active database system that is actively used by one or more users, organizations, or other entities. As an example, the primary database system replicates data to the secondary or inactive database system, and the secondary database system is intended to take over the primary database system in the event of failure or other event or circumstance.

[0041] Next, the operation of loading at least one database object into the primary in-memory store according to the first form is captured in the first log (block 810). The first log is then sent to the secondary database system (block 815). In response to receiving the first log, the secondary database system replays the first log (block 820). When replaying the first log, the secondary database system checks the value of the log replay configuration parameter (condition block 825). If the log replay configuration parameter is of the first value (condition block 825, "first" leg), the secondary database system loads at least one database object into the secondary in-memory store according to the second form, the second form being different from the first form (block 830). The term "secondary in-memory store" refers to the in-memory store in the secondary database system. After block 830, method 800 may terminate. As an example, the second form is a page-loadable form. Otherwise, if the log replay configuration parameter is the second value (condition block 825, "second" leg), the secondary database system loads at least one database object into the secondary in-memory store according to the first form (block 835). After block 835, method 800 may terminate.

[0042] Referring to Figure 9, the process for executing a failover load script in a secondary database system is shown. At the start of Method 900, a failover condition is detected in the primary database system (block 905). In response to the detection of the failover condition, the secondary database system executes a failover load script (block 910). For example, the failover load script may be integrated into an overall failover script executed by the secondary database system. In this example, the failover load script is executed only if the reset-to-primary configuration parameter is set.

[0043] As part of the execution of the failover load script, the secondary database system identifies one or more database objects to be loaded into the in-memory store according to a first form (block 915). For example, the first form is a page-loadable form. Next, the secondary database system unloads one or more database objects (block 920). Then, the secondary database system reloads one or more database objects corresponding to the primary-defined load unit into the in-memory store according to a second form (block 925). For example, the second form is a column-loadable form. After block 925, method 900 terminates.

[0044] In some implementations, the subject matter of the present invention may be configured to be implemented in a system 1000, as shown in Figure 10A. The system 1000 may include a processor 1010, memory 1020, storage device 1030, and input / output device 1040. Each of the components (e.g., 1010, 1020, 1030, and 1040) may be interconnected using a system bus 1050. The processor 1010 may be configured to process instructions for execution within the system 1000. In some implementations, the processor 1010 may be a single-threaded processor. In alternative implementations, the processor 1010 may be a multi-threaded processor. The processor 1010 may be further configured to process instructions stored in memory 1020 or storage device 1030, including receiving or transmitting information through the input / output device 1040. Memory 1020 may store information within the system 1000. In some implementations, memory 1020 may be computer-readable medium. In alternative implementations, memory 1020 may be a volatile memory unit. In some further implementations, memory 1020 may be a non-volatile memory unit. Storage device 1030 may be capable of providing large-capacity storage to system 1000. In some implementations, storage device 1030 may be a computer-readable medium. In alternative implementations, storage device 1030 may be a floppy disk device, a hard disk device, an optical disk device, a tape device, a non-volatile solid-state memory, or any other type of storage device. Input / output device 1040 may be configured to provide input / output operations for system 1000. In some implementations, input / output device 1040 may include a keyboard and / or a pointing device. In alternative implementations, input / output device 1040 may include a display unit for displaying a graphical user interface.

[0045] Figure 10B shows an exemplary implementation of a database 140 that provides database services. The database 140 may include physical resources 1080 such as at least one hardware server, at least one storage, at least one memory, and at least one network interface. The database 140 may also include infrastructure, as previously mentioned, at least one operating system 1082 for the physical resources and at least one hypervisor 1084 (which may create and run at least one virtual machine 1086). For example, each multitenant application may run on its corresponding virtual machine.

[0046] Referring to Figure 11, the process for determining the load unit when loading database objects into memory in the secondary database system is shown. At the start of method 1100, a first log is sent from the primary database system to the secondary database system, and the first log captures the loading of at least one database object into the primary in-memory store in the primary database system (block 1105). In response to receiving the first log, the secondary database system replays the first log (block 1110). When replaying the first log, the secondary database system checks the value of a log replay configuration parameter (condition block 1115). If the log replay configuration parameter is a first value (condition block 1115, "first" leg), the secondary database system loads at least one database object into the secondary in-memory store according to a first format (block 1120). For example, the first format is a page-loadable format, and if the log replay configuration parameter is set to a first value, it specifies or indicates that the database object must be loaded in a page-loadable format. In this case, the secondary database system can override the load unit decision made in the primary database system to load at least one database object into the primary in-memory store. After block 1120, method 1100 may terminate.

[0047] If the log replay configuration parameter is set to the second value (condition block 1115, “second” leg), the secondary database system loads at least one database object into the secondary in-memory store according to the second form (block 1125). It should be understood that the second value is different from the first value, and that the second value can be indicated, specified, or encoded in any appropriate way to indicate that it is different from the first value. For example, the second form is a column-loadable form, and if the log replay configuration parameter is set to the second value, it specifies or indicates that the database object must be loaded in a column-loadable form. Similarly, in this case, the secondary database system can also override any load unit decision made in the primary database system to load at least one database object into the primary in-memory store. In other words, regardless of any load unit decision made in the primary database system to load at least one database object into the primary in-memory store, because the log replay configuration parameter has the second value, the secondary database system loads at least one database object into the secondary in-memory store according to the second form. After block 1125, method 1100 may terminate.

[0048] If the log replay configuration parameter is set to a third value (condition block 1115, the "third" leg), the secondary database system loads at least one database object into the secondary in-memory store in the same format used by the primary database system to load at least one database object into the primary in-memory store (block 1130). For example, if in block 1130 at least one database object is loaded into the primary in-memory store in a page-loadable format, the secondary database system loads at least one database object into the secondary in-memory store in a page-loadable format. In this example, if in block 1130 at least one database object is loaded into the primary in-memory store in a column-loadable format, the secondary database system loads at least one database object into the secondary in-memory store in a column-loadable format. Note that the third value is different from the first and second values. In some cases, the third value may be called the default value or "default," and the secondary database system defaults to the same load unit as the primary database system. In other words, if the log replay configuration parameter is set to default, the secondary uses the same load unit format as used by the primary. After block 1130, method 1100 may terminate.

[0049] Referring to Figure 12, the process for adjusting the buffer cache size in the secondary database system is shown, based at least on the log replay configuration parameters. When the secondary database system replays logs generated in the primary database system, it determines which replay load unit is used to load database objects into the in-memory store in the secondary database system (block 1205). The secondary database system also determines the buffer cache size configuration parameter based at least on the replay load unit (block 1210). For example, if the replay load unit is a page-loadable load unit, the buffer cache size configuration parameter is set to its maximum value. If the replay load unit is configured to be page-loadable (condition block 1215, "page-loadable" leg), the secondary database system increases the size of the buffer cache (for example, buffer cache 233 in Figure 2) according to the buffer cache size configuration parameter (block 1220). Alternatively, in block 1220, the secondary database system may set the size of the buffer cache to a first capacity, which is a relatively high capacity. Otherwise, if the reload unit is configured to be column loadable (condition block 1215, "column loadable" leg), the secondary database system reduces the size of the buffer cache according to the buffer cache size configuration parameter (block 1225). Alternatively, in block 1225, the secondary database system may set the size of the buffer cache to a second capacity, which is a relatively lower capacity, and this second capacity is smaller than the first capacity used in block 1220. After blocks 1220 and 1225, method 1200 may terminate.

[0050] The systems and methods disclosed herein can be embodied in various forms, including, for example, data processors such as computers, which also include databases, digital electronic circuits, firmware, and software, or combinations thereof. Furthermore, the above-described features and other aspects and principles of the implementations of this disclosure can be implemented in various environments. Such environments and associated applications may be specifically constructed to perform various processes and operations according to the disclosed implementations and may include general-purpose computers or computing platforms that are selectively activated or reconfigured by code to provide the necessary functionality. The processes disclosed herein are not inherently related to any particular computer, network, architecture, environment, or other device, and can be implemented by a suitable combination of hardware, software, and / or firmware. For example, various general-purpose machines can be used with programs written according to the teachings of the disclosed implementations, but it may be more convenient to construct a dedicated device or system to perform the necessary methods and techniques.

[0051] Ordinal numbers such as "1st," "2nd," etc., can sometimes relate to order, but when used in a document, ordinal numbers do not necessarily imply order. For example, ordinal numbers can be used simply to distinguish one item from another. For instance, they can distinguish the first event from the second event, but do not necessarily imply a chronological order or a fixed reference system (therefore, the first event in one paragraph of the explanation may be different from the first event in another paragraph of the explanation).

[0052] The foregoing description is intended to illustrate the scope of the invention, not to limit it, and the scope of the invention is defined by the appended claims. Other implementations are within the scope of the following claims.

[0053] These computer programs, also called programs, software, software applications, applications, components, or code, contain program instructions (i.e., machine instructions) for a programmable processor and can be implemented in high-level procedural programming languages ​​and / or object-oriented programming languages, as well as / or assembly language / machine language. As used herein, the term “machine-readable medium” refers to any computer program product, apparatus, and / or device, such as magnetic disks, optical disks, memory, and programmable logic devices (PLDs), used to provide machine instructions and / or data to a programmable processor, and includes machine-readable medium that receives program instructions as machine-readable signals. The term “machine-readable signals” refers to any signals used to provide machine instructions and / or data to a programmable processor. Machine-readable medium can store such program instructions non-temporarily, for example, in non-temporarily solid-state memory or magnetic hard drives, or any equivalent storage medium. Alternatively, or additionally, machine-readable medium can store such machine instructions transiently, as in processor caches or other random-access memory associated with one or more physical processor cores.

[0054] To provide user interaction, the subject matter described herein can be implemented on a computer having, for example, a display device such as a cathode ray tube (CRT) or liquid crystal display (LCD) monitor for displaying information to the user, a keyboard on which the user can provide input to the computer, and a pointing device such as a mouse or trackball. Other types of devices can also be used to provide user interaction. For example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, such as acoustic, speech, or tactile input.

[0055] The subject matter described herein can be implemented in a computing system including, for example, one or more backend components such as data servers, or a computing system including, for example, one or more middleware components such as application servers, or a computing system including, for example, one or more client computers having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described herein, or in any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication, such as a communication network. Examples of communication networks include, but are not limited to, local area networks ("LANs"), wide area networks ("WANs"), and the Internet.

[0056] A computing system can include clients and servers. Clients and servers are generally geographically separated, but not necessarily so, and usually interact via a communication network. The client-server relationship arises from computer programs running on each computer that have a client-server relationship with one another.

[0057] In the above description and claims, linked lists of elements or features may be used following phrases such as “at least one” or “one or more.” The term “and / or” may also occur in lists of two or more elements or features. Unless implicitly or explicitly contradicted by the context in which it is used, such phrases are intended to mean any of the listed elements or features individually, or any of the enumerated elements or features in combination with any of the other enumerated elements or features. For example, the phrases “at least one of A and B,” “one or more of A and B,” and “A and / or B” are intended to mean “A only, B only, or both A and B,” respectively. A similar interpretation is intended for lists containing three or more items. For example, the phrases “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, and / or C” are intended to mean “A only, B only, C only, both A and B, both A and C, both B and C, or all of A, B and C,” respectively. The use of the term “based on” in the above and in the claims is intended to mean “based on at least in part,” and features or elements not enumerated are also permitted.

[0058] Taking into consideration the implementation forms of the subject matter described above, this application discloses the following list of examples, where one feature of an example alone, or a combination of several features of the aforementioned examples, optionally combined with one or more features of one or more further examples, constitutes a further example that falls within the scope of the disclosure of this application.

[0059] Example 1: A method comprising the steps of: receiving data replicated from a primary database system by a secondary database system; receiving a first log by the secondary database system that captures at least one database object loaded into a primary in-memory store in the primary database system according to a first format; determining a value for a log replay configuration parameter; and, in response that the log replay configuration parameter has a first value, replaying the first log on the secondary database system and loading at least one database object into a secondary in-memory store according to a second format.

[0060] Example 2: The method according to Example 1, further comprising the steps of detecting a failover condition, initiating a first failover reload script in a secondary database system in response to the detection of a failover condition, unloading at least one database object from a secondary in-memory store, and reloading at least one database object into the secondary in-memory store according to a first form.

[0061] Example 3: The method described in any one of Examples 1 or 2, wherein the second form is different from the first form.

[0062] Example 4: The method according to any one of Examples 1 to 3, wherein the first format is a column-loadable format.

[0063] Example 5: The method according to any one of Examples 1 through 4, wherein the second format is a page-loadable format.

[0064] Example 6: In the case of a page-loadable format, only the pages related to the corresponding query are loaded into the in-memory store, using one of the methods described in Examples 1 through 5.

[0065] Example 7: In the case of a column-loadable format, the entire corresponding database object is loaded into an in-memory store, as described in any one of Examples 1 through 6.

[0066] Example 8: The method according to any one of Examples 1 to 7, wherein the column loadable format has data subcomponents, dictionary subcomponents, and index subcomponents of the corresponding database object, serialized into a single page chain, and the page loadable format has a separate page chain for each of the data subcomponents, dictionary subcomponents, and index subcomponents of the first database object.

[0067] Example 9: The method according to any one of Examples 1 to 8, further comprising the step of replaying the first log on a secondary database system in response to a log replay configuration parameter having a second value, and loading at least one database object into a secondary in-memory store according to a first format.

[0068] Example 10: The method according to any one of Examples 1 to 9, further comprising the step of replaying a second log on a secondary database system in response to a log replay configuration parameter having a third value, and loading at least a second database object into the secondary in-memory store in the same format as used in the primary in-memory store.

[0069] Example 11: A system comprising at least one processor and at least one memory containing program instructions, wherein, when executed by at least one processor, the program instructions cause a secondary database system to perform an operation including receiving data replicated from a primary database system, receiving a first log capturing at least one database object loaded into a primary in-memory store in the primary database system according to a first form, determining a value for a log replay configuration parameter, and, in response to the log replay configuration parameter having a first value, replaying the first log on the secondary database system to load at least one database object into a secondary in-memory store according to a second form.

[0070] Example 12: The system as in Example 11, wherein when a program instruction is executed by at least one processor, it causes the system to further perform actions including detecting a failover condition, initiating a first failover reload script in a secondary database system in response to the detection of the failover condition, unloading at least one database object from a secondary in-memory store, and reloading at least one database object into the secondary in-memory store in a first form.

[0071] Example 13: A system described in any one of Examples 11 to 12, wherein the second form is different from the first form.

[0072] Example 14: A system described in any one of Examples 11 through 13, wherein the first format is a column-loadable format.

[0073] Example 15: A system described in any one of Examples 11 through 14, wherein the second format is a page-loadable format.

[0074] Example 16: In the case of a page-loadable format, only the pages related to the corresponding query are loaded into the in-memory store, as described in any one of Examples 11 through 15.

[0075] Example 17: In the case of a column-loadable format, the entire corresponding database object is loaded into an in-memory store, as described in any one of Examples 11 through 16.

[0076] Example 18: The system described in any one of Examples 11 through 17, wherein the column loadable format has data subcomponents, dictionary subcomponents, and index subcomponents of the corresponding database object, serialized into a single page chain.

[0077] Example 19: The system described in any one of Examples 11 to 18, wherein the page-loadable format has a separate page chain for each of the data subcomponents, dictionary subcomponents, and index subcomponents of the first database object.

[0078] Example 20: A non-temporary computer-readable medium storing instructions that, when executed by at least one data processor, cause a secondary database system to perform an action including receiving data replicated from a primary database system; receiving a first log capturing at least one database object loaded into a primary in-memory store in the primary database system according to a first form; determining a value for a log replay configuration parameter; and, in response to the log replay configuration parameter having a first value, replaying the first log on the secondary database system and loading at least one database object into a secondary in-memory store according to a second form.

[0079] The implementations described above do not represent all implementations that correspond to the subject matter described herein. Rather, the implementations described above are merely examples that correspond to aspects related to the subject matter described herein. While some variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations can be provided in addition to those described herein. For example, the implementations described above may cover various combinations and subcombinations of the disclosed features, as well as / or combinations and subcombinations of some of the further features disclosed above. Furthermore, the logic flows depicted in the accompanying drawings and / or described herein do not necessarily require a specific order or sequence shown to achieve the desired result. Other implementations may be included in the following claims. [Explanation of symbols]

[0080] 100 Database Systems 110 Client Devices 120 Networks 130 Database Execution Engine 140 Databases 150 tables 200 Systems 210 In-memory databases 225 Datastores 230 Main Storage 233 Buffer Cache 235 Delta Storage 240A-N Application 245 platforms 250 Data Management Services 260 Database Management Services 270 User Interface (UI) 280 persistent storage 300 Systems 310 In-memory load attribute 315 Page Load Attributes 320 Road Unit Decided 325 Persistence Store 330 Data Page Chains 340 Dictionary Page Chain 350 Index Page Chain 400 System 410 Persistence Store 420 data page chains 430 Dictionary Page Chain 440 Index Page Chain 450 Road Unit Decided 460 Page Load Attributes 470 In-memory load attribute 480 Delta Components 485 Delta Component 500 Systems 510 Primary Database 520 Secondary Databases 600 System 610 Primary Database 620 Secondary Databases 1000 systems 1010 Processor 1020 memory 1030 Storage Devices 1040 Input / Output Devices 1050 System Bus 1080 physical resources 1082 Operating Systems 1084 Hypervisor 1086 virtual machines

Claims

1. A method for a secondary database system to perform each step, The steps include receiving data that has been replicated from the primary database system, The steps include receiving a first log that captures at least one database object loaded into a primary in-memory store in the primary database system according to a first format, The steps include determining the values ​​of the log playback configuration parameters, The steps include: in response to the log replay configuration parameter having a first value, replaying the first log on the secondary database system and loading the at least one database object into the secondary in-memory store in a second format; Methods that include...

2. Steps to detect a failover state, The steps include: in response to the detection of the failover state, initiating a first failover load script in the secondary database system; The steps include unloading at least one database object from the secondary in-memory store, The steps of reloading the at least one database object into the secondary in-memory store in accordance with the first format described above, and The method according to claim 1, further comprising:

3. The method according to claim 2, wherein the second form is different from the first form.

4. The method according to claim 1, wherein the first format is a column-loadable format.

5. The method according to claim 4, wherein the second format is a page-loadable format.

6. The method of claim 5, wherein, in the case of the page-loadable format, only pages related to the corresponding query are loaded into the primary in-memory store and the secondary in-memory store.

7. The method of claim 6, wherein, in the case of the column-loadable format, the entire corresponding database object is loaded into the primary in-memory store and the secondary in-memory store.

8. The method according to claim 7, wherein the column loadable format comprises having data subcomponents, dictionary subcomponents, and index subcomponents of a corresponding database object, serialized into a single page chain, and the page loadable format comprises having a separate page chain for each of the data subcomponents, dictionary subcomponents, and index subcomponents of a first database object.

9. The method according to claim 1, further comprising the step of replaying the first log on the secondary database system in response to the log replay configuration parameter having a second value, and loading the at least one database object into the secondary in-memory store in the first format.

10. The method according to claim 1, further comprising the step of replaying a second log on the secondary database system in response to the log replay configuration parameter having a third value, and loading at least a second database object into the secondary in-memory store in the same format as used in the primary in-memory store.

11. At least one processor, At least one memory that stores program instructions and Equipped with, When the program instruction is executed by the at least one processor, The secondary database system receives data that has been replicated from the primary database system, The secondary database system receives a first log capturing at least one database object loaded into the primary in-memory store in the primary database system according to a first format, Determining the values ​​of the log playback configuration parameters, In response to the log replay configuration parameter having a first value, the first log is replayed on the secondary database system and the at least one database object is loaded into the secondary in-memory store in a second format. A system that performs actions including [specific actions].

12. When the program instruction is executed by the at least one processor, Detecting a failover state, In response to the detection of the failover state, the secondary database system starts a first failover load script, Unloading at least one of the database objects from the secondary in-memory store, Reloading the at least one database object into the secondary in-memory store in accordance with the first form described above. The system according to claim 11, further comprising the operation including the operation.

13. The system according to claim 11, wherein the second form is different from the first form.

14. The system according to claim 11, wherein the first format is a column-loadable format.

15. The system according to claim 14, wherein the second format is a page-loadable format.

16. The system according to claim 15, wherein, in the case of the page-loadable format, only pages related to the corresponding query are loaded into the primary in-memory store and the secondary in-memory store.

17. The system according to claim 16, wherein, in the case of the column-loadable format, the entire corresponding database object is loaded into the primary in-memory store and the secondary in-memory store.

18. The system according to claim 17, wherein the column loadable format comprises data subcomponents, dictionary subcomponents, and index subcomponents of the corresponding database object, serialized into a single page chain.

19. The system according to claim 18, wherein the page-loadable format has a separate page chain for each of the data subcomponents, dictionary subcomponents, and index subcomponents of the first database object.

20. A non-temporary computer-readable storage medium that stores instructions, The instruction, when executed by at least one data processor, The secondary database system receives data that has been replicated from the primary database system, The secondary database system receives a first log capturing at least one database object loaded into the primary in-memory store in the primary database system according to a first format, Determining the values ​​of the log playback configuration parameters, In response to the log replay configuration parameter having a first value, the first log is replayed on the secondary database system and the at least one database object is loaded into the secondary in-memory store in a second format. A non-temporary computer-readable storage medium that enables the operation of the following:

Images