Database Management System

Abstract

A method for managing log files is provided, wherein a set of log files is updated, the storage space allocated to the log files is monitored, and further storage space is allocated based on determining that the available storage space is below a predetermined size. A method for generating a snapshot is provided, wherein data entries are included in the snapshot according to a determined relative order of log records. A method for replicating a binary large object is provided, wherein the binary large object is sent to a second database in response to identifying a log record including data indicating the binary large object. A method for deleting one or more binary large objects is provided, wherein data indicating the binary large objects that have been logically deleted is stored in a snapshot, and the binary large objects are deleted after a previous snapshot is deleted.

Description

Technical Field

[0001] This application relates to database management systems, and more particularly to methods and systems for improving the efficiency of relational database management systems. Background Art

[0002] As technology advances, the amount of information stored in electronic form and the desire for the ability to search, organize, and / or manipulate such information in real-time or near real-time are increasing. Database management systems, sometimes also referred to as databases and data warehouses, are designed to organize data in a form that facilitates efficient searching, retrieval, and / or manipulation of selected information. A typical database management system allows a user to submit a "query" or invoke one or more functions in a query language to search, organize, retrieve, and / or manipulate information that meets specific criteria.

[0003] Certain databases can be transactional in nature and can record transactions in a log, including one or more operations performed on the data. The log can be considered a continuous stream of log records, each log record corresponding to a transaction. This can allow transactions to be replayed or undone after a crash. Logs can also be used to replicate a database by sending the logs between databases and executing the transactions recorded therein. Log records can be stored in a log file that is periodically updated and truncated. When a transaction is executed, the log file can grow as log records are generated. When the log file is full, a second log file can be generated. When the second log file is generated, transactions may not complete because their corresponding log records cannot be generated until they can be recorded in the second log file. In some database systems, snapshots can be used to restore or replicate a database by initializing the state of the database from the snapshot and only replaying and executing all transactions that occurred after the snapshot was taken. Binary large objects can be large files in binary format that are difficult to manage in a conventional database because their structure is unlike other data types typically managed by a database.

[0004] The requirements for a database system can vary and to handle increasing demands, the database system can be scalable. A scale-up database system handles increased demands on the database system by a user by increasing memory or upgrading the CPU of an existing server. A scale-out database system increases capacity by adding new nodes (i.e., new machines) to the database system. Using scale-out to scale a database system can increase the storage capacity of the database system while increasing the business capacity.

[0005] It would be advantageous to reduce the time spent growing the log files when executing transactions. It would be advantageous to efficiently generate a snapshot that can be efficiently restored or replicated of a database. It would also be advantageous to more efficiently handle binary large objects. Summary of the Invention

[0006] According to a first aspect of the present disclosure, there is provided a computer-implemented method for managing a log file for recording operations on data stored in a database, the operations including reading from or writing to the database, the method comprising: updating the log file set by writing data indicating one or more operations that have been performed on the data stored in the database to a set of log files, the set of log files having a first storage portion allocated thereto; monitoring the first storage portion when updating the log file set; and allocating a second storage portion to the log file set when updating the log file set based on determining that an available portion of the first storage portion is below a predetermined size.

[0007] Monitoring the available portion of the first storage portion and allocating a second storage portion when updating the log file set prevents the database system from blocking the execution of transactions when generating additional log files. A transaction cannot be executed and / or completed until it is logged in the log record, and thus allocating resources for recording additional log records when updating the first storage portion prevents transactions from being blocked once the first storage portion is full.

[0008] According to a second aspect of the present disclosure, there is provided a computer-implemented method for generating a snapshot representing the state of a database at a given time, the method comprising: generating data entries in the database, the data entries being associated with log records for recording at least one operation corresponding to the data entries, the log records corresponding to a log sequence indicator; selecting a snapshot cutoff log sequence indicator; determining the relative order of the log sequence indicator and the snapshot cutoff log sequence indicator; and generating a snapshot representing the state of the database at the time corresponding to the snapshot cutoff log sequence indicator, wherein the snapshot includes the data entries according to the determined relative order.

[0009] This allows the database system to replay any transactions corresponding to the log records in the log file set that are located after the selected snapshot cut-off log sequence indicator when using snapshot recovery or re-initializing the database. Additionally, including an entry in the snapshot based on the location of the corresponding log record for the entry prevents the database system from having to separately determine which log records to replay. Thus, the log records do not need to be associated with the time at which their corresponding data entries are generated in the database before the data entries are fully generated. This prevents dirty reads, where data entries are made visible to the end user or application before they are fully generated. Providing an efficient way to generate a snapshot such that log records can be easily replayed can be particularly beneficial in a distributed relational database system, where the database is replicated between partitions within a node or between nodes for durability. The ability to generate a snapshot from which the database can be more efficiently recovered or replicated can be of particular value for scale-out architecture database systems in which database replication occurs frequently.

[0010] According to a third aspect of the present disclosure, there is provided a computer-implemented method for replicating binary large objects stored at a first database to a second database, the method comprising: sending a set of log records corresponding to an operation performed on data stored at the first database to the second database; identifying log records in the set of log records that include an indication of a binary large object stored at the first database; and in response to identifying log records that include an indication of a binary large object stored at the first database, sending the binary large object stored at the first database to the second database.

[0011] Having a single process of sending the binary large object after the corresponding log record for the binary large object has been identified allows the transaction to commit faster and prevents the transaction corresponding to replicating the binary large object at the second database from remaining uncommitted for a long time, thereby reducing the number of open transactions and / or write operations running at any given time. Efficient replication provides increased durability and capacity in a distributed database system, where replication between databases often occurs during new resource provisioning and during steady-state operation.

[0012] According to a fourth aspect of the present disclosure, there is provided a computer-implemented method for physically deleting one or more binary large objects from a database, where the database has multiple states, and a first state of the database at a first time is represented by a first snapshot. The method includes: generating a second snapshot representing the state of the database at a second time, where the second time is later than the first time and the second snapshot includes data identifying one or more binary large objects that have been logically deleted from the database before the second time; deleting the first snapshot; and after deleting the first snapshot, using the data identifying one or more binary large objects that have been logically deleted before the second time to physically delete one or more binary large objects from the database.

[0013] Storing in the second snapshot data identifying one or more binary large objects that have been logically deleted before the second time allows for the quick and efficient location of the binary large objects once it is appropriate to physically delete them by accessing the data stored in the second snapshot. If a transaction corresponding to the logical deletion of a binary large object occurs before the time corresponding to the oldest snapshot, the binary large object that has been logically deleted can be physically deleted. When deleting a snapshot, a set of binary large objects that have been logically deleted after the generation of the snapshot and before the generation of the next sequential snapshot can be physically deleted. These binary large objects are located by looking at the data stored in the next sequential snapshot. Storing the data in the snapshot also allows for the periodic deletion of data when it is no longer needed. Deleting binary large objects when logically correct can more quickly free up resources in the database system so that they can be reallocated to handle the demands on the database system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In accordance with the following detailed description, taken in conjunction with the drawings that illustrate the features of the present disclosure, various features of the present disclosure will be apparent, and in which:

[0015] Figure 1 A schematic diagram of a database system according to an example is shown.

[0016] Figure 2 A flowchart of a method for managing a log file according to an example is shown.

[0017] Figure 3 A set of managed log files according to an example is schematically shown.

[0018] Figure 4A and Figure 4B An updated tracking file according to an example is schematically shown.

[0019] Figure 5 A flowchart of a method for generating a snapshot according to an example is shown.

[0020] Figure 6A Schematically shows a set of data entries and a set of log files at a first time during a snapshot process according to an example.

[0021] Figure 6B Schematically shows a set of data entries and a set of log files at a second time during a snapshot process according to an example.

[0022] Figure 7 Schematically shows a set of data entries and a set of log files during a snapshot process according to an example.

[0023] Figure 8 Schematically shows a set of data entries and a set of log files during a snapshot process according to an example.

[0024] Figure 9 Shows a flowchart of a method for copying one or more binary large objects according to an example.

[0025] Figure 10A Schematically shows according to Figure 9 a first database and a second database during a replication process of an example of the method shown in

[0026] Figure 10B Schematically shows according to Figure 9 a first database and a second database during a replication process of an example of the method shown in

[0027] Figure 11 Shows a flowchart of a method for deleting large binary objects according to an example.

[0028] Figure 12 Schematically shows according to Figure 11 a database and a snapshot during a binary large object deletion process of an example of the method shown in

[0029] Figure 13 Schematically shows according to Figure 11 a set of log files and a snapshot during a binary large object deletion process of an example of the method shown in

[0030] Figure 14 Schematically shows according to Figure 11 a set of log files and a snapshot during a binary large object deletion process of an example of the method shown in

[0031] Figure 15 Schematically shows according to Figure 11 a set of log files and a snapshot during a binary large object deletion process of an example of the method shown in

[0032] Figure 16 Schematically shows according to Figure 11 A set of log files and snapshots during a binary large object deletion process that are examples of the method shown.

[0033] Figure 17 Schematically shows a device according to an example. Detailed implementation manners

[0034] Figure 1 Schematically shows a database system 100 that may be related to the embodiments described herein. The database system 100 includes at least one database 102a and other data for managing the database 104a. Figure 1 Two databases 102a and 102b are shown, but it should be understood that the database system may include any number of databases. In some examples, the database is a virtual database stored on one machine. In other examples, the database is stored on separate machines, where the database system 100 is implemented in a networked computer. A database can generally be considered an organized collection of data stored electronically in a computer system. The databases 102a, 102b include a collection of data that includes structured data 104a, 104b, such as data sorted into tables in a row storage format, a column storage format, or a combination of both. The databases 102a, 102b also include unstructured data 106a, 106b. The unstructured data 106a, 106b includes one or more binary large objects. The binary large objects can be stored at the database, and pointers to the binary large objects can be stored in the structured data 104a, 104b so that the binary large objects 106a, 106b can be easily accessed. The databases 102a, 102b can include data stored according to the relational model. Any suitable query language can be used to access the data stored in the database according to the relational model. In an example, the Structured Query Language (SQL) can be used to operate on the data stored at the databases 102a, 102b.

[0035] The database system 100 includes a database management system. The database management system can be software and / or hardware configured to handle interactions from an application or an end user 110 with one or more databases 102a, 102b. The database management system also provides other functions to maintain one or more databases 102a, 102b. The database management system can be used to maintain the crash stability of one or more databases 102a, 102b by backing up the data stored in the one or more databases and information related to the structure of the data stored therein. The database management system can reinitialize or recover the database after a crash. The database management system can be used to manage multiple nodes. Each node can include one or more partitions, and each partition represents a version of the database. One or more partitions can be slave partitions configured to replicate the master partition. The database management system can include databases 102a, 102b, or can be used to control databases 102a, 102b, where databases 102a, 102b are from outside the database management system. The sum of the database management system and any number of databases can be referred to as the database system 100. When operating on databases 102a, 102b, the database management system accesses an instance of the database by opening or mounting the database to manipulate the data within the database. The database management system can provide various functions, including providing facilities for an end user 108 or an application to read, write, and modify the data stored in one or more databases 102a, 102b. The database management system receives queries from an end user 108 or an application accessing databases 102a, 102b and executes transactions on the data stored in databases 102a, 102b according to the queries. A transaction is a unit of work performed on the data stored within the database system. A transaction can be a single logical unit or unit of work performed on the data stored in the database and includes one or more operations on the data. To maintain reliability and consistency in the database, transactions are atomic, consistent, isolated, and persistent. These properties are generally referred to as the ACID properties.

[0036] The database system 100 maintains a log of operations performed on data stored in one or more databases 102a, 102b. The log may also be referred to as a transaction log, database log, binary log, or audit trail. Each database is associated with at least one log. The log is used to record the history of transactions performed by the database management system on the data stored at the databases 102a, 102b. The database system may maintain different logs for different types of transactions performed within the database system. Transactions are recorded in the log as log records. A log record is an entry in the log that includes an indicator of the relative position in the log and information related to the transaction associated with the log record. The log record 112a may include or be associated with a log sequence indicator, where each sequentially generated log record is assigned a log sequence indicator higher than the previous most recent log sequence indicator. The log record 112a may include an item indicator, where an item defines a portion of the log defined between a previous failover or crash of the database and the next subsequent failover or crash. When a failover or crash occurs in the database, a special item start log record indicating the start of a new item is generated. When an instance of the database 102a is opened such that the database management system performs operations on the data stored in the database 102a, the database management system may write the data to the log to record the operations that have been performed on the data. The log may be written to a set of log files to record the transactions that the database management system has performed. Each database 102a, 102b may be associated with a corresponding set of log files 110a, 110b. The database 102a may include the set of log files 110a, or the set of log files 110a may be stored remotely from the database 102a, and the association between the database 102a and the set of log files 110a may be maintained. The log may be a write-ahead log, where transactions are recorded before making changes to the data stored in the database 102a permanent. Changes to the data stored in the database 102a are made permanent by writing the data corresponding to the transaction to storage space. The storage space may be on disk or may be other forms of memory.

[0037] Figure 1A database 102a associated with a set of log files 110a is shown. The log file 110a includes one or more log records 112a. As described above, the log record 112a includes or is associated with an indicator of a relative position in the set of log files 110a and includes information related to a transaction associated with the log record. The indicator of the relative position in the log file 110a can be a log sequence indicator or a log sequence number. This can be stored in any suitable data format, such as an integer, a string, etc. The information related to the transaction associated with the log record 112a can be data indicating one or more operations associated with the transaction, such as an indicator of the type of operation performed. Alternatively, the log record 112a can include an indicator of the data state stored in the database before and after the transaction. The log record 112a can include an indicator of the transaction, such as a transaction ID. The transaction ID can be associated with a data entry 114a stored in the database 102a, which is generated as part of or in accordance with the transaction. The log record 112a can include an indicator of the most recent previous log record in the log file 112a, where the log file 110a can be a linked list of log records 112a.

[0038] The database system 100 includes a plurality of snapshots 116a, 116b corresponding to respective databases 102a, 102b. The snapshot represents the state of the database at a given time. For example, the snapshot can be a read-only copy of the data 104a, 106a stored in the database 102a at a given time. The snapshot combined with other information (such as log files) can be used to restore the state of the database after a crash.

[0039] Log File Allocation

[0040] As described above, the database system 100 maintains a set of log files 110a for recording operations on the data stored in the database 102a. The database system 100 prevents a transaction from being committed until a log record 112a corresponding to the transaction has been generated by writing the data to the set of log files 110a. A transaction is said to be committed if one or more operations on the data stored in the database 102a corresponding to the transaction have been made permanent. The last step in a transaction can involve committing the transaction, where the transaction cannot be committed until it has been recorded in the log record 112a. Maintaining the log record 112a corresponding to each transaction enables the database to be restored or rolled back to a stable state after a crash. After a crash, the database 102a can be restored to a state in which the transaction has been executed or not executed, but not to a state in which the transaction has been partially executed.

[0041] Figure 2A flowchart of a method 200 for managing a log file for recording operations on data stored in a database 102a is shown, where the operations include reading or writing data to the database 102a. The method starts at block 210, where the method 200 includes updating a set of log files 110a by writing data indicating one or more operations performed on the data stored in the database 102a to the set of log files 110a, and the set of log files 110a has a first storage portion allocated thereto. Writing data indicating one or more operations performed on the data stored in the database 102a to the set of log files 110a may involve generating one or more log records 112a in the set of log files 110a. As described above, the log record 112a may be related to a transaction including one or more operations on the data stored in the database. The log record 112a may have a fixed size and / or a predetermined structure and is recorded in one or more pages, which may also be referred to as memory pages, storage pages, or virtual pages. A page is a contiguous virtual memory block with a fixed length. A page may have a minimum size of 4KiB or 4kB. Data may be stored in the database 102a in pages, where a page map or page table is used to identify the physical location where the data is stored. Having a set of log files 110a with a first storage portion allocated thereto allows the database management system to write data indicating one or more operations performed on the data stored in the database 102a to the set of log files 110a without physically increasing the set of log files 110a before each write operation. When operating in a mode where transaction records must be recorded in the corresponding log record 112a before commit, increasing the set of log files 110a before recording the transaction delays the completion of the transaction in the database system. In an example where the first storage portion allocated is larger than the size of each portion of the data indicating the one or more operations written to the log file, the set of log files may record multiple transactions without increasing the set of log files.

[0042] Writing data indicating one or more operations performed on the data stored in the database 102a involves reserving a portion of the set of log files 110a after receiving a request to perform one or more operations and performing the one or more operations in an isolated manner. The portion of the set of log files 110a is suitable for recording one or more operations. Then, the data indicating the one or more operations is recorded in the reserved portion of the set of log files 110a. The one or more operations may not be committed until they are recorded in the set of log files 110a. At commit, the results of the one or more operations can be seen in the database, that is, the results can be queried and are visible to subsequent transactions.

[0043] At block 220, the method includes monitoring a first storage portion when updating the log file set 110a. Monitoring the first storage portion can be performed by a dedicated thread configured to periodically access the log file set. Alternatively, one or more variables are updated when data is written to the log file set, and monitoring the log file set includes accessing the one or more variables. Other suitable methods for monitoring the first storage portion can also be used. At block 230, method 200 involves, in response to determining that the available portion of the first storage portion is below a predetermined size, allocating a second storage portion to the log file set 110a when updating the log file set 110a. As described above, a transaction including one or more operations cannot be committed (i.e., completed) until it has been logged to the log file set 110a. Allocating a second storage portion to the log file set 110a when updating the log file set 110a allows the database system to continue processing transactions in database 102a without imposing a temporary block on active or incoming transactions while the database system grows the log file set 102a. Since the second storage portion is allocated when there is available space in the first storage portion, log records corresponding to the current and incoming transactions are logged in the allocated second storage portion once the first storage portion is full. Thus, by allocating the first storage portion and allocating the second storage portion when updating the log file set 110a, operations performed on the data stored in database 102a can continue to be logged without growing the log file set 110a when logging each transaction. Allocating the second storage portion can involve generating additional log files having one or more pages. The pages of the newly generated log files are considered invalid until data indicating one or more operations is written to the newly generated log files. An order for updating the additional log files can be assigned. For example, by associating with corresponding log file identifiers indicating the order in which the log files will be updated.

[0044] Monitoring the first storage portion when updating the log file set 110a can involve periodically determining the size of the available portion of the first storage portion. A thread running on the database management system can be configured to periodically access the log file set 110a and determine the size of the available portion of the first storage portion. Monitoring the first storage portion when updating the log file set 110a can be triggered in response to receiving a request to perform one or more operations on the data stored in database 102a. Operations on the data stored in the database can include generating new data and / or reading, writing, or modifying data previously stored in database 102a. Determining the size of the available portion of the first storage portion in response to receiving a request to perform an operation on the data stored at the database can be performed after a predetermined number of requests have been received or after each request is received.

[0045] The predetermined size can depend on the rate of operations performed on the data stored in database 102a. During a high-activity period in which the available portion of the first storage portion decreases more rapidly, the predetermined size is larger, such that the second storage portion is allocated more quickly. Allocating the second storage portion takes time, so starting to allocate the second storage portion more quickly during a high-activity period prevents the first storage portion from filling up before the second storage portion has been allocated. During a low-activity period, the second storage portion is not allocated until closer to the time it is needed, rather than being allocated very early before the first storage portion fills up. This prevents the second storage portion from being allocated but remaining unused for a long time. The predetermined size can depend on the size of the first storage portion that is allocated. The predetermined size can be a proportion of the size of the first storage portion, such as less than 20%. The predetermined size can be determined by an operator of database system 100, such as an administrator of database system 100.

[0046] The predetermined size can depend on the ratio between the size of the first storage portion and the size of data indicating that one or more operations have been performed on the data stored in database 102a. For example, if one or more log records 112a are written to log file set 110a, the predetermined size can be related to the ratio between the estimated number of other log records that can be written to log file set 110a and the number of log records currently written to log file set 110a. The estimated number of other log records that can be written to the log file set depends on the available portion of the first storage portion that is allocated.

[0047] The size of the second storage portion can depend on the rate of operations performed on the data stored in database 102a. Database system 100 can monitor the rate at which the update log file set 110a is updated, and the size of the second storage portion can depend on that rate. In this way, during a high-activity period, the allocated second storage portion can be large in order to handle the increased activity. This prevents the system from having to allocate additional storage portions soon after allocating the second storage portion. Alternatively or additionally, the size of the second storage portion can be determined in response to a request to perform one or more operations on the data stored in database 102a. A request to perform one or more operations on the data stored in database 102a can include an indicator to perform a plurality of operations on the data stored in the data. For example, an end user 108 of databases 102a, 102b can periodically indicate to database system to perform a plurality of operations on the data stored in database 102a. A first request to perform one or more operations on the data stored in the database can include an indication to receive a plurality of requests to perform operations on the data stored in the database. As a result, the size of the second storage portion is determined when it is expected to receive a plurality of requests to perform operations on the data stored in database 102a.

[0048] Figure 3Schematically shows a set of log files 300 with an assigned first storage portion 310. In Figure 3 , the set of log files 300 includes one log file. However, it should be understood that the set of log files may include multiple log files. The assigned first storage portion may include multiple pages of a predetermined size. Figure 3 The set of log files 300 at a first time is shown at 320. At the first time, data indicating one or more operations performed on the data stored in the database has not been written to the set of log files 300. The set of log files 300 at the first time includes multiple pages for storing data, and these pages are invalid before the data indicating one or more operations is written to them. The invalid pages may also be referred to as incomplete pages. A transaction related to one or more operations performed on the data stored in database 102a may be recorded on more than one page. For example, in the case where the data of the transaction to be written to the set of log files 300 is larger than the page size in the set of log files 300. A page is considered invalid if it is associated with or includes an indication of page invalidity. Before writing data to multiple pages in the set of log files 300, these pages are shown in dashed lines at 320. Updating the set of log files 310 involves writing data indicating one or more operations performed on the data stored in the database to the set of log files 300, which is shown in Figure 3 , where data 314 is being written to the first page 312 in the set of log files 300.

[0049] Figure 3 The set of log files 300 at a second time is shown at 330. The second time is later than the first time, and the set of log files 300 is updated by writing data indicating one or more operations performed on the data stored in the database. The available portion of the first storage portion can be defined by the amount of storage occupied by invalid pages in the first storage portion, and is shown in dashed lines. At the second time, the available portion has dropped below a predetermined size 340, and thus a second storage portion is to be assigned to the set of log files 300. Figure 3 The set of log files 300 at a third time is shown at 350. The third time is later than the second time, and after determining that the available storage portion has dropped below the predetermined size 340, a second storage portion 360 has been assigned to the set of log files 300.

[0050] In the event of a database crash, the database system attempts to locate the most recent data written to the log file set 300 that indicates one or more operations performed on the data stored in the database 102a in order to restore the database 102a to a known state. The database may crash due to a power failure, hardware failure (such as memory, disk, CPU), or network failure of one or more machines hosting the database system, or an operating system error or other software or hardware related issue. In the case where the log file set 300 has an allocated storage portion, the most recent data written to the log file set cannot be located by accessing the end of the log file set 300 because there will be invalid pages at the end of the log file set 300 before the data is written. Thus, the method may involve updating a tracking file that includes an indicator of a portion of the log file set 300 to indicate the most recently updated portion of the log file set. Maintaining an indication of the most recently updated portion of the log file set allows for the location of data that indicates the most recent operations performed on the data stored in the database. For example, by accessing the indicated portion of the log file set and traversing the set of log records until the most recently generated log record is found.

[0051] Figure 4A The log file set 300 is schematically illustrated, where data 316 indicating one or more operations performed on the data stored in the database is being written to the log file set 300 and the tracking file 318 is being updated. Updating the tracking file 318 includes updating an indicator such that it indicates the most recently updated portion of the log file set 300. The tracking file 318 can be updated periodically by using a thread running on the database management system, which will update the tracking file 318 after a predetermined time interval.

[0052] In some examples, writing data indicating one or more operations performed on data stored in a database to a set of log files 300 involves generating one or more log records in the set of log files. Each log record corresponds to a log sequence indicator indicating a relative sequence in the set of log files 300. An indicator of a portion of the set of log files 300 can correspond to a log record in the set of log files. As a result, when determining the most recently generated log record in the set of log files, the database system can start traversing the set of log files starting from the log record corresponding to the indicator in the trace file 318. Updating the indicator of a portion of the set of log files can involve using the log sequence indicator corresponding to the most recently generated log record as at least a portion of the indicator of a portion of the set of log files 300. For example, the trace file 318 can have log sequence indicators of log records that have been written to the set of log files 300. The indicator of the portion of the set of log files stored in the trace file can also include an indication of an entry in the most recently updated portion of the set of log files, where each log record can be associated with an entry. A thread configured to periodically update the trace file 318 can do so in response to the log record being generated.

[0053] In other examples, updating the indicator of a portion of the set of log files 300 can include selecting a log sequence indicator higher than the log sequence indicator corresponding to the most recently generated log record to be used as at least a portion of the indicator of a portion of the set of log files. To locate the most recently generated log record in the set of log files, the system can start traversing the set of log files 300 backward from the location in the set of log files indicated by the trace file until a first valid page is found. Periodically updating the trace file improves the efficiency of maintaining an indication of the most recently updated portion of the set of log files 300 because the database system does not need to write to the trace file 318 after generating each log record. Generally, the frequency of change of entries in the log is less than the frequency of trace file updates, and thus maintaining the entries allows the database system to quickly locate the first portion of the set of log files and then search for the most recently updated portion of the set of log files within this first portion based on the trace file.

[0054] Updating the trace file 318 can be performed in response to generating a log record. For example, the trace file can be updated after a predetermined number of log records have been generated in the set of log files 300. Thereafter, the trace file 318 can be continuously updated after another predetermined number of log records are generated in the set of log files 300. In the case where the trace file 318 includes the log sequence indicator of the most recently generated log record, updating the trace file 318 can be performed once the log sequence indicator of the currently generated log record is selected. Figure 4BAn example is schematically shown in which the trace file 318 is updated when specific log records 320a, 320b, 320c are generated in the log file set 300. Alternatively, an indicator of a portion of the log file set 300 can be updated in response to the generation of each log record. Updating the trace file 318 in response to the generation of each log record allows the most recent data indicating one or more operations performed on the data stored in the database to be found after a crash without having to traverse the log file set.

[0055] Each page in the log file set 300 can be associated with a log sequence indicator, and in the case where a log record is stored on more than one page, the log record can correspond to more than one log sequence indicator. Thus, updating the indicator of a portion of the log file set can involve using the highest log sequence indicator corresponding to the most recently generated log record as at least a portion of the indicator of the portion of the log file set. Each page can include an indicator of the log sequence indicator corresponding to the log record and an offset in the log record. In this case, updating the trace file involves using the log sequence indicator to update the trace file after all pages corresponding to the log record have been logged in the log file set.

[0056] In some operating modes of the database system, a transaction cannot be committed until the trace file 318 has been updated, such that the indicator of the portion of the log file set 300 in the trace file 318 indicates the portion of the log file set 300 at or after the portion that includes the log record corresponding to the transaction. Thus, generating a log record can involve determining that the highest log sequence indicator associated with the log record has been used to update the trace file 318, or that the trace file 318 includes a log sequence indicator higher than the highest log sequence indicator associated with the log record. Alternatively or additionally, a transaction can be committed if and only if the highest log sequence indicator associated with the transaction has been used to update the trace file. To facilitate this, the trace file 318 can act as a pseudo-slave, which means that a transaction is committed if and only if the trace file 318 has acknowledged the highest log sequence indicator associated with the transaction.

[0057] To improve storage utilization efficiency, allocating a second storage portion may include reallocating a portion of a first storage portion that has been allocated to a subset of an updated set of log files 300. It is not necessary to permanently store all log files, so the database system periodically purges the oldest log files so that the set of log files 300 does not grow too large. To this end, a subset of the updated set of log files (such as the oldest log files currently stored) can be reused to record further operations on the data stored in database 102a. The subset of log files to be reused is obsolete. A log file may be obsolete if the transaction associated with the log file has been completed and the log file is no longer used for replicating to other databases or restoring the state of the database. Depending on the time when the oldest snapshot was taken, log files may become obsolete, as will be discussed in the section titled Deleting Binary Large Objects later. In the case where a log file is associated with or includes a log file identifier, reallocating a portion of the first storage portion that has been allocated to a subset of the set of log files may involve modifying one or more log file identifiers of the subset of log files. Modify the log file identifier such that the log file identifier indicates which subset of log files is to be used to record further operations to be performed on the data stored in database 102a. Although the subset of log files may still include data corresponding to operations that have been performed on the data stored in the database, any page in a log file whose log sequence indicator does not match the log sequence indicator determined based on the log file identifier of the associated log file and the offset into the associated log file can be considered invalid. Before reallocating a portion of the first storage portion that has been allocated to an updated subset of log files, the method may include determining the size of the available storage portion and reallocating a portion of the first storage portion if the available storage portion is below a further predetermined value. Reallocating a portion of the first storage portion that has been allocated to an updated subset of log files can be performed independently of determining that the available storage portion is below a predetermined threshold.

[0058] In some examples, the tracking file 318 is the first tracking file that is the first indicator of a portion of a set of log files, and the method includes alternately updating the first tracking file that is the first indicator of a portion of a set of log files and updating a second tracking file that is the second indicator of a portion of a set of log files. In known systems, if the database system crashes while updating the tracking file, the indicators associated with a portion of the set of log files in the tracking file may be corrupted or inaccessible. Providing two tracking files that are updated alternately provides stronger resilience against corruption during a crash because only one tracking file is updated at a time. The first and second tracking files may be embodied as a single logical tracking file that includes two log sequence indicators for the most recently generated log records corresponding to updating each respective first and second tracking file, and a checksum for each of the first and second tracking files. After a crash, the database system treats any tracking file with a mismatched checksum as invalid. Additionally, when determining which tracking file to use to locate the most recently updated portion of the set of log files, the database system compares the log sequence indicators in each tracking file or block and uses the tracking file or block with the highest log indicator as the indicator of the most recently updated portion of the set of log files. In some examples, the first and second tracking files may be the first pair of tracking files, and the computer-implemented method includes updating multiple pairs of tracking files. In cases where the database system can have multiple operations writing to multiple pairs of tracking files simultaneously, there may be multiple pairs of tracking files. A pair of tracking files may be a single tracking file that includes two blocks, and multiple pairs of tracking files may be multiple block pairs of a single tracking file.

[0059] The set of log files may be truncated periodically or in response to a predetermined event. Truncating the log is generally a process of reducing the log size by removing log records corresponding to old transactions. In the examples described herein, when the log file has a predetermined size, it may not be possible to physically truncate the set of log files. In such cases, the set of log records may be logically truncated by treating any log records that would otherwise be physically deleted as invalid. Truncating the set of log files decrements the log sequence indicator used to record new log records. Therefore, selecting the tracking file with the highest log sequence indicator may not be sufficient to locate the most recently updated portion of the set of log files. Accordingly, the tracking file may include a version indicator that is incremented during each truncation. When accessing the tracking file, the tracking file with the highest version indicator can be used to identify the most recently updated portion of the set of log files. If multiple tracking files have the same version indicator, the tracking file including the highest log sequence indicator may be selected.

[0060] Generate a snapshot

[0061] During the restoration of database 102a or when re-provisioning nodes in database system 100, a snapshot of database 102a can be used. There can be a master node and one or more slave nodes, where the slave nodes are configured to replicate the master node. When restoring database 102a using the snapshot or re-provisioning nodes in database system 100, database system 100 first initializes the snapshot of database 102a and then replays the log records in the log file set that are related to certain transactions, the results of which are not included in the snapshot. To reliably restore database 102a or re-provision the nodes, the log records corresponding to the transactions whose results are included in the snapshot should not be replayed. The logical order of the transactions executed on the data stored in the database does not necessarily match the order of the corresponding log records stored in the log file. At the start of a transaction, one or more operations corresponding to the transaction are executed in isolation such that the results of one or more transactions are not visible or saved to the database. The log records that have positions in the log file are subsequently retained, and one or more operations are recorded in the retained log records. Once one or more operations have been recorded in the log records, the transaction can be committed. When the transaction is committed, a data entry corresponding to the transaction is generated. The transaction and the associated data entry are assigned a logical order upon commit, the logical order corresponding to the time when the data entry is generated. Once the transaction is committed, the log records can be considered to be fully generated. A first transaction can start at a first time and retain log records at a first position, and a second transaction can start at a second time after the first time and retain log records at a second position. If the second transaction is committed before the first transaction, the data entries generated according to the second transaction in the database can have a logical order in the database that is earlier than the data entries corresponding to the first transaction in the database. However, the positions of the log records are retained before each corresponding data entry is generated, and thus the log records may be out of order compared to the data entries in the database.

[0062] Figure 5A flowchart of a method 500 for generating a snapshot representing the state of a database 102a at a given time is shown. Method 500 will be described first in a particular order, but it should be understood that method 500 can be executed in an order different from that initially described, as will become apparent. Method 500 includes, at block 510, generating a data entry 114a in database 102a, where data entry 114a is associated with a log record 112a for recording at least one operation corresponding to data entry 114a. Log record 112a corresponds to a log sequence indicator. Data entry 114a is stored in a table in the database, and the data entry can be a row in the table or can include multiple rows each in a respective table. When a transaction corresponding to at least one operation on data in a table stored in the database is committed, a new row corresponding to the result of the at least one operation is generated and the original entry in the table is maintained for at least some time. In this way, each log record is associated with a data entry 114a in the database, and data entry 114a is generated once one or more operations corresponding to data entry 114a have been recorded in log record 112a. Method 500 includes selecting a snapshot cutoff log sequence indicator at block 520. Selecting the snapshot cutoff log sequence indicator involves reserving a log sequence indicator corresponding to a position in a set of log files such that no log record can be associated with the selected snapshot cutoff log sequence indicator. In some examples, generating the snapshot is performed as a transaction and the corresponding log record is generated in the set of log files and written out to the snapshot, where the snapshot cutoff log sequence indicator is the log sequence indicator of the log record corresponding to the snapshot. Method 500 includes determining the relative order of the log sequence indicator corresponding to the data entry and the snapshot cutoff log sequence indicator at block 530. At block 540, method 500 includes generating a snapshot representing the state of database 102a at the time corresponding to the snapshot cutoff log sequence indicator, where the snapshot includes data entry 114a according to the determined relative order. The log sequence indicator corresponding to the data entry is selected before data entry 114a is generated. Data entry is generated once at least one operation has been recorded in the reserved log record 112a. Selecting the snapshot cutoff log sequence indicator and including data entry 114a according to the relative order between log record 112a corresponding to the data entry and the snapshot cutoff log sequence indicator means that when restoring the state of the database by replaying log records, the system can replay any log records reserved after the snapshot cutoff log sequence indicator. Since there is no need to selectively replay log records, there is no need to select an indicator of the generation time of the associated data entry and store it in the log record before generating the data entry.If a snapshot is generated by selecting the position of a data entry in a database and including the data entry before that position, an indication of the relative position of the associated data entry with respect to the snapshot can be assigned to the corresponding log record. Similarly, if a snapshot is generated by selecting the time at which an entry in the database is generated, an indication of the time at which the associated data entry is generated relative to the selected time can be assigned to the corresponding log record. Since log records are retained before the data entry is generated, and since the database system must keep track of which log records are to be replayed after a crash, the position or the time of generation of a data entry in the database will be selected before the data entry is fully generated. Selecting the time of generation of a data entry before it is fully generated (i.e., committed) can result in a dirty read, where uncommitted data can be read by the end user or an application before it is fully generated.

[0063] Figure 6A Schematically shown is a set 600 of data entries stored in a database 610 at a first time T_1. Each data entry corresponds to one of a plurality of log records stored in a set 620 of log files. The log records are associated with log sequence indicators LSI_1 to LSI_6 corresponding to their positions in the set 620 of log files. As described above, log records can be retained at the start of a transaction, and data entries associated with the log records can be generated at the end of the transaction once one or more operations have been recorded in the corresponding log records. At time T_1, the database includes a set of data entries D_1 to D_5, each data entry being associated with a corresponding log record in the set 620 of log files, as indicated by the arrows shown in dashed lines Figure 6A in the figure. At time T_1, the log record corresponding to the log sequence indicator LSI_5 has been retained, but the transaction has not been committed and thus the corresponding data entry D_6 has not been generated. Data entries can be retained, but they may not be considered generated until they are saved to the database. At time T_1, a snapshot cut-off log sequence indicator S_LSI has been selected. We will now consider which data entries will be included in Figure 6A the snapshot.

[0064] Data entry D_4 was generated before time T_1 and is associated with the log record corresponding to LSI_4. After selecting the snapshot cut-off log sequence indicator S_LSI, the method includes determining the relative order of the log sequence indicator LSI_4 and the snapshot cut-off log sequence indicator S_LSI. If it is determined that LSI_4 is earlier than S_LSI in the set of log files 620, data entry D_4 will be included in the snapshot. In some examples, data entry D_4 may include the log sequence indicator LSI_4, and determining the relative order may include comparing the log sequence indicator LSI_4 in data entry D_4 with the snapshot cut-off log sequence indicator S_LSI.

[0065] The data entry D_6 shown in dashed lines has not been generated at time T_1 (which is the time when the snapshot cut-off log sequence indicator is selected). As described above, the log sequence indicator is selected before the data entry in the database is generated. In Figure 6A it, the log sequence indicator LSI_5 has been selected before the snapshot cut-off log sequence indicator S_LSI is selected but the data entry D_6 has not been generated. If the log record LSI_5 has an earlier order than the snapshot cut-off log sequence indicator S_LSI, the method may include waiting for the generation of the data entry D_6 corresponding to the log record LSI_5 before generating the snapshot. To facilitate this, the database system may maintain a list of one or more active transactions. The entries in the active transaction list are associated with the log sequence indicators of the log records that record the corresponding transactions. In this case, the method includes identifying the entries in the active transaction list that are associated with the log sequence indicators having an earlier order than the snapshot cut-off log sequence indicator and waiting for the identified transactions to complete before generating the snapshot.

[0066] Figure 6B Schematically shown is the set of data entries 600 stored in the database 610 at a later second time T_2. Once the snapshot cut-off log sequence indicator S_LSI is selected, the database system does not block the execution of any transactions and thus continuously adds log records to the set of log files 620. In Figure 6B it, after the snapshot cut-off log sequence indicator S_LSI is selected, the log record corresponding to LSI_8 has been retained, but the corresponding transaction has not been completed. The log records corresponding to LSI_9 and LSI_10 have been generated, as well as the associated data entries D_7 and D_8. Now the transaction corresponding to the log sequence indicator LSI_5 has been completed, and the associated data entry D_6 has been generated. At time T_2, after determining the relative order of the corresponding log sequence indicators of data entries D_1 to D_6 and the snapshot cut-off log sequence indicator S_LSI, a snapshot is then generated, and the snapshot includes data entries D_1 to D_6.

[0067] Reference Figure 7 , in some examples, a data entry includes an indication of the determined relative order of a log sequence indicator and a snapshot cut-off log sequence indicator. This allows the snapshot to selectively include data entries based on the indication of the determined relative order, without having to compare each data entry during the snapshot process. Data entries D_1 through D_8 include such an indication. In Figure 7 , generating data entries D_1 through D_9 involves generating indicators I_1 through I_8 of the relative order of the log sequence indicator and the snapshot cut-off log sequence indicator corresponding to the data entries based on a global reference variable. Selecting the snapshot cut-off log sequence indicator includes selecting the next available log sequence indicator and then immediately modifying the global reference variable. This will reference Figure 7 The explanation of D_6 shown in: First, perform at least one operation corresponding to data entry D_6, but it has not yet been saved to the database. Subsequently, retain the log record corresponding to log sequence indicator LSI_5 and read the value RV_1 from the global reference variable. Then record the at least one operation in the retained log record corresponding to log sequence indicator LSI_5. Then generate data entry D_6 including the value RV_1 read from the global reference value. When the snapshot cut-off log sequence indicator S_LSI is selected, the global reference variable is modified such that it becomes the value RV_2. After the snapshot cut-off log sequence indicator is selected, a transaction for which a log record has been retained and a log sequence indicator has been selected will read the value RV_2 from the global reference variable. For example, in the case where log sequence indicator LSI_9 is selected, the value RV_2 is read from the global reference variable. In this way, any data entry related to a transaction that starts recording its operations after the snapshot cut-off log sequence indicator S_LSI is selected will include a determined relative order indicator different from those of the transactions that started recording their operations before the snapshot cut-off log sequence indicator was selected. By reading the value from the global reference variable immediately after retaining the log record, and then modifying the global reference variable if the snapshot cut-off log sequence indicator S_LSI is selected and before the data entry is fully generated, the data entry will still include the value RV_1. The value RV_1 indicates that the transaction related to the data entry started before the snapshot cut-off log sequence indicator S_LSI was selected. Using the indicator of the determined relative order based on the global reference variable can be less memory-intensive and / or can use less storage space compared to storing the log sequence indicator in the data entry. In examples where there are millions or even billions of data entries, reducing the size of each data entry can be particularly beneficial.

[0068] A set of log files can have millions of log records, and correspondingly, a log sequence indicator can be a large variable, so storing the log sequence indicator uses a large amount of storage space. Therefore, using a snapshot cutoff log sequence indicator and an indicator of the relative order of the log sequence indicators corresponding to data entries may be a more efficient way to quickly determine the relative order when taking a snapshot. Writing the indicator to the data entry when generating the data entry prevents having to use an additional write operation to generate the indicator. A global reference variable can have a value determined from a set of snapshot values, and when a snapshot is generated, the global reference variable is modified such that it changes from a first snapshot value to a second snapshot value. The set of snapshot values can include enough values to represent each snapshot that is currently being stored. When a new snapshot is generated, the oldest snapshot can be deleted and the snapshot value corresponding to the oldest snapshot can be used as the snapshot value for the current snapshot. Reusing the snapshot value when deleting the corresponding snapshot allows the global reference variable to be smaller.

[0069] The indicator of the determined relative order of the log sequence indicator for each data and the snapshot cutoff log sequence indicator can include a first part and a second part. The first part is an indicator of the time when the data entry is generated, and the second part is generated based on the global reference variable. The indicator of the time when the data entry is generated can be a unique value such that no two data entries have the same indicator. When generating a data entry, a transaction ID can be selected to identify the time or logical order when the transaction is completed, and the data entry is generated. These transaction IDs can be generated for other purposes and are repurposed by the current method. By using a combination of the indicator of the time when the data entry is generated and the second part generated based on the reference variable, the data entry can store a small amount of redundant information to provide the functionality described herein. The second part based on the reference variable can be represented by a small portion of data representing a set of values, where the set of values is recycled. The first part may already be stored in the data entry for other purposes.

[0070] In an example, generating a snapshot involves selecting an indicator corresponding to the relative time of the snapshot with respect to the time when the data entry is generated, such as a transaction ID. Including a data entry in the snapshot can depend on determining that the indicator corresponding to the relative time of the snapshot with respect to the time when the data entry is generated indicates that the snapshot will be generated at a time later than the time when the data entry is generated. Alternatively or additionally, including a data entry in the snapshot can depend on determining that the second part generated based on the global reference variable corresponds to the value representing the global reference variable before the global reference variable is modified. The indicator corresponding to the relative time of the snapshot with respect to the time when the data entry is generated is selected before selecting the snapshot cutoff log sequence indicator.

[0071] Figure 8Schematically shows a set of data entries 800, each data entry corresponding to a log record in a set of log files 810. Each data entry in the set of data entries 800 includes an indicator of the determined relative order of the corresponding log sequence indicator and the snapshot cutoff log sequence indicators S_LSI_1, S_LSI_2. The indicator includes a first part ( Figure 8 in the "Transaction" column), which is an indicator of the time when the data entry was generated. For example, the time of the transaction that completed generating the data entry. The indicator also includes a second part ("Rev_Var") generated based on a global reference variable. The indicator of the time when the data entry was generated may not be an indicator of the actual time, but may be an indicator of the logical time in the database system when each subsequent transaction is assigned, or may be used to select a logical time higher than the previous transaction. Figure 8The global reference variable in is a bit with a value of 0 or 1, and modifying the global reference variable involves flipping the value of the bit. At some time before the selection of the snapshot cut-off log sequence indicator S_LSI_1, the global reference variable bit has a value of 0. For a transaction that starts and for which a log sequence indicator has been selected before the selection of the snapshot cut-off log record (e.g., the transaction corresponding to log record LSI_6), reading the global reference variable will result in a bit value of 0. The data entry D_5 generated according to such a transaction will include a bit value of 0. When generating the data entry, the data entry is written to the database using an indicator of the time of generating the data entry (such as the transaction ID). This indicator of the time of generating the data entry also indicates the relative order of the transaction that generated the data entry relative to other transactions. At the same time, or before the selection of the snapshot cut-off log sequence indicator, another indicator T_S_1, T_S_2 of the relative time of the snapshot relative to the time of generating the data entry can also be selected. This indicator is used to determine which transactions have been completed at the time when the snapshot starts. The data entry D_6 corresponding to the log sequence indicator LSI_5 is not generated before the time of selecting the snapshot cut-off log sequence indicator S_LSI_1, and thus is not generated before the indicator T_S_1 of the relative time of selecting the snapshot in this case. Therefore, when generating the data entry D_6, this data entry can have an indicator T_7 of a time after the snapshot time T_S_1 = T_6. However, since the global reference variable is read immediately after retaining the log record corresponding to LSI_5, and since the global reference variable is modified immediately after selecting the snapshot cut-off log sequence indicator S_LSI_1, the snapshot will include the data entry D_6 because the Ref_Var stored in the data entry D_6 is 0. As the system progresses and a further snapshot with a snapshot cut-off log sequence indicator S_LSI_2 is generated, the global reference variable bit value will switch back from 1 to 0. The data entries generated according to the transactions that start after the selection of the second snapshot cut-off log sequence indicator will have the same Ref_Var as those that started before the generation of the first snapshot cut-off log sequence indicator S_LSI_1. However, as discussed above, generating a snapshot involves waiting for the data entries corresponding to the log records associated with the log sequence indicators whose order is earlier than the snapshot cut-off log sequence indicator S_LSI_1 to be generated in the database before generating the first snapshot. Therefore, the data entries corresponding to the log records retained and / or generated before the selection of the first snapshot cut-off log sequence indicator will have an indicator of the time of generating the data entry, which corresponds to a time earlier than the relative time T_S_2 of the second snapshot.

[0072] In an example of restoring the state of a database at a time after the time corresponding to the snapshot cutoff log sequence indicator, the database can be restored by restoring the state of the database from the snapshot and replaying any log records retained after selecting the snapshot cutoff log sequence indicator. This improves the efficiency of replaying log records because the database system can replay any log records generated after selecting the snapshot log sequence indicator. The database system does not have to selectively replay log records based on further determining whether the log records correspond to data entries included in the snapshot.

[0073] Copying Binary Large Objects

[0074] The database system 100 can copy the first database 102a at the second database 102b, for example, in a case where the two databases 102a and 102b are a primary database and a secondary database, respectively. This allows users of the database to query the data stored in the database more efficiently without burdening the first database 102a. Having the second database 102b as a replica of the first database 102a also provides a backup. Copying the first database at the second database includes initializing a snapshot of the first database 102a at the second database 102b and sending log records 112a corresponding to operations that have been performed on the data stored at the first database 102a for replay on the second database 102b. Then, the second database replays the log records recorded after the snapshot was generated. When the log records are written to the set of log files 110b at the second database 102b, the transactions corresponding to the log records are executed. Sending the log records to the second database 102b can include sending the log records to be replayed at the second database 102b. The log records may not actually be received by the second database 102b, but may be received elsewhere and written to the set of log files 110b corresponding to the second database 102b at a later time. The combination of the set of log files, the database, and any number of snapshots can generally be referred to as a database. Binary large objects are typically large files, including images, sounds, videos, or other multimedia files, and are stored as a collection of binary data. It can be difficult to process binary large objects in a database because of the relatively lack of associated classification information compared to the data in the tables stored in the database. Binary large objects do not have any specific size. In some cases where the binary large object is a large object, the size of the binary large object further exacerbates the processing of the binary large object. However, in other cases, the binary large object can be a relatively small file.

[0075] Figure 9A flowchart of method 900 for copying binary large objects stored at a first database to a second database is shown. The first database is associated with a set of log records. The set of log records corresponds to operations performed on data stored at the first database. As described above, log records may include data indicating a transaction, where a transaction is one or more operations performed on data stored at a database. Operations performed on data stored at the first database include reading, writing, or modifying data stored at the first database, including writing new data to the database. The database system may receive a query from an end user or application accessing the database system, where the query specifies one or more transactions to be performed. For example, the query may be received in the form of suitable computer code, such as in the form of Structured Query Language (SQL) or any other suitable computer code.

[0076] Method 900 includes sending, at block 910, the set of log records corresponding to operations performed on data stored at the first database to the second database. Sending the set of log records to the second database includes sending the set of log records to be written to a set of log files associated with and / or stored at the second database. The set of log files associated with and / or stored at the second database is used to record operations performed on data stored at the second database.

[0077] At block 920, method 900 includes identifying log records of a set of log records that include an indication of a binary large object stored at a first database. When a binary large object is generated at the first database, a special log record is generated that includes data indicating that the binary large object has been generated at the first database. The special log record can be stored together with log records corresponding to operations performed on row-store data stored in the first database. When sending the log records to a second database, the special log record is identified. At block 930, method 900 includes, in response to identifying a log record that includes an indication of a binary large object stored at the first database, sending the binary large object stored at the first database to the second database. Sending the set of log records involves reading a set of log files into memory, identifying log records in the set of log files that include an indication of a binary large object stored at the first database, and then sending the set of log files and the binary large object to the second database. Sending the binary large object to the second database in response to identifying a log record that includes an indication of the binary large object allows the binary large object to be generated at the second database when writing the identified log record to a set of log files associated with and / or stored at the second database. A transaction corresponding to a log record that includes an indication of a binary large object stored at the first database cannot be committed at the second database until the second database has received the binary large object corresponding to the log record. In the case where the log record corresponds to a transaction for generating a binary large object, the transaction can be committed only when the second database has received the binary large object and the transaction has been recorded in a log file corresponding to the second database. Sending the binary large object in response to identifying the log record reduces the time that the database system must wait before committing a transaction corresponding to the identified log record at the second database. This increases the speed of replicating the first database at the second database. In the case where the binary large object and the log record are sent independently to the second database, the log record corresponding to the binary large object can be received at the second database, but the corresponding transaction cannot be committed until the binary large object has also been received at the second database, and thus the database system will have to maintain an active transaction until the binary large object is received.

[0078] Figure 10A A first database 1000a is schematically illustrated, and a first set of log files 1002a includes a plurality of log records, each log record corresponding to log sequence indicators LSI_1 through LSI_9. A binary large object BLOB_5 is stored at the first database 1000a. At Figure 10AIn [the figure], the log file set 1002a and the binary large object BLOB_5 are shown as stored in the first database 1000a, but it should be understood that they may not be stored within the first database 1000a. For example, the binary large object BLOB_5 may be stored separately from the first database 1000a in a separate directory or file system, but is associated with the database 1000a, such as in the case where the database includes data identifying the binary large object BLOB_5. Figure 10A The second database 1000b and the second log file set 1002b are also shown. The second database 1000b may be configured to replicate the first database 1000a. Figure 10A It may be related to the first time of sending the log record set in the log file set 1002a to the second database 1000b. In Figure 10A [the figure], the log record including an indication of the binary large object BLOB_5 is the log record corresponding to the log sequence indicator LSI_5.

[0079] The log record set corresponding to the operations performed on the data stored at the first database 1000a is sent to the second database 1000b in sequence, and sending the binary large object BLOB_5 to the second database includes inserting the binary large object BLOB_5 into the sequence after the log record including an indication of the binary large object BLOB_5. Inserting the binary large object into the sequence of the log file set may involve using the same process or sending the binary large object BLOB_5 as part of a transaction, which includes sending the log record set to send BLOB_5.

[0080] As Figure 10A shown, the binary large object BLOB_5 may be inserted into the sequence immediately after the log record including an indication of the binary large object stored at the first database. Inserting the binary large object BLOB_5 into the sequence immediately after the log record allows the binary large object to be received at the second database 1000a immediately after the log record including an indication of the binary large object (corresponding to the log sequence indicator LSI_5). This reduces the time taken to write the log record to the second log file set 1002b and commit the transaction corresponding to the log record.

[0081] The binary large object BLOB_5 stored at the first database is associated with the log sequence indicator LSI_5 in a table or file. Associating the binary large object with the log sequence indicator LSI_5 enables the system to locate the binary large object BLOB_5 after identifying the log record including an indication of the binary large object BLOB_5.

[0082] Figure 10BShows a process of sending a binary large object BLOB_5 to a second database in one or more parts. Each part is associated with a corresponding indicator including a first part and a second part. The first part is associated with a log sequence indicator LSI_5 corresponding to a log record including an indication of the binary large object BLOB_5. The second part indicates a part of the binary large object. More specifically, Figure 10B Shows sending the binary large object BLOB_5 to the second database in multiple parts 1004. Each part in the multiple parts 1004 is associated with an indicator having a first part LSI_5 and a second part indicating a part of the binary large object BLOB_5. The binary large object BLOB_5 is sent as one or more pages, and each page is associated with a two-part indicator as described above. By sending the binary large object as one or more parts (each part including an indicator having a first part and a second part), one or more parts of the binary large object can be sent to the second database in sequence in reverse order, or other log records can be sent between one or more parts of the binary large object.

[0083] Method 900 may also include reserving a portion of the storage space at the second database to store log records. For example, once a log record is identified, one or more pages in the log file set 1002b can be reserved to store the log record. This can be performed before sending the log record to the second database, such that the location and resources for storing the log record in the second log file set 1002b are reserved before sending the log record. Method 900 may also include generating a file for storing the binary large object at the second database 1000b before sending the binary large object BLOB_5 to the second database. The file at the second database 1000b can be associated with the binary large object BLOB_5. Generating the file includes reserving sufficient pages to store the binary large object BLOB_5 at the second database 1000b. This ensures that when the binary large object BLOB_5 and the log record are received at the second database 1000b, the resources for storing the binary large object BLOB_5 and the associated log record at the second database 1000b are available.

[0084] Method 900 may also include receiving each of one or more parts 1004 of the binary large object BLOB_5 at the second database 1000b and storing each of the one or more parts at the second database according to the corresponding indicator. The corresponding indicator has a first part and a second part, the first part is associated with a log sequence indicator and the second part indicates a part of the binary large object. The file generated at the second database can be associated with the log sequence indicator LSI_5 such that each part of the binary large object BLOB_5 is written to the file based on the indicator of the corresponding part.

[0085] Method 900 may also include storing a binary large object at a second database and maintaining an association between the binary large object and a log sequence indicator. The binary large object is stored and associated with the log sequence indicator of a log record that logs an operation to generate the binary large object by maintaining the log sequence indicator in a file that includes the binary large object. Alternatively, a table may be used to store the association between the log sequence indicator and the binary large object stored at the second database. This allows the binary large object to be easily located for deletion or replication when sending the log records in the log file set 1002b of the second database 1000b to another database. When writing data corresponding to a log record to the log file set 1002b of the second database, other log records may be generated in the log file set 1002b having log sequence indicators different from those of the log records in the first set 1002a. Accordingly, the binary large object BLOB_5 may be stored at the second database 1000b and associated with a log sequence indicator different from LSI_5.

[0086] The binary large object can be stored in a directory at the second database 1000b according to at least a portion of the log sequence indicator. In the case where more than one binary large object is stored or to be stored at the second database 1000b, storing the binary large objects according to at least a portion of their respective log sequence indicators allows for easy location of the binary large objects. When an operation is performed on a binary large object or when it is copied to a database, the relative binary large object can be easily located. When the second database 1000b is copied to a third database, the log record set 1002b can be sent to the third database, and once a log record including an indication of the binary large object is identified, the database system uses the log sequence indicator to locate the directory at the second database where the binary large object is stored. Once the binary large object is identified in its respective directory, it is then sent to the third database. The binary large objects can be stored in a multi-level directory at the second database 1000b, where each level of the multi-level directory corresponds to a respective portion of the log sequence indicator. For example, the first level of the multi-level directory can be associated with the first portion of the log sequence indicator, such as the first portion of a string, an integer, or any other suitable variable used to record the log sequence indicator. The log sequence indicator can typically be an incrementing value, and thus the first portion of the log sequence indicator can be used to define the first level of the multi-level directory and the second portion of the log sequence indicator can be used to define the second level of the multi-level directory. This stores the binary large objects efficiently such that the binary large objects can be located without having to access a single directory that includes all binary large objects and traverse the binary large objects therein. In the case where a large number of binary large objects are stored in a database, locating the binary large object associated with a log record can be non-trivial, and thus grouping the binary large objects according to their log sequence indicators provides an efficient way to sort the binary large objects. Storing the binary large objects in a multi-level directory as described above can also simplify the log truncation process, where when the log is truncated, the binary large objects referenced by the log records deleted during the log truncation should also be deleted. Instead of scanning the log records and deleting the binary large objects referenced by the log records, the binary large objects associated with the log sequence indicators below the truncation log sequence indicator can be deleted based on the directory where they are stored. Here, the truncation log sequence indicator indicates the position in the log file set below which all log records are to be truncated. In the presence of a large number of binary large objects and associated log records, this process is more efficient than scanning the truncated portion of the log file set and deleting the binary large objects indicated by the log records therein.

[0087] A log record including data indicating that a binary large object is stored at a first database may include a checksum generated from the binary large object. This allows the database system to check that the binary large object received at a second database matches the binary large object indicated by the log record. If the checksum corresponding to the binary large object is stored in the corresponding log record of the binary large object, the checksum does not need to be stored in the file containing the binary large object, or in the file name of that file.

[0088] The identified log record may include data indicating the size of the binary large object, and the method may include reserving space for one or more portions of the binary large object in a sequence for sending the binary large object based on the data indicating the size of the binary large object. Reserving sufficient space in the sequence for sending the binary large object ensures the reliability of sending the binary large object, since space is reserved in the sequence before sending the binary large object. The data indicating the size of the binary large object in the identified log record can be used to reserve a portion of the storage space at the second database to store the binary large object.

[0089] Method 900 may also include sending an indication of one or more portions of the binary large object to the second database. After one or more portions of the binary large object are received at the second database, an indication that one or more portions of the binary large object have been received is generated. A log record including an indication of the binary large object may include data indicating the size of the binary large object, and the number of pages for the binary large object may be determined based on the size of the binary large object and the size of a page in the database system. An indication is generated once all the pages of the binary large object to be received by the second database have been received. This allows the system to ensure that the binary large object is received at the second database. The indication can also be used to determine whether to allow a transaction corresponding to replicating the binary large object at the second database to be committed, where a transaction corresponding to replicating the binary large object cannot be committed until the binary large object is received and / or stored at the second database.

[0090] A log record including an indication of a binary large object stored at a first database may be made invalid or corrupted during storage or when the log record is sent to the second database. Thus, identifying a log record including an indication of a binary large object may include identifying an invalid log record and determining whether there is a binary large object associated with the log sequence indicator of the invalid log record.

[0091] Deleting a binary large object

[0092] When data is deleted from a database in a database system, the data is initially logically deleted, e.g., in response to a request from a user to delete the data or when undoing a transaction corresponding to the generation of the data. Logically deleting the data includes making the data inaccessible to end users or applications querying the database. Logically deleted data can still be maintained at the database for a period of time to enable the completion of pending transactions that depend on the data, such as when copying the data to another database. Logically deleted data can also be maintained for a period of time so that the database can be returned to the state it was in before the data was logically deleted. After a period of time, the data that has been logically deleted can be physically deleted, i.e., permanently removed from the database. The data is not physically deleted until a snapshot including the data is deleted and the log records referencing the data are deleted or invalidated. This helps ensure that it is appropriate to physically delete the data. As described above, a snapshot can represent the state of the database at a time corresponding to a snapshot cutoff log sequence indicator. A log record can be considered invalid if the order of the log sequence indicator corresponding to the log record is lower than the snapshot cutoff log sequence indicator of the oldest snapshot representing the state of the database at a given time.

[0093] Figure 11 A flowchart of a method 1100 for physically deleting one or more binary large objects from a database is shown, where the database has multiple states, and a first of the states of the database at a first time is represented by a first snapshot. The first snapshot is generated based on the above-described method 500. The first snapshot includes a copy of the data stored at the database at the first of the states. The first of the states of the database is the state of the database after transactions corresponding to log records in a log file set having an order earlier than the snapshot cutoff log sequence indicator have been committed. In some examples, generating a snapshot can include storing data from a first database in a format different from the format in which the data is stored in the database. The data stored at the database can be serialized into the snapshot, where serialization is the process of converting a data structure or object into a format that can be stored.

[0094] Method 1100 includes generating, at block 1110, a second snapshot representing a database state at a second time, the second time being later than a first time, and the second snapshot including data identifying one or more binary large objects that have been logically deleted from the database prior to the second time. When one or more binary large objects are logically deleted from the database, data identifying the one or more binary large objects that have been logically deleted is stored. The data identifying the one or more binary large objects that have been logically deleted can be stored in any suitable format. As described above, a binary large object can be associated with a log sequence indicator corresponding to a log record of a transaction performed to create the binary large object. Thus, storing the data identifying the binary large object that has been logically deleted can include storing the associated log sequence indicator in a list or table of logically deleted binary large objects. When a binary large object is logically deleted from the database, a log record can be generated that includes data indicating the deletion of the binary large object. Storing the data identifying the one or more binary large objects that have been deleted prior to the second time can include storing a list of log sequence indicators associated with the binary large objects that have been logically deleted prior to the second time. The list can be serialized into the second snapshot such that it is stored in a suitable storage format.

[0095] Method 1100 includes deleting, at block 1120, the first snapshot. Snapshots are periodically deleted from the database system so that storage space can be freed for subsequent snapshots or other uses. The database system can maintain multiple snapshots and periodically generate new snapshots. A new snapshot can be generated in response to a predetermined number of transactions that have been performed since the previous snapshot. The database system performs a snapshot cleanup process in which one or more of the oldest snapshots of the multiple snapshots are deleted, which can be performed periodically or in response to a predetermined event. At block 1130, method 1100 includes, after deleting the first snapshot, physically deleting one or more binary large objects from the database using the data identifying the one or more binary large objects that have been logically deleted prior to the second time. As described above, deleting the oldest snapshot invalidates the log records corresponding to the log sequence indicators between the snapshot cutoff log sequence indicator of the deleted oldest snapshot and the snapshot cutoff log sequence indicator of the next oldest snapshot in the log file set. Thus, it is allowed to physically delete the binary large objects that have been logically deleted between the oldest snapshot and the next oldest snapshot after deleting the oldest snapshot. Storing the data identifying the one or more binary large objects that have been logically deleted prior to the second time in the second snapshot allows the first snapshot to be safely deleted while still being able to locate the one or more binary large objects to be physically deleted. Grouping and storing the data identifying the one or more binary large objects that have been logically deleted in the second snapshot allows the binary large objects to be efficiently identified and permanently deleted.

[0096] Figure 12 Schematically shows a first snapshot S_1 representing the state of the database 1200 at the first time t_1. At the time t_1, the binary large objects B_1, B_2, B_3 have not been logically deleted from the database 1200. Figure 12 Shows a second snapshot S_2 representing the database 1200 at a second time t_2 after the first time. At the second time t_2, the binary large object B_1 has been logically deleted from the database and the second snapshot S_2 includes data identifying the binary large object B_1 that has been deleted prior to the second time. Subsequently, at some time after t_2, the first snapshot S_1 is deleted. After deleting the first snapshot S_1, the data identifying the binary large object B_1 stored in the second snapshot S_2 is used to delete the binary large object B_1.

[0097] As in the embodiments discussed above with respect to Figures 5 to 8 The database 1200 maintains multiple snapshots and periodically generates new snapshots and deletes old snapshots. The method 1100 may include deleting the second snapshot S_2 based on determining that all one or more binary large objects that have been logically deleted prior to the second time t_2 have been physically deleted. This ensures that the binary large object B_1 that can be physically deleted has been physically deleted before deleting the snapshot S_2 that stores a pointer to the binary large object B_1. The second snapshot S_2 may be where an indication of one or more binary large objects B_1 that have been logically deleted prior to the second time t_2 is stored. If the second snapshot S_2 is deleted before ensuring that all one or more binary large objects that have been logically deleted prior to the second time have been physically deleted, the binary large objects that should have been physically deleted will remain at the database, exhausting storage space.

[0098] Method 1100 may also include generating a third snapshot representing the database state at a third time, the third time being later than the second time t_2, and the third snapshot including data identifying one or more binary large objects that have been logically deleted from the database before the third time. The third time may be before the first snapshot is deleted and before the one or more binary large objects that have been logically deleted before the second time t_2 are physically deleted. Alternatively, the third time may be after the first snapshot has been deleted and after the one or more binary large objects that have been logically deleted before the second time t_2 have been physically deleted. The third snapshot may include data identifying binary large objects that have been deleted before the second time and before the first time. In this way, each subsequent snapshot may save a cumulative record of binary large objects that have been logically deleted, such that if they are not successfully deleted at the appropriate time, the database system can locate and delete them later. Deleting binary large objects at the appropriate time includes: deleting a binary large object if the oldest snapshot stored at the database system represents the database state at a time after the binary large object has been logically deleted. For example, this is done once the log record corresponding to the transaction including the logically deleted binary large object is invalid. Deleting the oldest snapshot and then deleting the binary large object (if it has been logically deleted before the next oldest snapshot is generated) allows for efficient deletion of binary large objects once it is appropriate to do so. The thread responsible for deleting snapshots may also perform the deletion of binary large objects that have been logically deleted after the time corresponding to the deleted snapshot and before the time corresponding to the next snapshot. For example, a transaction including the deletion of a snapshot may also include permanently deleting binary large objects that have been logically deleted after the deleted snapshot is generated and before the next snapshot is generated. This reduces the number of individual processes or threads running on the database system and allows for the reallocation of storage space used for storing binary large objects whenever it is logically correct to do so.

[0099] The second snapshot may include data identifying one or more binary large objects that have been logically deleted after a first time t_1 and before a second time t_2. Storing data identifying binary large objects that have been logically deleted between two snapshots can be a more efficient way of storing data indicating binary large objects that have been logically deleted. This is because each snapshot may store only data identifying a subset of all binary large objects that have been logically deleted. Method 1100 may include deleting the first snapshot S_1 based on determining that the second snapshot S_2 has been generated. Deleting the first snapshot S_1 after the second snapshot S_2 has been generated ensures that data identifying one or more binary large objects B_1 that have been logically deleted before the second time t_1 is stored in the second snapshot S_2 for deleting the one or more binary large objects B_1 before the first snapshot S_1 is deleted.

[0100] Figure 13 A set of log files 1300 is schematically illustrated, which includes a plurality of log records for recording operations performed on data stored at a database, a second snapshot S_2 representing the state of the database at a time corresponding to a snapshot cut-off log sequence indicator S_LSI_2, and a first type of log record 1310. The data stored in the second snapshot identifying one or more binary large objects B_1 that have been logically deleted before the second time includes data corresponding to the first type of log record 1310. In some examples, the first type of log record 1310 may include data indicating one or binary large objects that have been logically deleted before the second time. For example, the file name of a file including a binary large object, or a log sequence indicator corresponding to a log record generated when the binary large object was generated. In other examples, the first type of log record 1310 may include an indication of one or more other log records, as will be described later. Generating the second snapshot S_2 may include serializing the first type of log record 1310 to be stored in the second snapshot S_2. Alternatively, generating the second snapshot S_2 may include storing a log sequence indicator corresponding to the first type of log record 1310 in the snapshot S_2 such that the log record 1310 can be located based on the data stored in the second snapshot S_2. The first type of log record 1310 may not be stored in the set of log files 1300, but may be specifically generated to be serialized into the snapshot S_2.

[0101] Generate a first type of log record 1310 when generating the second snapshot. For example, once the generation of snapshot S_2 has started but before the snapshot cut-off log sequence indicator S_LSI_2 is selected, the first type of log record 1310 can be generated. In other examples, after the snapshot cut-off log sequence indicator S_LSI_2 for determining the time of snapshot S_2 is selected, the first type of log record 1310 can be generated. The log record 1310 includes data identifying one or more binary large objects that have been logically deleted before the selected snapshot cut-off log sequence indicator S_LSI_2. In such examples, the log record 1310 will occur after S_LSI_2 at Figure 13 After the snapshot cut-off log sequence indicator S_LSI_2 has been selected, the generation of the first type of log record 1310 can be performed before any other binary large objects are logically deleted. The generation of the first type of log record 1310 can be performed as part of generating snapshot S_2, such as part of the same transaction corresponding to generating snapshot S_2.

[0102] Figure 14A set of log files 1400 is schematically shown, which includes a plurality of log records including log records of a first type 1410, a first snapshot S_1, a second snapshot S_2 including data corresponding to the log records of the first type, and a first table 1420. The log records of the first type 1410 can be from one or more entries in the first table 1420 at a database and delete the one or more entries in the table 1420 after generating the log records of the first type 1410. The table 1420 initially includes an entry identifying a binary large object B_1 that has been logically deleted before a second time, and deletes the entry corresponding to B_1 once the log records of the first type 1410 are generated. Generating the log records 1410 can be triggered by other periodic processes in the database system, or a dedicated thread can control the periodic generation of the log records of the first type. The first table 1420 can be periodically updated by adding entries that identify binary large objects that have been deleted since the table was previously updated. Storing data identifying more than one binary large object that has been logically deleted in one log record 1410 provides a more efficient way to store data in the second snapshot S_2, which indicates binary large objects that have been logically deleted and will be physically deleted after or during the deletion of the first snapshot S_1. Preferably, one or more entries in the first table 1420 are deleted immediately after generating the log records of the first type 1410. Deleting the entries in the table 1420 immediately after generating the log records 1410 means that during subsequent snapshots when generating other log records of the first type, the other log records of the first type will include data corresponding to binary large objects that have been deleted after the second snapshot S_2. This also ensures that all entries deleted from the table 1420 are included in the log records of the first type 1410.

[0103] In some examples, at least one entry in the first table includes an indication of a corresponding log record of a second type, which includes data identifying one or more binary large objects that have been deleted before a second time. Figure 15A set of log files 1500 is schematically shown, which includes a plurality of log records, log records of a first type 1510, a first snapshot S_1, a second snapshot S_2 including data identifying one or more binary large objects that have been deleted before a second time, a first table 1520, and log records of a second type 1530. The log records of the second type 1530 include data identifying one or more binary large objects that have been logically deleted. The log records of the second type 1530 can be generated in response to logically deleting a binary large object (such as B_1) from a database. Alternatively, the log records of the second type, such as log records 1530, can be generated periodically. The log records of the second type can include data identifying binary large objects that have been logically deleted since the generation of the previous log record of the second type. By storing data identifying one or more log records of the second type in the log records of the first type, the log records of the first type can include a relatively small amount of data that can be used to locate a large number of binary large objects to be physically deleted. This reduces the size of the second snapshot S_2 while still maintaining the information necessary to locate and physically delete binary large objects.

[0104] When deleting the first snapshot, the invalid part of the set of log files based on the deletion of the first snapshot S_1 is not immediately deleted. The invalid part of the log file is used together with the data corresponding to the log records of the first type 1510 stored in the second snapshot S_2 to locate one or more binary large objects that have been logically deleted before the second time to physically delete the binary large objects. For example, the data corresponding to the log records of the first type 1510 can be used to locate one or more log records of the second type, such as log records 1530, that identify one or more binary large objects that have been logically deleted before the second time. After physically deleting the binary large objects based on one or more log records of the second type 1530, the invalid log records can then be deleted.

[0105] In some examples, one or more log records of a second type are generated from a second table, and one or more entries in the second table include indicators of binary large objects that have been logically deleted before a second time. After generating the log records of the second type, the entries in the second table used to generate the log records of the second type are deleted from the second table. This process can be performed during a snapshot process that occurs on a primary database, the primary database being linked to the database on which method 1100 is executed, such as in the case where the database is a slave database configured to replicate from the primary database. As a result, the number of threads or processes that need to run on the database system is reduced. This allows data identifying binary large objects that have been logically deleted and are to be physically deleted to be grouped and stored, thereby reducing the size of the storage space required for snapshot S_2. Deleting the entries in the second table after the entries are used to generate the log records of the second type prevents duplicate recording in the log records of the second type of the indicators of binary large objects that have been logically deleted. It also prevents the second table from growing too large and thus makes the generation of the log records of the second type slower. The entries in the second table can include data that can be used to locate and identify the binary large objects. For example, data indicating the size, checksum, and / or log sequence indicator corresponding to the log record generated for recording the corresponding binary large object.

[0106] Figure 16 Schematically shown is a set of log files 1600, which includes a plurality of log records including log records 1610 of a first type and log records 1630 of a second type, a first snapshot S_1, a second snapshot S_2, a first table 1620, and a second table 1640. The second table 1640 includes data identifying binary large objects that have been logically deleted in the slave database. The second table 1640 can include a list of log sequence indicators corresponding to the log records used to record the operations of generating the binary large objects. As described above, after storing the binary large objects, the association between the binary large objects and the corresponding log sequence indicators can be maintained, and the association is subsequently used to locate the binary large objects for deletion.

[0107] In some examples, more than one entry of the second table 1640 is used to generate the log records 1630 of the second type; for example, the log records 1630 can include data corresponding to a plurality of entries in the second table 1640, such as a plurality of log sequence indicators corresponding to binary large objects that have been logically deleted.

[0108] One or more other entries in the second table 1640 may be related to ongoing operations performed on binary large objects at the database. For example, one or more transactions related to operations on binary large objects and that have not been committed. Accordingly, the method 1100 may involve identifying entries in the second table 1640 that are related to binary large objects that have been logically deleted based on a comparison of the entries in the second table 1640 with data indicating one or more ongoing operations at the database. An operation may be considered ongoing if the transaction specifying the operation has not been committed. The second table may be used to store indications of ongoing operations performed on binary large objects and binary large objects that have been logically deleted. Comparing the entries in the second table with data indicating one or more ongoing operations at the database may include comparing the entries in the table with a list or set of active transactions that have not been committed. A binary large object will be considered to have been logically deleted if the binary large object corresponds to an entry in the second table 1640 indicating that it has been logically deleted or if the binary large object corresponds to an entry in the table 1640 that is not associated with an active transaction.

[0109] Figure 17 is a schematic diagram of an exemplary apparatus 1700 configured with software to perform the functions described herein. The apparatus 1700 has a computer-readable medium 1710 and a processor 1730. The computer-readable medium 1710 includes instructions 1720 that, when executed by the processor 1730, cause the processor 1730 to perform one or more of the previously described methods, namely method 200; method 500; method 900; and method 1100. The exemplary apparatus 1700 may be a single device or may include multiple devices that are stored locally or remotely from each other and are configured to communicate via any suitable wired or wireless device. References in the specification to "example" or similar language mean that a particular feature, structure, or characteristic described in connection with the example is included in at least that example, but not necessarily in other examples.

[0110] The examples described herein may have particular application to a scale-out architecture database system. A scale-out database system is a database system in which increased demand is met by adding new hardware resources. A scale-out architecture can increase capacity in response to a workload by provisioning a cluster of commodity hardware. A scale-out database system related to the examples herein can include one or more clusters. Each cluster includes at least one aggregator node and at least one leaf node. The aggregator node processes metadata related to the database system, routing queries, and the aggregated results of queries. The leaf node stores data in the cluster and executes queries issued by the aggregator node. The leaf nodes can be partitioned, where each partition in a leaf node is a database. To maintain persistence in the database system, the partitions in the leaf nodes can be arranged as a primary database and at least one secondary database. In such an arrangement, one or more secondary databases are configured to replicate the primary database.

[0111] The number of aggregator nodes and leaf nodes in a cluster determines the storage size and performance of that cluster. An application with larger storage requirements can have a higher ratio of leaf nodes to aggregator nodes than a more general application. An application with higher connection capacity requirements can have a higher ratio of aggregator nodes to leaf nodes than a more general application. Increasing the size of a database system using a scale-out architecture allows an administrator to add new nodes to the cluster and rebalance the data stored in the cluster through online operations without shutting down parts of the database system. Depending on how the database system needs to grow, the database system can be scaled in an appropriate manner. In the case where the number of queries increases, the amount of provisioned CPU and RAM can be increased. If the data range is expanding but the amount of data processing or queries does not increase significantly, storage can be increased. In other cases, such as when the number of objects stored in the database increases and thus increased processing and storage are required, a distributed scale-out architecture can allow new machines to be provisioned and thus more CPU, RAM, and storage space to be provisioned.

[0112] Improving the speed and reliability at which a database can be replicated (e.g., replicating binary large objects) is particularly important when considering provisioning new leaf nodes in a scale-out architecture and during replication between partitions in a node. Additionally, improving the reliability of generating snapshots (as in the snapshot design section) allows partitions in each leaf node to quickly generate reliable snapshots. Being able to load a snapshot and replay any log records that have been retained after selecting a snapshot cutoff log sequence indicator when replicating or reprovisioning a node or a partition in a node can improve the efficiency of replication or reprovisioning. Improving the efficiency and reliability of replication in a distributed database system improves the overall persistence of the database system as well as the system's ability to maintain ACID properties.

[0113] Processing deletions of binary large objects in a database system, such as a distributed scale-out database system, allows for the release of provisioned resources when they are no longer needed. By deleting data that is no longer needed for storage, the storage requirements of leaf nodes in a cluster are reduced, or in some cases the number of leaf nodes maintained is decreased, improving the efficiency of the database system and allowing resources to be redeployed to other areas of the database system where they are needed.

[0114] The above examples should be understood as illustrative. It should be understood that any feature described in connection with any one example can be used alone, or in combination with other features described, and can also be combined with one or more features of any other example or any combination of any other examples. In addition, equivalents and modifications not described above can also be employed.

[0115] Numbered clauses

[0116] The following numbered clauses describe various embodiments of the present disclosure.

[0117] 1. A computer-implemented method for managing a log file for recording operations on data stored in a database, the operations including reading or writing data to the database, the method comprising:

[0118] Updating a set of log files by writing data indicating one or more operations performed on data stored in the database to the set of log files, the set of log files having a first storage portion allocated thereto;

[0119] Monitoring the first storage portion when updating the set of log files; and

[0120] Allocating a second storage portion to the set of log files when updating the set of log files based on determining that an available portion of the first storage portion is below a predetermined size.

[0121] 2. The computer-implemented method according to clause 1, wherein monitoring the first storage portion when updating the set of log files includes periodically determining the size of the available portion of the first storage portion.

[0122] 3. The computer-implemented method according to clause 1, wherein monitoring the first storage portion when updating the set of log files includes determining the size of the available portion of the first storage portion in response to receiving a request to perform one or more operations on data stored in the database.

[0123] 4. The computer-implemented method according to clause 1, wherein the predetermined size depends on the rate of operations performed on data stored in the database.

[0124] 5. The computer-implemented method according to clause 1, which includes updating a tracking file, the tracking file including indicators of a portion of the set of log files to indicate the most recently updated portion of the set of log files.

[0125] 6. The computer-implemented method according to clause 5, wherein updating the tracking file is performed periodically.

[0126] 7. The computer-implemented method according to clause 5, wherein writing data indicating one or more operations performed on the data stored in the database to the set of log files includes generating one or more log records in the set of log files, each log record corresponding to a log sequence indicator indicating a relative sequence in the set of log files.

[0127] 8. The computer-implemented method according to clause 7, wherein the indicator of a portion of the set of log files corresponds to a log record in the set of log files.

[0128] 9. The computer-implemented method according to clause 7, wherein updating the indicator of a portion of the set of log files includes using the log sequence indicator corresponding to the most recently generated log record as at least a portion of the indicator of a portion of the set of log files.

[0129] 10. The computer-implemented method according to clause 7, wherein the indicator of a portion of the set of log files is updated in response to generating a log record.

[0130] 11. The computer-implemented method according to clause 7, wherein the indicator of a portion of the set of log files is updated in response to generating each log record.

[0131] 12. The computer-implemented method according to clause 8, wherein updating the indicator of a portion of the set of log files includes selecting a log sequence indicator higher than the log sequence indicator corresponding to the most recently generated log record to be used as at least a portion of the indicator of a portion of the set of log files.

[0132] 13. The computer-implemented method according to clause 1, wherein the size of the second storage portion depends on the rate of operations performed on the data stored in the database.

[0133] 14. The computer-implemented method according to clause 1, wherein the size of the second storage portion is determined in response to receiving a request to perform one or more operations on the data stored in the database.

[0134] 15. The computer-implemented method according to clause 1, wherein allocating the second storage portion includes reallocating a portion of the first storage portion that has been allocated to an updated subset of the set of log files.

[0135] 16. The computer-implemented method according to clause 5, wherein the tracking file is a first tracking file having a first indicator of a portion of the set of log files, and the computer-implemented method includes alternately updating the first tracking file having the first indicator of the portion of the set of log files and updating a second tracking file having a second indicator of the portion of the set of log files.

[0136] 17. The computer-implemented method according to clause 16, wherein the computer-implemented method includes updating a plurality of the first and second tracking files.

[0137] 18. A non-transitory computer-readable storage medium that includes computer-readable instructions that, when executed by a processor, cause the processor to perform the method according to clause 1.

[0138] 19. A database system that includes:

[0139] at least one processor; and

[0140] at least one memory that includes computer program code, the at least one memory and the computer program code being configured to, with the at least one processor, cause the database system to perform the method according to clause 1.

[0141] 20. A computer-implemented method for generating a snapshot representing the state of a database at a given time, the method including:

[0142] generating a data entry in the database, the data entry being associated with a log record for recording at least one operation corresponding to the data entry, the log record corresponding to a log sequence indicator;

[0143] selecting a snapshot cutoff log sequence indicator;

[0144] determining a relative order of the log sequence indicator and the snapshot cutoff log sequence indicator; and

[0145] generating a snapshot representing the state of the database at the time corresponding to the snapshot cutoff log sequence indicator, wherein the snapshot includes the data entries according to the determined relative order.

[0146] 21. The computer-implemented method according to clause 20, which includes selecting the log sequence indicator of the log record before generating the data entry.

[0147] 22. The computer-implemented method according to clause 21, wherein after selecting the snapshot cutoff log sequence indicator, if the log record indicator has an order earlier than the snapshot cutoff log sequence indicator, the method includes waiting for the generation of the data entry corresponding to the log record in the database before generating the snapshot.

[0148] 23. The computer-implemented method according to clause 21, wherein the data entry includes the log sequence indicator and determining the relative order includes comparing the log sequence indicator in the data entry with the snapshot cutoff log sequence indicator.

[0149] 24. The computer-implemented method according to clause 21, wherein the data entry includes an indicator of the determined relative order of the log sequence indicator and the snapshot cutoff log sequence indicator.

[0150] 25. The computer-implemented method according to clause 24, wherein generating the data entry includes generating an indicator of the relative order of the log sequence indicator and the snapshot cutoff log sequence indicator based on a global reference variable, and selecting a snapshot cutoff log sequence indicator includes selecting the next available log sequence indicator and immediately modifying the global reference variable thereafter.

[0151] 26. The computer-implemented method according to clause 24, wherein the indicator of the determined relative order of the log sequence indicator and the snapshot cutoff log sequence indicator includes a first part and a second part, the first part is an indicator of the time of generating the data entry, and the second part is generated based on a global reference variable.

[0152] 27. The computer-implemented method according to clause 26, wherein generating a snapshot includes selecting an indicator corresponding to the relative time of the snapshot relative to the time of generating the data entry and determining to include the data entry in the snapshot according to any of the following:

[0153] The indicator corresponding to the relative time of the snapshot relative to the time of generating the data entry indicates that the snapshot is to be generated at a time slightly later than the time of generating the data entry; or

[0154] The second part generated based on the global reference variable corresponds to the value of the global reference variable before modifying the global reference indicator.

[0155] 28. The computer-implemented method according to clause 26, wherein the global reference variable comprises one bit and modifying the global reference variable comprises flipping the value of the bit.

[0156] 29. A non-transitory computer-readable storage medium comprising computer-readable instructions that, when executed by a processor, cause the processor to perform the method according to clause 20.

[0157] 30. A database system comprising:

[0158] at least one processor; and

[0159] at least one memory comprising computer program code, the at least one memory and the computer program code being configured to, with the at least one processor, cause the database system to perform the method according to clause 20.

[0160] 31. A computer-implemented method for copying a binary large object stored at a first database to a second database, the method comprising:

[0161] sending a set of log records corresponding to an operation performed on data stored at the first database to the second database;

[0162] identifying log records of the set of log records that include an indication of the binary large object stored at the first database; and

[0163] in response to identifying the log records that include an indication of the binary large object stored at the first database, sending the binary large object stored at the first database to the second database.

[0164] 32. The computer-implemented method according to clause 31, wherein the set of log records corresponding to the operation performed on data stored at the first database is sent to the second database in a sequence, and sending the binary large object to the second database comprises inserting the binary large object into the sequence after the log record that includes the indication of the binary large object.

[0165] 33. The computer-implemented method according to clause 32, wherein the binary large object is inserted into the sequence immediately after the log record that includes the indication of the binary large object.

[0166] 34. The computer-implemented method according to clause 31, wherein the identified log records are associated with a log sequence indicator, and the binary large object stored at the first database is associated with the log sequence indicator.

[0167] 35. The computer-implemented method according to clause 34, wherein the binary large object is sent to the second database in one or more parts, each part being associated with a corresponding indicator, the corresponding indicator including a first part associated with the log sequence indicator and a second part indicating the part of the binary large object.

[0168] 36. The computer-implemented method according to clause 35, comprising:

[0169] receiving at the second database each of the one or more parts of the binary large object; and

[0170] storing each of the one or more parts at the second database according to the corresponding indicator.

[0171] 37. The computer-implemented method according to clause 35, comprising storing the binary large object at the second database and maintaining the association between the binary large object and the log sequence indicator.

[0172] 38. The computer-implemented method according to clause 37, wherein the binary large object is stored in the second database as a directory according to at least a part of the log sequence indicator.

[0173] 39. The computer-implemented method according to clause 37, wherein the binary large object is stored in the second database as a multi-level directory, each level of the multi-level directory corresponding to a corresponding part of the log sequence indicator.

[0174] 40. The computer-implemented method according to clause 31, wherein the identified log record includes a checksum generated from the binary large object.

[0175] 41. The computer-implemented method according to clause 31, comprising reserving a portion of storage space at the second database to store the log record after the log record has been identified.

[0176] 42. The computer-implemented method according to clause 31, comprising generating a file at the second database to store the binary large object before sending the binary large object.

[0177] 43. The computer-implemented method according to clause 32, wherein the identified log record includes data indicating the size of the binary large object, and the method includes reserving space for one or more portions of the binary large object in a sequence for transmitting the binary large object based on the data indicating the size of the binary large object.

[0178] 44. The computer-implemented method according to clause 43, which includes sending an indication of the one or more portions of the binary large object to the second database, wherein after receiving the one or more portions of the binary large object at the second database, an indication that the one or more portions of the binary large object have been received is generated.

[0179] 45. A non-transitory computer-readable storage medium, which includes computer-readable instructions that, when executed by a processor, cause the processor to perform the method according to clause 31.

[0180] 46. A database system, which includes:

[0181] at least one processor; and

[0182] at least one memory including computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the database system to perform the method according to clause 31.

[0183] 47. A computer-implemented method for physically deleting one or more binary large objects from a database, wherein the database has multiple states, and a first state of the database at a first time is represented by a first snapshot, the method includes:

[0184] generating a second snapshot representing the state of the database at a second time, the second time being later than the first time and the second snapshot including data identifying one or more binary large objects that have been logically deleted from the database before the second time;

[0185] deleting the first snapshot; and

[0186] after deleting the first snapshot, physically deleting the one or more binary large objects from the database using the data identifying the one or more binary large objects that have been logically deleted before the second time.

[0187] 48. The computer-implemented method according to clause 47, comprising deleting the second snapshot based on determining that all of the one or more binary large objects that have been logically deleted before the second time have been physically deleted.

[0188] 49. The computer-implemented method according to clause 47, comprising generating a third snapshot representing the state of the database at a third time, the third time being later than the second time and the third snapshot including data identifying one or more binary large objects that have been logically deleted from the database before the third time.

[0189] 50. The computer-implemented method according to clause 47, wherein the second snapshot includes data identifying one or more binary large objects that have been logically deleted after the first time and before the second time.

[0190] 51. The computer-implemented method according to clause 47, comprising deleting the first snapshot based on determining that the second snapshot has been generated.

[0191] 52. The computer-implemented method according to clause 47, wherein the data identifying one or more binary large objects that have been logically deleted before the second time includes data corresponding to a first type of log record.

[0192] 53. The computer-implemented method according to clause 47, wherein the first type of log record is generated when generating the second snapshot.

[0193] 54. The computer-implemented method according to clause 47, wherein the first type of log record is generated from one or more entries in a first table at the database, and the one or more entries in the table are deleted after generating the first type of log record.

[0194] 55. The computer-implemented method according to clause 54, wherein the one or more entries in the first table are deleted immediately after generating the first type of log record.

[0195] 56. The computer-implemented method according to clause 54, wherein at least one entry in the first table includes an indication of a corresponding second type of log record, the corresponding second type of log record including data identifying one or more binary large objects that have been deleted before the second time.

[0196] 57. The computer-implemented method according to clause 56, wherein the one or more log records of the second type are generated periodically.

[0197] 58. The computer-implemented method according to clause 57, wherein the one or more log records of the second type are generated from a second table, one or more entries in the second table include indicators of binary large objects that have been logically deleted before the second time, and after generating the log records of the second type, the entries in the second table used to generate the log records of the second type are deleted from the second table.

[0198] 59. The computer-implemented method according to clause 58, wherein after generating the log records of the second type according to the entries used to generate the log records of the second type, the entries in the second table used to generate the log records of the second type are immediately deleted.

[0199] 60. The computer-implemented method according to clause 58, wherein the log records of the second type are generated using more than one entry of the second table.

[0200] 61. The computer-implemented method according to clause 58, wherein one or more other entries in the second table are related to ongoing operations performed on binary large objects at the database, and the method includes identifying, based on a comparison of the entries in the second table with data indicating one or more ongoing operations in the database, the entries in the second table related to binary large objects that have been logically deleted.

[0201] 62. A non-transitory computer-readable storage medium, which includes computer-readable instructions that, when executed by a processor, cause the processor to perform the method according to clause 47.

[0202] 63. A database system, which includes:

[0203] at least one processor; and

[0204] at least one memory including computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the database system to perform the method according to clause 47.

Claims

1. A computer-implemented method for copying a binary large object stored at a first database to a second database, the method comprising: Sending a set of log records corresponding to an operation performed on data stored at the first database to the second database; And When sending the set of log records to the second database: Identifying log records of the set of log records that include an indication of a binary large object stored at the first database; And In response to identifying the log records that include an indication of the binary large object stored at the first database, sending the binary large object stored at the first database to the second database; Wherein the set of log records corresponding to the operation performed on the data stored at the first database is sent to the second database in a sequence, and sending the binary large object to the second database includes inserting the binary large object into the sequence after the log record that includes the indication of the binary large object.

2. The computer-implemented method according to claim 1, wherein the binary large object is inserted into the sequence immediately after the log record that includes the indication of the binary large object.

3. The computer-implemented method according to claim 1, wherein the identified log records are associated with a log sequence indicator, and the binary large object stored at the first database is associated with the log sequence indicator.

4. The computer-implemented method according to claim 3, wherein the binary large object is sent to the second database in one or more parts, each part being associated with a corresponding indicator, the corresponding indicator including a first part associated with the log sequence indicator and a second part indicating the part of the binary large object.

5. The computer-implemented method according to claim 4, comprising: Receiving at the second database each of the one or more parts of the binary large object; And Storing each of the one or more parts at the second database according to the corresponding indicator.

6. The computer-implemented method according to claim 4, comprising storing the binary large object at the second database and maintaining the association between the binary large object and the log sequence indicator.

7. The computer-implemented method according to claim 6, wherein the binary large object is stored in a directory at the second database according to at least a part of the log sequence indicator.

8. The computer-implemented method according to claim 6, wherein the binary large object is stored in a multi-level directory at the second database, each level of the multi-level directory corresponding to a corresponding part of the log sequence indicator.

9. The computer-implemented method according to claim 1, wherein the identified log records include a checksum generated from the binary large object.

10. The computer-implemented method according to claim 1, which includes, after the log record has been identified, reserving a portion of the storage space at the second database to store the log record.

11. The computer-implemented method according to claim 1, which includes, before sending the binary large object, generating a file at the second database to store the binary large object.

12. The computer-implemented method according to claim 1, wherein the identified log record includes data indicating the size of the binary large object, and the method includes reserving space for one or more parts of the binary large object in a sequence for sending the binary large object based on the data indicating the size of the binary large object.

13. The computer-implemented method according to claim 12, which includes sending an indication of the one or more parts of the binary large object to the second database, wherein after the one or more parts of the binary large object have been received at the second database, an indication that the one or more parts of the binary large object have been received is generated.

14. A non-transitory computer-readable storage medium, which includes computer-readable instructions that, when executed by a processor, cause the processor to perform the method according to claim 1.

15. A database system, which includes: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code being configured to, with the at least one processor, cause the database system to perform the method according to claim 1.

Images