Apply logs to buckets

Abstract

In a scenario where there is a primary computing system operating on data, and (multiple) secondary computing systems that are ready to be promoted to the new primary computing system if the old primary computing system fails, a log service computing system is used to apply logs to storage segments. Data is separated into storage segments that are collectively distributed among multiple storage segment servers, rather than having each of the primary and secondary computing systems locally maintain their own copy of the entire data.

Description

Background Art

[0001] Various services are now provided in a cloud computing environment. "Cloud computing" is a model for supporting ubiquitous, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage devices, applications, and services). The cloud computing model can also take the form of various application service models, such as, for example, software as a service ("SaaS"), platform as a service ("PaaS"), and infrastructure as a service ("IaaS"). The cloud computing model can also be deployed using different deployment models, such as private clouds, community clouds, public clouds, hybrid clouds, etc. One type of software as a service ("SaaS") is called database as a service (DBaaS).

[0002] Conventional DBaaS is typically implemented in an architecture that includes one primary computing node and one or more secondary computing nodes (each node is often provided as a virtual machine). The primary computing node operates a database server and has a directly attached solid-state drive from which the primary database server can read or write. The secondary computing nodes also operate database servers and also have solid-state drives from which corresponding secondary database servers can read or write. Data on the primary computing node is synchronized with the secondary computing nodes.

[0003] The secondary computing nodes provide redundancy for the purpose of meeting service availability goals. If a primary computing node were to fail, a failover of the database can occur by promoting one of the secondary computing nodes to the primary computing node and instantiating additional secondary computing nodes to maintain the required level of redundancy.

[0004] In order to synchronize the data at the primary computing node with the data at the secondary computing node, communication from the primary computing node to the secondary computing node is required. The primary computing node retains a log of the operations that occurred at the primary computing node. The primary computing node can send log records to the secondary computing nodes (either individually or as blocks). The secondary computing nodes each apply the operations specified by the primary computing node and confirm them back to the primary computing node. Once the primary computing node receives such a confirmation from a quorum (for example, two of the three secondary computing nodes), the primary computing node has synchronized its data with the secondary computing node to the extent that if the primary computing node fails, there is a guarantee that the majority of the secondary computing systems will apply the log records. Therefore, in the case of recovery, even if the newly promoted primary computing node has not yet applied the log records, the newly promoted primary computing node can still update itself with current data by consulting other secondary computing nodes.

[0005] The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as described above. Rather, this background is provided merely to illustrate one exemplary technology area in which some embodiments described herein may be practiced. Summary of the invention

[0006] At least some embodiments described herein relate to applying logs to storage segments in a scenario where there is a primary computing system operating on data, and (multiple) secondary computing systems that are ready to be promoted to the new primary computing system if the old primary computing system fails. However, instead of having each of the primary and secondary computing systems locally maintain their own replicated copies of the entire data, the data is separated into storage segments that are collectively distributed across multiple storage segment servers. The storage segment servers collectively act as a centralized repository for data. The log service computing system supports this according to the principles described herein.

[0007] For example, when the primary computing system operates on data, various write operations are logged in various log records. As an example, in order to perform a write operation on a storage segment, the primary computing system can read the storage segment from an appropriate storage segment server (or, if the storage segment is cached, from its own cache), and can perform a write operation on a copy of the storage segment. In addition, to log the write operation, the primary computing system identifies how the content of the storage segment changes, records the incremental changes and the identifier of the storage segment in the log record, and writes the log record to a persistent log (perhaps with other log records in the block). The log service computing system can either be notified of the presence of the log record by reading the log record from the persistent log or by receiving a communication from the primary computing system.

[0008] The log service computing system supports the application of log records by identifying the following specific storage segment servers: the storage segment server is assigned to store a specific set of storage segments including the storage segment to be written. The log service computing system then causes the specific storage segment server to perform the specific write operation specified in the log record. The log service computing system can also cause one or more secondary computing systems to perform specific write operations on the copies of the storage segments present on (multiple) secondary computing systems. As an example, the log service computing system can receive a request for a log record from (multiple) storage segment servers and (multiple) secondary computing systems. The log service computing system can then provide the entity requesting the log record with the log record it is aware of. In one embodiment, in order to avoid inconsistent recovery, the log service computing system does not provide log records that have not yet been confirmed to have been saved in the persistent log.

[0009] There are many technical advantages to handling logging in this manner. First, data (e.g., databases and logging of operations on that user data) is no longer confined to the primary or secondary computing systems themselves, which may have relatively limited storage capacity. For example, if the computing system is executed within a virtual machine, the size of the data is conventionally limited to the size of the maximum virtual machine storage size. Instead, logging is saved in a centralized persistent log. User data (e.g., databases) is stored between storage segment servers. The larger the user data, the more storage segment servers there can be. Therefore, the size of the data is potentially unlimited.

[0010] Furthermore, when the storage segment servers use the storage service provided by the cloud computing environment to store their respective data, there is no need to perform a data size provisioning operation. For example, such a data size provisioning operation may be the performance of a full backup (or snapshot) of all of the data. Conventionally, each of the primary computing system and the secondary computing system would have to perform a full backup separately by transferring the data over a network. Thus, the principles described herein avoid data size provisioning operations and therefore scale better for larger sizes of data.

[0011] Additionally, the time spent building a new secondary computing system is significantly reduced because the new secondary computing system no longer needs to have a directly attached volume that has an entire replica of the data. Instead, the secondary computing system may only need to operate an application (e.g., a database server), and a relatively small cache. Furthermore, in cases where the primary computing system writes to a volume attached to the storage service, the establishment of a quorum for the purpose of determining whether log records have been properly logged can now be handled by the storage service. In such a case, the write to the persistent log is completed only after the storage service has established that the log record has been durably written to the appropriate number of locations.

[0012] This summary introduces some concepts in a simplified form that will be further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to describe the manner in which the above and other advantages and features of the present invention can be obtained, the invention briefly described above will be described in more detail by reference to specific embodiments of the invention shown in the accompanying drawings. Therefore, these drawings only depict example embodiments of the present invention and should not be considered limiting of the scope of the present invention. With this in mind, example embodiments of the present invention will be described and explained with reference to the accompanying drawings, in which:

[0014] Figure 1 illustrates an environment in which the principles described herein may be employed, the environment comprising a primary computing system, one or more secondary computing systems, a storage segment server between which data is stored, persistent storage, and a logging service for applying log records generated by the primary computing system;

[0015] Figure 2 A flow chart of a method for generating log records and performing write operations is illustrated.

[0016] Figure 3 illustrates a log record including a log sequence number, a description of a change to an increment of a storage segment, and a storage segment identifier to which the change is applied;

[0017] Figure 4 illustrates a block filled with a plurality of log records;

[0018] Figure 5 illustrates a flow chart of a method for applying logging to a storage segment according to the principles described herein;

[0019] Figure 6 The diagram illustrates a log environment in which the log consists of the following two components: a fixed-size log part and a growable log part;

[0020] Fig. 7A The diagram shows Figure 6 A log environment of a specific example of a log environment of , and in which the fixed-size log portion includes three virtual log files (each having four blocks), and in which the growable log portion has eighteen virtual log files therein;

[0021] Figure 7B The diagram shows Fig. 7A A log environment in a subsequent state after the virtual log file VLF19 has been destaged and the storage location of the fixed-size log portion is reused by a subsequent virtual log file VLF22;

[0022] Figure 7C The diagram shows Figure 7BThe log environment of the checkpoint is after the checkpoint is taken at that point and the log environment of the subsequent state after the block is further written to block 22B and includes block 22B.

[0023] Fig.7D Illustrated with Xs and Os inserted into the block to indicate the write mark bit bound to the virtual log file Figure 7C The logging environment; and

[0024] Figure 8 An example computer system is illustrated in which the principles described herein may be employed. DETAILED DESCRIPTION

[0025] At least some embodiments described herein relate to applying logs to storage segments in a scenario where there is a primary computing system operating on data, and (multiple) secondary computing systems ready to be promoted to the new primary computing system if the old primary computing system fails. However, rather than having each primary and secondary computing system locally maintain their own replica of the entire data, the data is separated into storage segments that are collectively distributed across multiple storage segment servers. The storage segment servers collectively act as a centralized repository for data. The log service computing system supports this according to the principles described herein.

[0026] For example, when the primary computing system operates on data, various write operations are logged in various log records. As an example, to perform a write operation on a storage segment, the primary computing system can read the storage segment from the appropriate storage segment server (or, if the storage segment is cached, from its own cache), and can perform a write operation on a copy of the storage segment. In addition, to log the write operation, the primary computing system indicates how the contents of the storage segment have changed, records the incremental changes and the identifier of the storage segment in the log record, and writes the log record to a persistent log (perhaps other log records). The log service computing system can be aware of the existence of the log record either by reading the log record from the persistent log or by receiving a communication from the primary computing system.

[0027] The log service computing system supports the application of log records by identifying the following specific storage segment servers: the storage segment server is assigned to store a specific set of storage segments including the storage segment to be written. The log service computing system then causes the specific storage segment server to perform the specific write operation specified in the log record. The log service computing system can also cause one or more secondary computing systems to perform specific write operations on the copies of the storage segments present on (multiple) secondary computing systems. As an example, the log service computing system can receive a request for log records from (multiple) storage segment servers and (multiple) secondary computing systems. The log service computing system can then provide the log record it is aware of to the entity requesting the log record. In one embodiment, in order to avoid inconsistent recovery, the log service computing system does not provide log records that have not yet been confirmed to have been saved in the persistent log.

[0028] Handling logging in this manner has many technical advantages. First, data (e.g., databases and logging of operations on that user data) is no longer confined to the primary or secondary computing systems themselves, which may have relatively limited storage capacity. For example, if the computing system is executed within a virtual machine, the size of the data is conventionally limited to the size of the maximum virtual machine storage size. Instead, logging is saved in a centralized persistent log. User data (e.g., databases) is stored between storage segment servers. The larger the user data, the more storage segment servers there can be. Therefore, the size of the data is potentially unlimited.

[0029] Furthermore, when the storage segment servers use the storage service provided by the cloud computing environment to store their respective data, there is no need to perform data sizing operations. For example, such a data sizing operation may be the performance of a full backup (or snapshot) of the entire data. Conventionally, each of the primary computing system and the secondary computing system would have to perform a full backup separately by transmitting the data over the network. Thus, the principles described herein avoid data sizing operations and therefore scale better for larger sizes of data.

[0030] Additionally, the time spent building a new secondary computing system is significantly reduced due to the following: the new secondary computing system no longer needs to have a directly attached volume that has an entire copy of the data. Instead, the secondary computing system may only need to operate an application (e.g., a database server), and a relatively small cache. In addition, in the case where the primary computing system writes to a volume attached to the storage service, the establishment of a quorum for the purpose of determining whether log records have been properly logged can now be handled by the storage service. In such a case, the write to the persistent log is completed only after the storage service establishes that the log record has been durably written to the appropriate number of locations.

[0031] Figure 1 An environment 100 is shown in which the principles described herein may be employed. By way of example only, the environment 100 may be a network, such as perhaps a cloud computing environment. The environment 100 includes a primary computing system 110 on which an application 110A operates. By way of example, the application 110A may be a database server application. The primary computing system 110 may be a physical computing system (such as a computing node in a cloud computing environment), in which case the primary computing system 110 may be configured as follows with respect to the following: Figure 8 The computing system 800 is constructed as described. Alternatively or additionally, the main computing system 110 may be a virtual machine that simulates a computing system. The application 110A running on the main computing system 110 performs a write operation on the user data. For example, if the application 110A is a database server, the user data may be a database.

[0032] The environment 100 also includes (multiple) secondary computing systems 120. Each of the (multiple) secondary computing systems 120 operates a corresponding application, which may be an instance of the same application being executed on the primary computing system 110. For example, if the application 110A on the primary computing system 110 is a database server application, an instance of the database server application may be executed on each of the (multiple) secondary computing systems 120. Each of the (multiple) secondary computing systems 120 may be a physical computing system (such as a computing node in a cloud computing environment), in which case the secondary computing systems may be described below with respect to Figure 8 The computing system 800 is constructed as described. Alternatively or additionally, each (multiple) secondary computing system 120 may be a virtual machine emulating a computing system.

[0033] (Multiple) secondary computing systems 120 can include any number of secondary computing systems. In the illustrated embodiment, there are two secondary computing systems 121 and 122 shown, each of which operates corresponding applications 121A and 122A, respectively. Ellipsis 123 indicates that the principles described herein are not limited to the number of (multiple) secondary computing systems 120. If the primary computing system 110 fails so that it cannot be effectively recovered, one of the (multiple) secondary computing systems 120 will be promoted to the new primary computing system, thereby allowing the promoted computing system to perform write operations. Therefore, (multiple) secondary computing systems 120 allow built-in redundancy within the environment 100. If a secondary computing system is promoted to a primary computing system, a new secondary computing system can be started to replace the upgraded computing system, and thereby maintain the desired number of (multiple) secondary computing systems 120. On the other hand, independent of whether the primary computing system 110 has failed, the secondary computing system can be added or deleted from the set of (multiple) secondary computing systems at will.

[0034] When the primary computing system 110 performs write operations on user data (e.g., a database), the primary computing system 110 logs these data operations (as represented by arrow 101A) in a persistent log 131. The persistent log 131 is retained so that the environment 100 can be recovered in the event of a failure. The process of the primary computing system 110 writing to a storage segment (e.g., a page of user data) while persistently logging the write operation will now be described.

[0035] In particular, Figure 2 A flow chart of a method 200 for generating a log record and performing a write operation is illustrated. As an example, the method 200 may be performed by Figure 1 The primary computing system 110 performs a read operation. The primary computing system first reads a particular memory segment (action 201). This ensures that a copy of the memory segment is brought into the primary computing system's cache. The memory segment may have been previously read by the primary computing system, in which case the memory segment will already be in the primary computing system's cache. Otherwise, the primary computing system performs a read operation from the corresponding memory segment server containing the memory segment to be written to. For example, in Figure 1 , the primary computing system 110 reads (as represented by arrow 102 ) a memory segment (eg, a page) from the memory segment server 142 .

[0036] The primary computing system then writes to the read (e.g., cached) copy of the storage segment (act 202). And, to create a log record of the write operation (act 203), the incremental changes made or to be made as part of the write operation are determined (act 211). The primary computing system then creates a log record including the incremental changes (act 212). For example, Figure 3 A log record 300 is shown, which includes a log sequence number 301, a description of the incremental change 302, and a storage segment identifier 303 to which the change is applied. The log sequence number is a number that uniquely identifies the log record and represents the position of the log record within the log. The log sequence number is typically a monotonically increasing number, so that the higher the log sequence number, the more recently the log record was placed within the log. The log record is written to a persistent log (action 204). For example, in Figure 1 , as represented by arrow 101A, primary computing system 110 writes a log record (eg, log record 300 ) to persistent log 131 .

[0037] In one embodiment, log records are organized into blocks. A block is a unit of storage that can be written to and read from atomically (i.e., at a time - so that the write or read is either completed for the entire block, or abandoned if the write is not performed at all). Typical block sizes will be constant in a computing system, but example block sizes include 512 bytes and 4096 bytes. Because log records can be much smaller than blocks, a block can contain multiple log records. Figure 4 A block 400 is shown populated with a plurality of log records 401, 402, 403, and 404. As an example, Figure 3 The logging record 300 can be Figure 4 402 of log records. Ellipsis 405 indicates that box 400 may include any number of blocks that are suitable for the size of a given log record and the size of the block. It can be said that each log record occupies a "slot" within a block. When a log record is written to a persistent log (such as by Figure 1 Arrows 101A and Figure 2 ), which may be a block of log records written to the persistent log (such as block 400). Thus, the primary computing system 110 may wait for some number of log records to be generated before writing the log records as a block to the persistent log 131.

[0038] Return to Figure 1 , the environment 100 also includes a log service computing system 132. The log service computing system 132 can be a physical computing system, such as the following Figure 8Alternatively or additionally, the log service computing system 132 may be a virtual machine, or perhaps an executable component, such as the following: Figure 8 Described executable component 806. The log service computing system 132 facilitates write operations executed and logged by the primary computing system 110 to be propagated to the appropriate storage segment servers 140 and to the secondary computing system(s) 120.

[0039] The environment 100 also includes a plurality of segment servers 140. The data operated by the application 110A on the primary computing system 110 is spread across the segment servers 140. Thus, each segment server is used to serve a segment corresponding to only a portion of the user data. A segment may be, for example, a page of data. Each page of data may include a plurality of blocks.

[0040] For example, if Figure 1 1, the segment server 141 performs data operations only for the assigned segment set 151. In addition, arrow 162 indicates that the segment server 142 performs data operations only for its assigned segment set 152, arrow 163 indicates that the segment server 143 performs data operations only for its assigned segment set 153, and arrow 164 indicates that the segment server 144 performs data operations only for its assigned segment set 154. The ellipsis 145 and 155 indicate that the principles described herein are not limited to the number of segment servers. In addition, the segment sets assigned to any given segment server need not be contiguous segments within the address space.

[0041] However, if the memory segments (e.g., pages) assigned to any given memory segment are contiguous within the address space of the user data, it is simpler to identify which memory segment server is assigned to perform data operations for a given memory segment. In addition, since a single read operation can be issued to potentially return many contiguous memory segments that are contiguous in the address space of the user data, read operations on contiguous memory segments can be performed more efficiently on contiguous memory segments. In one embodiment, the user data is represented within a storage service 170 located below the memory segment servers 140. The user data distributed across the memory segment servers 140 can maintain the same layout and structure as the user data on the storage service 170.

[0042] The storage segment set 151 to the storage segment set 155 are collectively illustrated as the storage segment set 150. When the storage segment server 140 uses the storage service 170 provided by the cloud computing environment to store their corresponding data, the need to perform data size operations is eliminated. For example, such a data size operation can be the performance of a complete backup of all data. Conventionally, each of the primary computing system and the secondary computing system would have to perform a complete backup separately by transmitting data over the network. Therefore, the principles described herein allow for an excellent ability to scale data upward. However, a copy of the storage segment set 151 to the storage segment set 155 can also be retained in a cache at each of the storage segment servers 141 to the storage segment server 145. This allows for fast access to the storage segment. However, if the storage segment is lost in the cache of the storage segment server 140 (or even if one of the storage segment servers itself is down), the storage segment is still available in the storage service 170 in the cloud computing environment.

[0043] Figure 5 A flow chart of a method 500 for applying log records to storage segments according to the principles described herein is illustrated. The method 500 can be executed for each log record in a plurality of log records identified to be applied. Execution of the method 500 multiple times allows the log to be applied to the appropriate storage segment. That is, the write operation specified in the log record is applied to the storage segment within the appropriate storage segment server. As an example, the method 500 can be executed by Figure 1 The method 500 may be performed by the log service computing system 132. Figure 1 is executed within the environment 100 so that the application Figure 3 Thus, we will now refer to Figure 1 and Figure 3 To describe Figure 5 .

[0044] When it is determined that a particular log record is to be applied, method 500 is initiated (act 501). For example, the particular log record may be Figure 3 In response to identifying that a particular log record is to be applied (act 501), the particular log record is applied (act 510). This can be done with respect to a storage segment server corresponding to the storage segment. This can also be done with respect to a secondary computing system.

[0045] Thus, the application of logging (action 510) includes identifying a segment server that is assigned to a particular set of segments, the particular set of segments including the segments to be written to in a particular write operation (action 511). In the case where the set of segments is contiguous in address space, the assignment of the set of segments to the segment servers can be aligned with the most significant bits of the segment identifiers. For example, assume that the segment identifiers range from binary 00000, 00000 (commas are added every five bits to aid the reader) to binary 01101, 01111. In this case, there may be seven segment servers, one for a segment with an identifier having four most significant bits of 0000, another for a segment with an identifier having four most significant bits of 0001, and so on until the seventh segment is for a segment with an identifier having four most significant bits of 0110. Therefore, determining which segment server is assigned to store a particular segment becomes computationally efficient. The log service computing system 132 can therefore determine the appropriate bucket server for the log record 300 based on the bucket identifier 303 .

[0046] The particular storage segment server is then caused to perform a write operation to the storage segment specified in the particular log record (action 512). For example, log service computing system 132 may provide (e.g., as represented by arrow 103B) log record 300 to the appropriate storage segment server (e.g., storage segment server 142). This would indicate the completion of the write operation that was initiated when primary computing system 110 read (e.g., as represented by arrow 102) the storage segment from the storage segment server.

[0047] In one example, the notification (represented by arrow 103B) is not pushed to the appropriate bucket server. Rather, it is up to the bucket server to make a query to the log service computing system 132 for the appropriate log record. Thus, providing the appropriate log record to the bucket server may be in response to a request from a particular bucket server. Thus, after the bucket server 142 queries the applicable log entry (as represented by arrow 103A), the bucket server 142 assigned to the bucket being written to may be notified of the write operation.

[0048] The log service computing system 132 may have a broker component 133 thereon that handles the processing of incoming requests for log records. The broker 133 may be an executable component on the log service computing system. For example, the broker 133 may be as follows for Figure 8806. Instead of providing one log record at a time, the agent 133 may provide one block at a time. For example, if the agent 133 determines that there are any log records (e.g., any of log records 401, 402, 403, 404, or 405) in a block of log records (e.g., block 400) that have a bucket identifier within a set of buckets assigned to a particular bucket server, the agent 133 may send the entire block to the particular bucket server.

[0049] In addition, the agent 133 can ensure that the log record is not communicated to any entity until the log service computing system 132 has been notified that the log record has been securely written to the persistent log 131 (i.e., the log record has become public). This helps the recovery process to be consistent. During recovery, the recovery uses the persistent log 131 to determine which operations have been completed. If other components in the system (e.g., a secondary computing system or a storage segment server) have performed operations that the persistent log 131 is not aware of, the recovery will not be able to reach a reliable state. Then, the user data becomes ambiguous and therefore corrupted.

[0050] The agent 133 may also perform other tasks. For example, the agent 133 may anticipate that the primary computing system 110 (which is the source of the generated log records) is unreliable. To compensate for this, the agent 133 may implement different strategies for deduplicating log records that have been received multiple times from the primary computing system 110. The agent 133 may perform this deduplication by tracking primary computing system generation and by retaining only the most recent generation for all log records. The agent 133 may also anticipate log record loss and compensate for it by filling in the lost log records described below (e.g., by reading from a fixed-size log portion 601).

[0051] Return to Figure 5 In method 500, the log service computing system also causes the secondary computing system(s) to perform a specific write operation on the copy of the storage segment at the secondary computing system (action 520). This can also be done by communicating to each secondary computing system 121 and 122 (e.g., by Figure 1104B and 105B in the secondary computing system 120 (e.g., log record 300). Although the log record can be pushed to the secondary computing system 120, the agent 131 can similarly handle requests for log records from the secondary computing system 120 (as represented by arrows 104A and 105A). Again, the log records can be provided as blocks. As a reminder, the agent 133 can ensure that the log record is not communicated to the secondary computing system 120 unless the log record has been confirmed to be written to the persistent log 131. As a possible optimization, if the log record is for a write to a storage segment that has not yet been cached, the secondary computing system may also ignore the log record (and therefore cause a read from the storage segment server). In this case, if the secondary computing system later wants to use the storage segment, the secondary computing node can read the storage segment from the storage segment server (which has already applied the write of the skipped log).

[0052] The persistent log 131 may be stored within a volume 135 within a storage service 170 of the cloud computing environment. The storage service 170 ensures that the log records (e.g., each log record block) are redundantly saved before providing confirmation that the log record block has been written to the primary computing system 110. Once the storage service 170 ensures that the appropriate minimum number of copies of the log have been written to the volume 135, the storage service 170 confirms that the write has been completed. The persistent log 131 may then immediately confirm to the primary computing system 110 (as represented by arrow 101B) that the write back has been completed, allowing the primary computing system 110 to continue to perform operations that depend on the completion.

[0053] Volume 135 may be mounted to multiple computing systems. For example, a single mounted volume 135 may be included as a read-write volume within a drive mounted to primary computing system 110 while being mounted to (multiple) secondary computing systems 120 as a read-only volume. Because only one computing system is mounted to the volume as a write volume, multiple computing systems may be mounted to volume 135 without the risk of data corruption due to competing or inconsistent write operations. The volume may be part of a cloud storage service within a cloud computing environment. For example, storage segment servers 140 and / or secondary computing systems may read log records directly from persistent log 131, rather than reading log records from agent 131.

[0054] In some embodiments, the storage segment set 150 can be managed by a cloud service in a cloud computing environment. In this case, the cloud service itself can ensure that a minimum number (or quorum) of redundant copies of the storage segment are stored in the cloud storage device before the write operation is deemed successful. The quorum is established so that in the event of a failure, the quorum of redundant copies of the storage segment allows a decisive determination of what the contents of the storage segment are. Therefore, when the storage segment server confirms the completion of the write operation, the write operation has been performed on at least a quorum of storage locations. Because the secondary computing system is no longer responsible for maintaining the quorum, the number of secondary computing systems is highly flexible. Instead, the secondary computing system is more often used to allow local secondary computing systems to be available for processing, and is also responsible for failover, and not to ensure the durability of the storage device itself.

[0055] Thus, the principles described herein describe an architecture in which data is separated from the primary computing system and the secondary computing system. Because a relatively smaller amount of data is actually cached by the secondary computing system, building a new secondary computing system becomes relatively fast. The cache of the secondary computing system can be empty, which only means that some of the initial read operations may take longer. For example, when further redundancy is needed to increase the level of availability, a new secondary computing system can be established. Alternatively, a new secondary computing system can be established to replace the newly promoted secondary computing system that is now acting as the primary computing system in order to maintain the current level of availability.

[0056] Because the new secondary computing system no longer needs to have a directly attached volume with an entire copy of the data, the time spent building a new secondary computing system is significantly reduced. Instead, the secondary computing system may only require an application, and a relatively small cache. In addition, in the case where the primary computing system writes to a volume attached to the storage service, the establishment of a quorum for the purpose of determining whether log records have been properly logged can now be handled by the storage service. In such a case, the write to the log is completed only after the storage service has established that the log record has been durably written to the appropriate number of locations.

[0057] Already referenced Figures 1 to 5 Having described the environment 100, reference will now be made to Figures 6 to 7DDescribe more about the log within volume 135. Recall that volume 135 is a write volume for primary computing system 110, but a read-only volume for secondary computing system(s) 120. This is possible because only primary computing system 110 writes to the log within volume 135, while secondary computing system(s) 120 only read volume 135 from persistent log 131 within volume 135. Described herein are improvements on how an unbounded log (a log that can grow within limits) can be accommodated based on volume 135 while allowing all of the entities reading from the log to know what address to read from within the log.

[0058] Figure 6 A log environment 600 is illustrated in which there is a log consisting of two components - a fixed-size log portion 601 and a growable log portion 602. As symbolically represented by boundaries 611 and 612 as thicker, the size of the fixed-size log portion is fixed. The fixed-size log portion 601 includes the more recent log records recorded in the log (and includes the tail of the log). As will become further apparent below, the advantage of the fixed-size log portion 601 being fixed in size is that the reader of the fixed-size log portion 601 can deterministically know where the log record is located in the fixed-size log portion 601 using only the log sequence number of the log record. In addition, the fixed-size log portion is in a format designed to be securely shared between the reader and the writer. This simplifies the process of enabling the reader to read log records from the log.

[0059] As indicated by arrow 621, the fixed-size log portion 601 is written sequentially. When the end of the fixed-size log portion 601 (e.g., end 612) is encountered, the bend is written back (as indicated by dashed arrow 622) to the beginning of the fixed-size log portion 601 (e.g., beginning 611) to continue writing to the log. Therefore, writing to the fixed-size log portion 601 occurs in a circular manner. Because of the circular write pattern, older log records will be overwritten by newer log records. Before that happens, the downstager component 610 writes those older log records sequentially to the end of the growable log portion 602 (e.g., end 613). In this way, the growable log portion 602 grows sequentially in direction 623.

[0060] Therefore, the fixed-size log portion 601 includes newer log records, which are the log records that are most often read from it. On the other hand, the growable log portion 602 includes older log records that are less often read. In addition, the fixed-size log portion 601 will include the tail of the log, which is the last log record written to the log as a whole. In the event of a failure, it is important to be able to identify the tail of the log, because the log record is the last log record that the log service has guaranteed to be executed, even if a failure occurs before the data operation represented by the log record has been completed. During recovery, the recovery process restores the last checkpoint, and re-performs the data operations of each log record one at a time until the tail of the log is encountered.

[0061] Fig. 7A The diagram shows Figure 6 A specific example of the logging environment 600 is a logging environment 700A. Fig. 7A The fixed-size log section 701 indicates Figure 6 An example of a fixed-size log portion 601. Fig. 7A The growable log portion 702A indicates Figure 6 6. An example of a growable log portion 602 of a fixed size. In this example, and in one embodiment, the fixed-size log portion 701A is sized to include an odd number of sub-portions. For example, the sub-portions can be virtual log files. As will become apparent from the following description, the advantage of the log portion 701A being sized to include an odd number of virtual log files is that the normal operation of initializing the sub-portions is simplified while still allowing the recovery process to find the tail of the log. In this particular example, the fixed-size log portion 701A is sized to include three virtual log files.

[0062] Assume that the log consists of 21 virtual log files (or "VLFs") so far, and that the virtual log files are identified in sequential order as VLF1, VLF2, VLF3, etc. Fixed-size log portion 701A would include the last three virtual log files VLF19, VLF20, and VLF21. Older virtual log files VLF1 to VLF18 would have previously been downgraded by downgrader 710 to growable log portion 701A.

[0063] In this example, and as is most apparent from the fixed-size log portion 701A, each portion (e.g., virtual log file) includes a fixed number of blocks. Although virtual log files can typically have a very large number of blocks, to keep the example simple, each virtual log file (e.g., VLF19 to VLF21) is illustrated as having four (4) blocks. For example, virtual log file VLF19 consists of blocks 19A to 19D sequentially, virtual log file VLF20 consists of blocks 20A to 20D sequentially, and virtual log file VLF21 consists of blocks 21A to 21D sequentially.

[0064] When a log record is written into a persistent log, the block including the log record is written into a fixed-size log file. Each log record in a block appears in a slot in the block. The log sequence number can therefore be composed of a concatenation of a virtual log file identifier, a block identifier, and a slot identifier. Note that any reader can obtain any log record in any virtual file in those virtual files using only the log sequence number, knowing that the fixed-size log portion 701A has VLF19, VLF20, and VLF21 arranged sequentially in this order. Blocks can be identified from the log sequence number, which allows appropriate blocks to be read. Then, the slot identifier from the log sequence number can be used to extract appropriate log records from the block.

[0065] In any case, blocks are written one block at a time sequentially by the primary computing system 110 to a fixed-size log file. Fig. 7A In the example above, when a block has an asterisk in the upper right corner, it indicates that it has been written. Fig. 7A At the point in time illustrated in , because block 21A is the last block written, the tail of the log (represented by pointer 711) is just after block 21A.

[0066] The demotion device 710 monitors the position of the tail of the log in the fixed-size log part 701A (i.e., the position most recently written to), and ensures that any virtual log file that is about to be overwritten is demotioned to the growable log part 702A. Then, the storage position of the fixed-size log part 701A that was once used to store the newly demotioned virtual log file can be reused for subsequent virtual log files.

[0067] Figure 7BIllustrated is a log environment 700B, which represents a subsequent state of the log environment 700B after the virtual log file VLF19 has been demoted, and the storage location of the fixed-size log file 701A (now referred to as being labeled 701B) is reused by the virtual log file VLF22 having blocks 22A, 22B, 22C, and 22D. In this example, each time a new sub-portion (e.g., virtual log file) is initialized within the fixed-size log portion, the sub-portion identifier (e.g., virtual log file identifier) ​​for each successively initialized sub-portion (e.g., virtual log file) is incremented. Also note that the growable log portion 702B has now grown by one virtual log file to now include virtual log file VLF19. In this example, Fig. 7A Compared with the tail of the log in Figure 7B The tail 711 of the log has not moved yet. That is to say, the tail 711 of the log is still just after block 21A.

[0068] Now assume that a checkpoint is taken at this point (just after block 21A is written to fixed-size log portion 701B). Then, consistent with the sequential and round-robin write pattern, assume that the next blocks are then written to fixed-size log portion 701B in order: block 21B, block 21C, block 21D, block 22A, and block 22B. The result will be Figure 7C 700C. Now note that the fixed-size log portion 701C has up to block 22B, shown with an asterisk, and therefore the tail 711 of the log is now just after block 22B. In addition, for convenience, a snapshot pointer 712 is indicated for the convenience of the reader and will be referenced further below.

[0069] Now assume that when the tail 711 of the log is just after block 22B (such as Figure 7C The first task during recovery will be to restore the most recent snapshot. Figure 7C In the example of FIG. 7A , the state of the data is brought to the state just after the data operations for all the log records in the log record in block 21A are performed. But to make the data completely current, the recovery process performs all subsequent data operations represented by the subsequent log records until the tail of the log is encountered (at pointer 711). But the problem is to identify when the tail of the log is encountered. After all, there is still data in blocks 22C and 22D, even though they are log records that have been downgraded as part of blocks 19C and 19D. Therefore, it is important to identify what blocks have been written as part of the current virtual log file (VLF22) and to distinguish those blocks from those blocks with stale data from previous cyclic write cycles.

[0070] To allow the recovery process to make this distinction, there is new marking data within each block, which, together with the current virtual log file identifier, allows the recovery process to deterministically conclude whether new data has been written to the block as part of the current virtual log file. In one embodiment, the new marking data within the block can be two bits. The value of these bits, combined with the sub-part identifier, allows the recovery process to determine which blocks have been written to.

[0071] For example, assume that when writing to a block of a subpart with an odd subpart identifier (e.g., VLF19 or VLF21), the two bits are written with a first possible value (e.g., 01). In this case, when writing to a block of a subpart with an even subpart identifier (e.g., VLF20 or VLF22), the two bits are written with a second possible value (e.g., 10).

[0072] Fig.7D is with Figure 7C The same logging environment 700D as environment 700C except that "O" fills those blocks that would have position 01 using the convention described in the previous paragraph, and "X" fills those blocks that would have position 10 using the convention described in the previous paragraph. Because the asterisks are used only for the convenience of the reader, they have been removed.

[0073] Subpart VLF20 has had all of its blocks written (because the tail of the log is forward in VLF22). Therefore, because subpart identifier VLF20 is even, the two bits will be 10, and thus blocks 20A, 20B, 20C, and 20D are shown with Xs. Of course, recovery should look for the tail of the log in the most recent subpart VLF22. However, while recovery does not need to look for the tail of the log in any of the previous subparts, if recovery did check subpart VLF20, recovery would know that the tail of the log is not there.

[0074] Subpart VLF21 has had all of its blocks written (because the tail of the log is forward in VLF22). Therefore, because subpart identifier VLF21 is odd, the two bits will be 01, and thus blocks 21A, 21B, 21C, and 21D are shown as having Os. Therefore, if recovery did check subpart VLF21, recovery would know that the tail of the log is not there.

[0075] Because the tail 711 of the log is within sub-part VLF22, sub-part VLF22 only has some of its blocks written. Specifically, because the sub-part identifier VLF is an even number, the two bits of the two blocks 22A and 22B that are written to will be 10, and are therefore shown as "X" inside. However, note that the old data from VLF19 is still in block 22C and block 22D. When the main computing system is writing to blocks 19C and 19D that are part of sub-part VLF19, the old data is written to those blocks. Because the data has not changed at all, the two bits of blocks 22C and 22D remain 01, and are therefore marked with "O" inside them. Therefore, when storage space begins to be reused for sub-part VLF22, the recovery process can still find the tail of the log without having to reformat the storage space used for sub-part VLF19. This effect is achieved precisely because there are an odd number (three) of subsections within the fixed-size log section, and the subsections are added with identifiers that alternate between even and odd numbers (which occurs when they monotonically increase by one at a time).

[0076] Thus, the log being written to volume 135 can be optimized so that the more frequently accessed blocks (including the most recent log records) are optimized so that any reader can read those blocks using only the log sequence number and access the appropriate log record. Furthermore, the writes can occur during normal operation so that in a subsequent restore, the tail of the log can be found quickly without having to expend effort to reformat the storage space in the fixed-size portion of the log. Thus, indeed, once a block with that log record is written to Figure 6 In the log environment 600, the log service can confirm that data operations are guaranteed to occur.

[0077] Additionally, because the size of the fixed-size log portion is fixed, the management data of the driver 135 remains unchanged. And, as described above, because the content is self-describing, the fixed-size log portion is designed to be safely shared between readers and writers. Even if the reader and writer do not need to communicate with each other, this allows the reader and writer to be protected from torn writes and other concurrent access distortions. Therefore, the driver 135 can be installed on multiple computing systems.

[0078] Because the principles described herein operate in the context of a computing system, reference will be made to Figure 8Describe a computing system. Computing systems are now increasingly taking a variety of forms. For example, a computing system may be a handheld device, an appliance, a laptop computer, a desktop computer, a mainframe, a distributed computing system, a data center, or even a device that is not conventionally considered to be a computing system, such as a wearable device (e.g., glasses, a watch, a band, etc.). In this specification and claims, the term "computing system" is broadly defined to include any device or system (or combination thereof) that includes at least one physical and tangible processor and a physical and tangible memory on which computer executable instructions executable by the processor can be located. The memory may take any form and may depend on the nature and form of the computing system. The computing system may be distributed over a network environment and may include multiple constituent computing systems.

[0079] like Figure 8 As shown, in its most basic configuration, the computing system 800 generally includes at least one hardware processing unit 802 and memory 804. The memory generally includes at least one hardware processing unit 802 and memory 804. The memory 804 can be physical system memory, which can be volatile, non-volatile, or some combination of the two. The term "memory" can also be used herein to refer to non-volatile mass storage devices such as physical storage media. If the computing system is distributed, the processing, memory and / or storage capabilities can also be distributed.

[0080] The computing system 800 has a number of structures thereon that are often referred to as "executable components." For example, the memory 804 of the computing system 800 is illustrated as including an executable component 806. The term "executable component" is a name for a structure that is well known to those of ordinary skill in the computing arts, which structure may be software, hardware, or a combination thereof. For example, when implemented in software, one of ordinary skill in the art will understand that the structure of the executable component may include a software object, routine, method that can be executed on the computing system, whether such executable component is present in the computing system's stack or whether the executable component is present on a computer-readable storage medium.

[0081] In such cases, one of ordinary skill in the art will recognize that the structure of the executable component exists on a computer-readable medium such that the structure of the executable component, when interpreted by one or more processors of a computing system (e.g., by a processor thread), causes the computing system to perform a function. Such a structure may be directly computer-readable by the processor (e.g., if the executable component is a binary). Alternatively, the structure may be organized as interpretable and / or compiled (whether in a single stage or in multiple stages) to generate such a binary that is directly interpretable by the processor. When the term "executable component" is used, such an understanding of the example structure of an executable component is well within the understanding of one of ordinary skill in the computing arts.

[0082] The term "executable component" is also well understood by those of ordinary skill in the art to include structures that are implemented exclusively or almost exclusively in hardware, such as in a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other special circuit. Therefore, the term "executable component" is a term for a structure that is well understood by those of ordinary skill in the computing field, whether it is implemented in software, hardware, or a combination. In this specification, the terms "component" or "vertex" and the like may also be used. As used in this specification, and in this case, these terms (whether or not the terms are modified by one or more modifiers) are also intended to be synonymous with the term "executable component" or a specific type of such an "executable component", and therefore also have a structure that is well understood by those of ordinary skill in the computing field.

[0083] In the following description, embodiments are described with reference to actions performed by one or more computing systems. If such actions are implemented in software, then in response to having executed the computer-executable instructions constituting the executable component, one or more processors (of the associated computing system that performs the action) direct the operation of the computing system. For example, such computer-executable instructions may be embodied on one or more computer-readable media forming a computer program product. Examples of such operations involve manipulation of data.

[0084] Computer-executable instructions (and manipulated data) may be stored in memory 804 of computing system 800. Computing system 800 may also contain communication channels 808 that allow computing system 800 to communicate with other computing systems over network 810, for example.

[0085] Although not all computing systems require a user interface, in some embodiments, computing system 800 includes a user interface 812 for interfacing with a user. User interface 812 may include output mechanism 812A and input mechanism 812B. The principles described herein are not limited to accurate output mechanism 812A or input mechanism 812B, because this will depend on the nature of the device. However, output mechanism 812A may include, for example, a speaker, a display, a tactile output, a hologram, a virtual reality, etc. The example of input mechanism 812B may include, for example, a microphone, a touch screen, a hologram, a virtual reality, a camera, a keyboard, a mouse of other pointer inputs, any type of sensor, etc.

[0086] The embodiments described herein may include or utilize a dedicated or general-purpose computing system including computer hardware (such as, for example, one or more processors and system memory), as discussed in more detail below. The embodiments described herein also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media can be any available media that can be accessed by a general or dedicated computing system. The computer-readable medium storing computer-executable instructions is a physical storage medium. The computer-readable medium carrying computer-executable instructions is a transmission medium. Therefore, by way of example and not limitation, an embodiment may include at least two obviously different types of computer-readable media: a storage medium and a transmission medium.

[0087] Computer-readable storage media includes RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other physical and tangible storage media that can be used to store desired program code components in the form of computer-executable instructions or data structures and can be accessed by a general-purpose or special-purpose computing system.

[0088] A "network" is defined as one or more data links that support the transport of electronic data between computing systems and / or components and / or other electronic devices. When information is transmitted or provided to a computing system via a network or another communication connection (hardwired, wireless, or a combination of hardwired or wireless), the computing system appropriately treats the connection as a transmission medium. The transmission medium may include a network and / or a data link that may be used to carry desired program code in the form of computer executable instructions or data structures and may be accessed by a general or special computing system. Combinations of the above should also be included within the scope of computer-readable media.

[0089] In addition, when implementing various computing system components, program code components in the form of computer executable instructions or data structures can be automatically transferred from a transmission medium to a storage medium (or vice versa). For example, a computer executable instruction or data structure received over a network or data link can be buffered in RAM within a network interface component (e.g., a "NIC") and then ultimately transferred to a computing system RAM and / or a less volatile storage medium at the computing system. Therefore, it should be understood that a readable medium can be included in a computing system component that also (or even primarily) utilizes a transmission medium.

[0090] Computer executable instructions include, for example, instructions and data that, when executed at a processor, cause a general purpose computing system, a special purpose computing system, or a special purpose processing device to perform certain functions or groups of functions. Alternatively or additionally, the computer executable instructions may configure a computing system to perform certain functions or groups of functions. Computer executable instructions may be, for example, binary or even instructions that have undergone some translation (e.g., compilation) before being directly executed by a processor, such as intermediate format instructions (such as assembly language), or even source code.

[0091] Those skilled in the art will recognize that the present invention can be practiced in a network computing environment with many types of computing system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, pagers, routers, switches, data centers, wearable devices (e.g., glasses or watches), etc. The present invention can also be practiced in a distributed system environment, in which local and remote computing systems that are linked (or by a hardwired data link, a wireless data link, or a combination of a hardwired link and a wireless data link) through a network both perform tasks. In a distributed system environment, program components can be located in both local memory devices and remote memory storage devices.

[0092] Those skilled in the art will also appreciate that the present invention may be practiced in a cloud computing environment supported by one or more data centers or portions thereof. The cloud computing environment may be distributed, although this is not required. When distributed, the cloud computing environment may be distributed internationally within an organization and / or have components owned across multiple organizations.

[0093] In this specification and the appended claims, "cloud computing" is defined as a model for supporting on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage devices, applications, and services). The definition of "cloud computing" is not limited to any of the other numerous advantages that can be derived from such a model when properly deployed.

[0094] For example, cloud computing is currently being adopted in the market to provide ubiquitous and convenient on-demand access to a shared pool of configurable computing resources. Furthermore, the shared pool of configurable computing resources can be quickly provisioned through virtualization, released with low management effort or service provider interaction, and then scaled accordingly.

[0095] The cloud computing model can consist of various features, such as on-demand, self-service, broad network access, resource pools, rapid elasticity, measured services, etc. The cloud computing model can also take the form of various application service models, such as software as a service ("SaaS"), platform as a service ("PaaS"), and infrastructure as a service ("IaaS"). It can also be deployed using different deployment models such as private cloud, community cloud, public cloud, hybrid cloud, etc. In this specification and claims, a "cloud computing environment" is an environment in which cloud computing is employed.

[0096] The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the present invention. The described embodiments are considered to be illustrative and non-restrictive in all respects. Therefore, the scope of the present invention is indicated by the appended claims rather than the foregoing description. All changes falling within the meaning and equivalent scope of the claims should be included within their scope.

Claims

1. A computing system comprising: one or more processors; as well as One or more computer-readable storage media having computer-executable instructions thereon, the computer-executable instructions being structured such that when executed by the one or more processors, the computing system is caused to perform the following operations: identifying one or more log records to apply, the one or more log records comprising a specific log record that is logged by the primary computing system and that specifies a specific write operation to a specific storage segment; and In response to identifying the specific log record to be applied, applying the specific log record by performing the following operations: identifying a specific storage segment server, the specific storage segment server being assigned to a specific storage segment set, the specific storage segment set including the specific storage segment to be written to in the specific write operation; causing the specific storage segment server to perform the specific write operation on the specific storage segment specified in the specific log record; and A secondary computing system is caused to perform the particular write operation on a cached copy of the memory segment stored within a cache of the secondary computing system, the secondary computing system being configured to be promoted to replace the primary computing system in the event of a failure of the primary computing system.

2. The computing system of claim 1 , wherein in response to receiving a request for the specific log record from the specific storage segment server, the causing the specific storage segment server to perform the specific write operation is performed, and wherein the causing the specific storage segment server to perform the specific write operation is performed by providing the specific log record to the specific storage segment server in response to the request from the specific storage segment server.

3. The computing system of claim 2, wherein the specific log record comprises a plurality of log records organized in a block, and the specific log record is provided to the specific storage segment server by providing the block to the specific storage segment server.

4. The computing system of claim 1 , wherein the specific log record is generated in response to the primary computing system reading the specific storage segment from the specific storage segment server, determining an incremental change to be made to the specific storage segment, making the incremental change to the replica of the specific storage segment, and identifying the incremental change and an identifier of the specific storage segment within the specific log record.

5. The computing system of claim 1, wherein causing the specific storage segment server to perform the specific write operation on the specific storage segment occurs after confirming that the specific log record has been saved to a centralized log.

6. The computing system of claim 1 , wherein the causing the specific storage segment server to perform the specific write operation is performed in response to receiving a request for the specific log record from the secondary computing system, and the causing the secondary computing system to perform the specific write operation is performed by providing the specific log record to the secondary computing system in response to the request from the secondary computing system. 7 . The computing system of claim 6 , the specific log record comprising a plurality of log records organized in a block, the specific log record being provided to the secondary computing system by providing the block to the secondary computing system.

8. The computing system of claim 1, wherein causing the secondary computing system to perform the specific write operation occurs after verifying that the specific log record has been saved to a centralized log.

9. The computing system of claim 1, wherein the one or more log records to be applied are logged within a volume that is mounted to the primary computing system as a read and write volume and is simultaneously mounted to the secondary computing system as a read-only volume.

10. A method for applying a log to a storage segment, the method comprising: identifying one or more log records to apply, the one or more log records comprising a specific log record specifying a specific write operation to a specific storage segment; and In response to identifying the specific log record to be applied, applying the specific log record by performing the following operations: identifying a specific storage segment server, the specific storage segment server being assigned to a specific storage segment set, the specific storage segment set including the specific storage segment to be written to in the specific write operation; Avoid causing the specific storage segment server to perform the specific write operation on the specific storage segment specified in the specific log record until after confirming that the specific log record has been saved to the centralized log; and In response to determining that the specific log record has been saved to the centralized log, causing the specific storage segment server to perform the specific write operation on the specific storage segment specified in the specific log record.

11. The method of claim 10, wherein the specific log record is a first log record, the specific write operation is a first write operation, the specific storage segment is a first storage segment, the specific storage segment server is a first storage segment server, the specific storage segment set is a first storage segment set, the one or more log records further include a second log record, the second log record specifies a second write operation to a second storage segment, and the method further includes: identifying a second segment server assigned to a second segment set, the second segment set including the second segment to be written in the second write operation, the second segment server being different from the first segment server, the second segment set being different from the first segment set; and The second storage segment server is caused to perform the second write operation on the second storage segment specified in the second log record.

12. The method of claim 10, wherein in response to receiving a request for the specific log record from the specific storage segment server, the causing the specific storage segment server to perform the specific write operation is performed, and by providing the specific log record to the specific storage segment server in response to the request from the specific storage segment server, the causing the specific storage segment server to perform the specific write operation is performed.

13. The method of claim 12, wherein the specific log record comprises a plurality of log records organized in a block, and the specific log record is provided to the specific storage segment server by providing the block to the specific storage segment server.

14. The method of claim 10, wherein the particular log record is generated in response to a primary computing system reading the particular storage segment from the particular storage segment server, determining an incremental change to be made to the particular storage segment, making the incremental change to a copy of the particular storage segment, and identifying the incremental change and an identifier of the particular storage segment within the particular log record.

15. The method according to claim 10, further comprising, in response to identifying the specific log record to be applied, performing the following operations: A secondary computing system is caused to perform the particular write operation on the copy of the storage segment at the secondary computing system.

16. The method of claim 15, wherein the causing the specific storage segment server to perform the specific write operation is performed in response to receiving a request for the specific log record from the secondary computing system, and the causing the secondary computing system to perform the specific write operation is performed by providing the specific log record to the secondary computing system in response to the request from the secondary computing system.

17. The method of claim 16, the specific log record comprising a plurality of log records organized in a block, the specific log record being provided to the secondary computing system by providing the block to the secondary computing system.

18. The method of claim 15, said causing the secondary computing system to perform the particular write operation occurs after verifying that the particular log record has been saved to a centralized log.

19. A computer program product comprising one or more hardware storage devices having computer executable instructions thereon, the computer executable instructions executable by one or more processors of a computing system to configure the computing system to perform the following operations: identifying one or more log records to apply, the one or more log records comprising a specific log record specifying a specific write operation to a specific storage segment; and In response to identifying the specific log record to be applied, applying the specific log record by performing the following operations: identifying a specific storage segment server, the specific storage segment server being assigned to a specific storage segment set, the specific storage segment set including the specific storage segment to be written to in the specific write operation; Avoid causing the specific storage segment server to perform the specific write operation on the specific storage segment specified in the specific log record until after confirming that the specific log record has been saved to the centralized log; and In response to determining that the specific log record has been saved to the centralized log, causing the specific storage segment server to perform the specific write operation on the specific storage segment specified in the specific log record.

Images