Synchronized database replication using erasure codes

Abstract

Synchronous replication for a distributed database system may be performed using an erasure code scheme. A request to cause a write to a hosted database in a distributed database system is received. A replicated message for a synchronous replication technique is generated, and then the replicated message is divided and encoded into a plurality of chunks according to an erasure code scheme that allows the replicated message to be recombined in less than the plurality of chunks. The chunk is sent to another instance of the database that receives and reorganizes the copy message from the chunk and responds to a confirmation that the write is submitted.

Description

Background Technology

[0001] The commoditization of computer hardware and software components has spurred the rise of service providers offering computing and storage capabilities as a service. At least some of these services, such as database services, are implemented in a distributed manner to provide data persistence and availability. In this way, the workload of client applications can be distributed across multiple components of a distributed database system to provide consistent performance. Attached Figure Description

[0002] Figure 1 This is a logic block diagram illustrating synchronous database replication using erasure coding according to some embodiments.

[0003] Figure 2 This is a block diagram illustrating a provider network area that can implement database services according to some embodiments, the database services implementing techniques for synchronous database replication using erasure coding.

[0004] Figure 3 This is a block diagram illustrating various components of a database service and storage service for a database accessible to a managed database client according to some embodiments.

[0005] Figure 4 This is a block diagram illustrating the interaction between a database service and a separate storage service according to some embodiments.

[0006] Figure 5 This is a block diagram illustrating cross-region replication of a database hosted in a database service, according to some embodiments.

[0007] Figure 6 This is a high-level flowchart illustrating various methods and techniques for implementing synchronous database replication using erasure coding, according to some embodiments.

[0008] Figure 7 This is a high-level flowchart illustrating various methods and techniques for reassembling replicated message blocks using erasure coding, according to some embodiments.

[0009] Figure 8 This is a block diagram illustrating an example computer system according to various embodiments.

[0010] Although embodiments have been described herein with reference to several examples and illustrative drawings, those skilled in the art will recognize that the embodiments are not limited to the described embodiments or drawings. It should be understood that the drawings and detailed descriptions thereof are not intended to limit the embodiments to the specific forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling within the spirit and scope defined by the appended claims. The headings used herein are for organizational purposes only and are not intended to limit the scope of the specification or claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning “possibly”) rather than a mandatory sense (i.e., meaning “must”). The word “include / including / includes” indicates an open relationship and therefore means including but not limited to. Similarly, the word “have / having / has” also indicates an open relationship and therefore means having but not limited to. The terms “first,” “second,” “third,” etc., used herein serve as markers for the nouns that follow them and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.) unless such ordering is explicitly stated otherwise.

[0011] "Based on." As used herein, this term describes one or more factors that influence the determination. This term does not exclude other factors that may influence the determination. That is, the determination may be based solely on those factors or at least partially on those factors. Consider the phrase "A is determined based on B." While B may be a factor influencing the determination of A, this phrase does not exclude the possibility that the determination of A is also based on C. In other instances, A may be determined solely on B.

[0012] The scope of this disclosure includes any feature or combination of features disclosed herein (express or implicit) or any generalization thereof, whether or not it alleviates any or all of the problems set forth herein. Therefore, during the examination of this application (or an application claiming priority thereto), new claims may be formulated for any such combination of features. Specifically, features of the appended claims and supplementary claims may be combined with features of the independent claims, and features of the respective independent claims may be combined in any suitable manner, not merely the specific combinations listed in the appended claims. Detailed Implementation

[0013] This article describes various embodiments of synchronous database replication using erasure coding. Databases synchronously replicated over large networks (such as wide area networks, like the Internet) are often highly sensitive to anomalous latency along the path on the WAN. This is likely because database updates may need to be replicated sequentially, so a slow update slows down all updates replicated in the database (or a subset of updates in "fragments" or other parts of the database).

[0014] WANs and other large networks can include multiple redundant paths that provide alternative routes between destinations, such as between different instances of a replicated updated database. Although these redundant paths often have similar overall frequency distributions, their anomalous latencies may not be highly correlated over time.

[0015] In various embodiments, techniques are described that utilize multiple redundant paths using an erasure coding scheme. For example, by breaking an update to be replicated into multiple chunks k, where a replicated message describing the update can be recreated with N chunks, where N < k, the k chunks can be sent over redundant network paths and the message reconstructed upon arrival of the first N. This significantly reduces the tail / anomalous latency experienced by the chunks end-to-end. Thus, the anomalous latency can be treated as a binary erasure, and erasure coding applied as a forward error correction scheme. Such techniques can reduce the overall latency for replication schemes for distributed database systems, particularly synchronous replication schemes or other replication schemes that rely on applying replicated updates in order. For example, allowing distributed database systems with a very wide distribution (e.g., across different data centers, geographical regions or continents) to have high performance synchronous replication that does not sacrifice database performance for consistency.

[0016] In various embodiments, different erasure coding schemes can be used. In some embodiments, simple parity check (for N = k + 1) techniques, Reed-Solomon techniques, and other maximum distance separable code (for any N > k) techniques can be applied. In some embodiments, rateless erasure coding schemes such as fountain codes and Raptor codes can be implemented, which can further reduce overhead by avoiding sending redundant information unless a response is not received within a reasonable time.

[0017] Figure 1 is a logical block diagram showing synchronous database replication using erasure coding according to some embodiments. In Figure 1 , the distributed database system 110 can be a stand-alone database system (e.g., implemented on a dedicated network system or service or by a user of a cloud or other provider network, as discussed in detail below with respect to Figure 2 ). In some embodiments, the distributed database system 110 can be a database service, as discussed below with respect to Figures 2 to 5The database service 210, discussed in detail, can be implemented and managed by a provider network. The distributed database system 110 can be one of many different types of databases, including those supporting different kinds of access to database data, such as through query languages ​​like Structured Query Language (SQL) or APIs, or other commands providing NoSQL interfaces. Different types of databases can store database data in different formats and according to different data models. For example, one type of database can use a relational data model that imposes a common schema on tables in a relational database, while another type of database can use a non-relational data model that imposes a flexible schema, which may not be common among different items or objects in the database. Databases can store various types of data, including, but not limited to, graph databases that store data using graph data models, time-series databases that store time-series data, key-value databases that use unique keys to look up data objects of various data types or formats in the database, or document databases that store data as documents (including nested data) with changing attributes.

[0018] Distributed database 110 can store the database in a storage system that can be attached to or accessed by database instances 120, 130, and 140. In some embodiments, a non-distributed storage system can be implemented to store the database. In other embodiments, the database can be stored in a distributed data storage system, such as those described below. Figures 2 to 5 The storage service 220 discussed. Database instances (e.g., database instances 120, 130, and 140) can be implemented as instances, nodes, or other standalone applications, which can comprise one or more tiers in a database system application stack, implemented together on common nodes, servers, or instances, or separated across different tiers across nodes, servers, or other host systems. In various embodiments, the instances, nodes, or other standalone applications may or may not implement virtualization to execute some or all components, functions, or workflows as virtual machines, instances, containers, microVMs, or other forms of computing power. For example, as described below regarding... Figure 3In addition to the database journal, the database service may also implement query processors, front-ends, and adjudicators on the respective host systems. In some embodiments, these host systems may be distributed (e.g., in different data centers or other different locations, such as on different server racks) to avoid common failure scenarios where two host systems may experience related failures simultaneously. In some embodiments, a database instance may include multiple features or functionalities (or all of the various features or functionalities discussed, such as handling write adjudication, query processing / execution, and replication). Therefore, in some embodiments, the term database instance may not be limited to a single component but may refer to the location of a database functionality in a given location, such as a database instance in a region of the database service.

[0019] like Figure 1 As depicted, database instance 120 can receive write request 112. Write request 112 can be a standalone request (e.g., an UPDATE or INSERT statement in SQL) or can be received as part of a larger transaction containing multiple read and / or other write requests. Database instance 120 can process write request 112 and perform synchronous replication to database instances 130 and 140. As indicated at 102, communication between database instances 120 can utilize redundant network path 102, which can provide alternative routes for communication between database instances 120, 130, and 140.

[0020] In the example shown, to perform synchronous replication of writes, database instance 120 may partition and encode replication messages (e.g., communications instructing updates to be included in journals or other database system data stores that authoritatively commit writes to the database) into multiple chunks (e.g., 122a, 122b, 122c, 122d, and 122e), which may be sent individually to database instances 140 and 130, respectively, across redundant network paths 102. Various protocols may respond to the synchronous execution of replication messages, and replication messages may be in various formats (e.g., single-phase or multi-phase commits or other protocols or conflict resolution protocols). In various embodiments, replication messages may be generated for processing as an ordered update stream, such that for a replication message update to be applied, all previous updates in the ordered update stream described in an earlier replication message may have to be received.

[0021] Even if some message chunks (e.g., 122d of database instance 130 and 122b and 122d of database instance 140) are not received at the database instance, each database instance can still reassemble the replication message, apply the write, and return message acknowledgments, as indicated at 132 and 142. In this way, database instance 120 can return a response 114 indicating that the write has been committed to the database after receiving acknowledgments from database instances 130 and 140. This technique not only reduces the potential loss of individual replication messages due to network failures but also mitigates the negative impact of such loss as a single replication message in an ordered update stream, where their loss would hinder the application of all subsequent updates.

[0022] Please note, Figure 1 Provided as a logical diagram of a distributed database system and its corresponding components and interactions, and not intended to limit the physical arrangement, size, or number of components, modules, or devices used to implement such features. For example, cross-region replication technology may also involve sending messages across provider network regions (although internal region replication can also be performed). Multiple additional components may be involved, thus spanning multiple components (e.g., query processors, adjudicators, etc., as discussed below). Figures 3 to 5 As explained, the responsibility for assigning database instance 120 is as follows.

[0023] This specification continues to describe an example network-based database service implemented as part of a provider network that uses erasure coding to perform synchronous database replication. The description of the example database service includes various aspects of the example database service, such as the database instance and standalone storage service. This specification then describes flowcharts of various embodiments of a method for synchronous database replication using erasure coding. Next, this specification describes example systems that can implement the disclosed technology. Various examples are provided throughout this specification.

[0024] Figure 2 This is a block diagram illustrating a provider network area that can implement database services according to some embodiments, the database services implementing techniques for synchronous database replication using erasure coding. A provider network (sometimes referred to as a "cloud provider network" or "cloud") refers to a pool of network-accessible computing resources (e.g., computing, storage, and networking resources, applications, and services), which may be virtualized or bare metal. A provider network can provide convenient, on-demand network access to a shared pool of configurable computing resources that can be programmatically provided and released in response to user commands. These resources can be dynamically provided and reconfigured to adapt to variable loads. Therefore, cloud computing can be viewed both as applications delivered as a service over a publicly accessible network (e.g., the Internet or cellular communication networks) and as the hardware and software in the cloud provider's data center providing these services.

[0025] A provider network can be structured into multiple regions, where a region is a separate geographical area containing a cluster of cloud provider data centers. Each region can contain two or more availability zones connected to each other via a dedicated high-speed network (e.g., fiber optic communication connection). An availability zone (also referred to as an availability domain, or simply a "zone") is an isolated fault domain containing one or more data center facilities with separate power supplies, separate networking, and separate cooling from data center facilities in another availability zone. A data center is a physical building or enclosure that houses and provides power and cooling to the servers of the cloud provider network. Preferably, availability zones within a region are far enough apart that the same natural disaster should not simultaneously take down more than one availability zone. Users can connect to availability zones of the provider network via a transit center (TC) through a publicly accessible network (e.g., the Internet, cellular communication network). A TC can be considered as a backbone location linking users to the provider network and can be co-located and securely connected (e.g., via VPN or direct connection) to availability zones from other network provider facilities (e.g., Internet service providers, telecommunications providers). Each region can operate two or more TCs for redundancy. Regional networks connect to a global network that links each region to at least one other region. The provider network can deliver content from points of presence (PoPs) outside these regions but networked with them, via edge locations and regional edge caching servers. This segmentation and geographical distribution of computing hardware enables the provider network to provide users with low-latency resource access globally with high fault tolerance and stability.

[0026] The provider network can implement various computing resources or services, which may include virtual computing services, data processing services (e.g., map downscaling, data streaming, and / or other large-scale data processing technologies), data storage services (e.g., object storage services, block-based storage services, or data warehouse storage services), and / or any other type of network-based service (which may include various other types of storage, processing, analytics, communication, event processing, visualization, and security services not shown). The resources required to support the operation of such services (e.g., computing and storage resources) may be provided in an account associated with the cloud provider, which differs from the resources requested by users of the cloud provider network, which may be provided in user accounts.

[0027] In various embodiments, the services and operations of the provider network can be broadly categorized into two types: control plane operations carried on the logical control plane and data plane operations carried on the logical data plane. The data plane represents the movement of user data through a distributed computing system, while the control plane represents the movement of control signals through a distributed computing system. The control plane typically contains one or more control plane components distributed across and implemented by one or more control servers. Control plane services typically include administrative operations such as system configuration and management (e.g., resource placement, hardware capacity management, diagnostic monitoring, system status information). The data plane contains customer resources implemented on the cloud provider network (e.g., compute instances, containers, block storage volumes, databases, file storage). Data plane services typically include non-administrative operations such as moving customer data to and from customer resources. Some control plane components (e.g., first-tier control plane components, such as the control plane for virtualized computing services) are typically implemented on a separate set of servers from the data plane servers, while other control plane components (e.g., second-tier control plane components, such as analytics services) may share virtualization servers with the data plane, and control plane services and data plane services may be transmitted over separate / dissimilar networks.

[0028] like Figure 2 As depicted, the exemplary provider network may include multiple provider network regions 220a, 220b, etc., which may include one or more data centers hosting various resource pools, such as a collection of physical and / or virtualized computer servers, storage devices, networking devices, etc., required to implement and distribute the infrastructure and storage services provided by the provider network within provider network region 200 (e.g., hereinafter referred to as...). Figure 8 The described computing system 1000).

[0029] In the illustrated embodiment, multiple clients (shown as client 250) can interact with provider network 200 via network 260. The provider network can implement corresponding instantiations of the same (or different) services, database service 210a for region 200a and database service 210b for region 200b, storage service 220a for region 200a and storage service 220b for region 220b, and various other virtual computing services 230a and 230b. It should be noted that references to a given component may be in either the singular or plural form, where one or more instances of a given component may exist. However, the use of either form is not intended to exclude the other.

[0030] In various embodiments, Figure 2The components shown can be implemented directly within computer hardware, as instructions that can be executed directly or indirectly by computer hardware (e.g., a microprocessor or computer system), or in combination using these technologies. For example, Figure 2 The components can be implemented by a system containing multiple computing nodes (or simply nodes), each of which can be similar to Figure 8 The computer system embodiments shown and described below. In various embodiments, the functionality of a given service system component (e.g., a component of a database service or a component of a storage service) may be implemented by a specific node or may be distributed across several nodes. In some embodiments, a given node may implement the functionality of more than one service system component (e.g., more than one database service system component).

[0031] Generally, client 250 can encompass any type of client configurable to submit web-based service requests (including requests for database services) to one or more of provider network zones 200a or 200b via network 260. For example, a given client 250 may include a suitable version of a web browser, or may include plug-in modules or other types of code modules that can execute as an extension of or within an execution environment provided by a web browser. Alternatively, client 250 (e.g., a database service client) can encompass applications such as database applications (or their user interfaces), media applications, office applications, or any other applications that can utilize persistent storage resources to store and / or access one or more database tables. In some embodiments, such applications may include sufficient protocol support (e.g., for a suitable version of Hypertext Transfer Protocol (HTTP)) for generating and processing web-based service requests without necessarily implementing full browser support for all types of web-based data. That is, client 250 can be an application that can directly interact with services in a provider network zone. In some embodiments, client 250 may generate web-based service requests based on a Representational State Transfer (REST) ​​style web service architecture, a document- or message-based web-based service architecture, or another suitable web-based service architecture. Although not shown, in some embodiments, some clients of the provider network 200 services may be implemented within the provider network's services (e.g., a client application for database service 210a may be implemented on one of the other virtual computing services 230a in region 200a). Therefore, in some embodiments, various examples of the interactions discussed with respect to client 250 may also be implemented for internal clients.

[0032] In some embodiments, client 250 (e.g., a database service client) can provide network-based storage access to database data to other applications in a manner transparent to those applications. For example, client 250 can integrate with an operating system or file system to provide storage according to a suitable variant of the storage model described herein. However, the operating system or file system may provide different storage interfaces to applications, such as a traditional file system hierarchy of files, directories, and / or folders. In this embodiment, it is not necessary to modify the application to utilize the storage system service model, as described above. Instead, the details of interfacing with the provider network can be coordinated by client 250 and the operating system or file system on behalf of the application executing within the operating system environment.

[0033] Client 250 can transmit network-based service requests to and receive responses from areas of the provider network via network 260. In various embodiments, network 260 can encompass any suitable combination of networking hardware and protocols necessary to establish network-based communication between client 250 and provider network areas 200a and 200b. For example, network 260 can typically encompass various telecommunications networks and service providers that commonly implement the Internet. Network 260 can also include private networks, such as local area networks (LANs) or wide area networks (WANs), and public or private wireless networks. For example, given client 250 and provider network areas, both may be provided within an enterprise with their own internal networks. In this embodiment, network 260 can include the hardware (e.g., modems, routers, switches, load balancers, proxy servers, etc.) and software (e.g., protocol stacks, billing software, firewalls / security software, etc.) necessary to establish networking links between given client 250 and the Internet, and between the Internet and provider network areas 200a and 200b. It should be noted that in some embodiments, client 250 may use a private network instead of the public Internet to communicate with areas of the provider network. For example, client 250 can be provided within the same enterprise as the database service. In this case, client 250 can communicate entirely with the provider's network area via a private network 260 (e.g., a LAN or WAN that can use Internet-based communication protocols but is not publicly accessible).

[0034] Generally, provider network areas 200a and 200b may implement one or more service endpoints capable of receiving and processing network-based service requests, such as requests to access a database (e.g., query, insert, update, etc.) and / or manage a database (e.g., create a database, configure a database, etc.). For example, the provider network area may include hardware and / or software that implements specific endpoints to correctly receive and process HTTP-based network-based service requests for said endpoints. In one embodiment, the provider network area may be implemented as a server system capable of receiving network-based service requests from client 250 and forwarding them to components of a system implementing database service 210a or 210b, storage service 220a or 220b, and / or another virtual computing service 230a or 230b for processing. In other embodiments, the provider network area may be configured to implement multiple load-balanced systems (e.g., in a cluster topology), and other request management features may dynamically manage large-scale network-based service request processing loads. In various embodiments, the provider network area may support REST-style or document-based (e.g., SOAP-based) type network-based service requests.

[0035] In addition to acting as an addressable endpoint for client-side network-based service requests, in some embodiments, the provider network area may implement various client management features. For example, provider network area 200a may coordinate the metering and billing of client usage of network-based services (including storage resources) by tracking, for instance, the identity of requesting client 250, the number and / or frequency of client requests, the size of data tables (or records thereof) stored or retrieved on behalf of client 250, the total storage bandwidth used by client 250, the storage category requested by client 250, or any other measurable client usage parameters. The provider network area may also implement a financial accounting and billing system, or may maintain a database of usage data that can be queried and processed by external systems to report and bill client usage activities. In some embodiments, the provider network area may collect, monitor, and / or aggregate various storage service system operation metrics, such as metrics reflecting the rate and type of requests received from client 250, the bandwidth utilized by such requests, the system processing latency of such requests, system component utilization (e.g., target capacity determined for individual database engine head node instances), network bandwidth and / or storage utilization, the rate and type of errors caused by requests, storage and database characteristics (e.g., size, data type, etc.), or any other suitable metrics. In some embodiments, system administrators may use such metrics to tune and maintain system components, while in other embodiments, such metrics (or relevant portions thereof) may be exposed to client 250 to enable such clients to monitor their use of database service 210a or 210b, storage service 220a or 220b, and / or another virtual computing service 230a or 230b (or the underlying system implementing those services).

[0036] In some embodiments, the provider network area may also implement user authentication and access control procedures. For example, for a given web-based service request to access a specific database table, the provider network area verifies whether the client 250 associated with the request is authorized to access the specific database table. The provider network area may determine such authorization by, for example, evaluating identity, password, or other credentials against credentials associated with the specific database table, or by evaluating the requested access to the specific database table against an access control list for the specific database table. For example, if client 250 does not have sufficient credentials to access the specific database table, the provider network area may, for example, deny the corresponding web-based service request by returning a response indicating an error condition to the requesting client 250. Various access control policies may be stored as records or lists of access control information by the database service 210a or 210b, storage service 220a or 220b, and / or other virtual computing services 230a or 230b.

[0037] It should be noted that in many of the examples described herein, services such as database services or storage services may reside within the computing system or enterprise system providing database services to client 250 and may not be exposed to external clients (e.g., users or client applications). In such embodiments, internal "clients" (e.g., database service 210a) may access storage service 220a via a local or private network (e.g., via an API directly between systems implementing these services). In such embodiments, the use of storage service 220a in storing database tables on behalf of client 250 may be transparent to those clients. In other embodiments, storage service 220a or 220b may be exposed to client 250 via provider network area 200a or 200b, thereby providing storage of database tables or other information for applications other than those that rely on database service 210a or 210b for database management. In such embodiments, clients of storage service 220a or 220b may access storage service 220a or 220b via network 260 (e.g., via the Internet). In some embodiments, virtual computing services 230a or 230b may receive or use data from storage services 220a or 220b (e.g., via an API directly between virtual computing services 230a or 230b and storage services 220a or 220b) to store objects on behalf of client 250 for performing computing services 230a or 230b. In some cases, billing and / or credentialing services within a provider network area may be unnecessary for, for example, internal clients managing clients or service components within the same enterprise.

[0038] Figure 3 This is a block diagram illustrating various components of a database service and storage service for a database accessible to a hosted database client according to some embodiments. Database service 210 (instantiated as database service 210a in region 200a and database service 210b in region 200b) may implement control plane 340, which may manage the creation, provisioning, deletion, or other characteristics of the database hosted in database service 210. For example, control plane 340 may monitor hosts (e.g., as described below regarding...). Figure 8The high workload (e.g., hot) performance of the discussed computing system (or apparatus such as computing system 1000) and the removal or redirection of database engine head node instances from some hosts to avoid overloading host 310. Control plane 340 can handle various management requests, such as requests to create and manage the database (e.g., by configuring or modifying performance, such as by enabling "serverless" or other automated management functions in response to requests that may cause the database to enable in-situ resource scaling). Control plane 340 can guide the placement of database engine head node instances on hosts to distribute workloads across hosts to avoid failure scenarios such as insufficient memory.

[0039] Database service 210 may implement one or more different types of database systems with corresponding query processors for accessing database data as part of the database. For example, database service 210 may implement various types of connection-based (e.g., a network connection has already been established between the database client and database instance 320) database systems, which may, for example, facilitate the execution of various operations continuing on multiple communications between the database client and the connected database instance 320. In at least some embodiments, database service 210 may be a relational database service representing a client-hosted relational database.

[0040] Database service 210 may implement host group 310, which in various embodiments can provide a multi-tenant configuration, allowing different database instances (e.g., database instances 320a and 320b) to be hosted on the same host 310, but providing access to different databases on behalf of different clients through different connections. In some embodiments, host 310 may not be multi-tenant.

[0041] In various embodiments, the host may implement virtualization technologies, such as virtual machine-based virtualization or container-based virtualization, wherein the database instance 320 may be a different corresponding virtual machine, a microVM that can provide a reduced or lightweight virtual machine implementation that retains the use of individual kernels within a microVM, or a container that provides virtualization of the operating system using a shared kernel. The host may implement a virtualization manager that can support hosting one or more individual query processors 321 as different corresponding VMs, microVMs, or containers. The virtualization manager can support increasing or decreasing the resources available to the host for other tasks.

[0042] The host system can support various features for accessing the database, such as those mentioned below. Figures 4 to 5The query processor 321 and the adjudicator 323 are discussed in detail. The query processor 321 can implement agents, interfaces or other controls, and other supporting aspects of virtualization, depending on the type of virtualization used to collect and facilitate communications for utilization metrics for in-situ scaling.

[0043] In some embodiments, the database data for the database used in database service 210 may be stored in a separate storage service 220. In some embodiments, storage service 220 may be implemented to store the database data as a virtual disk or other persistent storage drive. In other embodiments, storage service 220 may use a log-structured storage to store the database data.

[0044] For example, data can be organized across various logical volumes, segments, and pages for storage on one or more storage nodes 360 of the storage service 220. For example, in some embodiments, each database may be represented by a logical volume, and each logical volume may be segmented across a set of storage nodes 360. In some embodiments, each segment, possibly located on a specific storage node within the storage nodes, may contain a set of contiguous block addresses. In some embodiments, each segment may store a set of one or more data pages and a change log (also called a redo log) for each data page it stores (e.g., a log of redo log entries). Storage nodes 360 may receive redo log entries and merge them to create new versions of the corresponding data pages and / or append or replace log entries (e.g., lazily and / or in response to a request for a data page or database crash). In some embodiments, data pages and / or change logs may be mirrored across multiple storage nodes, depending on a variable configuration (which may be specified by a client maintaining the database on its behalf in the database system). For example, in different embodiments, one, two, or three copies of data or change logs may be stored in each of one, two, or three different availability zones or areas, depending on the default configuration, dedicated persistence preferences, or client-specified persistence preferences.

[0045] In some embodiments, a volume can be a logical concept representing a highly persistent storage unit as understood by the user / client / application of a storage system. In some embodiments, a volume is a distributed storage area that, from the user / client / application's perspective, is a single, consistent, ordered log of write operations on various user pages of a database. In some embodiments, each write operation can be encoded in a log record (e.g., a redo log record), which can represent a logically ordered mutation of the content of a single user page within the volume. In some embodiments, each log record can contain a unique identifier (e.g., a logical sequence number (LSN)). In some embodiments, each log record can be persisted to one or more synchronization segments forming a Protection Group (PG) in the distributed storage area, thereby providing high durability and availability for the log record. In some embodiments, a volume can provide an LSN-type read / write interface for a variable-size contiguous byte range.

[0046] In some embodiments, a volume may consist of multiple segments, each segment becoming persistent through a protection group. In such embodiments, a volume may represent a storage unit consisting of a variable sequence of contiguous volume segments. Reads and writes to a volume can be mapped to corresponding reads and writes to the segments constituting the volume. In some embodiments, the size of a volume can be changed by adding or removing volume segments from the end of the volume.

[0047] In some embodiments, a segment may be a finite persistent storage unit assigned to a single storage node. In some embodiments, a segment may provide limited best-effort persistence for a specific fixed-size byte range of data (e.g., as a persistent but non-redundant single point of failure for the storage node). In some cases, this data may be a mirror of user-addressable data, or in various embodiments, the data may be other data, such as volume metadata or erasure bits. In some embodiments, a given segment may reside on exactly one storage node. In some embodiments, multiple segments may reside on each storage device (e.g., SSD) within a storage node, and each segment may be limited to one SSD (e.g., a segment may not span multiple SSDs). In some embodiments, a segment may not need to occupy a contiguous region on an SSD; in fact, an allocation map describing the region owned by each segment may exist in each SSD. As described above, in some embodiments, a protection group may consist of multiple segments distributed across multiple storage nodes. In some embodiments, a segment may provide an LSN-type read / write interface for a fixed-size contiguous byte range (wherein the size is defined at creation time). In some embodiments, each segment may be identified by a segment UUID (e.g., a universally unique identifier for a segment).

[0048] In some embodiments, a page can be a storage block that is typically of a fixed size. In some embodiments, each page can be a storage block (e.g., a block of storage in virtual memory, disk, or other physical memory) of a size defined by the operating system, and may also be referred to herein as a "data block". In some embodiments, a page can be a set of contiguous sectors. In some embodiments, a page can serve as an allocation unit in a storage device, and as a unit in a log page containing headers and metadata. In some embodiments, the term "page" or "storage page" can be a similar block of a size defined by the database configuration, which is typically a multiple of 2, such as 4096, 8192, 16384, or 32768 bytes.

[0049] In some embodiments, the storage node 360 ​​of the storage service 220 can perform some database system responsibilities, such as updating data pages in the database and, in some cases, performing some query processing on the data. Figure 3 As shown, storage node 360 ​​can implement data page request processing 361 and data management 365 to implement various features of these characteristics regarding data pages 367 and page logs 369 of redo log records stored in a database volume within a log-structured storage service. For example, data management 365 can perform at least some of any or all of the following operations: replication (locally, e.g., within the storage node), merging redo logs to generate data pages, snapshots (e.g., creation, recovery, deletion, etc.), clone volume creation, log management (e.g., manipulating log records), crash recovery, and / or space management (e.g., for segments). In some embodiments, each storage node may also have multiple attached storage devices (e.g., SSDs) on which data blocks may be stored on behalf of clients (e.g., users, client applications, and / or database service subscribers). Data page request processing 361 can process requests to return data pages of records from a database volume and can perform operations to merge redo log records or otherwise generate the data pages to be returned in response to a request.

[0050] In at least some embodiments, storage node 360 ​​can provide multi-tenant storage, such that data stored in part or all of a storage device can be stored for a different database, database user, account, or entity than data stored on the same storage device (or other storage device) attached to the same storage node. In some embodiments, various access control and security mechanisms can be implemented to ensure that data is not accessed at the storage node except for authorized requests (e.g., for users authorized to access the database, the database owner, etc.).

[0051] In some embodiments, the corresponding database journals, such as database journals 330a to 330d, may be hosted in a database service that stores ordered updates to a database (e.g., a database volume). The adjudicator 323 may be responsible for deciding whether a transaction can be committed or written (while adhering to isolation rules), working in conjunction with database journals 330a to 330d to order transactions, and for ensuring that committed data is strongly consistent.

[0052] Front-end 320 may implement proxy, request router, or other load balancing features that route database requests to one or more query processors 321. For example, front-end 320 may be responsible for authenticating requests to connect to a database at a specific network endpoint and assigning query processors 321 to connections (or, for example, specific requests for queries or transactions). Front-end 320 may maintain connections (e.g., as a proxy) so that separate connections do not have to be established if different query processors are used for different requests to the database.

[0053] Figure 4 This is a block diagram illustrating various interactions for handling database client requests according to some embodiments. In this example, one or more client applications 410 may store data into one or more databases maintained by database service 210. Client application 410 may submit database requests 414 (e.g., requests inducing reads, such as queries or read-only transactions, or requests inducing writes, such as update, insert, delete, or transactions containing write statements) and receive responses 417 from frontend 320.

[0054] Frontend 320 can dispatch database request 415 to query processor 430, which can parse the request and interact with different components depending on the type of request. For read requests, query processor 430 can submit a read request 431 for a data page, relying on the local cache and / or accessing storage node 425. The read request is returned 432 and used to perform the read. For write requests, write requests can be sent to adjudicator 440, which determines whether a conflict exists, and if not, writes 434 to log 450 and confirms the write 435 to query processor 430. Response 416 can then be sent to frontend 320 for response 417 to client application 410.

[0055] As mentioned above Figure 1 The discussed topic is the ability to replicate a database. In some scenarios, this replication can span regions. Figure 5This is a block diagram illustrating cross-region replication of a database hosted in a database service, according to some embodiments. In the example shown here, multiple query processors 532a and 532b of one or more database instances may be hosted in corresponding services in database services 509a and 509b in regions 505a and 505b to provide database services to clients accessing databases in different regions. It should be noted that although the example shown illustrates two regions, any number of regions can be implemented. As mentioned above regarding... Figure 4 As discussed, the query processor can provide read and write capabilities to the database in different areas using query processor 532a and adjudicator 534a.

[0056] In some embodiments, the adjudicator 534 may implement protocols to support cross-region transactions. For example, adjudicator 534a may communicate with adjudicator 534b and / or query processors 532a and 532b to determine whether a given write is conflicting or can be committed. Once committed, these changes can be written to journal 534a or 534b, which ultimately allows the corresponding journal applicators 550a and 550b to apply the changes to the corresponding copies of database volumes 540a and 540b in each region 505a and 505b.

[0057] like Figure 5 As shown, various communications can occur across the wide area network 560 between different components (e.g., query processors 532a and 532b, adjudicators 534a and 534b, loggers 536a and 536b, and log application processors 550a and 550b). Replication messages describing updates to the database can be sent between these components according to various types of synchronous replication technologies that can be implemented. Therefore, the above regarding... Figure 1 The techniques discussed can be applied to various possible replication messages exchanged across WAN 560 or within zones 505a and 505b.

[0058] For example, replication messages may include shared updates as part of a check to determine if conflicts exist (e.g., between query processor 532 and the arbiter, and between the arbiters (e.g., 534a and 534b)). In some embodiments, replication messages may be exchanged between the journal app and the storage node storing the database volume to ensure that updates are applied to the database volume in an orderly manner.

[0059] Figures 2 to 5 The database services and storage services discussed provide examples of systems that can minimize connection loss when changing the database query processor version. However, various other types of distributed database systems (such as those discussed above) also present challenges. Figure 1 These technologies can be implemented (as discussed). Figure 6This is a high-level flowchart illustrating various methods and techniques for implementing synchronous database replication using erasure coding, according to some embodiments. Various different systems and apparatuses of a distributed database system may implement the various methods and techniques described below, individually or together. For example, the control plane components of a distributed database system and / or database instance may implement various methods. Alternatively, combinations of different systems and apparatuses may implement these techniques. Therefore, the examples above and / or any other systems or apparatus described as performing the illustrated methods are not intended to limit the configuration of other different components, modules, systems, or systems and apparatuses.

[0060] As indicated at 610, according to some embodiments, a request initiating a write to the database is received at a first instance of the distributed database system. The write request may include various API commands, query language statements, transactions or other multipart operations, or various other types of requests supported by the distributed database system. In some embodiments, the write may include the writing of system metadata or other information not visible to clients of the distributed database system.

[0061] As indicated at 620, according to some embodiments, replication messages for writing to a database can be divided and encoded into multiple chunks according to an erasure coding scheme that allows the replication message to be reassembled in fewer than the number of chunks. (See above regarding...) Figure 1 The discussion can apply to many different erasure coding schemes, including but not limited to simple parity check, Reed-Solomon or other maximum distance separable codes (for any N>k), or rateless erasure coding schemes, such as fountain codes and slash dragon codes.

[0062] As indicated at 630, according to some embodiments, chunks of replication messages can be sent across a network to a second database instance of a distributed database system, the network containing multiple redundant paths providing alternative routes between the first and second database instances. The second database instance can be identified according to the appropriate replication protocol or technology applied. For example, the second database instance can be a follower node applying writes committed by a leader node (e.g., the first database instance). In some embodiments, the second database instance can be a participant in conflict checking or other write resolution techniques that satisfy a conflict check or other write resolution technique if a certain number (or all) of other database instances agree to commit or execute the update described in the replication message. Replication messages can be sent to more than one second database instance (e.g., one or more additional database instances). The same erasure coding technique and chunks can be reused, or in some embodiments, different erasure coding techniques or chunks can be used.

[0063] As indicated at 640, it can be determined whether an acknowledgment for the write has been received. If so, a response to the request can be sent, indicating that the write has been submitted to the database, as indicated at 650. If not, in some embodiments, a response to the request can be sent, indicating that the write failed. As discussed above, different types, quantities, and sources of acknowledgments can be received depending on the type of replication protocol implemented.

[0064] Figure 7 This is a high-level flowchart illustrating various methods and techniques for reassembling replication message chunks using erasure coding, according to some embodiments. As indicated at 710, according to some embodiments, chunks of replication messages for writing to a database in a distributed database system can be received separately. For example, chunks can be received directly from the sending database instance. In some embodiments, one or more chunks can be received from other recipients who also acknowledge the replication message chunks, using rumors or other protocols that exchange information between receiving instances of the replication messages.

[0065] As indicated at 720, it can be determined that a sufficient number of chunks for recompiling the replication has been received. If not, a 760 error response can be sent (or further waiting for other chunks). In some embodiments, additional chunks can be received from other receiving database instances. In some embodiments, a request for lost chunks required for the recompiling replication message can be sent to the transport database instance.

[0066] As indicated at 730, chunks can be reassembled into the replication message according to an erasure coding scheme. In some embodiments, parameters, flags, or other information regarding which erasure coding scheme to apply can be specified or indicated to the receiving database instance. In some embodiments, a default erasure coding scheme can be applied. As indicated at 740, in some embodiments, a write indicated by the replication message can be applied. As indicated at 750, an acknowledgment indicating the application of the write can be sent.

[0067] In various embodiments, the methods described herein can be implemented using any combination of hardware and software. For example, in one embodiment, the methods can be implemented on one or more computer systems (e.g., Figure 8The method is implemented on or across one or more computer systems, which include one or more processors that execute program instructions stored on one or more computer-readable storage media coupled to the processors. The program instructions can implement the functionality described herein (e.g., the functionality of various servers and other components of a network-based virtual computing resource provider as described herein). The various methods illustrated in the accompanying drawings and described herein represent example embodiments of the methods. The order of any method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0068] Figure 8 This is a block diagram illustrating an example computer system according to various embodiments. For example, in various embodiments, computer system 1000 may use erasure coding to implement synchronous database replication. Computer system 1000 may be any of a variety of types of devices, including but not limited to personal computer systems, desktop computers, laptop or notebook computers, mainframe computer systems, handheld computers, workstations, network computers, consumer devices, application servers, storage devices, telephones, mobile phones, or generally any type of computing device.

[0069] Computer system 1000 includes one or more processors 1010 (each processor may contain multiple cores, which may be single-threaded or multi-threaded) coupled to system memory 1020 via input / output (I / O) interface 1030. Computer system 1000 further includes a network interface 1040 coupled to I / O interface 1030. In various embodiments, computer system 1000 may be a single-processor system containing one processor 1010, or a multiprocessor system containing several processors 1010 (e.g., two, four, eight, or another suitable number). Processor 1010 may be any suitable processor capable of executing instructions. For example, in various embodiments, processor 1010 may be a general-purpose or embedded processor implementing any of a variety of instruction set architectures (ISAs), such as x86, PowerPC, SPARC, or MIPS ISA or any other suitable ISA. In a multiprocessor system, each of the processors 1010 may typically, but not necessarily, implement the same ISA. Computer system 1000 also includes one or more network communication devices (e.g., network interface 1040) for communicating with other systems and / or components via a communication network (e.g., the Internet, LAN, etc.). For example, a client application executing on system 1000 may use network interface 1040 to communicate with a server application executing on a single server or server cluster, said single server or server cluster implementing one or more components of the database system described herein. In another example, an instance of a server application executing on computer system 1000 may use network interface 1040 to communicate with other instances of a server application (or another server application) that may be implemented on other computer systems (e.g., computer system 1090).

[0070] In the illustrated embodiment, the computer system 1000 further includes one or more persistent storage devices 1060 and / or one or more I / O devices 1080. In various embodiments, the persistent storage device 1060 may correspond to a disk drive, tape drive, solid-state storage, other mass storage device, or any other persistent storage device. The computer system 1000 (or a distributed application or operating system running thereon) can store instructions and / or data in the persistent storage device 1060 as needed and can retrieve the stored instructions and / or data as needed. For example, in some embodiments, the computer system 1000 may host a storage system server node, and the persistent storage device 1060 may include an SSD attached to the server node.

[0071] Computer system 1000 includes one or more system memories 1020 storing instructions and data accessible by processor 1010. In various embodiments, system memory 1020 may be implemented using any suitable memory technology (e.g., one or more of the following: cache, static random access memory (SRAM), DRAM, RDRAM, EDO RAM, DDR10 RAM, synchronous dynamic RAM (SDRAM), Rambus RAM, EEPROM, non-volatile / flash memory, or any other type of memory). System memory 1020 may contain program instructions 1025 executable by processor 1010 to implement the methods and techniques described herein. In various embodiments, program instructions 1025 may be platform-native binary, such as Java. TM Any interpreted language such as bytecode, or languages ​​like C / C++ and Java. TM It can be encoded in any other language or any combination thereof. In some embodiments, program instructions 1025 may be implemented as multiple separate client, server nodes and / or other components.

[0072] In some embodiments, program instructions 1025 may include instructions executable to implement an operating system (not shown), such as UNIX, LINUX, or Solaris. TM MacOS TM Windows TMAny of various operating systems. Any or all of the program instructions 1025 may be provided as a computer program product or software that may include a non-transitory computer-readable storage medium on which the instructions are stored, said instructions being used to program a computer system (or other electronic device) to perform processes according to various embodiments. The non-transitory computer-readable storage medium may include any mechanism for storing information in a machine-readable form (e.g., software, processing application). Generally, the non-transitory computer-accessible medium may include a computer-readable storage medium or memory medium, such as a magnetic or optical medium, for example, a disk or DVD / CD-ROM coupled to the computer system 1000 via I / O interface 1030. The non-transitory computer-readable storage medium may also include any volatile or non-volatile medium (such as RAM (e.g., SDRAM, DDR SDRAM, RDRAM, SRAM, etc.), ROM, etc.), which may be included as system memory 1020 or another type of memory in some embodiments of the computer system 1000. In other embodiments, program instructions may be transmitted using optical, acoustic, or other forms of propagation signals (e.g., carrier waves, infrared signals, digital signals, etc.) transmitted via communication media such as networks and / or wireless links (e.g., implemented via network interface 1040).

[0073] In some embodiments, system memory 1020 may include data storage area 1045 that can be configured as described herein. In one embodiment, I / O interface 1030 may coordinate I / O traffic between processor 1010, system memory 1020, and any peripheral device in the system, including via network interface 1040 or other peripheral interfaces. In some embodiments, I / O interface 1030 may perform any necessary protocols, timing, or other data transformations to convert data signals from one component (e.g., system memory 1020) into a format suitable for use by another component (e.g., processor 1010). In some embodiments, I / O interface 1030 may include support for devices attached via various types of peripheral buses, such as the Peripheral Component Interconnect (PCI) bus standard or variants of the Universal Serial Bus (USB) standard. In some embodiments, the functionality of I / O interface 1030 may be divided into two or more separate components, such as a northbridge and a southbridge. Moreover, in some embodiments, some or all of the functionality of I / O interface 1030 (e.g., the interface to system memory 1020) may be directly incorporated into processor 1010.

[0074] For example, network interface 1040 may allow data exchange between computer system 1000 and other devices attached to the network, such as other computer systems 1090 (which may implement one or more storage system server nodes, master nodes, read-only nodes, and / or clients of the database systems described herein). Additionally, network interface 1040 may allow communication between computer system 1000 and various I / O devices 1050 and / or remote storage devices 1070. In some embodiments, input / output device 1050 may include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other device suitable for inputting or retrieving data by one or more computer systems 1000. Multiple input / output devices 1050 may reside within computer system 1000 or may be distributed across various nodes in a distributed system containing computer system 1000. In some embodiments, similar input / output devices may be separate from computer system 1000 and may interact with one or more nodes in a distributed system containing computer system 1000 via wired or wireless connections (such as through network interface 1040). Network interface 1040 typically supports one or more wireless networking protocols (e.g., Wi-Fi / IEEE 802.11 or another wireless networking standard). However, in various embodiments, network interface 1040 can support communication over any suitable wired or wireless general-purpose data network (e.g., other types of Ethernet). Additionally, network interface 1040 can support communication via telecommunications / telephone networks, such as analog voice networks or digital fiber optic communication networks, via storage area networks (SANs), such as Fibre Channel, or via any other suitable type of network and / or protocol. In various embodiments, computer system 1000 may include more than Figure 8 The components shown may include more, fewer, or different components (e.g., monitor, graphics card, sound card, peripherals such as ATM interface, Ethernet interface, Frame Relay interface, and other network interfaces).

[0075] It should be noted that any one or any component of the distributed system embodiments described herein can be implemented as one or more network-based services. For example, read-write nodes and / or read-only nodes within the database layer of a database system can present database services and / or other types of data storage services using the distributed storage system described herein to clients as network-based services. In some embodiments, the network-based service can be implemented by software and / or hardware systems designed to support interoperable machine-to-machine interactions on a network. The network service may have an interface described in a machine-processable format such as a Network Services Description Language (WSDL). Other systems can interact with the network-based service in a manner specified by the description of the network-based service's interface. For example, the network-based service may define various operations that other systems can invoke and may define specific application programming interfaces (APIs) that other systems can be expected to follow when requesting various operations.

[0076] In various embodiments, a network-based service can be requested or invoked using a message containing parameters and / or data associated with the network-based service request. Such a message can be formatted according to a specific markup language such as Extensible Markup Language (XML), and / or encapsulated using a protocol such as Simple Object Access Protocol (SOAP). To execute a network-based service request, the network-based service client can assemble a message containing the request using an internet-based application-layer transport protocol such as Hypertext Transfer Protocol (HTTP) and deliver the message to an addressable endpoint (e.g., a Uniform Resource Locator (URL)) corresponding to the network service.

[0077] In some embodiments, web-based services may be implemented using expressive state transfer (“RESTful”) techniques instead of message-based techniques. For example, a web-based service implemented according to RESTful techniques may be invoked using parameters included in HTTP methods such as PUT, GET, or DELETE, rather than encapsulated in a SOAP message.

[0078] Although the embodiments described above have been described in considerable detail, many variations and modifications will become apparent to those skilled in the art once the foregoing disclosure is fully understood. It is intended that the appended claims be construed as covering all such modifications and changes, and therefore, the above description should be considered illustrative rather than restrictive.

[0079] Embodiments of this disclosure may be described in accordance with the following terms: Clause 1. A system comprising: Multiple computing devices, each including at least one processor and memory configured to implement a distributed database system; The distributed database system includes multiple database instances, and the first database instance among the multiple database instances is configured as follows: Receive a request that causes a write to the database hosted by the distributed database system; The writes to the database are synchronously replicated to a second or more database instances among the plurality of database instances, wherein, in order to synchronously replicate the writes, the first database instance among the plurality of database instances is configured to: Generate a replication message for the write to include the write as part of an ordered replication stream from the first database instance in the database instance to the second or more database instances in the plurality of database instances; An erasure coding scheme is applied to the replication message, wherein the erasure coding is used to divide and encode the replication message written to the database into multiple chunks, wherein the erasure coding scheme allows the replication message to be reassembled with fewer than the number of chunks; and The replication message chunks are sent individually across a network to a second or more database instances among the plurality of database instances, the network comprising multiple redundant paths providing alternative routes between the first database instance and the second or more database instances among the plurality of database instances; and Receive corresponding confirmation that the write has been applied to the second or more database instances among the plurality of database instances; and After receiving the corresponding confirmation that the write has been applied to the second or more database instances among the plurality of database instances, a response to the request is sent, the response indicating that the write has been submitted to the database.

[0080] Clause 2. The system according to Clause 1, wherein the first database of the plurality of database instances is further configured to: After a period of time without receiving at least one of the corresponding acknowledgments, one or more of the blocks of the copy message are resent, or one or more additional blocks of the copy message determined according to the erasure coding scheme are sent.

[0081] Clause 3. The system according to any one of Clauses 1 to 2, wherein the second or more database instances of the plurality of database instances are configured to: At least some of the blocks that receive the copy message; Determine that a sufficient number of the blocks have been received for reassembling the replication message; Reassemble the copied message; The write indicated in the copy message; and Send confirmation that the write has been submitted to the first database instance among the plurality of database instances.

[0082] Clause 4. The system according to any one of Clauses 1 to 3, wherein the first database instance of the plurality of database instances is in a first region of the provider network implementing the distributed database system, and wherein the second or more database instances of the plurality of database instances are in a second region of the provider network.

[0083] Clause 5. A method comprising: Receive the request that triggers a write to the database at the first database instance of the distributed database system; The distributed database system synchronously replicates the writes to the database to a second database instance of the distributed database system, including: The replication message used for writing to the database is divided and encoded into multiple chunks according to an erasure coding scheme, wherein the erasure coding scheme allows the replication message to be reassembled in fewer than the multiple chunks; The replication message chunks are sent across a network to the second database instance, the network comprising multiple redundant paths providing alternative routes between the first and second database instances; and Receive confirmation at the first database instance that the write has been applied to the second database instance; and The first database instance sends a response to the request, the response indicating that the write has been submitted to the database.

[0084] Clause 6. The method described in Clause 5 further includes: If no confirmation is received after a period of time, one or more of the blocks of the copy message are resent, or one or more additional blocks of the copy message determined according to the erasure coding scheme are sent.

[0085] Clause 7. The method according to any one of Clauses 5 to 6 further comprises: The distributed database system synchronously replicates the writes to the database to a third database instance of the distributed database system, including: The replication message used for writing to the database is divided and encoded into the multiple blocks according to the erasure coding scheme. The replication message chunk is sent across the network to the third database instance, the network including multiple additional redundant paths that provide alternative routes between the first and third database instances; and At the first database instance, an acknowledgment is received that the write has been applied to the third database instance.

[0086] Clause 8. The method according to any one of Clauses 5 to 7 further comprises: At the first database instance of the distributed database system, a second request is received that causes a second write to the database. According to the erasure coding scheme, the second copy message used for the second write to the database is divided and encoded into a second number of blocks; Send the chunk of the second replication message to the second database instance across the network; and After failing to receive confirmation that the second write has been applied to the second database instance, the first database instance sends a response to the request, indicating that the second request failed.

[0087] Clause 9. The method according to any one of Clauses 5 to 8, wherein the adjudicator for the second database reorganizes at least some of the blocks of the replication message to apply the write to the second database instance.

[0088] Clause 10. The method according to any one of Clauses 5 to 9, wherein the erasure coding scheme is a rateless erasure coding scheme.

[0089] Clause 11. The method according to any one of Clauses 5 to 10, further comprising: At least some of the blocks of the replication message are received by the second database instance; The second database instance determines that a sufficient number of the blocks for reassembling the replication messages have been received; The replication message is reassembled by the second database instance; The write indicated in the replication message is applied by the second database instance; and The second database instance sends an acknowledgment that the write has been submitted to the first database instance among the plurality of database instances.

[0090] Clause 12. The method according to any one of Clauses 5 to 11, wherein the first database instance is in a first region of a provider network implementing the distributed database system, and wherein the second database instance is in a second region of the provider network.

[0091] Clause 13. The method according to any one of Clauses 5 to 13, wherein the distributed database system is a non-relational database service implemented as part of a provider network.

[0092] Clause 14. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform: Receive the request that triggers a write to the database at the first database instance of the distributed database system; The replication message used for writing to the database is divided and encoded into multiple chunks according to an erasure coding scheme, wherein the erasure coding scheme allows the replication message to be reassembled in fewer than the multiple chunks; The replication message chunks are sent across a network to a second database instance, the network comprising multiple redundant paths providing alternative routes between the first and second database instances; and Receive confirmation at the first database instance that the write has been applied to the second database instance; and After receiving confirmation that the write has been applied to the second database instance, the first database instance sends a response to the request, the response indicating that the write has been submitted to the database.

[0093] Clause 15. One or more non-transitory computer-readable storage media as described in Clause 14, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: If no confirmation is received after a period of time, one or more of the blocks of the copy message are resent, or one or more additional blocks of the copy message determined according to the erasure coding scheme are sent.

[0094] Clause 16. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 15, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: At the first database instance of the distributed database system, a second request is received that causes a second write to the database. According to the erasure coding scheme, the second copy message used for the second write to the database is divided and encoded into a second number of blocks; Send the chunk of the second replication message to the second database instance across the network; and After failing to receive confirmation that the second write has been applied to the second database instance, the first database instance sends a response to the request, indicating that the second request failed.

[0095] Clause 17. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 15, wherein a arbiter for the second database reorganizes at least some of the blocks of the replication message to apply the write to the second database instance.

[0096] Clause 18. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 17, wherein the erasure coding scheme is a maximum distance separable erasure coding scheme.

[0097] Clause 19. One or more non-transitory computer-readable storage media according to any one of Clauses 14 to 18, wherein the first database instance is in a first area of ​​a provider network implementing the distributed database system, and wherein the second database instance is in a second area of ​​the provider network.

[0098] Clause 20. One or more non-transitory computer-readable storage media pursuant to any one of Clauses 14 to 19, wherein the distributed database system is a relational database service implemented as part of a provider network.

Claims

1. A system comprising: Multiple computing devices, each including at least one processor and memory configured to implement a distributed database system; The distributed database system includes multiple database instances, and the first database instance among the multiple database instances is configured as follows: Receive a request that causes a write to the database hosted by the distributed database system; The writes to the database are synchronously replicated to a second or more database instances among the plurality of database instances, wherein, in order to synchronously replicate the writes, the first database instance among the plurality of database instances is configured to: The replication message used for writing to the database is divided and encoded into multiple chunks according to an erasure coding scheme, wherein the erasure coding scheme allows the replication message to be reassembled in fewer than the multiple chunks; The chunk of the replication message is sent across a network to a second or more database instances among the plurality of database instances, the network including multiple redundant paths that provide alternative routes between the first database instance among the plurality of database instances and the second or more database instances among the plurality of database instances; as well as Receive the corresponding confirmation that the write operation has been submitted to the database; as well as Send a response to the request, the response indicating that the write has been submitted to the database.

2. The system according to claim 1, wherein the first database in the plurality of database instances is further configured to: After a period of time without receiving at least one of the corresponding acknowledgments, one or more of the blocks of the copy message are resent, or one or more additional blocks of the copy message determined according to the erasure coding scheme are sent.

3. The system according to any one of claims 1 to 2, wherein the second or more database instances of the plurality of database instances are configured to: At least some of the blocks that receive the copy message; Determine that a sufficient number of the blocks have been received for reassembling the replication message; Reassemble the copied message; The write indicated in the copy message; as well as Send confirmation that the write has been submitted to the first database instance among the plurality of database instances.

4. The system according to any one of claims 1 to 3, wherein the first database instance of the plurality of database instances is in a first region of the provider network implementing the distributed database system, and wherein the second or more database instances of the plurality of database instances are in a second region of the provider network.

5. A method comprising: Receive the request that triggers a write to the database at the first database instance of the distributed database system; The distributed database system synchronously replicates the writes to the database to a second database instance of the distributed database system, including: The replication message used for writing to the database is divided and encoded into multiple chunks according to an erasure coding scheme, wherein the erasure coding scheme allows the replication message to be reassembled in fewer than the multiple chunks; The replication message chunks are sent across a network to the second database instance, the network comprising multiple redundant paths providing alternative routes between the first and second database instances; and Receive confirmation at the first database instance that the write has been applied to the second database instance; and The first database instance sends a response to the request, the response indicating that the write has been submitted to the database.

6. The method of claim 5, further comprising: If no confirmation is received after a period of time, one or more of the blocks of the copy message are resent, or one or more additional blocks of the copy message determined according to the erasure coding scheme are sent.

7. The method according to any one of claims 5 to 6, further comprising: The distributed database system synchronously replicates the writes to the database to a third database instance of the distributed database system, including: The replication message used for writing to the database is divided and encoded into the multiple blocks according to the erasure coding scheme. The replication message chunk is sent across the network to the third database instance, the network including multiple additional redundant paths that provide alternative routes between the first and third database instances; and At the first database instance, an acknowledgment is received that the write has been applied to the third database instance.

8. The method according to any one of claims 5 to 7, further comprising: At the first database instance of the distributed database system, a second request is received that causes a second write to the database. According to the erasure coding scheme, the second copy message used for the second write to the database is divided and encoded into a second number of blocks; The chunk of the second replication message is sent across the network to the second database instance; as well as After failing to receive confirmation that the second write has been applied to the second database instance, the first database instance sends a response to the request, indicating that the second request failed.

9. The method according to any one of claims 5 to 8, wherein the adjudicator for the second database reorganizes at least some of the blocks of the replication message to apply the write to the second database instance.

10. The method according to any one of claims 5 to 9, wherein the erasure coding scheme is a rateless erasure coding scheme.

11. The method according to any one of claims 5 to 10, further comprising: At least some of the blocks of the replication message are received by the second database instance; The second database instance determines that a sufficient number of the blocks for reassembling the replication messages have been received; The replication message is reassembled by the second database instance; The write indicated in the replication message is applied by the second database instance; as well as The second database instance sends an acknowledgment that the write has been submitted to the first database instance among the plurality of database instances.

12. The method according to any one of claims 5 to 11, wherein the first database instance is in a first region of a provider network implementing the distributed database system, and wherein the second database instance is in a second region of the provider network.

13. The method according to any one of claims 5 to 12, wherein the distributed database system is a non-relational database service implemented as part of a provider network.

14. One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform: Receive the request that triggers a write to the database at the first database instance of the distributed database system; The replication message used for writing to the database is divided and encoded into multiple chunks according to an erasure coding scheme, wherein the erasure coding scheme allows the replication message to be reassembled in fewer than the multiple chunks; The chunk of the replication message is sent across a network to a second database instance, the network containing multiple redundant paths that provide alternative routes between the first database instance and the second database instance; as well as At the first database instance, receive confirmation that the write has been applied to the second database instance; as well as After receiving confirmation that the write has been applied to the second database instance, the first database instance sends a response to the request, the response indicating that the write has been submitted to the database.

15. One or more non-transitory computer-readable storage media according to claim 14, storing additional program instructions that, when executed on or across the one or more computing devices, cause the one or more computing devices to further perform: If no confirmation is received after a period of time, one or more of the blocks of the copy message are resent, or one or more additional blocks of the copy message determined according to the erasure coding scheme are sent.

Images