Processing delete requests based on updated change feeds

Abstract

The disclosure herein describes the use of change feeds to process delete requests using sequence numbers. When a delete occurs on a source data repository, a delete notification is created in the change feed on the source server. The delete notification includes a set of delete record IDs that identify the set of records to be deleted, a tombstone sequence number (TSN) that identifies the sequence of delete notifications in the delete notification set, and / or a delete sequence number (DSN). The DSN is incremented by 1 each time a new delete notification is created. A delete notification can represent the deletion of a single record or a set of records. Each delete notification is assigned a time to live (TTL) value. The delete notification is deleted when the TTL expires. The TSN and DSN entries are used to determine whether any delete updates have been missed to prevent silent failures.

Description

Background Art

[0001] Some database systems list changes to database data in order of their modification time. This list of changes, or change feed, includes writes and other modifications to stored data. However, these systems do not record or log deletions of stored data. A deleted marker can be added to the change feed record to simulate an update upon deletion. Depending on the configured expiration time, the deleted marker can be permanently removed after a period of time. However, users or applications might not be able to read the change feed until the deleted marker's expiration time expires. In this case, the user or application might never know that the document was deleted, which can cause silent failures. Summary of the Invention

[0002] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0003] A computerized method for creating a delete notification in a change feed associated with a source data repository storing data. The delete notification includes a delete record ID set identifying a set of records to be deleted and a tombstone sequence number (TSN) identifying the sequence of delete notifications within the delete notification set. The current delete index, the delete notification sequence number (DSN), in the change feed is incremented by one. A time-to-live (TTL) value is assigned to the delete notification. The delete notification is removed upon expiration of a period corresponding to the TTL value. The record set associated with the delete notification is permanently deleted from the source data repository. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The disclosed examples are described in detail below with reference to the accompanying drawings:

[0005] Figure 1 is an exemplary block diagram illustrating a system 100 for processing delete requests using a change feed.

[0006] Figure 2 is an exemplary block diagram illustrating a system 200 for updating a target data repository based on changes to a source data repository.

[0007] Figure 3 is an exemplary block diagram illustrating a source server creating a delete notification reflecting changes to a source data repository that is used to update a target data repository.

[0008] Figure 4 is an exemplary block diagram illustrating a target server updating a data repository and initiating a recovery process when data becomes out of sync with a source data repository.

[0009] Figure 5 is an exemplary block diagram illustrating deletion notifications with tombstone sequence numbers (TSNs).

[0010] Figure 6 is an exemplary flow chart illustrating the operation of a computing device to delete a record and update a change feed including a TSN.

[0011] Figure 7 is an exemplary flow diagram illustrating the operation of a computing device to update a local data repository based on change feed updates received from a source server.

[0012] Figure 8 is an exemplary flow diagram illustrating the operation of a computing device to update a local data repository based on updates from a change feed.

[0013] Figure 9 is an exemplary flow chart illustrating the operation of a computing device to detect data inconsistencies between a source data repository and a local read-only copy of the data.

[0014] Figure 10 is an exemplary block diagram illustrating an example computing environment suitable for implementing some of the various examples disclosed herein.

[0015] Corresponding reference numerals indicate corresponding parts throughout the drawings. DETAILED DESCRIPTION

[0016] Some services (such as, but not limited to, Azure IoT Hub) use database systems with a change feed feature to log changes to data as persistent storage. In one example, a database system (such as, but not limited to, the CosmosDB system) provides this change feed logging feature to enable users to obtain a list of items in order of their modification time. The change feed also supports the ability to read the feed from an offset called a transaction commit sequence number (LSN) / transaction timestamp "etag", which can be saved and used to continue reading the feed at the next unprocessed change entry. However, the change feed does not support deletions. Applications that are reading and processing entries in the change feed sequence are unaware of deletions because deleted records do not appear in the change feed. This can lead to missed updates and incorrect update behavior.

[0017] A "deleted" flag can be added to documents to simulate an update upon deletion. When a document is deleted, the "deleted" flag on the record is toggled or set to "true." This can result in a growing number of deleted document entries, where the number of records with deleted documents may exceed the number of active records. This can lead to data explosion unless the deleted item data is eventually purged.

[0018] If the service supports record expiration, a time-to-live (TTL) field can be added to the record. Deleted item records are automatically deleted when the TTL expires. However, this approach presents a problem: users might not be able to read the change feed until after the deleted document's TTL expires. In this case, users or applications reading the change feed might never know that the document was deleted and thus be unable to determine which deletion updates were missed.

[0019] In one example, instead of issuing a DELETE / recordId request, the request is PUT / recordId with a body of {deleted:true,ttl:3600} indicating an expiration time of one hour. If the application or the server hosting the application is down for a period exceeding the TTL, the application might not be able to read the change feed until after the expiration time, in which case the entry is deleted from the change feed. The application will never be aware of the deletion.

[0020] Aspects of the present disclosure provide a change feed component that creates delete notifications in a change feed associated with a source data repository that stores data. The delete notification includes a set of delete record IDs that identify the set of records to be deleted and a tombstone sequence number (TSN) that identifies the sequence of delete notifications in the delete notification set. The TSN is used to verify that no delete notifications have been missed. This enables more accurate updating of data using change feeds while reducing silent failures and reducing the number of times that an entire snapshot of the source data must be re-copied to a target data repository. This reduces processor usage, reduces network bandwidth usage, reduces response time, and improves the reliability and accuracy of read-only data in the target data repository.

[0021] Another aspect provides a delete sequence number (DSN) within the change feed that is incremented each time a record set is deleted. The DSN is used to detect delete update failures and prevent silent failures. This reduces failures in the target data repository, reduces processor resource usage, and reduces the error rate of data updates in the target data repository.

[0022] In other examples, this change feed component that provides delete updates enables deletions from a source data repository to a target data repository to be processed consistently and without any timing dependencies.DSN and TSN also provide a method to detect failure scenarios and initiate recovery from failure events.

[0023] In other examples, the current DSN within the change feed is incremented by 1 each time a delete occurs. This helps detect missed delete notifications, especially when all notifications are due.

[0024] The change feed operates in an unconventional manner by including delete notification records for incremental updates of a target data repository and providing a delete sequence to identify missed delete updates. In this manner, the change feed is used in an unconventional manner and allows delete record updates and detection of silent failures to reduce the load on the system, thereby improving the functionality of the underlying computing device and preventing users from reading incorrect or erroneous data.

[0025] Figure 1 is an exemplary block diagram illustrating a system 100 for processing delete requests using a change feed. In some examples, system 100 includes a source database 102 storing data 103. Data 103 in source database 102 is continually modified by change requests 110 received from a set of services. Change requests 110 can include put, commit, and / or delete requests for writing (modifying) or deleting at least a portion of data 103 in database 102.

[0026] Database 102 can be any type of database or other data repository on one or more data storage devices.The data storage device associated with database 102 can include one or more different types of data storage devices, such as, for example, one or more rotating disk drives, one or more solid-state drives (SSDs) and / or any other type of data storage device. In some non-limiting examples, the data storage device includes a redundant array of independent disks (RAID) array. The data storage device can be included in a computing device, attached to a computing device, inserted into a computing device, or otherwise associated with a computing device. In other examples, the database can be supported on one or more remote data repositories accessed via a network, such as a data storage device on a remote computing device, a data storage in a remote data center, or cloud storage.

[0027] A change feed 104 is maintained to track updates 106 (changes) to data 103. Change feed 104 is an index of changes performed on data 103, listed in the order in which the modifications were made to data 103. Changes or updates 106 are made in response to change requests 110 received from service set 108.

[0028] The set of services 108 may include applications or other clients sending requests to read at least some data 103 , write (modify) at least some data 103 , and / or delete at least some data 103 (one or more records) in the database 102 .

[0029] A read-only copy of data 103 in source database 102 is maintained by one or more applications in application set 112 within one or more partitions. In some examples, the read-only copy of data 103 is maintained by an application or service, such as, for example, but not limited to, cache 114, data warehouse 116, and / or search index 118. In some non-limiting examples, dozens or hundreds of servers and / or clients maintain read-only copies of data 103 in source database 102.

[0030] Application set 112 uses updates 106 received from change feed 104 to update a local read-only copy of data 103 to keep the local copy synchronized with the data in source database 102. In other words, information from the change feed is used to replicate updates to data 103 that occurred in the source database within a read-only copy maintained at a target data repository, such as cache 114, data warehouse 116, search index 118, or any other target data repository.

[0031] Figure 2 is an exemplary block diagram illustrating a system 200 for updating a target data repository based on changes to a source data repository. In this non-limiting example, one or more clients 202 make changes to data 204 stored in a record set 206 in a source database 208 or other source data repository. The source database 208 is a database for storing data implemented on one or more data storage devices, such as, but not limited to, Figure 1 The client set 202 may modify the data 204 via one or more change requests, such as, but not limited to, a commit 210 request, a put 212 request, and / or a delete 214 request.

[0032] In some examples, submit 210 creates a new record, put 212 updates a pre-existing record stored in the service, and delete 214 removes or erases a pre-existing record from the service. Submit 210 and put 212 requests can also insert an entry into a sequence consisting of record changes with new or updated records, while delete 214 cannot.

[0033] In this example, change feed 216 is an index or log record or other means of tracking changes to data 204. In other examples, the change feed is not an explicit component of the database. The database stamps each record with a monotonically increasing transaction time during creation or update. If the same record is updated twice, it will only appear once, and deletions will never appear.

[0034] When data is written (modified) or deleted from record set 206, an entry is added to change feed 216 to record the change. In some examples, change feed 216 includes a list 218 of updates made to data 204 in modification order 220.

[0035] In some non-limiting examples, multiple servers 228 maintain one or more read-only copies of data 204 in record set 206 within one or more partitions. Multiple servers 228 send read queries 230 to change feed 216 to obtain update lists 218. In some examples, the query to change feed 216 includes a timestamp 232 or LSN. The query requests a list of changes or records that record changes made to data 204 since the given timestamp 232 was included in the query. If the timestamp in the query is logical time "t4" and the current timestamp is "t7," the change feed sends update lists 218 for all changes made to data 204 during the interval starting at "t4" and ending at "t7." Therefore, if one record is deleted at timestamp "t5" and two records are added at timestamp "t6," the update list sent to the request target server includes a delete notification identifying the record deleted at "t5" and one or more updates identifying the two records added at "t6."

[0036] Update list 218 is used to update the read-only 222 copy of data 204 in record set 224 maintained at target database 226. When clients in client set 236 send one or more requests 238 to read data 204 from target database 226, updates based on update list 218 obtained from change feed 216 can ensure that the data provided to client set 236 in response is accurate (synchronized) with the source copy on database 208.

[0037] In some examples, if timestamp 232 is at time "t0," a query is sent to read all changes with a timestamp after t0. If the page size is 2, the returned updates return a maximum of two records. If the page size is 1, a single record is returned. The next time a read query is sent at timestamp t2, the query requests all changes made to the source data repository after time t2. In this way, the system incrementally scans the data for changes that occurred in the order in which they were made. If the target server is down, one or more delete notifications may be deleted before the target server can query them. When the target server comes back online and queries for changes since the last update at t2, the returned updates do not include the delete notification records that have timed out and been removed from the change feed 216. In this case, the target server among the multiple servers can detect missed delete notifications by comparing the previous TSN for the last delete notification processed at the target server with the TSN for the next delete notification to identify the missed notification.

[0038] If all delete notifications have expired, the server can query the DSN in the change feed to detect missed delete notifications. The DSN is the most recent value of the TSN set on the source server. In some examples, there may be delete notifications with a TSN value less than the DSN. In this example, only the last delete notification has a TSN set equal to the DSN.

[0039] In some examples, the system uses a delete notification at the end that contains a DSN with no TTL. This final delete notification is updated with the new DSN on each delete operation, and therefore always appears after all other delete notifications. In this way, the target server identifies situations in which the target database 226 becomes out of sync with the source database 208.

[0040] Figure 3 3 is an exemplary block diagram illustrating a source server 302 creating a delete notification reflecting changes to a source data repository for updating a target data repository. In some examples, source server 302 includes a data repository 304 storing data 306 that changes due to write requests and delete requests received from one or more services, such as applications or clients.

[0041] The change feed component 308 maintains a change feed list or change log, such as but not limited to Figure 1 Change log 104 and / or Figure 2 When a write request (put or commit) is received to modify data 306, the change feed component 308 creates a record in the change feed identifying the record and the data entry changed.

[0042] In some examples, when a delete request is received from an application, user, or service, the change feed component creates a delete notification 312 in the change feed. A delete notification is a record associated with the deletion of one or more records. The delete notification is added to the change feed update. The delete notification can be called a tombstone record.

[0043] In other examples, the deletion notification includes an identifier for the deleted record and a TSN that provides a sequence or index number for the deletion. Delete notification 312 also includes a TTL 316. The TTL is a threshold waiting time until the deletion notification is deleted. In other words, when a deletion notification 312 is created, it is allowed to remain in the updated change feed list until the TTL expires. The TTL can be any configurable time period. In one non-limiting example, the TTL can be 24 hours.

[0044] When the TTL for a given delete notification expires, the delete notification is deleted by the delete manager component 310. When the original record is deleted and the delete notification identifying the deleted record is also deleted at the end of the TTL period, there is no additional data deletion record in the change feed or any other updated data provided to the target data repository 326.

[0045] The delete manager component 310 periodically checks the TTL of each record in the change feed. The delete manager component may be referred to as a garbage collector component.

[0046] In one example, the delete manager component can check for expiration of records every hour, every minute, every second, or any other periodic time period. When the TTL for a given record expires, the delete manager component 310 deletes the record. In one example, when the TTL assigned to a given delete notification expires, the delete manager component 310 deletes the delete notification from the change feed.

[0047] In some examples, the source DSN 320 is also maintained within the DSN entry in the change feed. The DSN is an index or sequence number that identifies the number of delete records for each partition. Because delete records expire after a TTL, this is only the total number of delete records ever created, not the total number of delete records that existed in the change feed at a given point in time. The system maintains a local copy of the DSN for each data partition. When something is deleted in a given partition, the DSN for that partition is incremented by one.

[0048] Each time a delete notification is created, the DSN update component 318 increments the source DSN 320 by 1. Thus, if the current source DSN 320 value is twelve (12), and a new delete notification is created representing the deletion of a set of three records from the source data repository 304, the DSN update component 318 increments the source DSN 320 value by 1, resulting in a new value of thirteen (13).

[0049] The updates are transmitted from the source server to the target server hosting the target data repository 326. In some examples, the change feed updates are received by the target server from the source server via a network 334. The network can be implemented by one or more physical network components, such as, but not limited to, routers, switches, network interface cards (NICs), and other network devices. The network 334 can be any type of network for enabling communication with remote computing devices, such as, but not limited to, a local area network (LAN), a subnet, a wide area network (WAN), a wireless (Wi-Fi) network, or any other type of network. In this example, the network 334 can be a WAN, such as the Internet. However, in other examples, the network is a local or private LAN.

[0050] Target data repository 326 maintains a read-only copy 328 of data 306 in source data repository 304. A local delete sequence number (DSN) 330 is maintained that can be compared to the source DSN 320 value. Each time target data repository 326 is updated to delete some data based on delete notification 312, local DSN 330 is incremented by one or set equal to the TSN in the delete record, since the TSN should monotonically increase if there are no missed delete records. In this way, local DSN 330 should be equal to source DSN 320 or just one less than source DSN 320, since each time a delete occurs on the source database, an update is sent to the target data repository, which increments the local DSN in response. Ultimately, the local DSN on the target should be equal to the source DSN.

[0051] The target repository may be significantly behind in processing the change feed, and in this case, its local DSN may be less than the source DSN by more than one. For example, if the local DSN is 5 and the source DSN is 9, then delete records with TSNs between 5 and 9 (such as values ​​6, 7, 8, or 9) will appear in the change feed (assuming they have not expired). Once the target repository has processed all delete records, the local DSN becomes equal to the source DSN.

[0052] If the value of local DSN 330 differs from the value of source DSN 320 by more than 1, and there are no delete records to be read from the change feed, this indicates that the target data repository is out of sync with source data repository 304. Therefore, if local DSN 330 is equal to or only one less than the value of source DSN 320, target data repository 326 has not missed any delete notifications, and read-only data 328 should be an exact copy of source data 306. If local DSN 330 is two or more less than the value of source DSN, target data repository 326 has missed one or more delete notifications and no longer contains a faithful copy of source data 306. Because delete notifications are deleted after the TTL time, and the original record represented by the delete notification was deleted when the delete notification was created, the read-only data in target data repository 326 cannot be updated using updates from the change feed. This enables the system to accurately detect missed delete notifications and develop recovery procedures to reconstruct the target data repository using an exact copy of data 306 from source data repository 304.

[0053] In other examples, the target data repository 326 maintains a record of the previous TSN 332. The previous TSN 332 is the tombstone sequence number from the last delete notification processed at the target data repository. The TSN 314 of the newly received delete notification 312 is compared to the previous TSN 332 to determine whether the new TSN 314 is in sequence with the previous TSN 332. For example, if the previous TSN 332 is seven (7) and the new TSN 314 of the delete notification received in the update from the change feed component 308 is eight (8), then the new delete notification is in the correct sequence and no delete notification was missed due to a server failure or loss of communication with the source server. However, if the new TSN 314 is nine (9) and the previous TSN 332 was only seven (7), it indicates that the missed delete notification with TSN eight (8) was never received. This gap in the tombstone sequence number indicates a missed delete notification. The system can initiate a failover process.

[0054] In some examples, the change feed component 308 receives as input a delete transaction request that includes a list of IDs of records to be deleted in a variable called "ids_to_delete". The change feed component 308 can read the delete notification (tombstone) record with the ID of the most recent (latest) delete notification and store it in the object "ctr". The change feed component 308 creates a new object of type "Tombstone" associated with the delete notification. The representation of the DSN state on the source is as follows:

[0055] {dsn:u64}.

[0056] After the delete operation, the status becomes:

[0057] dsn=dsn+1.

[0058] The most recent delete notification (tombstone) is updated with the new delete period and deleted record ID. Again, this is equivalent to the DSN on the source data repository. In the design, we create this record so that it appears in the change feed as the last deleted record. Since this record has no TTL, it will not expire, and the target will always eventually read it in the feed. The change feed component 308 creates a new object of type Tombstone as follows:

[0059]

[0060] After a delete operation, a delete notification is added to the changes feed, which can look like the following:

[0061]

[0062] The object is stored in a delete notification (tombstone) record. This is equivalent to the delete notification discussed above. In this non-limiting example, the TTL provides a one-day time-to-live for the record. The change feed component deletes the record with the ID in "ids_to_delete".

[0063] In some non-limiting examples, if a transaction is successfully executed, the change feed component 308 stores the latest transaction sequence number / transaction timestamp as the LSN. In other examples, the only state maintained on the target is the TSN of the last processed delete record and the checkpoint from which to resume reading the change feed. When the application starts, it initializes the last known delete notification (tombstone) state in memory and periodically flushes this state to disk or permanent storage.

[0064] While processing the delete notification, it gets updated like this:

[0065]

[0066] On a cold start, the LSN and TSN values ​​are both zero. The value(s) may be initialized to the last known value on disk or in persistent storage.

[0067] The change feed reader component does not rely on any temporal semantics to ensure correctness and provides a deterministic way to handle deleted records, identify potential system failures, and / or initiate recovery mechanisms. For example, a failure recovery process can cause the target data store 326 to be taken out of service due to bad data and / or restore the data at the target to a correct state. The target data store 326 can be restored by deleting all data from the target data store and copying a new snapshot of all data from the source data store 304 to the target data store 326.

[0068] Figure 4 4 is an exemplary block diagram illustrating a target server 400 updating a data repository and initiating a recovery process when data becomes out of sync with a source data repository. In some examples, the target server 400 is a server hosting a target data repository 422. The target server can be implemented as any database system, such as, but not limited to, an application or system that uses a change feed processor or similar technology and is built to process change streams to maintain state.

[0069] The target data repository 422 stores data, such as, but not limited to, a copy of data in the source data repository. The target data repository 422 may be implemented in a partition of the plurality of partitions. The target data repository 422 may optionally include a database, such as, but not limited to, Figure 2 Database 226 in.

[0070] In some examples, the target server 400 includes a change feed reader component 402. The change feed reader component 402 sends a query to a change feed component on the source server, such as, but not limited to, Figure 3 The change feed reader component 402 receives updates from the change feed on the source server.

[0071] If a delete notification is received in an update, the change feed reader component 402 compares 408 the locally maintained DSN 410 value with the value of the current DSN provided in the update. The current DSN value is the source DSN indicating the delete sequence, such as, but not limited to, Figure 3 320 in the source DSN. A delete is up-to-date if the local DSN and source DSN are equal or if the local DSN is only one less than the source DSN. A difference of 1 indicates that no deletes have been missed. A delete is up-to-date after the delete record is applied, after which the local DSN and source DSN become identical. Only when the local DSN and source DSN are equal can we say that the target and source are synchronized with respect to deleted records.

[0072] However, if the local DSN 410 value is two or more less than the current DSN provided in the most recent delete notification from the change feed, then the target server has missed one or more delete notifications. In this case, the recovery component 418 initiates a recovery process 420. The recovery process takes the local data repository 422 offline or otherwise makes the local data in the local data repository 422 unavailable to users, applications, or other services that read data from the data repository 422 because the data is not an exact copy of the source data.

[0073] In other examples, the change feed reader component compares a local copy of the previous TSN 412 obtained from the last processed delete notification (tombstone) record with the current TSN in the most recently received delete notification provided by the change feed update from the source server. The current TSN is obtained from the delete notification received from the change feed in the change feed update. When a given delete operation occurs on the source server, the system creates a delete notification with a TTL that specifies how long the delete notification remains in the change feed before being deleted. The TSN on the source server is incremented by one to reflect the delete operation.

[0074] The previous TSN value is the TSN associated with the read-only version of the data on the target data repository. When the last delete notification in the change feed is processed on the target server, the previous TSN should be equal to the DSN on the source data repository. The change feed is sorted by modification time. Delete notifications waiting to be processed on the target server have no TTL associated with them. Delete notifications are retained until the target server processes the delete notification to update the read-only data on the target data repository.

[0075] If the current TSN in the new (unprocessed) delete notification is equal to the previous TSN or is only one greater than the previous TSN of the last processed delete notification (deletion notification), this indicates that no delete notifications from the change feed have been lost or otherwise missed. Therefore, the local data in the local data repository is accurate. However, if the current TSN of the newly received and unprocessed delete notification from the change feed is two or more greater than the previous TSN, this indicates that one or more delete notifications have been missed. In this case, recovery component 418 initiates recovery process 420.

[0076] In some examples, each time data on the data store 422 is updated based on a delete notification, the increment component 414 performs an increment operation 416. The increment operation may include incrementing the DSN by 1 each time a record set deletion occurs.

[0077] Each time a deletion notification is created in the change feed, the deletion manager component 310 on the source server 302 assigns a TSN value to the deletion notification that is incremented by one from the previous TSN of the previous deletion notification, as described above in Figure 3 As shown in . Incrementing TSN on delete operation is done at the source. However, in some non-limiting examples, each time a new delete notification is generated, the incrementing component increments the value of TSN.

[0078] When a record set is deleted, in some examples, the change feed reader component 402 reads the delete notification record from the change feed. The DSN and / or TSN associated with the delete notification are used by the change feed reader component to handle the processing of the deleted record and detect whether the system has reached an unrecoverable failure state. When such a failure is detected, the system abandons its internal state and begins to rebuild the server.

[0079] In some examples, the change feed reader component 402 reads records from the change feed and processes the records. After an application fails abnormally, it loses its memory state and initializes itself from a persistent state. In some examples, the memory state is saved.

[0080] In other non-limiting examples, the change feed read by the application is a continuous subsequence of the source database's change feed. The DSN and / or TSN are used to verify whether the application / target database state and the source database state are synchronized with each other, or whether there are pending updates in the change feed that need to be processed by the application. If a change feed with a delete notification associated with a change update with an expired TTL is removed from the source change feed before being processed by the target server, data may be missed or out of sync. In this case, a recovery operation will be run on the application. The TSN associated with the delete notification is cleared from the source server when the TTL expires to avoid data explosion.

[0081] In this way, change feed readers leveraging DSN and TSN help build highly available services that provide a strongly consistent view of data stored in a persistent, read-only data repository in a very performant manner.

[0082] Figure 55 is an exemplary block diagram illustrating a delete notification 500 having a TSN 502. Delete notification 500 can represent the deletion of a single record or the deletion of multiple records (two or more records). Deleted record ID set 504 included in delete notification 500 includes a record identifier (ID) for each deleted record. If a single record is deleted, deleted record ID set 504 includes only the single ID for that record. If three records are deleted during a single transaction, deleted record ID set 504 includes three record IDs identifying each of the three deleted records.

[0083] TSN 502 is the tombstone sequence number 502 of the delete notification 500. The tombstone sequence number is incremented each time another delete notification is created. The delete notification does not include a DSN.

[0084] The last committed transaction sequence number (LSN) 506 is the sequence number of each transaction. In some examples, the deletion notification is a "TombstoneRecord" which can be represented as follows:

[0085] struct TombstoneRecord

[0086] {

[0087] id: String,

[0088] delete_epoch:u64,

[0089] deleted_record_ids: Vec <string>,

[0090] _lsn:u64

[0091] }

[0092] The term "delete_epoch" represents the transaction sequence number (TSN), a monotonically increasing number that increments each time a record set is deleted. The term "deleted_record_ids" represents the list of identifiers (IDs) of the deleted records. The transaction sequence number (LSN) is the commit sequence number of the transaction that performed the delete operation.

[0093] When a document set is deleted, then in the same transaction, the delete notification record is updated with a list of IDs of the deleted records for a single transaction or single delete request. If a single delete notification record has a list of all the record IDs that have been deleted since the beginning, then that list will grow, leading to another data explosion situation, even though it is in only one record.

[0094] In one example, a delete notification record with a known identifier "CURRENT_TOMBSTONE" is provided. During a cold start, it can be represented as:

[0095] id: 'CURRENT_TOMBSTONE',

[0096] delete_epoch: 0, deleted_record_ids: [],

[0097] _lsn:0}.

[0098] For each delete transaction, a copy of the record is created with a different ID. The record is updated with the new set of deleted record IDs. A TTL is added to the replicated record so that the record is automatically deleted after the expiration time. This prevents data explosion, as all deleted records and their deletion notification records are eventually removed from the system. This also helps identify applications reading the change feed if they fall behind by more than the TTL duration and must initiate recovery actions.

[0099] Deletion notifications that provide logical version numbers for both deletion notifications and missed deletion detection have no dependencies on time or other such non-deterministic factors. This deletion notification provides a deterministic method for consistently handling deletion requests using updated change feeds. It further provides a method for detecting if the system is in a failed state and a method for recovering from it.

[0100] Figure 6 is an exemplary flow chart illustrating the operation of a computing device to delete a record and update a change feed including a TSN. Figure 6 The process shown is performed by a computer (such as Figure 10 The change feed component is executed on the computing device 1000).

[0101] The process begins at 602 by receiving a set of record IDs for deletion. The set of record IDs is received in a delete request from a user, application, or service. At 604, the DSN is incremented immediately before or upon creating a delete notification record. At 606, the change feed component creates the delete notification record. A delete notification record can be referred to as a tombstone record. At 608, the change feed component creates (assigns) a TTL for the delete notification record. At 610, the set of records is deleted from the source data repository.

[0102] At 612, it is determined whether a next delete request has been received. If so, the change feed component iteratively executes 602 to 612 until no additional delete requests are received. The process then terminates.

[0103] The delete notification record contains the latest value of the DSN as the TSN. The record deletion, creation of the delete notification record, TTL creation, and DSN incrementing occur as a transaction. If any step fails, the transaction is rolled back to the previous state.

[0104] Although Figure 6 The operations shown are performed by a computing device, but aspects of the present disclosure contemplate other entities performing the operations. In a non-limiting example, a cloud service performs one or more of the operations. In another example, one or more computer-readable storage media storing computer-readable instructions may be executed to cause at least one processor to implement Figure 6 The operation shown.

[0105] Figure 7 is an exemplary flow diagram illustrating the operation of a computing device to update a local data repository based on change feed updates received from a source server. Figure 7 The process shown is performed by a computer (such as Figure 10 The change feed component or change feed reader component executed on the computing device 1000) is executed.

[0106] The process begins by sending a read query to the source change feed at 702. The read query requests all updates to the source data repository after a given timestamp. At 704, a determination is made as to whether an update has been received. If so, at 706, a determination is made as to whether the update is an unprocessed update. An unprocessed update is one that has not been previously received and processed to update the local data repository. If so, at 708, a determination is made as to whether the update includes an unprocessed delete notification. If not, at 710, the update is processed, and the process terminates thereafter.

[0107] Returning to 708, if the update includes a delete notification, the change feed reader compares the new TSN in the delete notification with the previous TSN at 712. At 714, it is determined whether the new TSN is greater than the previous TSN by at least one. If so, a restore operation is initiated at 716. The process then terminates.

[0108] If the TSN of the new delete notification is not greater than the previous TSN, then the record is deleted from the local data store at 718. At 720, the DSN is incremented by 1. Thereafter, the process terminates.

[0109] Although Figure 7 The operations shown are performed by a computing device, but aspects of the present disclosure contemplate other entities performing the operations. In a non-limiting example, a cloud service performs one or more of the operations. In another example, one or more computer-readable storage media storing computer-readable instructions may be executed to cause at least one processor to implement Figure 7 The operation shown.

[0110] Figure 8 is an exemplary flow diagram illustrating the operation of a computing device to update a local data repository based on updates from a change feed. Figure 8 The process shown is performed by a computer (such as Figure 10 The change feed component or change feed reader component executed on the computing device 1000) is executed.

[0111] The process begins by receiving an update from a source change feed at 802. At 804, a determination is made as to whether a DSN with an update has been received. If so, at 806, the change feed reader component calculates the difference between the received source DSN and the local DSN. At 808, a determination is made as to whether the difference is less than or equal to 1. If so, at 810, the deletion is processed. At 812, the local DSN is incremented to reflect the processed deletion notification.

[0112] If the difference between the source DSN and the local DSN is greater than 1, then at 814, a determination is made as to whether there are any unprocessed delete notifications. If not, then at 816, a recovery process is initiated. The process terminates thereafter. If there are any unprocessed delete notifications, the data may still be out of sync. For example, if the target DSN is 6 and the source DSN is 8, the difference between the source DSN value and the target DSN value is greater than 1. However, if there are two unprocessed delete notifications remaining in the change feed waiting to be processed at the target, the target will not be out of sync with the source because the target DSN will be incremented by 2 when the two pending delete notifications are fully processed to update the data at the target.

[0113] Although Figure 8 The operations shown are performed by a computing device, but aspects of the present disclosure contemplate other entities performing the operations. In a non-limiting example, a cloud service performs one or more of the operations. In another example, one or more computer-readable storage media storing computer-readable instructions may be executed to cause at least one processor to implement Figure 8 The operation shown.

[0114] Figure 9 is an exemplary flow chart illustrating the operation of a computing device to detect data inconsistencies between a source data repository and a local read-only copy of the data. Figure 6 The process shown is performed by a computer (such as Figure 10 The change feed component or change feed reader component executed on the computing device 1000) is executed.

[0115] The process begins by receiving a delete notification at 902. At 904, the new TSN of the delete notification is compared to the local copy of the previous TSN. At 906, it is determined whether the difference between the previous TSN and the new TSN is greater than 1. If not, at 908, the change feed reader component compares the local DSN to the updated source DSN provided by the change feed update to determine if the data is out of sync. At 910, it is determined whether there is a gap greater than 1 between the two sequences and there are no pending delete notifications remaining in the change feed. If not, at 912, the record set identified in the delete notification is deleted from the local data repository. At 914, the local DSN is incremented by 1. At 916, the previous TSN is updated to the TSN value of the delete notification.

[0116] If the gap between the TSN and the previous TSN at 906 is greater than 1, then a restore operation is initiated at 918. Similarly, if the gap between the local DSN and the source DSN received in the change feed update is greater than 1, then a restore operation is initiated at 918. The process terminates thereafter.

[0117] Although Figure 9 The operations shown are performed by a computing device, but aspects of the present disclosure contemplate other entities performing the operations. In a non-limiting example, a cloud service performs one or more of the operations. In another example, one or more computer-readable storage media storing computer-readable instructions may be executed to cause at least one processor to implement Figure 9 The operation shown.

[0118] Other Examples

[0119] Some aspects and examples disclosed herein are directed to a system, method, and / or computer-executable instructions for processing a delete request with a change feed, comprising: a processor; and a computer-readable medium storing instructions that, when executed by the processor, are operable to: create a delete notification in a change feed associated with a source data repository storing data, the delete notification including a set of delete record IDs identifying a set of records to be deleted and a TSN identifying a sequence of delete notifications in the set of delete notifications; increment a current DSN in the change feed, the DSN identifying the number of delete notifications generated on the change feed; assign a TTL value to the delete notification, wherein the delete notification is removed upon expiration of a time period corresponding to the TTL value; and delete the set of records associated with the delete notification.

[0120] Additional aspects and examples disclosed herein are directed to a system, method, or computer-executable instructions for creating a delete notification in a change feed associated with a source data repository storing data, the delete notification including a set of delete record IDs identifying a set of records to be deleted and a TSN identifying a sequence of delete notifications in the set of delete notifications; incrementing a current DSN in the change feed, the DSN identifying the number of delete notifications generated on the change feed; assigning a TTL value to the delete notification, wherein the delete notification is removed upon expiration of a time period corresponding to the TTL value; and deleting the set of records associated with the delete notification.

[0121] Other aspects and examples disclosed herein are directed to a system, a method, and / or one or more computer storage devices having computer-executable instructions stored thereon for processing delete requests using a sequence number with a change feed that, when executed by a computer, causes the computer to perform operations comprising: creating a delete notification in a change feed associated with a source data repository storing data, the delete notification comprising a set of delete record IDs identifying a set of records to be deleted and a TSN identifying a sequence of delete notifications in the set of delete notifications; incrementing a current DSN in the change feed, the DSN identifying the number of delete notifications generated on the change feed; assigning a TTL value to the delete notification, wherein the delete notification is removed upon expiration of a time period corresponding to the TTL value; and deleting the set of records associated with the delete notification.

[0122] In an example scenario, database systems such as CosmosDB use the LSN, DSN, and / or TSN in the change feed for each transaction's delete and update operations to detect failures. Any application that reads the update change feed to maintain local state in the database can utilize the DSN and / or TSN for each partition to process delete notifications and detect missed delete notifications to trigger recovery operations.

[0123] In another example, the system performs transactions on data stored in a service. Each record updated in a transaction is assigned a monotonically increasing LSN, which is the transaction commit sequence number. Entries in the change feed include the latest LSN for the record. The LSN is used to track updates associated with put and commit requests and deletes.

[0124] When a data document is deleted, a temporary deletion notification record is inserted. This inserts a temporary deletion notification record for the deleted document. The permanent deletion notification record is updated with the latest deletion epoch TSN and transaction LSN. It also updates the change feed and inserts the new and updated documents. Documents inserted into the change feed are batched per transaction, but their order within the batch may not be guaranteed.

[0125] The other systems simulate a restore operation on the application, which is equivalent to clearing all application state and rehydrating data from the database by reading the change feed from scratch. After the restore is complete, the in-memory application state is synchronized with the database state and all records from the change feed are processed.

[0126] In another non-limiting example, records from the change feed are only processed if their LSN is greater than or equal to the last checkpoint LSN. The change feed is read only from the last checkpoint LSN. If the read record is a data document, the application updates its in-memory state. Otherwise, if it is a delete notification record and the tombstone epoch is at most one older than the applied delete epoch, the record is processed and the application deletes the document from its in-memory state. If the tombstone epoch is two or more older than the applied delete epoch, the application missed the delete and a recovery operation is run.

[0127] In the example scenario, when record "R1" is deleted, a delete notification record (TM) is created. The sequence number for the deletion is incremented by 1. An entry is created for the delete notification, which includes the TSN, and another entry is created for the DSN. The target server reads both entries and keeps track of the sequence numbers. If the DSN in the new update is 22, but the target server's local DSN is only 20, the system knows about the missing number in the delete notification sequence. When this happens, the recovery process can reconfigure the server to prevent serving bad data to the requesting client.

[0128] Similarly, in another example, if the TSN of a delete notification is TM 7 and the previous TSN on the target database is TM 5, the system recognizes that the delete of TM 6 was missed or not received. This indicates that the data is out of sync with the source data repository. This provides a simple, scalable system that is computationally more efficient due to less data travel between the source and target. This is an incremental approach that prevents security violations and accurately detects missed deletes without sacrificing data quality.

[0129] Alternatively, or in addition to other examples described herein, examples include any combination of:

[0130] - comparing a local DSN value associated with the read-only version of the data on the source data repository with a current DSN obtained from a delete notification in a change update received from the change feed to identify a difference between the local DSN value and the current DSN value;

[0131] - in response to a difference between the local DSN value and the current DSN value being equal to 1, deleting from the target data repository the set of records identified in the set of deleted record IDs in the delete notification and incrementing the local DSN value by 1;

[0132] - in response to a difference between the local DSN value and the current DSN value being equal to or greater than 2, initiating a restore operation;

[0133] - comparing a previous TSN value associated with the read-only version of the data on the target data repository with the TSN obtained from the delete notification received from the change feed in the change update to identify a difference between the previous TSN value and the current TSN value;

[0134] - in response to the difference between the previous TSN and the current TSN being equal to 1, deleting from the target data repository the set of records identified in the set of deleted record IDs in the deletion notification;

[0135] - replace the value of the previous TSN with the value of the current TSN value; and

[0136] In response to a difference between the previous TSN value and the current TSN value being equal to or greater than 2, initiating a recovery operation.

[0137] Although aspects of the disclosure have been described in terms of various examples and their related operations, those skilled in the art will understand that combinations of operations from any number of the different examples are also within the scope of aspects of the disclosure.

[0138] Sample operating environment

[0139] Figure 10 1 is a block diagram of an example computing device 1000 for implementing aspects disclosed herein and is generally designated as computing device 1000. Computing device 1000 is an example of a suitable computing environment and is not intended to imply any limitation on the scope of use or functionality of the examples disclosed herein. Computing device 1000 should also not be interpreted as having any dependency or requirement related to any one or combination of the components / modules shown. The examples disclosed herein can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions (such as program components) executed by a computer or other machine (such as a personal data assistant or other handheld device). Generally, program components, including routines, programs, objects, components, data structures, etc., refer to code that performs specific tasks or implements specific abstract data types. The disclosed examples can be practiced in various system configurations, including personal computers, laptops, smartphones, mobile tablets, handheld devices, consumer electronics, professional computing devices, etc. When tasks are performed by remote processing devices linked via a communication network, the disclosed examples can also be practiced in a distributed computing environment.

[0140] The computing device 1000 includes a bus 1010 that directly or indirectly couples the following devices: computer storage memory 1012, one or more processors 1014, one or more presentation components 1016, I / O ports 1018, I / O components 1020, a power supply 1022, and a network component 1024. Although the computing device 1000 is depicted as appearing to be a single device, multiple computing devices 1000 can work together and share the depicted device resources. For example, the memory 1012 can be distributed across multiple devices, and the processor(s) 1014 can be housed with different devices.

[0141] Bus 1010 represents anything that may be one or more buses, such as an address bus, a data bus, or a combination thereof. Figure 10 The various blocks of the computer are shown with lines for clarity, but depicting the various components can be accomplished with alternative representations. For example, in some examples, components such as display devices are I / O components, and some examples of processors have their own memory. There is no distinction between categories such as "workstation," "server," "laptop," "handheld device," etc., because all of these categories are in the Figure 10 and within the scope of references to "computing devices" herein. Memory 1012 may take the form of computer storage media referenced below and is operable to provide storage of computer-readable instructions, data structures, program modules, and other data for computing device 1000. In some examples, memory 1012 stores one or more of an operating system, a general-purpose application platform, or other program modules and program data. Memory 1012 is thus capable of storing and accessing data 1012a and instructions 1012b executable by processor 1014 and configured to perform the various operations disclosed herein.

[0142] In some examples, memory 1012 includes computer storage media in the form of volatile and / or non-volatile memory, removable or non-removable memory, data disks in a virtual environment, or a combination thereof. Memory 1012 may include any number of memories associated with or accessible by computing device 1000. Memory 1012 may be internal to computing device 1000 (e.g., Figure 10 1000 ); external to computing device 1000 (not shown); or both (not shown). Examples of memory 1012 include, but are not limited to, random access memory (RAM); read-only memory (ROM); electrically erasable programmable read-only memory (EEPROM); flash memory or other storage technology; CD-ROM, digital versatile disc (DVD) or other optical or holographic media; magnetic cassettes, tape, disk storage or other magnetic storage devices; memory wired to an analog computing device; or any other medium for encoding desired information and accessed by computing device 1000. Additionally or alternatively, memory 1012 may be distributed across multiple computing devices 1000, for example, in a virtualized environment where instruction processing is performed on multiple computing devices 1000. For purposes of this disclosure, "computer storage media," "computer storage memory," "memory," and "memory device" are synonyms for computer storage memory 1012, and none of these terms include carrier waves or propagated signals.

[0143] (Multiple) processors 1014 can include any number of processing units that read data from various entities such as memory 1012 or I / O components 1020, and can include a CPU and / or a GPU. Specifically, (multiple) processors 1014 are programmed to execute computer-executable instructions for implementing aspects of the present disclosure. The instructions can be executed by a processor, multiple processors within the computing device 1000, or a processor external to the client computing device 1000. In some examples, (multiple) processors 1014 are programmed to execute instructions such as those shown in the accompanying drawings. In addition, in some examples, (multiple) processors 1014 represent an implementation of simulation technology for performing the operations described herein. For example, the operations can be performed by a simulated client computing device 1000 and / or a digital client computing device 1000. (Multiple) presentation components 1016 present data indications to a user or other device. Exemplary presentation components include a display device, a speaker, a printing component, a vibration component, etc. Those skilled in the art will understand and appreciate that computer data can be presented in a variety of ways, such as visually in a graphical user interface (GUI), audibly through speakers, wirelessly between computing devices 1000, through wired connections, or otherwise. I / O ports 1018 allow computing device 1000 to be logically coupled to other devices, including I / O components 1020, some of which may be built-in. Example I / O components 1020 include, for example, but not limited to, microphones, joysticks, game controllers, satellite dishes, scanners, printers, wireless devices, and the like.

[0144] The computing device 1000 can operate in a network environment using logical connections to one or more remote computers via a network component 1024. In some examples, the network component 1024 includes a network interface card and / or computer executable instructions (e.g., a driver) for operating a network interface card. Communications between the computing device 1000 and other devices can occur over any wired or wireless connection using any protocol or mechanism. In some examples, the network component 1024 is operable to transmit data using a transport protocol over public, private, or hybrid (public and private), using short-range communication technologies (e.g., near field communication (NFC), Bluetooth), or other similar communication protocols. TM The network component 1024 communicates with cloud resources 1028 via a network 1030 via wireless communication links 1026 and / or wired communication links 1026a. Various different examples of communication links 1026 and 1026a include wireless connections, wired connections, and / or dedicated links, and in some examples, at least a portion is routed through the Internet.

[0145] Although described in conjunction with the example computing device 1000, the examples of the present disclosure can be implemented using many other general or special computing system environments, configurations, or devices. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with aspects of the present disclosure include, but are not limited to, smartphones, mobile tablets, mobile computing devices, personal computers, server computers, handheld or laptop computers, multiprocessor systems, game consoles, microprocessor-based systems, set-top boxes, programmable consumer electronics, mobile phones, wearable or portable forms of mobile computing and / or communication devices (e.g., watches, glasses, headphones or earphones), network PCs, minicomputers, mainframes, distributed computing environments including any of the above systems or devices, virtual reality (VR) devices, augmented reality (AR) devices, mixed reality (MR) devices, holographic devices, and the like. Such systems or devices may accept input from a user in any manner, including from input devices such as a keyboard or pointing device, through gesture input, proximity input (such as by hovering), and / or voice input.

[0146] Examples of the present disclosure can be described in the general context of computer-executable instructions (such as program modules), which are executed by one or more computers or other devices with software, firmware, hardware or a combination thereof. Computer-executable instructions can be organized into one or more computer-executable components or modules. Typically, program modules include but are not limited to routines, programs, objects, components and data structures that perform specific tasks or implement specific abstract data types. Various aspects of the present disclosure can be implemented with such components or modules of any number and organization. For example, various aspects of the present disclosure are not limited to the specific computer-executable instructions or specific components or modules shown in the figures and described herein. Other examples of the present disclosure may include different computer-executable instructions or components with more or less functions than those shown and described herein. In examples relating to general-purpose computers, when configured to execute the instructions described herein, various aspects of the present disclosure convert the general-purpose computer into a special-purpose computing device.

[0147] As an example and not limitation, computer-readable media include computer storage media and communication media. Computer storage media include volatile and non-volatile removable and non-removable memories implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, etc. Computer storage media are tangible and mutually exclusive with communication media. Computer storage media are implemented in hardware and do not include carrier waves and propagation signals. Computer storage media for the purposes of this disclosure are not signals themselves. Exemplary computer storage media include hard disks, flash drives, solid-state memories, phase change random access memories (PRAMs), static random access memories (SRAMs), dynamic random access memories (DRAMs), other types of random access memories (RAMs), read-only memories (ROMs), electrically erasable programmable read-only memories (EEPROMs), flash memory or other storage technologies, compact disc read-only memories (CD-ROMs), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by a computing device. In contrast, communication media typically embodies computer-readable instructions, data structures, program modules, etc. in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0148] The execution order or execution order of the operations in the examples of the present disclosure shown and described herein are not necessary and can be performed in different sequential ways in various examples.For example, it is contemplated that before, simultaneously or after another operation, (execute) or execution (perform) a specific operation is performed within the scope of the various aspects of the present disclosure. When introducing the elements of various aspects of the present disclosure or its example, the articles "a (a)", "an (an)", "the (the)" and "said (said)" are intended to represent that there are one or more elements. The terms "comprising (comprising)", "including (including)" and "having (having)" are intended to be inclusive and represent that there may be other elements in addition to the listed elements. The term "exemplary" is intended to represent "example". The phrase "one or more of A, B and C" refers to "at least one of A and / or at least one of B and / or at least one of C".

[0149] Having described various aspects of the disclosure in detail, it will be apparent that modifications and variations can be made without departing from the scope of the various aspects of the disclosure as defined in the appended claims. As various changes can be made in the above constructions, products, and methods without departing from the scope of the various aspects of the disclosure, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.< / string>

Claims

1. A system for processing a delete request using a sequence number using a change feed, the system comprising: A computer-readable medium storing instructions that, when executed by a processor, are operable to: creating a delete notification in a change feed associated with a source data repository storing the data, the delete notification including a delete record ID set identifying a set of records for deletion and a tombstone sequence number (TSN) identifying a sequence of the delete notification in the delete notification set; assigning a time-to-live (TTL) value to the deletion notification, wherein the deletion notification includes the TSN, and the deletion notification is removed from the change feed upon expiration of a time period corresponding to the TTL value; In response to creating the delete notification, incrementing a current delete notification sequence number (DSN) within the change feed, wherein the DSN is present in the change feed to identify a total number of delete notifications generated on the change feed; as well as In response to receiving a query from a target data store, providing a change update to the target data store to delete the record set associated with the delete notification.

2. The system of claim 1 , wherein the instructions are further operable to: Comparing a local DSN value associated with the read-only version of the data on the target data repository with the current DSN obtained from the delete notification in the change update received from the change feed to identify a difference between the local DSN value and the value of the current DSN.

3. The system of claim 2, wherein the instructions are further operable to: In response to the difference between the local DSN value and the value of the current DSN being equal to 1, deleting from the target data repository the set of records identified in the deleted record ID set in the deletion notification; and The value of the local DSN value is incremented by one, wherein the local DSN value is incremented by one each time the target data repository is updated based on a new delete notification in a new change update received from the change feed.

4. The system of claim 2, wherein the instructions are further operable to: In response to the difference between the local DSN value and the value of the current DSN being equal to or greater than 2, a recovery operation is initiated.

5. The system of claim 1 , wherein the instructions are further operable to: A previous TSN value associated with the read-only version of the data on the target data repository is compared with the TSN obtained from the delete notification received from the change feed in a change update to identify a difference between the previous TSN value and a current TSN value.

6. The system of claim 5, wherein the instructions are further operable to: In response to the difference between the previous TSN value and the current TSN value being equal to 1, deleting from the target data repository the set of records identified in the set of deleted record IDs in the deletion notification; and The value of the previous TSN is replaced with the value of the current TSN value.

7. The system of claim 5, wherein the instructions are further operable to: In response to the difference between the previous TSN value and the value of the current TSN being equal to or greater than 2, a recovery operation is initiated.

8. A method for processing a delete request using a sequence number using a change feed, the method comprising: creating, by a change feed component, a delete notification in a change feed associated with a source data repository storing the data, the delete notification including a delete record ID set identifying a set of records for deletion and a tombstone sequence number (TSN) identifying a sequence of the delete notification in the delete notification set; assigning a time-to-live (TTL) value to the deletion notification, wherein the deletion notification includes the TSN, and the deletion notification is removed from the change feed upon expiration of a time period corresponding to the TTL value; In response to creating the delete notification, incrementing a current delete notification sequence number (DSN) within the change feed, wherein the DSN is present in the change feed to identify a total number of delete notifications generated on the change feed; as well as In response to receiving a query from a target data store, providing a change update to the target data store to delete the record set associated with the delete notification.

9. The method according to claim 8, further comprising: A change feed reader compares a local DSN value associated with the read-only version of the data on the target data repository with the current DSN obtained from the delete notification in the change update to identify a difference between the local DSN value and the value of the current DSN, the change update being received from the change feed.

10. The method according to claim 9, further comprising: In response to the difference between the local DSN value and the value of the current DSN being equal to 1, deleting from the target data repository the set of records identified in the deleted record ID set in the deletion notification; as well as Increment the value of the Local DSN value by 1.

11. The method according to claim 9, further comprising: In response to the difference between the local DSN value and the value of the current DSN being equal to or greater than 2, a recovery operation is initiated.

12. The method according to claim 8, further comprising: A previous TSN value associated with the read-only version of the data on the target data repository is compared with the TSN obtained from the delete notification received from the change feed in a change update to identify a difference between the previous TSN value and a current TSN value.

13. The method according to claim 12, further comprising: In response to the difference between the previous TSN value and the current TSN value being equal to 1, deleting from the target data repository the set of records identified in the set of deleted record IDs in the deletion notification; as well as The value of the previous TSN is replaced with the value of the current TSN value.

14. The method according to claim 12, further comprising: In response to the difference between the previous TSN value and the value of the current TSN being equal to or greater than 2, a recovery operation is initiated.

15. One or more computer storage media having stored thereon computer-executable instructions for processing a deletion request using a sequence number using a change feed, the computer-executable instructions, when executed by a computer, causing the computer to perform operations comprising: creating a delete notification in a change feed associated with a source data repository storing the data, the delete notification including a set of deleted record IDs identifying a set of records to be deleted and a tombstone sequence number (TSN) identifying a sequence of the delete notifications in the set of delete notifications; assigning a time-to-live (TTL) value to the deletion notification, wherein the deletion notification includes the TSN, and the deletion notification is removed from the change feed upon expiration of a time period corresponding to the TTL value; In response to creating the delete notification, incrementing a current delete notification sequence number (DSN) within the change feed, wherein the DSN is present in the change feed to identify a total number of delete notifications generated on the change feed; as well as In response to receiving a query from a target data store, providing a change update to the target data store to delete the record set associated with the delete notification.

16. The one or more computer storage media of claim 15, wherein the operations further comprise: Comparing a local DSN value associated with the read-only version of the data on the target data repository with the current DSN obtained from the delete notification in the change update received from the change feed to identify a difference between the local DSN value and the value of the current DSN.

17. The one or more computer storage media of claim 16, wherein the operations further comprise: In response to the difference between the local DSN value and the value of the current DSN being equal to 1, deleting from the target data repository the set of records identified in the deleted record ID set in the deletion notification; as well as Increment the value of the Local DSN value by 1.

18. The one or more computer storage media of claim 16, wherein the operations further comprise: In response to the difference between the local DSN value and the value of the current DSN being equal to or greater than 2, a recovery operation is initiated.

19. The one or more computer storage media of claim 15, wherein the operations further comprise: A previous TSN value associated with the read-only version of the data on the target data repository is compared with the TSN obtained from the delete notification received from the change feed in a change update to identify a difference between the previous TSN value and a current TSN value.

20. The one or more computer storage media of claim 19, wherein the operations further comprise: In response to the difference between the previous TSN value and the current TSN value being equal to 1, deleting from the target data repository the set of records identified in the set of deleted record IDs in the deletion notification; as well as The value of the previous TSN is replaced with the value of the current TSN value.

Images