Progressive object refresh in content distribution networks

Abstract

Embodiments herein describe a CDN that uses revalidation to force edge servers to refresh their cached objects (i.e., download a new copy of the object from the origin data center). The edge servers can use revalidation to determine whether an object currently cached at the edge server has been updated or changed in the origin data center. Embodiments herein utilize revalidation to perform a forced refresh to force edge servers to refresh their cached objects regardless of whether those cached objects match the objects stored in the origin data center. Forced refresh can be used when there are network connections that can cause the cached objects to become corrupted. Furthermore, forced refresh can be performed in batches so as not to overwhelm the network and computing resources in the CDN.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of and priority to U.S. Patent Application Serial No. 17 / 518,764, filed November 4, 2021. The above-identified related patent application is incorporated herein by reference in its entirety. BACKGROUND

[0003] A content delivery network or content distribution network (CDN) is a geographically distributed network of edge servers and their origin data centers. Their goal is to provide high availability and performance by distributing services spatially relative to end users. As the Internet has become a mission-critical medium for individuals and businesses, CDNs have emerged to alleviate performance bottlenecks. Since then, CDNs have evolved to serve a large portion of today’s Internet content, including web objects (text, graphics, and scripts), downloadable objects (media files, software, documents), applications (e-commerce, portals), live streaming, on-demand streaming, and social media websites. BRIEF DESCRIPTION OF DRAWINGS

[0004] In order to more fully understand the above-described concepts, embodiments thereof can be described by reference to the drawings, wherein:

[0005] It should be noted, however, that the attached drawings illustrate only typical embodiments and should not be considered as limiting the application; the application can be embodied in other equally effective embodiments.

[0006] Figure 1 is a block diagram of a CDN performing a forced refresh using revalidation according to one embodiment.

[0007] Figure 2 shows a forced refresh performed during revalidation according to one embodiment.

[0008] Figure 3 is a flowchart for performing a forced refresh in batches according to one embodiment.

[0009] Figure 4 shows batching content using object tags according to one embodiment.

[0010] Figure 5 is a flowchart for performing a forced refresh in batches according to one embodiment. DETAILED DESCRIPTION

[0011] Embodiments herein describe a CDN that uses revalidation to force edge servers to refresh their cached objects (i.e., download a new copy of the object from the origin data center). An edge server (also referred to as a proxy server) responds to a user's request for data (commonly referred to as an "object") by retrieving the object from an origin data center and caching the object. The edge server can then respond to future user requests for the object without re-downloading the object from the origin data center. If the user demand for the object subsides, the edge server will no longer cache the object (i.e., the edge server deletes the object), but the object is still stored in the origin data center. If the user demand remains high, the edge server continues to cache the object. However, the origin data center can update the object, thereby making the object cached in the edge server stale. Thus, the edge server can periodically perform a so-called revalidation process to ensure that its copy of the object matches the most recent version of the object stored at the origin. If not, the edge server can download and cache an updated version of the object.

[0012] As described above, revalidation can be used by an edge server to determine whether a currently cached object at the edge server has been updated or changed in the origin data center. For example, a cached news article can be updated as unstable situations change, or to correct errors in the article. The updated article is stored in the origin data center. The edge server can use revalidation to check whether its cached copy of the news article still matches the article stored at the origin, and if not, download a new copy of the article (i.e., refresh the object).

[0013] Embodiments herein utilize revalidation to perform forced refreshes to force edge servers to refresh their cached objects, regardless of whether those cached objects match the objects stored in the origin data center. Forced refreshes can be used when there are network connections that can have caused the cached objects to become corrupted. For example, when an edge server first retrieves an object from an origin, network connection problems can cause the cached object to lose data, or the data can be rearranged relative to the copy of the object stored at the origin. Given that an edge server can cache hundreds of thousands or millions of different objects, it is often difficult, if not impossible, to determine which cached objects are corrupted. Moreover, a primary entity that controls the origin data center can rely on third parties to operate the edge servers. Thus, the primary entity can not have access to the edge servers, and thus, can not know which objects are corrupted or uncorrupted.

[0014] To recover the corrupted cached objects, embodiments herein enforce a flush by changing the tags corresponding to the objects. When an object is stored in the origin data center, it is assigned a tag that uniquely identifies the object (e.g., a timestamp or hash of the underlying data). When the object is updated at the origin, its tag is also modified. Thus, when the edge server sends a revalidation request using the old tag of the object, the origin data center will notify the edge server that the tags do not match; in response, the edge server downloads a fresh copy of the object and its updated tag. To enforce a flush, the origin data center can spontaneously change the tags of objects even if the objects have not been updated. When the edge server performs a revalidation, none of the tags match, triggering the edge server to download the objects to replace any cached objects that can be corrupted.

[0015] Because this process can be used to flush a large number (or all) of the cached objects in the edge servers, this can place significant stress on the network connecting the origin data center to the edge servers. To control or limit this stress, the origin data center can perform the enforced flush in batches by changing only a fraction of the tags at a time. For example, the origin data center can change only a quarter of the tags each week, thereby enforcing a flush of all the cached objects in the edge servers over four weeks. Advantageously, using embodiments herein, a CDN can recover from small or large data corruption events by performing an enforced flush in a controlled manner that does not overwhelm the network.

[0016] Figure 1 is a block diagram of a CDN 100 that uses revalidation to perform an enforced flush according to one embodiment. Generally, the CDN 100 includes an origin data center 105 that stores a master copy of media assets 115 (or more generally, objects) that are cached at edge servers 130 distributed at various geographic locations. The edge servers 130 serve requests for the media assets 115 by users 150. For example, edge server 130A can serve requests initiated by user 150A in one geographic region, while edge server 130B serves requests initiated by user 150B in a different geographic region. Although two edge servers 130 are shown, the CDN 100 can include any number of edge servers 130. Moreover, the CDN 100 can have multiple origin data centers 105. For example, one set of edge servers 130 (e.g., edge servers in a first country) can use one origin data center 105, while a different set of edge servers 130 (e.g., servers in a second country) use a different origin data center 105.

[0017] While embodiments herein are described in the context of a streaming environment where a user 150 requests a media asset 115 (e.g., a movie, a television show, a video clip, a news article, etc.), the embodiments can be applied to a CDN that provides any kind of object (e.g., an application, a web object, a live stream, etc.). Typically, when a user request is received, an edge server 130 checks whether the requested media asset 115 is located in its own cache 135, and if so, provides the asset 115 to the user 150. But if not, the edge server 130 requests a copy of the media asset 115 from the origin data center 105. The edge server 130 then caches the media asset 115 for a period of time so that the next time the user 150 requests the asset 115, the edge server 130 can provide the asset 115 without having to download it again from the origin data center 105.

[0018] In this example, the origin data center 105 includes a library 110 that stores media assets 115. The origin data center 105 (also referred to as the origin 105) can include one or more computing systems (e.g., servers) that store the library 110. In one embodiment, the library 110 stores all media assets 115 in the CDN 100. For example, if the origin 105 is owned by a media company, the origin 105 can store all media assets 115 that are available to the users 150 as part of a subscription-based (or free) streaming service, for example. In contrast, the edge servers 130 can store only a portion of the media assets 115 that are available to the users 150. Moreover, each edge server 130 can store different portions of the media assets 115 depending on the requests made by the users 150 located in its respective geographic location. That is, the edge server 130A can store a different subset of the media assets 115 in its cache 135A than the edge server 130B stores in its cache 135B.

[0019] In this embodiment, each media asset 115 is assigned a tag 120 when it is first stored in the library 110. The tag 120 (also referred to as an electronic tag (Etag)) can be a timestamp that indicates when the media asset 115 was stored (or updated), or a hash that is derived from the underlying data in the media asset 115. In any case, the tag 120 can be different from all other tags 120 for media assets 115 in the CDN 100.

[0020] The tags 120 can be used to revalidate the media assets 115 cached at the edge servers 130. When a media asset 115 is cached, the edge server 130 stores a time to live (TTL) 140 to each media asset 115. The TTL 140 determines a time (or a count value) at which the edge server 130 is to perform a revalidation process on the media asset 115 to determine whether it is stale and should be refreshed. For example, when the TTL 140 expires, a revalidator 145 (e.g., a software application) in the server 130 sends the tag 120 to the source 105, which in turn checks whether the tag 120 matches the tag 120 of the copy of the media asset 115 stored in the repository 110. If so, the source 105 notifies the edge server 130 that the tags 120 match; in response, the edge server 130 can reset the TTL 140 of the media asset 115. If the tag 120 in the edge server 130 does not match the tag 120 in the source 105, the edge server 130 refreshes the asset 115 by downloading or receiving the media asset 115 and its new tag 120 from the source 105.

[0021] As noted above, the revalidation process can be utilized to perform forced refreshes on some or all of the media assets 115 cached in the edge servers 130. To this end, the source 105 includes a restoration module 125 (e.g., a software application or code) that changes the tags 120 of some or all of the media assets 115 in the repository. Unlike other implementations of revalidation in which the tags 120 are changed only when the underlying media asset 115 has been changed or updated, the restoration module 125 changes the tags 120 of at least some of the media assets 115 even if those assets 115 have not been changed. Advantageously, this allows the CDN 100 to update the media assets 115 cached in the edge servers 130 without having to evaluate the copies to determine which copies are corrupted due to, for example, network errors.

[0022] For example, when a cached media asset 115 is corrupted, the edge server 130 or origin 105 can not know. Because revalidation is performed by comparing tags 120, it is possible that the underlying media asset 115 in the edge server 130 is corrupted while the tag 120 is not. Thus, performing the normal revalidation process will not detect the error. Moreover, given the amount of data that is typically cached in a modern CDN 100, manually checking media assets 115 can be too time consuming and difficult. In contrast, the recovery module 125 can modify the revalidation process to force the edge servers 130 to update all (or a portion of) their cached media assets 115 by changing the corresponding tags 120 in the origin. Thus, when the revalidator 145 performs revalidation at the expiration of the TTL 140, the locally stored tag 120 will not match the changed tag 120 in the origin 105. In response, the edge server 130 will flush the cached media asset 115, thereby replacing any corrupted media asset 115.

[0023] Although the term edge server 130 is used, each edge server 130 can be a collection of servers at a particular location. For example, depending on the size of the CDN 100, each edge server 130 itself can be a data center that includes multiple computing systems.

[0024] Moreover, although not discussed in detail herein, the edge servers 130 or origin 105 can establish eviction rules for when the edge servers 130 evict (or delete) media assets 115 from their cache 135. These rules can be based on the number of user requests, the rate of user requests, time thresholds, etc. The revalidation process can be separate from the eviction process. After a media asset 115 is evicted / deleted, if the edge server 130 receives another user request for the asset 115, the edge server 130 again downloads the asset 115 from the origin 105.

[0025] Although not shown, the origin data center 105 and edge servers 130 can include at least one processor and memory for storing and executing software applications (e.g., the revalidator 145 and recovery module 125). Moreover, in another embodiment, the revalidator 145 and recovery module 125 can be stored in a computing system separate from the edge servers 130 and origin data center 105.

[0026] Figure 2 A forced flush performed during revalidation is shown in accordance with one embodiment. Figure 2 A portion of the CDN 100 in Figure 1 the edge server's cache 135, the recovery module 125 in the origin data center, and the library 110 in the origin data center.

[0027] Whether automatically or in response to a system administrator, the recovery module 125 begins a forced refresh by performing a tag change 205. As shown, the recovery module 125 generates a modified tag 210, which is then used as a modification to the original tag 140B stored in the repository 110. In this example, the recovery module 125 adds a prefix "Gl" to the original tag "123456," so that the modified tag 210 for this media asset 115 is now "Gl-123456." Again, the recovery module 125 modifies the tag 140B even though the underlying media asset 115 can not have been changed or otherwise updated. In another example, the recovery module 125 adds or changes a suffix of the original tag to produce the modified tag 210.

[0028] Thereafter, for example, when the TTL for the media asset 115 expires, a revalidator in the edge server determines to perform a revalidation process on the media asset 115 stored in the cache 135. A revalidation request 215 is sent from the edge server that includes the tag 140A stored in the cache 135. As shown, the value of the tag 140A (i.e., "123456") matches the original tag 140B stored in the repository 110. However, this tag 140B has been changed to the modified tag 210 by the recovery module 125 during the tag change 205. Thus, the origin informs the edge server that the tag 140A does not match the modified tag 210 now stored in the repository 110. In response, the edge server performs a forced refresh 220 in which the edge server downloads the media asset 115 from the repository 110 and stores it in the cache 135, thereby replacing the copy of the media asset 115 previously stored in the cache 135. If the cached media asset 115 was corrupted, the forced refresh 220 replaces the corrupted copy with an uncorrupted copy. If the cached media asset 115 was not corrupted, the forced refresh 220 simply replaces the media asset 115 with the same copy.

[0029] In one embodiment, the revalidation request 215 includes an "If-None-Match: Gl-123456" header that indicates a conditional request to the origin. The origin, in response, returns a 304 Not Modified response, which indicates that the cached copy of the media asset 115 is valid and should be used. In this case, the edge server does not perform the forced refresh 220. <etag>HTTP request header. If the value of the etag in the If-None-Match header matches the current value of the tag in the library 110, the origin responds affirmatively with a "304 Not Modified" indicating that the content is up to date. Otherwise, if the tags do not match, the origin responds with a "200 OK" and sends a new copy of the object. This HTTP procedure can be used during both normal revalidation requests as well as forced refresh 220.

[0030] Furthermore, as part of the forced refresh 220, the modified tag 210 is stored in the cache 135. Thus, when the edge server initiates another revalidation procedure for the asset, because both now store the modified tag 210 - i.e., "Gl-123456" - the tag in the edge server will match the tag in the origin. Of course, this assumes that the media asset 115 has not been updated in the origin, which would change the tag stored in the library 110. If the tags do not match, then the edge server performs a "normal" refresh instead of a forced refresh 220 since the edge server is responding to a change in the underlying media asset 115 stored at the origin, and not as part of a data recovery procedure to correct an error in the media asset 115 stored at the edge server.

[0031] Figure 3 is a flowchart of a method 300 for performing forced refreshes in batches, according to one embodiment. At block 305, the recovery module (e.g., the recovery module 125 in the origin 105 and the edge server 115) identifies a potentially corrupted cached object at an edge server of a CDN. In one embodiment, a corrupted object can be detected based on a user complaint that a retrieved object has an error, such as poor media asset quality or missing scenes, or a downloaded software application that cannot be installed. In another embodiment, an automated quality control application can randomly retrieve and inspect cached objects at an edge server to determine whether they have any errors. Figure 1 and Figure 2 In another embodiment, a network monitoring application can monitor the status of the network used to transfer objects from the origin data center to the edge server. If the network monitoring application detects an error in the network that can cause an object to be corrupted while being transferred over the network (e.g., packet loss, connection loss, etc.), the application can notify the recovery module. In this example, the recovery module can identify a situation in which an object has been corrupted without any person or application inspecting the object - i.e., the recovery module can assume that the object is corrupted due to the network error and perform a forced refresh.

[0032] In another embodiment, a network monitoring application can monitor the status of the network used to transfer objects from the origin data center to the edge server. If the network monitoring application detects an error in the network that can cause an object to be corrupted while being transferred over the network (e.g., packet loss, connection loss, etc.), the application can notify the recovery module. In this example, the recovery module can identify a situation in which an object has been corrupted without any person or application inspecting the object - i.e., the recovery module can assume that the object is corrupted due to the network error and perform a forced refresh.

[0033] In another embodiment, the recovery module can initiate a forced refresh in response to a predefined time period. For example, a forced refresh can be performed once a year, regardless of whether any potential corruption has been identified in the cached objects. In another embodiment, a forced refresh can also be manually triggered on all content or a subset of content.

[0034] At block 310, the recovery module subdivides the tags corresponding to the cached objects into batches. In one embodiment, the recovery module decides to perform a forced refresh on all objects in the CDN. In this case, the recovery module can subdivide the tags of these objects into batches, where a forced refresh is performed on each batch at a different time. However, in another embodiment, the recovery module performs a forced refresh on only a portion of the objects in the CDN. For example, the recovery module can exclude some objects, which will be discussed later in Figure 5 In another example, the recovery module can know that only a portion of the objects are corrupted (e.g., only a particular type of object is corrupted), and perform a forced refresh on only these objects. In any case, the recovery module identifies the tags corresponding to those objects, and uses the tags to batch the objects.

[0035] Figure 4 Batches of Content Using Object Tags Figure 4 shows the use of object tags to batch content, according to one embodiment. That is, the graph 400 shows data about tags that can be used to divide objects into different batches. Columns 405A-D define different ranges of the last value in the tag (e.g., EtagRegex) of an object. In this example, the tag is a series of hexadecimal digits, where each value is either 0-9 or A-F - e.g., "1F05027D9245". In this example, the last value (e.g., the value of the least significant bit) is used to subdivide the tag, and thus the object, into different batches. While the least significant bit is described in this example, in another embodiment, the most significant bit (or bits) can be used instead.

[0036] The first row of column 405A represents all tags ending in the value 0, the second row of column 405A represents all tags ending in the value 0 or 1, the third row of column 405A represents all tags ending in the value 0, 1, or 2, and the fourth row of column 405A represents all tags ending in the value 0, 1, 2, or 3. Similarly, the first row of column 405B represents all tags ending in the value 0, 1, 2, 3, or 4, the second row of column 405B represents all tags ending in the value 0, 1, 2, 3, 4, or 5, and so on, until reaching the last row in column 405D, which represents all tags ending in the value 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, or F (i.e., all tags).

[0037] Columns 410A-D indicate the proportion of content in the CDN (i.e., the proportion of objects) that has those labels. For example, according to the first row of column 410A, 6.25% of the objects in the CDN have labels that end in the value 0. According to the second row of column 410A, 12.5% of the objects in the CDN have labels that end in the value 0 or 1, and so on. Thus, chart 400 represents a CDN in which the last values of the labels evenly distribute the objects or content in the CDN.

[0038] By using chart 400, if the recovery module wants to divide the objects into four batches, it can select all labels with values ending in 0, 1, 2, or 3 into a first batch, all labels with values ending in 4, 5, 6, or 7 into a second batch, all labels with values ending in 8, 9, A, or B into a third batch, and all labels with values ending in C, D, E, or F into a fourth batch. According to chart 400, these four batches would each contain 25% of the content in the CDN. Thus, to force a refresh on only the first batch, the recovery module can change all labels ending in 0, 1, 2, or 3 to a different value (e.g., prefix these labels with a prefix, such as "Gl" as shown Figure 2 Thus, when an edge server sends a revalidation request for an object with a label ending in 0, 1, 2, or 3, the labels will not match, and the edge server will download a new copy of the corresponding object. However, when an edge server sends a revalidation request for an object with a label ending in any other value (i.e., 4-9 or A-F), the labels will match, and the edge server will not refresh the object (assuming the object has not been modified). In this way, the amount of objects that are forced to refresh using revalidation can be controlled by batching the objects. If the system administrator is concerned about network overload, the batches can be smaller (e.g., each of the potential values of the last digit in the label can be a different batch).

[0039] Using the last value of the tag to subdivide the objects into different batches is just one example. If more granularity is desired, the last two values of the tag can be used. Or, the first value of the tag can be used. In one embodiment, the recovery module can use any value that provides an average distribution of the tags. For example, by evaluating the tags, the recovery module or system administrator can determine that the last value of the tags is different than what is shown in chart 400, and that there is not an even distribution of the content. For example, 11.7% of the content can have tags with a last value of 0, while the remaining tags are evenly distributed with the remaining last values (i.e., 5.9% of the content has tags with a last value of 1, 5.9% of the content has tags with a last value of 2, and so on). To obtain 17 equal batches, the recovery module can assign tags with a penultimate value of 0-7 and a last value of 0 to one batch (assuming the content with the last tag value of 0 is evenly distributed, this would be 11.7 / 2 = 5.9% of the content), and assign tags with a penultimate value of 8-F and a last value of 0 to a second batch (this would be another 5.9% of the content). The other 15 batches can be formed using the remaining potential last values of the tags (i.e., values 1-F). Thus, the recovery module can use any number of values (and any combination) in the tags to form the batches.

[0040] Further, the recovery module need not ensure that each batch covers an equal amount of content in the CDN. For example, three batches can each include 20% of the content, while a fourth batch can have 40% of the content. The recovery module can perform forced refreshes on the fourth batch during times of low network traffic between the origin and edge servers, while the other three batches can be performed during peak or normal times.

[0041] Returning to the method 300, at block 315, the recovery module modifies the tags in the selected batch to force a refresh of the cached objects during the revalidation. As described above, the recovery module can modify only the tags in this batch and not the tags of objects in other batches. For example, the recovery module can add an additional prefix to the tags, add an additional suffix to the tags, or otherwise change the tags so that they do not match the tags of cached objects in the edge server. But embodiments herein are not limited to prefixes, the recovery module can change the tags so that they are not inadvertently refreshed when performing subsequent batch processing. For example, if the recovery module is to change the last value of the tags (e.g., the suffix) and the last value is used to divide the tags into batches, then the objects can be unnecessarily updated when performing subsequent batches. For example, if the recovery module changes the last value of the tags ending in "0" to "1" when performing a forced refresh of objects with tags ending in "0," then the objects will be forced to refresh again when performing a forced refresh of objects with tags ending in "1" in the next batch. Adding a value to the prefix instead of changing the last value (e.g., the suffix) is an easy way to avoid this problem and ensure that the tags remain unique. Moreover, the new prefix can be an easy way to identify which objects have been refreshed and which have not (i.e., objects with shorter tags).

[0042] At block 320, the recovery module determines when the selected batch is complete - i.e., the objects in the batch have been forced to refresh. In one embodiment, the recovery module waits until the TTL of the objects has elapsed. For example, if the TTL is one week, then the recovery module waits until one week has elapsed from its performance of block 315. Doing so ensures that the edge server performs the revalidation process at least once on all of its cached objects, and thus, any cached objects with mismatched tags will be refreshed.

[0043] In another embodiment, the recovery module can monitor the amount of network traffic associated with the revalidation requests and when the traffic returns to the normal historical average, assume that all forced refreshes have been performed and move to the next batch.

[0044] In another embodiment, the recovery module can evaluate the values of the tags sent in the revalidation requests. If the expected proportion of tags in the revalidation request include the prefix added to the tags to force a refresh (e.g., "Gl"), then the recovery module knows that the batch is complete. For example, if this is the first batch and it is assumed that each batch has 6.25% of the content in the CDN, then when 6.25% of the tags in the revalidation request have the prefix, the recovery module knows that the batch is complete. If it is the second batch, then when approximately 12.5% of the tags in the revalidation request have the prefix, the recovery module knows that the batch is complete, and so on.

[0045] In addition, the forced refresh can be performed in conjunction with the normal revalidation process. For example, if one of the objects at the source data center is updated and the tag is changed, then the edge server can recognize that the tag has changed and download the updated object, whether or not the updated object was part of the batch.

[0046] Assuming that the current batch is complete, the method 300 proceeds to block 325, where the recovery module determines whether all of the batches are complete. If not, the method proceeds to block 330, where the recovery module selects a different batch and repeats block 315, but this time without modifying the tags of the new batch. Thus, the edge server will force refresh the objects in this batch, but the objects in the previous batch will not be refreshed (assuming they have not been updated at the source).

[0047] When all of the batches have been forced refreshed, at block 335, the recovery module can continue to perform revalidation without forcing refresh of the cached objects. That is, the revalidation can continue to refresh any objects that are updated at the source. Notably, as described above, this "normal” revalidation process can occur in parallel with the forced refresh.

[0048] In one embodiment, the recovery module can repeat the method 300 in order to restore the modified tags to their original values. This will require another forced refresh of the batches, but it will also restore the CDN to the previous state with the original tags and without any corrupted data. This can be preferable if the system administrator is concerned that the modified tags (e.g., the added prefix) can cause problems later on. However, the system administrator can prefer to keep the modified tags and let these tags be removed over time in the normal revalidation process as objects are updated at the source.

[0049] Figure 5 is a flowchart of a method 500 for performing forced refresh in batches, according to one embodiment. As shown, the method 500 begins at block 505, where the recovery module receives a request to perform a forced refresh of a batch of objects. Figure 3 The process begins after block 310, in which the recovery module subdivides the tags into batches. In block 505, the recovery module identifies any objects in the selected batch that should be excluded from the forced refresh. For example, if the CDN is part of a media streaming service, the recovery module can leave the tags of media assets that are in high demand (e.g., popular shows or new releases) unmodified so as not to interfere with the CDN's ability to service user requests for these assets. In another example, the recovery module can exclude any objects that were added to the CDN after the event that triggered the forced refresh. For example, if the recovery module determines that a network outage a week ago corrupted some objects, the recovery module can not perform a forced refresh on any objects that were added to the CDN since then. Thus, the recovery module can select which objects in the CDN should or should not be subject to a forced refresh.

[0050] In one embodiment, the recovery module can use the path or name of an object to select which objects should be excluded from the forced refresh. For example, objects corresponding to a particular movie or television show can all share the same path or name. Before modifying the tags of the objects, the recovery module can perform a check to determine whether its path or name matches the ones identified in block 505, and if so, leave its tag unmodified.

[0051] In block 510, the recovery module modifies all the tags in the selected batch, except for the ones corresponding to the excluded objects. The tags can be modified using any of the techniques discussed above. Because the tags of the excluded objects are not forced to change, the edge servers will only download new copies of these objects during revalidation if the object is changed at the origin and given a new tag.

[0052] In block 515, the recovery module can determine whether it should stop the forced refresh when performing the forced refresh on the objects of the selected batch. For example, the recovery module can monitor network traffic between the origin data center and the edge servers, or can monitor compute usage in the origin data center, to determine whether the forced refresh is overwhelming network resources or local resources in the origin data center. In another example, a system administrator can use the recovery module to stop the forced refresh.

[0053] If the forced refresh should be stopped, in block 520, the recovery module acknowledges any tags sent in the revalidation requests. That is, the recovery module can instruct the origin data center to respond to each revalidation request from the edge servers by declaring that the tags match, even if they do not. Doing so prevents the edge servers and the origin from performing any additional forced refreshes and normal revalidation of the content.

[0054] At block 525, the recovery module (or system administrator) determines whether to resume the forced flush. For example, the recovery module can resume the forced flush at a time of reduced network demand (e.g., during early morning or late night hours) or after additional computing resources are launched at the source data center.

[0055] In another example, the recovery module can reduce the size of the selected batch. For example, the recovery module can change the tags of some objects back to their original values—i.e., undo the modifications of block 510— which effectively reduces the size of the batch. However, this can cause some objects that have already been forced flushed to be flushed again later, as the edge server now stores the modified tags instead of the original tags. Nonetheless, if the recovery module stops the forced flush shortly after the batch processing is initiated, this strategy can reduce the size of the batch and reduce the strain on the network or source resources.

[0056] After the forced flush is resumed, at block 320, the recovery module determines whether the batch is complete using any of the techniques discussed in Figure 3 The batch can be the original batch selected at block 510, or a reduced size batch implemented after the forced flush was stopped. If the batch is complete, the method 500 proceeds to block 325 of method 300 to determine whether all batches are complete, and if not, a new batch is selected. If the current batch is not complete, the method 500 returns to block 515 to continue monitoring the forced flush of the current batch for whether it should be stopped. Figure 3

[0057] In summary, the method 500 describes techniques for excluding particular objects from a forced flush and stopping a current batch in response to network or computing problems.

[0058] In the present disclosure, reference is made to various embodiments. However, it should be understood that the disclosure is not limited to the specifically described embodiments. Rather, any combination of the following features and elements, whether related to a different embodiment or not, is contemplated to implement and practice the teachings provided herein. Additionally, as used in the description herein, the ​

[0059] As those skilled in the art will appreciate, the embodiments described herein can be embodied as a system, method, or computer program product. Accordingly, the embodiments can take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.) or an embodiment combining software and hardware aspects that can all generally be referred to herein as a "circuit," "module" or "system." Furthermore, embodiments described herein can take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

[0060] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0061] Computer program code for carrying out operations of embodiments of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0062] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart or block diagram block or blocks.

[0063] These computer program instructions can also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function / act specified in the flowchart or block diagram block or blocks.

[0064] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus or other device implement the functions / acts specified in the flowchart or block diagram block or blocks.

[0065] The flow and block diagrams in the drawings show the architectural, functional and operational views of possible implementations of systems, methods and computer program products according to the various embodiments of the present disclosure. In this regard, each block in the flow or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending on the functionality involved. It will also be noted that each block in the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0066] While the forgoing is directed to embodiments of the present disclosure, other and further embodiments can be devised without departing from the basic scope of the present disclosure, and the scope of the present disclosure is determined by the appended claims.< / etag>

Claims

1. A method for forced refresh, comprising: At the source data center, the tags corresponding to objects in the Content Delivery Network (CDN) are further subdivided to form multiple batches; A forced refresh of objects in the first batch is performed at the source data center by spontaneously modifying the tags corresponding to objects in the first batch at the source data center without considering whether objects in the first batch have been updated, as part of a re-verification process associated with edge servers in the CDN, wherein the modified tags are stored at the source data center in the CDN. as well as After determining that a forced refresh of the objects in the first batch has been completed at the source data center, the tags in the second batch of the plurality of batches are selected and modified at the source data center to perform a forced refresh of the objects in the second batch.

2. The method according to claim 1, further comprising: After the labels for the second batch have been modified, a forced refresh is performed on the second batch using the re-verification process initiated by the edge server in the CDN.

3. The method of claim 1, wherein, A copy of the object is stored at both the source data center and the edge server.

4. The method of claim 3, further comprising, before segmenting the labels: Identify potential corruption that could cause a mismatch between a copy of an object stored at the edge server and a corresponding copy of an object stored at the source data center.

5. The method of claim 3, wherein, The re-verification process includes: A request is received from the edge server at the source data center, the request being used to determine whether the label of a copy of an object stored at the edge server matches the corresponding label of a copy of an object stored at the source data center; and When it is determined that the label at the edge server does not match the label at the source data center, the copy of the object stored at the edge server is refreshed using the copy of the object stored at the source data center.

6. The method of claim 1, wherein, Subdividing tags includes: Tags with different least significant bit values ​​or different most significant bit values ​​are assigned to different batches within the plurality of batches.

7. The method of claim 1, wherein, Modifications to the tags include: Add a new prefix or suffix to each tag in the first batch, wherein the corresponding tag stored in the edge server does not have the new prefix or suffix.

8. The method according to claim 1, further comprising: Based on the path or name corresponding to the object, identify at least one object in the first batch to be excluded from the forced refresh, wherein the label of the at least one object is not modified.

9. The method according to claim 1, further comprising: Determine to stop the forced refresh before the first batch is completed; as well as Accept any tags in the re-verification request received at the source data center to stop the forced refresh.

10. A non-transitory computer-readable medium comprising computer program code, said computer program code, when executed by operations of one or more computer processors, performing operations including: subdividing, at a source data center, tags corresponding to objects in a content delivery network (CDN) to form a plurality of batches; performing, at the source data center, a forced refresh of objects in a first batch of the plurality of batches as part of a revalidation process associated with an edge server in the CDN by spontaneously modifying, at the source data center, tags corresponding to objects in the first batch without regard to whether the objects in the first batch have been updated, wherein the modified tags are stored at the source data center in the CDN; and after determining that the forced refresh of objects in the first batch is complete at the source data center, selecting and modifying, at the source data center, tags in a second batch of the plurality of batches to perform a forced refresh of objects in the second batch.

11. The non-transitory computer-readable medium of claim 10, the operations further comprising: after the tags of the second batch have been modified, performing a forced refresh of the second batch using the revalidation process initiated by the edge server in the CDN. a copy of the object is stored at both the source data center and the edge server.

12. The non-transitory computer-readable medium of claim 10, wherein, 13. The non-transitory computer-readable medium of claim 12, the operations further comprising, prior to subdividing tags: identifying potential corruption that causes a copy of an object stored at the edge server to not match a corresponding copy of the object stored at the source data center. subdividing tags includes:

14. The non-transitory computer-readable medium of claim 10, wherein, assigning tags having one of different least significant bit values or different most significant bit values to different batches of the plurality of batches. modifying tags includes:

15. The non-transitory computer-readable medium of claim 10, wherein, adding a new prefix or suffix to each tag in the first batch, wherein the corresponding tag stored in the edge server does not have the new prefix or suffix.

16. A system for forced refresh, comprising: a processor; and a memory, the memory including computer program code that, when executed by operation of the one or more computer processors, performs operations comprising: subdividing, at a source data center, tags corresponding to objects in a content delivery network (CDN) to form a plurality of batches; performing, at the source data center, a forced refresh of objects in a first batch of the plurality of batches as part of a revalidation process associated with an edge server in the CDN by spontaneously modifying, at the source data center, tags corresponding to objects in the first batch without regard to whether the objects in the first batch have been updated, wherein the modified tags are stored at the source data center in the CDN; and after determining that the forced refresh of objects in the first batch is complete at the source data center, selecting and modifying, at the source data center, tags in a second batch of the plurality of batches to perform a forced refresh of objects in the second batch.

17. The system of claim 16, the operations further comprising: ​ ​ After the labels of the second batch have been modified, a forced refresh is performed on the second batch using the revalidation process initiated by the edge server in the CDN.

18. The system of claim 17, wherein, A copy of the object is stored at both the source data center and the edge server, wherein the operations further comprise, prior to subdividing the labels: Identifying potential corruption that causes a copy of the object stored at the edge server to not match a corresponding copy of the object stored at the source data center.

19. The system of claim 16, wherein, Subdividing the labels comprises: Assigning labels having one of different least significant bit values or different most significant bit values to different ones of the plurality of batches.

20. The system of claim 16, wherein, Modifying the labels comprises: Adding a new prefix or suffix to each label in the first batch, wherein a corresponding label stored in the edge server does not have the new prefix or suffix.

Images