Data caching method, device and system for content distribution network

Abstract

The invention provides a content delivery network-oriented data caching method, device, system and equipment, a readable storage medium and a program product, and relates to the field of intelligent cloud, in particular to cloud storage, cloud computing, cloud service, a CDN, a content delivery network and the like. The method comprises the following steps: sending a second data request to a source station based on a first length range and a first data request for acquiring target data; wherein the second data request is used for requesting at least one data block with the first data length in the target data; the first data length is within a first length range; receiving at least one data block of a first data length; sending target data comprising at least one data block of the first data length; and performing data splicing based on the at least one data block with the first data length, and storing the spliced target data into a cache. And more flexible and efficient source returning and cache management can be realized.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent cloud, specifically to technical fields such as cloud storage, cloud computing, cloud services, CDN, and content distribution network, and in particular to a data caching method, device, system, electronic device, computer-readable storage medium, and program product for a content distribution network. Background Art

[0002] CDN (Content Delivery Network) is a network architecture that deploys distributed servers globally and caches content to nodes closest to users, thereby accelerating content transmission and improving user access speed and experience. Summary of the Invention

[0003] The embodiments of the present disclosure provide a data caching method, apparatus, electronic device, computer-readable storage medium, and computer program product for a content distribution network, which can achieve more flexible and efficient back-to-source and cache management.

[0004] In a first aspect, an embodiment of the present disclosure proposes a data caching method for a content distribution network, comprising: sending a second data request to a source station based on a first length range and a first data request for obtaining target data; wherein the second data request is used to request at least one data block of the first data length in the target data; the first data length is within the first length range; receiving at least one data block of the first data length from the source station; sending target data including at least one data block of the first data length to a device corresponding to the first data request; performing data splicing based on the at least one data block of the first data length to obtain the spliced target data, and storing the spliced target data in a cache.

[0005] In a second aspect, an embodiment of the present disclosure proposes a data caching device for a content distribution network, comprising: a first sending unit, configured to send a second data request to a source station based on a first length range and a first data request for obtaining target data; wherein the second data request is used to request at least one data block of the first data length in the target data; the first data length is within the first length range; a first receiving unit, configured to receive at least one data block of the first data length from the source station; a second sending unit, configured to send target data including at least one data block of the first data length to a device corresponding to the first data request; a splicing cache unit, configured to perform data splicing based on at least one data block of the first data length to obtain the spliced target data, and store the spliced target data in a cache.

[0006] In a third aspect, embodiments of the present disclosure provide a data caching system for a content distribution network, comprising: a cache node and a source station. The cache node is configured to: send a second data request to the source station based on a first length range and a first data request for obtaining target data; the second data request is used to request at least one data block of the first data length in the target data; the first data length is within the first length range; receive at least one data block of the first data length from the source station; send target data including at least one data block of the first data length to a device corresponding to the first data request; perform data splicing based on the at least one data block of the first data length to obtain the spliced target data, and store the spliced target data in the cache.

[0007] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it can implement the data caching method for a content distribution network as described in any implementation method in the first aspect.

[0008] In a fifth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the data caching method for a content distribution network as described in any implementation method of the first aspect when executed.

[0009] In a sixth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the data caching method for a content distribution network as described in any implementation manner in the first aspect.

[0010] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 is an exemplary system architecture in which the present disclosure may be applied; Figure 2 A flowchart of a data caching method for a content distribution network provided by an embodiment of the present disclosure; Figure 3 A flowchart of another data caching method for a content distribution network provided by an embodiment of the present disclosure; Figure 4 A structural block diagram of a data caching device for a content distribution network provided by an embodiment of the present disclosure; Figure 5 A schematic structural diagram of an electronic device suitable for executing a data caching method for a content distribution network, provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other unless there is a conflict.

[0013] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0014] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the content distribution network-oriented data caching method, apparatus, system, electronic device, and computer-readable storage medium of the present disclosure can be applied.

[0015] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, a cache node 105, and an origin station 106. Network 104 is a medium for providing communication links between terminal devices 101, 102, and 103 and cache node 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0016] Users can use terminal devices 101, 102, and 103 to interact with cache nodes 105 via network 104 to receive or send messages, etc. Terminal devices can also be referred to as clients. Cache nodes 105 can interact with origin servers 106 via a network, which can include various connection types, such as wired or wireless communication links or fiber optic cables. Various applications, such as video applications and instant messaging applications, can be installed on terminal devices 101, 102, and 103, cache nodes 105, and origin servers 106 to facilitate information communication between them.

[0017] Terminal devices 101, 102, 103, cache node 105, and source station 106 can be either hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules, or as a single software program or software module, without specific limitations here. When cache node 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module, without specific limitations here. When source station 106 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module, without specific limitations here.

[0018] The cache node 105 can provide various services through various built-in applications. Taking an application that can provide services based on client requests as an example, the cache node 105 can achieve the following effects when running such applications: based on the first length range and the first data request for obtaining target data, a second data request is sent to the source station; wherein the second data request is used to request at least one data block of the first data length in the target data; the first data length is within the first length range; at least one data block of the first data length is received from the source station; target data including at least one data block of the first data length is sent to the device corresponding to the first data request; data splicing is performed based on at least one data block of the first data length to obtain the spliced target data, and the spliced target data is stored in the cache.

[0019] It should be noted that, in addition to being obtained from the source station 106, the target data can also be pre-stored locally in the cache node 105 through various means. Therefore, when the cache node 105 detects that the data is already stored locally, it can choose to directly obtain the data from the local location. In this case, the exemplary system architecture 100 may not include the source station 106.

[0020] Because the target data requires a significant amount of storage resources, the data caching methods for content distribution networks provided in the subsequent embodiments of this disclosure are generally performed by cache nodes 105 with relatively strong storage capabilities. Accordingly, data caching devices for content distribution networks are generally also located in cache nodes 105. However, it should also be noted that when terminal devices 101, 102, and 103 also have sufficient storage resources, terminal devices 101, 102, and 103 can also use applications installed thereon to complete the various operations previously assigned to cache node 105, thereby outputting the same results as cache node 105. In particular, in the case of multiple terminal devices with varying storage capabilities, if an application (such as a video application) determines that its terminal device has a significant amount of remaining storage resources, it can allow the terminal device to perform the aforementioned operations, thereby appropriately alleviating the computational pressure on cache node 105. Accordingly, data caching devices for content distribution networks can also be located in terminal devices 101, 102, and 103. In this case, exemplary system architecture 100 may also exclude cache nodes 105 and network 104.

[0021] It should be understood that Figure 1 The number of terminal devices, networks, cache nodes, and source stations in the embodiment is merely illustrative. Any number of terminal devices, networks, and servers may be used as required.

[0022] In some implementations, CDN systems typically utilize cache nodes to respond to user requests. For the initial request, the cache node retrieves the data from the source and caches it locally. Subsequent requests directly hit the cache. Most cache nodes support Hypertext Transfer Protocol (HTTP) range requests, enabling range-based fragmented retrieval and storing file resources on disk. Implementing flexible and efficient retrieval and cache management is a pressing issue.

[0023] For example, a cache node can use a fixed-size back-to-origin range for shard requests and store shards of the same size on disk. For example, if a file is 20MB in size and the client initiates a range request for a file range of [4.2MB–6.6MB], the cache node's processing flow is as follows: Step 1: After receiving the client's request, the cache node calculates the corresponding cache key based on the Uniform Resource Locator (URL) of the requested data and queries the cache index management system to see if the data has been cached on the local disk.

[0024] Step 2: If the range of data requested by the client has been completely cached in the local disk, the following steps 2.1 to 2.4 may be included.

[0025] Step 2.1: The cache node splits the client request range into multiple fixed-size shard intervals, such as [4MB–5MB], [5MB–6MB], and [6MB–7MB], and calculates the index information corresponding to these three shards respectively.

[0026] Step 2.2: Find the index information of the corresponding shard in the cache index management system. The index information includes the storage starting position and offset of the shard on the local disk.

[0027] Step 2.3: The cache node initiates three read operations on the disk, reading the data of the corresponding shards and loading them into the memory.

[0028] Step 2.4: Extract and concatenate the multiple sharded data into the [4.2MB–6.6MB] data range requested by the client, and then respond to the client with this data.

[0029] Step 3: If the disk does not cache the data range requested by the client at all, the following steps 3.1 to 3.3 may be included.

[0030] Step 3.1: The cache node determines the number of back-to-origin shards based on the preset back-to-origin range size. Assuming the back-to-origin range size is 1MB, three requests need to be made to the origin server, namely [4MB–5MB], [5MB–6MB], and [6MB–7MB].

[0031] Step 3.2: After the cache node downloads the required shards, it extracts and concatenates the multiple shards into the [4.2MB–6.6MB] data range requested by the client and responds to the client request.

[0032] Step 3.3: The cache node stores the three shards one by one on the local disk and creates corresponding index information for each shard in the cache index management system.

[0033] Step 4: If the file range requested by the client has been partially cached, the following steps 4.1 to 4.4 may be included.

[0034] Step 4.1: If the local disk has cached [4MB–5MB] of data, the cache node only needs to return the uncached portion to the source.

[0035] Step 4.2: Based on the size of the back-to-origin range, the cache node initiates two requests, [5MB–6MB] and [6MB–7MB], to obtain the uncached shards from the origin server.

[0036] Step 4.3: After the download is complete, the cache node extracts and reassembles the data in the entire required range to respond to the client request.

[0037] Step 4.4: Store the two newly restored shards on the local disk and create corresponding index entries for them.

[0038] For example, improperly configuring the fixed back-to-source segment size described above can lead to the following problems: For example, if the back-to-source segment size is too large and covers too wide a range, the actual required data often only occupies a small portion of the data block, thereby wasting a large amount of back-to-source bandwidth and storage resources. Alternatively, if the back-to-source segment size is too small, excessive fragmentation can be introduced, causing a surge in the number of data blocks that cache nodes must manage, increasing the load on cache index information and memory consumption, while also accelerating cache eviction rates and reducing overall hit rates. Furthermore, if the back-to-source segment size is too small, it can also lead to high disk input (I) / output (O) pressure. Overly sharded storage can result in multiple random disk I / Os, impacting overall system performance.

[0039] Please refer to Figure 2 , Figure 2 This is a flow chart of a data caching method for a content distribution network provided by an embodiment of the present disclosure, wherein process 200 includes the following steps: Step 201: Send a second data request to a source station based on a first length range and a first data request for obtaining target data.

[0040] This step is intended to be performed by the execution subject of the data caching method for content distribution network (such as Figure 1 The cache node 105 shown sends a second data request to the origin server based on the first length range and the first data request for the target data. The origin server is a server or server cluster that stores original content (such as web pages, images, videos, and files). For example, it is the ultimate source of all cached content in a CDN network. When a user requests content, if the cache node does not have the content locally or the cache is invalid, it can obtain the content from the origin server and synchronize it to the cache node to respond to the user request.

[0041] Exemplarily, the first length range may be pre-set, and the value interval of the first length range may be a range between the minimum value of the first length range and the maximum value of the first length range, for example, a≤x≤b, where x represents the first length range, a represents the minimum value of the first length range, and b represents the maximum value of the first length range. The first length range may be used to divide data intervals of the target data.

[0042] Optionally, the first data request may be received from a client, where the user requests to obtain target data.

[0043] Exemplarily, the second data request may be used to request at least one data block of a first data length in the target data, wherein the first data length is within a first length range.

[0044] For example, after the target data interval is divided into a data interval using a first length range, the divided data interval of the target data includes: one or more first data lengths, where a≤L≤b, L represents the first data length, a represents the minimum value of the first length range, and b represents the maximum value of the first length range.

[0045] Optionally, the number of the second data request may be one or more.

[0046] For example, a second data request is used to request a data block of a first data length in the target data, and the second data request corresponds one-to-one to each data block of the first data length. In this way, when the data interval of the divided target data includes multiple first data lengths, the number of second data requests can be multiple.

[0047] For another example, a second data request supports requesting multiple data blocks of the first data length in the target data. A second data request may correspond to multiple data blocks of the first data length, and the number of second data requests may be one.

[0048] Step 202: Receive at least one data block of a first data length from a source station.

[0049] Exemplarily, the above-mentioned execution entity sends a second data request to the source station, requesting to obtain at least one data block of the first data length in the target data. After receiving the second data request, the source station sends at least one data block of the first data length, and the above-mentioned execution entity receives at least one data block of the first data length from the source station.

[0050] For example, if there are multiple first data lengths, the execution entity may receive the data block of the first data length from the source station once, or may receive the data block of the first data length from the source station multiple times.

[0051] Step 203: Send target data including at least one data block of the first data length to the device corresponding to the first data request.

[0052] Exemplarily, the device corresponding to the first data request may be a client, and the execution subject sends the target data including at least one data block of the first data length to the client.

[0053] For example, the execution entity may splice at least one data block of the first data length based on the data interval of the target data and send the spliced data block to the client, thereby reducing the delay.

[0054] Optionally, the target data may include at least one data block of the first data length, and may also include data obtained by the execution entity from a local cache. For example, the execution entity caches part of the data requested in the first data request.

[0055] Step 204: performing data splicing based on at least one data block of the first data length to obtain spliced target data, and storing the spliced target data in a cache.

[0056] Exemplarily, the above-mentioned execution entity performs data splicing based on at least one data block of a first data length, for example, merging multiple adjacent or continuous segments of data into a larger data piece to obtain spliced target data, which may include one or more segments of data, and stores the spliced target data in a cache, for example, the cache may refer to a disk.

[0057] In the method provided by the embodiment of the present disclosure, the data block size (first data length) used during retrieval is different from the size of the locally stored data slices, which can achieve more flexible and efficient retrieval and cache management.

[0058] For example, using a smaller data block size during the source return phase (e.g., step 201) compared to the storage phase (e.g., step 204) can avoid downloading excessive data not actually needed by the client and reduce bandwidth waste. During the storage phase, splicing and caching are performed first, allowing for larger data blocks to be stored. When writing data back to the source into the cache, multiple adjacent or continuous data blocks are merged to form a larger data block, rather than writing them block by block. This significantly reduces the number of writes and improves subsequent read performance.

[0059] Please refer to Figure 3 , Figure 3 This is a flow chart of another data caching method for a content distribution network provided by an embodiment of the present disclosure, wherein process 300 includes the following steps 301 to 303. In some embodiments, the above step 201 may include steps 301 to 303.

[0060] Step 301: Determine a first data length based on a first length range and a data interval of target data.

[0061] Optionally, the data interval of the target data may include one or more first data lengths.

[0062] For example, the above-mentioned execution entity determines the size of the first data length based on the first length range and the data interval of the target data, and the first data length must be within the first length range, for example, the data interval of the target data is divided into one or more intervals of the first data length.

[0063] Optionally, step 301 may further include determining the first data length based on the real-time bandwidth, the first length range, and the data interval of the target data. This allows determining the segment length based on the network environment, further improving the flexibility of determining the first data length and achieving more efficient return to the source.

[0064] Step 302: Generate a second data request based on the first data length and the data interval of the target data.

[0065] For example, the second data request is used to request at least one data block of the first data length in the target data. The number of generated second data requests can be one or more. For details, please refer to the description of the second data request in step 201 above.

[0066] Step 303: Send a second data request to the source station.

[0067] Exemplarily, when there are multiple second data requests, the second data requests may be initiated in parallel to shorten the overall time consumption.

[0068] The method provided in the embodiments of the present disclosure can reasonably and flexibly determine the size of the first data length based on the first length range and the data interval of the target data, thereby avoiding the unreasonable configuration of a fixed back-to-source segment size that leads to problems such as bandwidth waste, excessive load consumption, or excessive disk pressure, and can improve overall system performance.

[0069] In some embodiments, the above step 301 determines the first data length based on the first length range and the data interval of the target data, and may include: in response to not querying all the target data from the cache, determining the first data length based on the first length range and the data interval of the target data.

[0070] Exemplarily, before determining the first data length based on the first length range and the data interval of the target data, it can be determined whether all the target data is queried from the cache. The cache may store part of the target data, all the target data, or no target data.

[0071] For example, the execution subject queries the cache for target data; in response to not finding the target data in the cache, determines that the target data is not stored in the cache; and in response to finding the target data in the cache, further determines whether all the target data is found in the cache.

[0072] In this way, if there is no target data or only part of the target data in the cache, the segmentation strategy based on the first length range is activated to determine a reasonable segment size to achieve more efficient return to the source.

[0073] In some embodiments, the above step 301 of determining the first data length based on the first length range and the data interval of the target data may include: In response to the minimum value of the first length range being greater than or equal to the length of the data interval of the target data, determining that the first data length is equal to the minimum value of the first length range; or, In response to determining that the minimum value of the first length range is greater than or equal to the length of the data interval of the target data, and that the third data request and the first data request come from the same device and are the next data request of the first data request, the first data length is determined based on the first length range, the data interval of the target data and the data interval of the target data in the third data request.

[0074] For example, even if the target data interval length is less than the preset minimum segmentation threshold (the minimum value of the first length range), segmentation is still performed based on the minimum value of the first length range, thereby avoiding efficiency losses caused by fragmented requests. Alternatively, segmentation can be performed based on the actual length of the target data interval, with the first data length being determined to be equal to the length of the target data interval.

[0075] For example, the third data request and the first data request come from the same device, or the third data request and the first data request come from the same device's consent session, such as the same mobile phone accessing a video application, ensuring that the segmentation policy is bound to the device or user behavior to avoid policy interference across user requests.

[0076] The third data request is the next data request of the first data request, which may refer to the continuity in the request sequence (eg, the first request obtains segment 1, and the third request obtains segment 2), and the data intervals have a front-to-back connection relationship.

[0077] Optionally, the interval between the arrival time of the third data request and the arrival time of the first data request is less than or equal to a time threshold. The first data request and the third data request are continuous, such as frame-by-frame transmission of video playback or segmented file downloading.

[0078] As another example, if the length of the target data interval is less than the preset minimum segmentation threshold (the minimum value of the first length range), based on the continuous requests of the same device, cross-request linkage optimization can be performed, which can not only reduce the overhead of small requests, but also achieve session-level optimization of data transmission and improve overall link efficiency.

[0079] In other embodiments, the above step 301 of determining the first data length based on the first length range and the data interval of the target data may include: In response to a minimum value of the first length range being less than a length of the data interval of the target data, determining a first ratio of the length of the data interval of the target data to the minimum value of the first length range; In response to the first ratio being greater than a first threshold, determining a first data length based on the first ratio and a first length range; In response to the first ratio being less than or equal to a first threshold, it is determined that the first data length is equal to a minimum value of the first length range.

[0080] Exemplarily, the difference between the minimum value of the first length range and the length of the data interval of the target data is determined. If the minimum value of the first length range is less than the length of the data interval of the target data, the ratio of the length of the data interval of the target data to the minimum value of the first length range (referred to as a first ratio) is determined. If the first ratio is determined to be greater than a first threshold, indicating a large number of segments, the first data length is dynamically determined based on the first ratio and the first length range. If the first ratio is determined to be less than or equal to the first threshold, indicating a small number of segments, the first data length can be directly set to the minimum value of the first length range.

[0081] For example, if it is determined that the first ratio is greater than the first threshold, indicating a large number of segments, the number of requests can be reduced by increasing the segment length. For example, the segment length can be dynamically increased in proportion to the first ratio. The larger the first ratio, the larger the first data length, ensuring that the first data length is less than or equal to the maximum value of the first length range. Alternatively, if it is determined that the first ratio is greater than the first threshold, the maximum value or middle value of the first length range can be directly used as the first data length.

[0082] In this way, based on the first ratio and the first threshold, data volume fluctuations can be flexibly addressed, reducing network overhead, source server load, and user access latency. When the first ratio is less than or equal to the first threshold, segments are segmented using the minimum length within a range to ensure flexibility. When the first ratio is greater than the first threshold, the segment length is increased, fully utilizing bandwidth and improving request efficiency while also avoiding request overload caused by excessive segmentation.

[0083] In some embodiments, step 302 of generating the second data request based on the first data length and the data interval of the target data may include: In response to not finding the target data in the cache, splitting the data interval of the target data into one or more first data lengths based on the first data length; Based on the one or more first data lengths, a second data request is generated.

[0084] Exemplarily, when the cache node does not cache the target data, the entire target data interval is split into a plurality of subintervals according to the first data length. The subintervals may be of equal or approximately equal length, or may be of unequal length. A separate second data request is generated for each subinterval, or a single second data request is generated for all subintervals.

[0085] In this way, if the target data is not found in the cache, the data interval of the target data is dynamically split based on the dynamically determined first data length, which can achieve more efficient return to the source.

[0086] In some embodiments, the above-mentioned splitting of the data interval of the target data into one or more first data lengths based on the first data length may include: in response to the second ratio of the length of the data interval of the target data to the first data length being not an integer and the first data length being less than the maximum value of the first length range, based on the first data length, splitting the data interval of the target data into one or more expanded first data lengths using a proportional expansion method.

[0087] Optionally, the length of the first data after expansion is less than or equal to the maximum value of the first length range.

[0088] Exemplarily, in response to the second ratio of the length of the data interval of the target data to the first data length not being an integer, and the first data length being less than the maximum value of the first length range, the first data length is expanded using a proportional expansion method based on the first data length to obtain an expanded first data length, and the data interval of the target data is split into one or more expanded first data lengths. Segments of approximately equal length can evenly distribute source station requests, avoid resource waste caused by the last small segment, and improve the transmission efficiency of individual segments.

[0089] For example, if the length of the target data interval is 15KB and the length of the first data is 4KB, direct splitting will result in three 4KB segments and one 3KB segment (4×3+3=15). The last 3KB segment has the following problems: a high proportion of HTTP request header overhead (the header overhead in a 3KB request may account for 20%); cache fragmentation (a 3KB segment occupies a 4KB cache unit, wasting 1KB of space).

[0090] After expanding to the first data length = 5KB, it is split into three 5KB segments, the number of requests is reduced by 1, the header overhead is reduced by 33%, and the cache space utilization is increased by 25%.

[0091] For another example, if the target data interval length is 17KB, the first data length is 4KB, and the maximum value of the first length range is 5KB, the target data interval length / first data length = 4.25. If expansion is attempted, the expanded first data length = 17KB / 4 = 4.25KB, and the final split is 4.25KB × 4. Alternatively, the expanded first data length can be rounded to 5KB, and the final split is 5KB × 3 + 2KB × 1.

[0092] In other embodiments, step 302 of generating the second data request based on the first data length and the data interval of the target data may include: In response to only part of the target data being found in the cache, splitting the interval corresponding to the unqueried data in the target data into one or more first data lengths based on the first data length; Based on the one or more first data lengths, a second data request is generated.

[0093] For example, when a cache node only caches a portion of the target data, it locates the miss interval (the interval corresponding to data not found in the target data) and splits the miss interval into several subintervals based on the first data length. The subintervals can be of equal, approximately equal, or unequal lengths. A separate second data request is generated for each subinterval, or a single second data request is generated for all subintervals. This ensures that only necessary data is returned to the source, avoiding wasted bandwidth and origin server resources.

[0094] Optionally, the above-mentioned splitting of the interval corresponding to the unqueried data in the target data into one or more first data lengths based on the first data length may include: in response to the fact that a third ratio of the length of the interval corresponding to the unqueried data in the target data to the first data length is not an integer, and the first data length is less than the maximum value of the first length range, splitting the interval corresponding to the unqueried data in the target data into one or more expanded first data lengths based on the first data length using a proportional expansion method. The specific implementation method is similar to the above-mentioned description of splitting the data interval of the target data into one or more expanded first data lengths using a proportional expansion method, and will not be repeated here.

[0095] In some embodiments, the above-mentioned step 204 performs data splicing based on at least one data block of the first data length to obtain the spliced target data, and stores the spliced target data in the cache, which may include: based on the second length range, sequentially splicing the data blocks of each interval in the target data to obtain one or more first continuous data; storing the one or more first continuous data in the cache.

[0096] Optionally, the length of the first continuous data is within a second length range, and the minimum value of the second length range is greater than the maximum value of the first length range.

[0097] Exemplarily, the second length range may be preset, and the value interval of the second length range may be a range between a minimum value of the second length range and a maximum value of the second length range.

[0098] Exemplarily, sequentially joining the data blocks of each interval in the target data may be joining the data blocks of each interval in the target data according to interval identifiers, timestamps, address offsets, and the like.

[0099] In this way, by splicing data blocks from each interval of the target data based on the second length range, and with the minimum value of the second length range being greater than the maximum value of the first length range, the number of disk writes can be significantly reduced, improving subsequent read performance. The minimum value of the second length range can prevent the spliced data blocks from being too small, reducing cache fragmentation, while the maximum value of the second length range can prevent the spliced data blocks from being too large, avoiding response delays caused by excessive data volume during cache queries.

[0100] Optionally, if the continuous data obtained after a single concatenation is less than the minimum length of the second length range, it may be concatenated with subsequent data blocks until the minimum length of the second length range is met. If the final remaining block does not meet the minimum length, it may be temporarily stored, such as by special marking, and subsequently merged with the new data block before being cached.

[0101] In some embodiments, storing the one or more first consecutive data into the cache may include: Creating index information corresponding to one or more first continuous data respectively; One or more first consecutive data and a first correspondence between the first consecutive data and the index information are stored in a cache.

[0102] Optionally, the index information may include but is not limited to at least one of the following metadata: a cache key, a storage location, and a data interval. The cache key, storage location, and data interval corresponding to each first continuous data may be determined to query the first continuous data from the cache based on the index information.

[0103] For example, the cache may include: one or more first continuous data and one or more first corresponding relationships. The corresponding relationships between the first continuous data and the index are stored in the cache to support fast query.

[0104] Optionally, if the first continuous data content is updated, the index information may be recalculated and the corresponding relationship may be updated to avoid index invalidation.

[0105] In this way, by generating and storing a unique index for each first continuous data, the data can be queried through the index later, which can greatly improve the query speed.

[0106] In some embodiments, the method provided by the embodiments of the present disclosure may also include: in response to receiving a fourth data request for obtaining a data block of a second data interval, querying the second data interval from a correspondence library including the first correspondence; in response to querying the second data interval from the correspondence library, obtaining the data block of the second data interval from the cache.

[0107] Illustratively, the fourth data request may be a request initiated by a user or a client to obtain a data block in the second data interval.

[0108] The correspondence library may include a correspondence between at least one continuous data and at least one index information.

[0109] For example, upon receiving the fourth data request, the execution entity extracts the target data interval (the second data interval) and searches the corresponding relationship database for a matching record. If a match is found, the corresponding data block is retrieved from the cache based on the matching relationship. This not only avoids repeated requests for cached data but also enables "zero-latency" data retrieval, meeting real-time interaction requirements. Furthermore, contiguous data blocks after splicing are identified by indexes, allowing queries to directly return complete blocks, improving data integrity (e.g., after splicing video segments, the index is a continuous interval).

[0110] Optionally, in response to not finding the second data interval in the query from the corresponding relationship library, the back-to-source and storage process is triggered, for example, executing the above step 201-step 201.

[0111] To deepen understanding, the present disclosure also provides a specific implementation solution in combination with a specific application scenario.

[0112] For example, assuming the client's first data request requests a file range of [4MB–10MB] and a first length range of 1 to 2 MB, the cache node sends a second data request to the origin based on the first length range and [4MB–10MB]. Assuming the first data length is 1.5 MB, the second data request requests data blocks in the interval [4MB–5.5MB] and data blocks in the interval [5.6MB–7MB]. The origin sends the data blocks in the interval [4MB–5.5MB] and the data blocks in the interval [5.6MB–7MB] to the cache node.

[0113] The cache node sends target data including the data block at the interval [4MB–5.5MB] and the data block at the interval [5.6MB–7MB] to the client. The target data may also include the data block at [7.1MB–10MB]. For example, the data block at [7.1MB–10MB] is stored in the local cache and is not obtained from the source station.

[0114] The cache node performs data splicing based on the data blocks in the interval [4MB–5.5MB] and the data blocks in the interval [5.6MB–7MB]. Assuming that the second length range is 2 to 4 MB, for example, the data blocks in the interval [4MB–5.5MB] and the data blocks in the interval [5.6MB–7MB] can be spliced into a continuous data block with a length of 3 MB, and 3 MB is within the second length range, and the spliced 3 MB continuous data block is stored in the cache.

[0115] In this way, the data block size used during retrieval (1.5MB) is different from the data slice size of local storage (3MB), which can achieve more flexible and efficient retrieval and cache management.

[0116] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a data caching device for a content distribution network. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0117] like Figure 4 As shown, the data caching device 400 for a content distribution network of this embodiment may include: a first sending unit 401, a first receiving unit 402, a second sending unit 403, and a splicing cache unit 404. The first sending unit 401 is configured to send a second data request to the source station based on the first length range and the first data request for obtaining the target data. The second data request is used to request at least one data block of the first data length in the target data; the first data length is within the first length range. The first receiving unit 402 is configured to receive at least one data block of the first data length from the source station. The second sending unit 403 is configured to send target data including at least one data block of the first data length to the device corresponding to the first data request. The splicing cache unit 404 is configured to perform data splicing based on at least one data block of the first data length, obtain the spliced target data, and store the spliced target data in the cache.

[0118] In this embodiment, in the data caching device 400 for content distribution network, the specific processing of the first sending unit 401, the first receiving unit 402, the second sending unit 403 and the splicing cache unit 404 and the technical effects thereof can be referred to respectively. Figure 2 The relevant descriptions of steps 201-204 in the corresponding embodiment are not repeated here.

[0119] In some optional implementations of this embodiment, the data caching device 400 for a content delivery network may further include a determination unit and a generation unit. The determination unit is configured to determine a first data length based on a first length range and a data interval of the target data; wherein the data interval of the target data includes one or more first data lengths. The generation unit is configured to generate a second data request based on the first data length and the data interval of the target data. The first sending unit 401 is further configured to send the second data request to the source station.

[0120] In some optional implementations of this embodiment, the determining unit is further configured to determine the first data length based on the first length range and the data interval of the target data in response to not finding all the target data from the cache.

[0121] In some optional implementations of this embodiment, the determining unit is further configured to: In response to the minimum value of the first length range being greater than or equal to the length of the data interval of the target data, determining that the first data length is equal to the minimum value of the first length range; or, In response to determining that the minimum value of the first length range is greater than or equal to the length of the data interval of the target data, and the third data request and the first data request come from the same device and are the next data request of the first data request, the first data length is determined based on the first length range, the data interval of the target data and the data interval of the target data in the third data request; wherein the interval between the arrival time of the third data request and the arrival time of the first data request is less than or equal to the time threshold.

[0122] In some optional implementations of this embodiment, the determining unit is further configured to: in response to the minimum value of the first length range being smaller than the length of the data interval of the target data, determine a first ratio of the length of the data interval of the target data to the minimum value of the first length range; In response to the first ratio being greater than a first threshold, determining a first data length based on the first ratio and a first length range; In response to the first ratio being less than or equal to a first threshold, it is determined that the first data length is equal to a minimum value of the first length range.

[0123] In some optional implementations of this embodiment, the generation unit is further configured to: in response to not finding the target data from the cache, split the data interval of the target data into one or more first data lengths based on the first data length; and generate a second data request based on the one or more first data lengths.

[0124] In some optional implementations of this embodiment, the generation unit is further configured to: in response to the second ratio of the length of the data interval of the target data to the first data length being not an integer and the first data length being less than the maximum value of the first length range, based on the first data length, split the data interval of the target data into one or more expanded first data lengths using a proportional expansion method; wherein the expanded first data length is less than or equal to the maximum value of the first length range.

[0125] In some optional implementations of this embodiment, the generation unit is further configured to: in response to only part of the target data being queried from the cache, split the interval corresponding to the data that has not been queried in the target data into one or more first data lengths based on the first data length; and generate a second data request based on the one or more first data lengths.

[0126] In some optional implementations of this embodiment, the splicing cache unit 404 is further configured to: based on the second length range, sequentially splice data blocks in each interval of the target data to obtain one or more first continuous data; wherein the length of the first continuous data is within the second length range, and the minimum value of the second length range is greater than the maximum value of the first length range; and store the one or more first continuous data in the cache.

[0127] In some optional implementations of this embodiment, the splicing cache unit 404 is further configured to: create index information corresponding to one or more first continuous data respectively; wherein the index information includes at least one of the following: a cache key, a storage location, and a data interval; and store the one or more first continuous data, and the first correspondence between the first continuous data and the index information in the cache.

[0128] In some optional implementations of this embodiment, the data caching device 400 for a content delivery network may further include a query unit and an acquisition unit. The query unit is configured to, in response to receiving a fourth data request for acquiring data blocks of the second data interval, query the second data interval from a correspondence database including the first correspondence. The acquisition unit is configured to, in response to finding the second data interval in the correspondence database, acquire data blocks of the second data interval from the cache.

[0129] This embodiment exists as an apparatus embodiment corresponding to the above-mentioned method embodiment. The technical effects brought about by the embodiments of the data caching device for a content distribution network provided by this embodiment can be referred to the corresponding relevant descriptions in the above-mentioned method embodiments respectively, and will not be repeated here.

[0130] According to an embodiment of the present disclosure, the present disclosure further provides a data caching system for a content distribution network, comprising: a cache node and a source station. The cache node is configured to: send a second data request to the source station based on a first length range and a first data request for obtaining target data; the second data request is used to request at least one data block of the first data length in the target data; the first data length is within the first length range; receive at least one data block of the first data length from the source station; send target data including at least one data block of the first data length to a device corresponding to the first data request; perform data splicing based on the at least one data block of the first data length to obtain the spliced target data, and store the spliced target data in the cache.

[0131] Optionally, the cache node can implement the method performed by the cache node described in any of the above embodiments. The source station can implement the method performed by the source station described in any of the above embodiments.

[0132] Optionally, the data cache system for a content delivery network may further include a client, which can implement the method described in any of the above embodiments and executed by the client or terminal device. The device corresponding to the first data request may be the client.

[0133] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the data caching method for a content distribution network described in any of the above embodiments when executing.

[0134] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium storing computer instructions, which are used to enable a computer to implement the data caching method for a content distribution network described in any of the above embodiments when executed.

[0135] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, which, when executed by a processor, can implement the data caching method for a content distribution network described in any of the above embodiments.

[0136] Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0137] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. Computing unit 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0138] Various components in device 500 are connected to I / O interface 505, including: an input unit 506, such as a keyboard, mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, optical disk, etc.; and a communication unit 509, such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0139] The computing unit 501 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the data caching method for a content delivery network. For example, in some embodiments, the data caching method for a content delivery network can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the data caching method for a content delivery network described above can be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the data caching method for a content delivery network in any other appropriate manner (for example, by means of firmware).

[0140] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0143] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0144] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0145] A computer system may include a client and a server. The client and server are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and virtual private server (VPS) services. The server may also be a server in a distributed system or a server integrated with blockchain.

[0146] According to the technical solution of the embodiment of the present disclosure, the data block size (first data length) used when returning to the source is different from the size of the data slice stored locally, which can achieve more efficient return to the source and cache management.

[0147] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0148] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A data caching method for a content distribution network, comprising: Sending a second data request to the source station based on the first length range and the first data request for obtaining the target data, wherein the second data request is used to request at least one data block of the target data having a first data length, and the first data length is within the first length range; receiving the at least one data block of the first data length from the source station; Sending target data including the at least one data block of the first data length to a device corresponding to the first data request; Data splicing is performed based on the at least one data block of the first data length to obtain spliced target data, and the spliced target data is stored in a cache.

2. The method according to claim 1, wherein The sending, to the source station, a second data request based on the first length range and the first data request for obtaining the target data includes: Determine the first data length based on the first length range and the data interval of the target data; wherein the data interval of the target data includes one or more first data lengths; generating the second data request based on the first data length and the data interval of the target data; Send the second data request to the source station.

3. The method according to claim 2, wherein: The determining the first data length based on the first length range and the data interval of the target data includes: In response to not finding all of the target data from the cache, the first data length is determined based on the first length range and a data interval of the target data.

4. The method according to claim 2 or 3, wherein: The determining the first data length based on the first length range and the data interval of the target data includes: In response to the minimum value of the first length range being greater than or equal to the length of the data interval of the target data, determining that the first data length is equal to the minimum value of the first length range; or, In response to determining that the minimum value of the first length range is greater than or equal to the length of the data interval of the target data, and that the third data request and the first data request come from the same device and are the next data request of the first data request, the first data length is determined based on the first length range, the data interval of the target data and the data interval of the target data in the third data request; wherein the interval between the arrival time of the third data request and the arrival time of the first data request is less than or equal to a time threshold.

5. The method according to claim 2 or 3, wherein: The determining the first data length based on the first length range and the data interval of the target data includes: In response to a minimum value of the first length range being smaller than a length of a data interval of the target data, determining a first ratio of a length of the data interval of the target data to a minimum value of the first length range; In response to the first ratio being greater than a first threshold, determining the first data length based on the first ratio and the first length range; In response to the first ratio being less than or equal to a first threshold, it is determined that the first data length is equal to a minimum value of the first length range.

6. The method according to claim 2 or 3, wherein: The generating the second data request based on the first data length and the data interval of the target data includes: In response to not finding the target data in the cache, splitting the data interval of the target data into one or more first data lengths based on the first data length; Based on the one or more first data lengths, the second data request is generated.

7. The method according to claim 6, wherein: The step of splitting the data interval of the target data into one or more first data lengths based on the first data length includes: In response to the fact that the second ratio of the length of the data interval of the target data to the first data length is not an integer, and the first data length is less than the maximum value of the first length range, based on the first data length, the data interval of the target data is split into one or more expanded first data lengths using a proportional expansion method; wherein the expanded first data length is less than or equal to the maximum value of the first length range.

8. The method according to claim 2 or 3, wherein: The generating the second data request based on the first data length and the data interval of the target data includes: In response to finding only part of the target data from the cache, splitting, based on the first data length, an interval corresponding to the unqueried data in the target data into one or more first data lengths; Based on the one or more first data lengths, the second data request is generated.

9. The method according to any one of claims 1 to 3, wherein The performing data splicing based on the at least one data block of the first data length to obtain spliced target data, and storing the spliced target data in a cache, includes: Based on the second length range, sequentially splicing data blocks in each interval of the target data to obtain one or more first continuous data; wherein the length of the first continuous data is within the second length range, and the minimum value of the second length range is greater than the maximum value of the first length range; The one or more first consecutive data are stored in the cache.

10. The method according to claim 9, wherein: The storing the one or more first continuous data into the cache comprises: Creating index information corresponding to the one or more first continuous data respectively; wherein the index information includes at least one of the following: a cache key, a storage location, and a data interval; The one or more first continuous data and a first correspondence between the first continuous data and the index information are stored in the cache.

11. The method according to claim 10, wherein: The method further comprises: In response to receiving a fourth data request for obtaining a data block of a second data interval, querying the second data interval from a correspondence relationship library including the first correspondence relationship; In response to querying the second data interval from the correspondence library, data blocks of the second data interval are obtained from the cache.

12. A data caching device for a content distribution network, comprising: a first sending unit configured to send a second data request to a source station based on the first length range and the first data request for obtaining target data, wherein the second data request is for requesting at least one data block of the target data having a first data length, and the first data length being within the first length range; a first receiving unit, configured to receive the at least one data block of the first data length from the source station; a second sending unit, configured to send target data including the at least one data block of the first data length to a device corresponding to the first data request; The splicing cache unit is configured to perform data splicing based on the at least one data block of the first data length to obtain spliced target data, and store the spliced target data in a cache.

13. A data caching system for a content distribution network, comprising: Cache nodes and origin servers; among them, The cache node is configured to: Sending a second data request to the source station based on the first length range and the first data request for obtaining target data, wherein the second data request is used to request at least one data block of the target data having a first data length, and the first data length is within the first length range; receiving the at least one data block of the first data length from the source station; Sending target data including the at least one data block of the first data length to a device corresponding to the first data request; Data splicing is performed based on the at least one data block of the first data length to obtain spliced target data, and the spliced target data is stored in a cache.

14. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to execute the method according to any one of claims 1 to 11.

16. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 11.

Images