Method and system for improving cache hit rate of edge CDN sink node

Abstract

The present application relates to the technical field of content delivery networks (CDNs), and provides a method and system for improving a cache hit rate of an edge CDN sink node. The method comprises: sink node cache servers each recording each URL request on the basis of a request sequence and saving same in a file; deploying an agent on each cache server to perform collection on the file, and collecting a compressed file every set time and uploading the compressed file to a central cache analysis service cluster; saving the number of access times of all URLs in a region, updating the access frequencies of the URLs in real time and sorting same, and screening for a most popular URL list; acquiring, from a database every set time, top N data records among data records that are sorted in a descending sequence on the basis of historical access times; using a CF to convert the top N data records into resource hotspot bitmap information, and issuing the resource hotspot bitmap information to each sink node cache server; and saving the resource hotspot bitmap information in a memory, and persisting the resource hotspot bitmap information in a magnetic disk.

Description

Method and system for improving cache hit rate of edge CDN sink node

[0001] Related applications

[0002] The present application claims priority to the Chinese patent application No. 202410450418.6, filed on April 15, 2024, and entitled "Method and system for improving cache hit rate of edge CDN sink node", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of content delivery network, and in particular to a method and system for improving cache hit rate of edge CDN sink node. BACKGROUND

[0004] The full name of CDN is Content Delivery Network, which is a content distribution network built on the network, with two key elements of content storage and distribution. The cache server is distributed to a location closer to the user, and the content of the source station is published to the node closest to the user edge, so that the user can obtain the required content nearby, and the response speed of user accessing website resources is improved.

[0005] The basic idea of CDN is to avoid as much as possible the bottlenecks and links on the Internet that may affect the speed and stability of data transmission, so that the content transmission is faster and more stable. The principle of CDN is to deploy a large number of distributed cache servers in the network. When a user requests a website, the global load balancing technology is used to direct the user's access to the cache server closest to the user, and the cache server proxies the source station to respond to the user's access request, so as to reduce the network delay and provide a smoother access experience for the user. Whether the content accessed by the user is in cache hit, that is, the cache hit rate of the cache server greatly affects the user's access experience.

[0006] At present, due to the limited storage space of the cache server of CDN, the cache server cluster is generally used in nodes to expand the storage space. By doing consistent hash on the request URL, the storage space of the cache servers in the cluster is aggregated into a large storage space. A cluster can deploy dozens of servers, and a single node can reach P-level storage space or above, so as to improve the cache hit rate of CDN cache server. Large-scale cache server clusters are generally deployed in Internet Data Center (IDC) machine rooms.

[0007] However, with the continuous enrichment of business scenarios and the continuous improvement of user experience, users have higher and higher requirements for low latency, for example: VR video, if the low latency requirement cannot be met, dizziness will occur when watching, which greatly affects the user experience, and the distance between the IDC room and the end user is not always the most perfect, in order to meet the user's demand, it is necessary to sink the cache server to a location closer to the end user, such as sinking to the operator's county-level room, or even further sinking to the location of the broadband access server (BRAS) and optical line terminal (OLT).

[0008] However, due to the limitations of sinking the room and the constraints of cost, the number of server clusters of the sinking node will be very limited, generally not more than 4, which will cause the single-node storage space to drop sharply, and then greatly reduce the hit rate, resulting in a large number of requests that cannot be hit in the cache server, and need to pull the content back to the source, losing the meaning of sinking.

[0009] A current optimization method is to optimize the hit rate through global HTTP 302, that is, in the case that the cache server cannot hit, instead of back to the source, the central 302 cluster is queried for the distribution of the request URL in other nodes; if it is queried, the node corresponding to the request of the client is responded through HTTP 302 redirection, and the client initiates a request to the redirected node again. This optimization method can improve the hit rate, but there are many problems:

[0010] First, this scheme requires that the client must support HTTP 302;

[0011] Second, HTTP 302 increases the round-trip time RTT of redirection and the round-trip time RTT of querying the 302 cluster, increasing the network delay;

[0012] Third, the redirected node may be far from the user terminal, which violates the meaning of node sinking;

[0013] Fourth, it is highly dependent on the stability and processing performance of the central 302 cluster.

[0014] Therefore, the current optimization of hit rate through global HTTP 302 is not the optimal solution for the node sinking scenario. SUMMARY

[0015] Therefore, the present application aims at the problems of the conventional technology when the node is sunken, and provides a method and system for improving cache hit rate of a sunken node of an edge CDN. The user access resource requests of multiple nodes in a region are combined and counted, and the hot resource distribution in the entire region including other sunken nodes can be known in a single cache server of the sunken node. The hottest resource cache list is calculated by an offline algorithm, and the resource cache list is transmitted to the cache server of the sunken node at a small cost. When the cache server of the sunken node has a new access request which is not hit, the resource list with high heat is searched in the cache server of each sunken node, so that only the resource with high heat is cached, and the content with higher heat can be cached in the limited storage space of the sunken node. Therefore, more hot resources are cached, and the cache hit rate is improved.

[0016] The present application provides a method for improving cache hit rate of a sunken node of an edge CDN, which comprises the following steps:

[0017] S1, recording each URL request in order by a cache server of a sunken node, and saving in an independent file;

[0018] S2, deploying an independent agent on the cache server to collect the file, collecting a compressed file every set time (preferably 5 minutes), and uploading to a centralized central cache analysis service cluster;

[0019] S3, saving global URL access times in a region by the central cache analysis service cluster, updating the access frequency of the URL in real time, and sorting the URL according to the access frequency to screen out the hottest URL list;

[0020] S4, obtaining the TOP N data records sorted from high to low according to the historical access times from the database at a set time interval (preferably 5 minutes), wherein the size of N represents how many hot resources the cache server of the sunken node needs to cache, and the size of N is calculated by dividing the storage space of the cache server by the average file size; converting the collection information of the MD5 fingerprint information corresponding to the N URL into resource hotspot bitmap information by using the data structure of a cuckoo filter, and delivering the resource hotspot bitmap information to each cache server of the sunken node;

[0021] The present application saves space, converts the collection information of the N records into resource hotspot bitmap information by using the data structure of a cuckoo filter, and sends the resource hotspot bitmap information to the cache server of the sunken node.

[0022] CF (Cuckoo Filter) is a hash table-based data structure used to determine whether an element exists in a set. The basic principle of CF is to map elements to different locations in a hash array through multiple hash functions. If all hash functions point to the same location, it is considered that the element exists in the set.

[0023] The present application uses the Cuckoo filter algorithm to distribute and search for hot resource distribution. The Cuckoo filter determines whether an element exists in a set through a bitmap. This scheme has two advantages:

[0024] First, the bitmap greatly compresses the storage space, saving the memory space of the cache server and improving the transmission efficiency of the central cache analysis service cluster in distributing cache resource hotness distribution.

[0025] Second, it improves the query efficiency and quickly locates whether the requested URL is a hot resource, without reducing the concurrent processing capacity and throughput of the cache server.

[0026] In the existing conventional situation, cache hotness statistics are usually based on single machines as the granularity, and cache replacement is performed through LRU algorithms. The first request resource, even if it is a cold resource, needs to be cached until it is replaced by a hotter resource. In this way, a large number of long-tail cold resources will occupy storage space before being replaced, which greatly damages the hit rate. The present application combines the resource request of user access in multiple nodes in a region to perform statistical analysis, calculates the hottest resource cache list through offline algorithms, and transmits the resource cache list to the cache server of the sink node at a small cost. When the cache server of the sink node has a new access request that is not hit, the resource list with high heat is found in each cache server of the sink node to ensure that only high-heat resources are cached, thereby improving the cache hit rate.

[0027] S5, after receiving the resource hotspot bitmap information through the sink node cache server, the resource hotspot bitmap information is saved in the memory, and is also persisted to the disk to prevent the loss of resource hotspot bitmap information caused by cache server restart.

[0028] In some embodiments, the method of recording each URL request in a separate file in the S1 step includes: performing MD5 operation on each URL request to generate a 128-bit MD5 fingerprint saved in the file. Whether the URL is involved in the MD5 operation is determined by the cache configuration. Each record in the file includes request timestamp + MD5 fingerprint information. The record file is cut every 5 minutes and compressed for saving. The file name is determined by the corresponding time period.

[0029] In order to save space and transmission time, MD5 operation is performed on each URL request to generate 128-bit MD5 fingerprints for storage; since the number of requests can be large, the application cuts and compresses the record file every 5 minutes.

[0030] In some embodiments, the method of updating the access frequency of the URL in real time in the S3 step comprises:

[0031] The central cache analysis service uses a database to save the access frequency of the URL, and the database table includes three fields, namely MD5 fingerprint information, last access time and access times, with the MD5 fingerprint information as the index.

[0032] In some embodiments, the method of sorting by the access frequency of the URL in the S3 step comprises:

[0033] The central cache analysis cluster collects the uploaded compressed files from each sink node cache server, decompresses the compressed files, reads the MD5 fingerprint information and corresponding time stamp in sequence, updates the database to increase the historical access times of the corresponding MD5 fingerprint information, judges the time period from the last access time to the current time stamp while updating the access times, decays the original access times with large time span to ensure that the access frequency is reflected in the most recent time period, and adopts a strategy that the longer the Key is not accessed, the greater the decay degree.

[0034] In some embodiments, the algorithm for decaying the access times comprises:

[0035] The current time stamp is subtracted from the last access time stamp to calculate the time length T in minutes, the decay factor m and the decay step n are set, the decay step n is obtained by dividing the time length T by the decay factor m, if the historical access times > the decay step n, the latest decayed historical access times are obtained by subtracting the decay step n from the historical access times, and then 1 is added to update the historical access times, if the historical access times ≤ the decay step n, the historical access times are directly decayed to 0, and then 1 is added to update the historical access times, the decay factor m is adjustable, the smaller the value of m, the more intense the decay of the historical access times, so as to ensure that the historical access times can more effectively represent the recent access frequency of the resource.

[0036] The application attenuates the historical access times of resources according to time by a resource hotness attenuation algorithm in the central cache analysis service cluster, and the longer the time from the last access time of the resource to the current time, the more intense the attenuation of the statistical historical access times of the resource, so that the resources with high access frequency in the recent period are finally defined as hot resources, instead of the resources with high access frequency in a long period but low access frequency in the recent period.

[0037] In some embodiments, the above method further comprises, after the S5 step:

[0038] When the URL access request of the resource hotspot bitmap information enters the cache server, the MD5 fingerprint information is generated by using the MD5 hash algorithm, and it is checked by the cuckoo filter algorithm whether the MD5 fingerprint information exists in the resource hotspot bitmap information. If it exists, it proves that the resource hotspot bitmap information is a hotspot resource in the region, and the resource hotspot bitmap information is cached. If it does not exist, it is considered that the resource hotspot bitmap information has not reached the standard in the region, and it is not cached, but only the MD5 fingerprint information is recorded in the file.

[0039] The application aggregates and statistically analyzes the access resource URLs of multiple sink nodes according to regions, and statistically analyzes the hotspot resource distribution of the region, which breaks through the following bottleneck of the conventional cache server that can only statistically analyze the access requests of a single machine. In the case of single machine granularity, the requests that are not hit for the first time need to be cached, and the same URL requests are used to heat them up. For the resources with insufficient hotness, they need to be removed by writing a ring on the disk, which will occupy storage space during this period. According to the application, the resource hotness in the region is statistically analyzed, and in the case of first-time request for a resource, it is determined whether the resource is a hot resource by querying the resource hotness bitmap. Only the hot resources are cached. According to the application, in the case of limited storage space of the sink node, the resources with low access hotness in the region are discarded, and more hot resources are cached, so that the cache hit rate is improved.

[0040] In some embodiments, the resource hotspot bitmap information is periodically updated by the central cache analysis service cluster.

[0041] The application also provides a system for improving the cache hit rate of the edge CDN sink node, which executes the method for improving the cache hit rate of the edge CDN sink node as described above, and comprises:

[0042] The URL request recording module is used for recording each URL request by the sink node cache server according to the request order, and saving in an independent file.

[0043] File collection compression upload module: used for deploying independent agent on the cache server to collect the file, collecting the compressed file every set time interval (preferably 5 minutes) and uploading to the centralized central cache analysis service cluster;

[0044] URL access frequency updating and sorting module: used for saving the global URL access times in the region by the central cache analysis service cluster, updating the URL access frequency in real time, sorting by the URL access frequency and screening out the hottest URL list;

[0045] Conversion resource hotspot bitmap information module: used for obtaining the TOP N data records sorted from high to low according to the historical access times from the database according to the set time interval (preferably 5 minutes), the size of N representing how many hotspot resources the sink node cache server needs to cache, the size of N being calculated by dividing the storage space of the cache server by the average file size; converting the set information of the MD5 fingerprint information corresponding to the N URL into resource hotspot bitmap information using the data structure of the cuckoo filter and delivering the resource hotspot bitmap information to each sink node cache server;

[0046] Resource hotspot bitmap information saving and persisting module: used for saving the resource hotspot bitmap information in the memory and persisting to the disk after the sink node cache server receives the resource hotspot bitmap information, so as to prevent the resource hotspot bitmap information from being lost due to the restart of the cache server.

[0047] The application further provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the method for improving the cache hit rate of the edge CDN sink node as described above.

[0048] The application further provides a computer device including a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method for improving the cache hit rate of the edge CDN sink node as described above when executing the program.

[0049] Compared with the prior art, the application has the following beneficial effects:

[0050] The method and system for improving cache hit rate of edge CDN sink node provided in the application can know the hot resource distribution in the whole region including other sink nodes in the single cache server of the sink node by combining and counting the user access resource requests of multiple nodes in the region, and calculate the hottest resource cache list by offline algorithm, and deliver the resource cache list to the cache server of the sink node at a small cost, and when the cache server of the sink node has a new non-hit access request, find the high-heat resource list in the cache server of each sink node, so that only the high-heat resource is cached, and the content with higher heat can be cached in the limited storage space of the sink node, so that more hot resources are cached, the cache hit rate is effectively improved, and the user access quality and use experience are improved. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0052] Fig. 1 is a CDN network interaction scenario diagram according to an embodiment of the application.

[0053] Fig. 2 is a cache server hot cache flowchart according to an embodiment of the application.

[0054] Fig. 3 is a resource hot spot analysis service flowchart according to an embodiment of the application.

[0055] Fig. 4 is a method flowchart for improving cache hit rate of edge CDN sink node according to an embodiment of the application.

[0056] Fig. 5 is a schematic diagram of the computer device according to an embodiment of the application. DETAILED DESCRIPTION

[0057] The exemplary embodiments will be described in detail herein below with reference to the drawings. The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, and uses to which it is subject. Rather, the exemplary embodiments are provided so that one of ordinary skill in the art will understand various aspects of at least one embodiment of the present application and will be able to utilize such in appropriate situations. Those skilled in the art will readily recognize a wide variety of

[0058] The terminology used in the disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0059] It is to be understood that, although the terms first, second, third, etc. can be used herein to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one piece of information from another. For example, a first information can also be termed a second information, and, similarly, a second information can also be termed a first information, without departing from the scope of the present disclosure. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "in response to determining" or "in response to ascertaining".

[0060] The embodiments of the present application are further described below.

[0061] The embodiments of the present application provide a method and system for improving cache hit rate of edge CDN sink node, to solve the problem that cache replacement of cache hotness statistics of conventional cache server occupies large storage space, and in the case of limited storage space, the hit rate is greatly reduced.

[0062] Method embodiment

[0063] The embodiments of the present application provide a method for improving cache hit rate of edge CDN sink node, as shown in FIG. 4, which includes the following steps S1 to S5.

[0064] S1: The sink node cache server records each URL request in order according to the request and saves it in a separate file.

[0065] The method of recording each URL request in a separate file includes: performing MD5 operation on each URL request to generate a 128-bit MD5 fingerprint saved in the file, and determining whether the URL participates in the MD5 operation with parameters according to the cache configuration. Each record in the file includes a request timestamp + MD5 fingerprint information, and the record file is cut every 5 minutes and compressed for saving. The file name is determined by the corresponding time period.

[0066] In order to save space and transmission time, the embodiments of the present application perform MD5 operation on each URL request to generate a 128-bit MD5 fingerprint saved in the file. Since the number of requests can be large, the record file is cut every 5 minutes and compressed for saving.

[0067] S2: Deploy independent agent on the cache server to collect the file, collect compressed file every 5 minutes, and upload to centralized central cache analysis service cluster.

[0068] S3: The central cache analysis service cluster saves the global URL access times in the region, updates the URL access frequency in real time, sorts the URL access frequency, and screens out the hottest URL list.

[0069] In one embodiment, the method of updating the URL access frequency in real time comprises:

[0070] The central cache analysis service uses a database to save the URL access frequency, and the database table includes three fields, namely MD5 fingerprint information, last access time, and access times, with MD5 fingerprint information as the index.

[0071] In one embodiment, the method of sorting by URL access frequency comprises:

[0072] The central cache analysis cluster collects uploaded compressed files from each sink node cache server, decompresses the compressed files, reads MD5 fingerprint information and corresponding time stamp in sequence, updates the database to increase the historical access times of the corresponding MD5 fingerprint information, judges the time period from the last access time to the current time stamp while updating the access times, decays the original access times with large time span, ensures that the access frequency can be reflected in the recent time period as much as possible, and adopts the strategy that the longer the Key is not accessed, the greater the decay degree.

[0073] In one embodiment, the algorithm for decaying the access times is:

[0074] The current time stamp is subtracted from the last access time stamp to calculate the time length T in minutes, the decay factor m and the decay step n are set, the decay step n value is obtained by dividing the time length T by the decay factor m; if the historical access times > decay step n, the latest decayed historical access times are obtained by subtracting the decay step n from the historical access times, and then 1 is added to update the historical access times; if the historical access times ≤ decay step n, the historical access times are directly decayed to 0, and then 1 is added to update the historical access times; the decay factor m is adjustable, the smaller the value of m, the more intense the decay of the historical access times. This ensures that the historical access times can more effectively represent the recent access frequency of the resource.

[0075] S4: Obtain the top N data records ranked from high to low according to the historical access times from the database at a time interval of 5 minutes, the size of N represents how many hot resources the sink node cache server needs to cache, the size of N is calculated by dividing the storage space of the cache server by the average file size; convert the set information of the MD5 fingerprint information corresponding to the N URL into resource hotspot bitmap information using the data structure of the cuckoo filter, and deliver the resource hotspot bitmap information to each sink node cache server (see FIG. 3).

[0076] The embodiment saves space, converts the set information of the N records into resource hotspot bitmap information using the data structure of the cuckoo filter, and sends the resource hotspot bitmap information to the sink node cache server.

[0077] In the existing conventional case, the cache heat statistics are usually taken as a single machine as a granularity, and the cache replacement is performed through an LRU algorithm. The first requested resource needs to be cached even if it is a cold resource, and is replaced by a hotter resource later. Thus, a large number of cold resources in the long tail will occupy the storage space before being replaced, which greatly damages the hit rate in the case of limited storage space. The embodiment combines the user access resource requests of multiple nodes in the region to perform statistical calculation, calculates the hottest resource cache list through an offline algorithm, and delivers the resource cache list to the sink node cache server at a small cost. When the sink node cache server has a new access request that is not hit, the resource list with high heat is searched in each sink node cache server, so that only the resource with high heat is cached, thereby improving the cache hit rate.

[0078] S5: After the sink node cache server receives the resource hotspot bitmap information, the resource hotspot bitmap information is saved in the memory and is persisted to the disk to prevent the loss of the resource hotspot bitmap information due to the restart of the cache server.

[0079] When the URL access request of the resource hotspot bitmap information enters the cache server, the MD5 fingerprint information is generated using the MD5 hash algorithm, and it is checked whether the MD5 fingerprint information exists in the resource hotspot bitmap information through the cuckoo filter algorithm (see FIG. 2). If it exists, it proves that the resource hotspot bitmap information is a hot resource in the region, and the resource hotspot bitmap information is cached. If it does not exist, it is considered that the resource hotspot bitmap information does not meet the heat in the region, and is not cached, but only the MD5 fingerprint information is recorded in the file.

[0080] The resource hotspot bitmap information is periodically updated by the central cache analysis service cluster.

[0081] FIG. 1 shows the CDN network interaction scenario of the embodiment.

[0082] System embodiments

[0083] The embodiments of the present application also provide a system for improving cache hit rate of a sink node of an edge CDN, which executes the method for improving cache hit rate of a sink node of an edge CDN as described above, and the system comprises a URL request recording module, a file collection and compression uploading module, a URL access frequency updating and sorting module, a conversion resource hotspot bitmap information module, and a resource hotspot bitmap information saving and persisting module.

[0084] The URL request recording module is used for recording each URL request in a request order by a sink node cache server and saving in an independent file.

[0085] The file collection and compression uploading module is used for collecting the file by deploying an independent agent on the cache server, collecting the compressed file every 5 minutes, and uploading to a centralized central cache analysis service cluster.

[0086] The URL access frequency updating and sorting module is used for saving the global URL access times in a region by the central cache analysis service cluster, updating the URL access frequency in real time, sorting by the URL access frequency, and screening the hottest URL list.

[0087] The conversion resource hotspot bitmap information module is used for obtaining the TOP N data records sorted from high to low according to the historical access times from the database according to the time interval of 5 minutes, the size of N representing how many hotspot resources the sink node cache server needs to cache, the size of N being calculated by dividing the storage space of the cache server by the average file size; converting the set information of the MD5 fingerprint information corresponding to the N URL into resource hotspot bitmap information using the data structure of a cuckoo filter, and delivering the resource hotspot bitmap information to each sink node cache server.

[0088] The resource hotspot bitmap information saving and persisting module is used for saving the resource hotspot bitmap information in the memory and persisting to the disk after the sink node cache server receives the resource hotspot bitmap information, so as to prevent the resource hotspot bitmap information from being lost due to the restart of the cache server.

[0089] The embodiment of the present application further provides a computer device, and FIG. 5 is a structural schematic diagram of the computer device according to the embodiment of the present application; referring to FIG. 5, the computer device comprises an input system 23, an output system 24, a memory 22 and a processor 21; the memory 22 is used for storing one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the method for improving cache hit rate of an edge CDN sink node provided in the above embodiment; the input system 23, the output system 24, the memory 22 and the processor 21 are connected through a bus or other manners, and the connection through the bus is taken as an example in FIG. 5.

[0090] The memory 22 is a readable and writable storage medium of a computer device, and is used for storing software programs, computer executable programs, and program instructions corresponding to the method for improving cache hit rate of an edge CDN sink node according to the embodiment of the present application; the memory 22 can mainly comprise a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application program required by a function; the data storage area can store data created according to the use of the device, etc.; in addition, the memory 22 can comprise a high-speed random access memory, and can further comprise a nonvolatile memory, for example, at least one magnetic disk storage device, a flash memory device or other nonvolatile solid-state storage device; in some examples, the memory 22 can further comprise a memory remotely arranged relative to the processor 21, and the remote memory can be connected to the device through a network. Examples of the network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network and a combination thereof.

[0091] The input system 23 is used for receiving input digital or character information, and generating key signal input related to user settings and function control of the device; the output system 24 can comprise a display device such as a display screen.

[0092] The processor 21 executes various function applications and data processing of the device by running the software programs, instructions and modules stored in the memory 22, that is, implements the method for improving cache hit rate of an edge CDN sink node.

[0093] The computer device provided above can be used for executing the method for improving cache hit rate of an edge CDN sink node provided in the above embodiment, and has corresponding functions and advantages.

[0094] The embodiments of the present application further provide a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to perform the method for improving cache hit rate of a sink node of an edge CDN provided by the above embodiments. The storage medium is any of various types of memory devices or storage devices, and includes: installation media such as CD-ROM, floppy disks or tape systems; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements; and the like. The storage medium can also include other types of memory or combinations thereof. In addition, the storage medium can be located in the first computer system in which the program is executed, or can be located in a second computer system that is connected to the first computer system through a network (such as the Internet), and the second computer system can provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (e.g., in different computer systems connected through a network). The storage medium can store program instructions (e.g., embodied as a computer program) that can be executed by one or more processors.

[0095] Of course, the storage medium containing computer executable instructions provided by the embodiments of the present application is not limited to the method for improving cache hit rate of a sink node of an edge CDN as described in the above embodiments, and can also perform related operations in the method for improving cache hit rate of a sink node of an edge CDN provided by any of the embodiments of the present application.

[0096] The embodiments of the present application have been described in detail with reference to the accompanying drawings and the embodiments of the present application. However, those skilled in the art should understand that the above embodiments are only preferred embodiments of the present application, and the detailed description is only to help the reader better understand the spirit of the present application, and is not a limitation on the protection scope of the present application. On the contrary, any improvement or modification made on the basis of the spirit of the present application should fall within the protection scope of the present application.

[0097] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. Any changes or replacements easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for improving the cache hit rate of edge CDN sinking nodes, comprising: S1. The sinking node cache server records each URL request in the order of requests and saves them in a separate file; S2. Deploy an independent agent on the cache server to collect the files. Collect compressed files once at a set time interval and upload them to a centralized central cache analysis service cluster. S3. The central cache analysis service cluster stores the global URL access count within the region, updates the URL access frequency in real time, sorts the URLs by access frequency, and filters out the list of the most popular URLs. S4. Retrieve the TOP N data records from the database according to the set time interval, sorted from highest to lowest based on the number of historical accesses. The size of N represents how many hot resources the sinking node cache server needs to cache. The size of N is calculated by dividing the storage space of the cache server by the average file size. Convert the set of MD5 fingerprint information corresponding to the N URLs obtained into resource hotspot bitmap information using the data structure of the Cuckoo Filter, and send it to each sinking node cache server. S5. After the sinking node cache server receives the resource hotspot bitmap information, it saves the resource hotspot bitmap information in memory and persists it to disk to prevent the resource hotspot bitmap information from being lost due to a restart of the cache server.

2. The method according to claim 1, wherein the method of recording each URL request and storing it in a separate file in step S1 comprises: Perform an MD5 operation on each URL request to generate a 128-bit MD5 fingerprint, which is stored in a file. The cache configuration determines whether the URL includes parameters for the MD5 operation. Each record in the file includes the request timestamp and the MD5 fingerprint information. The record file is split and compressed every 5 minutes, and the filename is determined by the corresponding time period.

3. The method according to claim 2, wherein the method for real-time updating the access frequency of the URL in step S3 includes: The central cache analysis service uses a database to store the access frequency of URLs. The database table contains three fields: MD5 fingerprint information, last access time, and access count, with MD5 fingerprint information as the index.

4. The method according to claim 3, wherein the method for sorting by URL access frequency in step S3 includes: The central cache analysis cluster collects the uploaded compressed files from the cache servers of each sinking node, decompresses the compressed files, and sequentially reads the MD5 fingerprint information and the corresponding timestamp. It then updates the database with the historical access count of the corresponding MD5 fingerprint information. While updating the access count, it determines the time period between the last access time and the current timestamp, and attenuates the access counts with large time spans to ensure that the access frequency is reflected in the most recent time period. The algorithm for attenuating the access count adopts the strategy that the longer the key has not been accessed, the greater the attenuation.

5. The method according to claim 4, wherein the algorithm for attenuating the number of accesses is: The duration T in minutes is calculated by subtracting the last access timestamp from the current timestamp. A decay factor m and a decay step size n are set. The decay step size n is obtained by dividing the duration T by the decay factor m. If the number of historical accesses is greater than the decay step size n, the latest decayed number of historical accesses is obtained by subtracting the decay step size n from the number of historical accesses, and then incremented by 1 to update the number of historical accesses. If the number of historical accesses is less than or equal to the decay step size n, the number of historical accesses is directly decayed to 0, and then incremented by 1 to update the number of historical accesses. The decay factor m is adjustable; the smaller the value of m, the more drastic the decay of the number of historical accesses.

6. The method according to claim 5, wherein after step S5, the method further comprises: When a URL access request for resource hotspot bitmap information enters the cache server, an MD5 hash algorithm is used to generate MD5 fingerprint information. The Cuckoo Filter algorithm is then used to check whether the MD5 fingerprint information exists in the resource hotspot bitmap information. If it exists, it proves that the resource hotspot bitmap information is a hot resource in the region, and the resource hotspot bitmap information is cached. If it does not exist, it is considered that the resource hotspot bitmap information has not yet reached the required popularity in the region, and it is not cached. Only the MD5 fingerprint information is recorded to a file.

7. The method according to claim 6, wherein the resource hotspot bitmap information is periodically updated and synchronously distributed by the central cache analysis service cluster.

8. A system for improving the cache hit rate of edge CDN sinking nodes, comprising: The URL request recording module is used by the sinking node caching server to record each URL request in the order of requests and save it in a separate file; The file collection, compression, and upload module is used to deploy an independent agent on the cache server to collect the files. It collects compressed files at set time intervals and uploads them to a centralized central cache analysis service cluster. The URL access frequency update and sorting module is used to update the URL access frequency in real time by storing the global URL access count in the region by the central cache analysis service cluster, and sort the URLs by access frequency to filter out the list of the most popular URLs. The resource hotspot bitmap information conversion module is used to retrieve the TOP N data records from the database according to the historical access count, sorted from highest to lowest, at set time intervals. The size of N represents how many hot resources the sinking node cache server needs to cache. The size of N is calculated by dividing the storage space of the cache server by the average file size. The module converts the set of MD5 fingerprint information corresponding to the N URLs obtained into resource hotspot bitmap information using the data structure of the Cuckoo Filter and sends it to each sinking node cache server. The resource hotspot bitmap information persistence module is used to save the resource hotspot bitmap information in memory after the sinking node cache server receives the resource hotspot bitmap information, and at the same time persist it to disk to prevent the loss of resource hotspot bitmap information due to the restart of the cache server.

9. A computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method of any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any one of claims 1-7.

Images