A method and system for improving the cache hit rate of edge CDN sinking nodes

Abstract

The present invention provides a method and system for improving the cache hit rate of edge CDN sinking nodes, belonging to the technical field of content delivery networks. The method includes: the sinking node cache server records each URL in order of requests and saves it in a file; the cache server deploys an agent to collect the file, and uploads the compressed file to the central cache analysis service cluster at regular intervals; the global URL access times within the saved area are saved, the access frequency of the URL is updated in real time and sorted, and the hottest URL list is screened out; at regular intervals, the TOP N data records sorted from high to low according to historical access times are obtained from the database; it is converted into resource hot spot bitmap information using CF and sent to each sinking node cache server; the resource hot spot bitmap information is saved in the memory and persisted to the disk at the same time. The present invention can cache more popular content within the limited storage space of the sinking nodes, effectively improving the cache hit rate.

Description

Technical Field

[0001] The present invention relates to the technical field of content delivery networks, and in particular, to a method and system for improving the cache hit rate of edge CDN sinking nodes. Background Art

[0002] The full name of CDN is Content Delivery Network. CDN is a content delivery network built on the network, which has two key elements: content storage and distribution. The cache servers are distributed closer to users, and the content of the source station is published to the nodes closest to the edge of the users, so that users can obtain the required content nearby and improve the response speed of users accessing website resources.

[0003] The basic idea of CDN is to avoid as much as possible the bottlenecks and links that may affect the data transmission speed and stability on the Internet, so that the content can be transmitted faster and more stably. The principle of CDN is to deploy a large number of distributed cache servers in the network. When a user requests a website, the global load balancing technology is used to direct the user's access to the cache server closest to the user. The cache server proxies the source station to respond to the user's access request, so as to reduce the network latency and provide a smoother access experience for users. Whether the content accessed by the user is cached, that is, the cache hit rate of the cache server, greatly affects the user's access experience.

[0004] At present, due to the limited single-machine storage space of CDN cache servers, generally, a cache server cluster method is adopted at the node to expand the storage space. By using the method of consistent hashing for the requested URL, the storage spaces of the cache servers in the cluster are aggregated into a large storage space. A cluster can deploy dozens of servers, and the single-node storage space can reach more than P level to improve the cache hit rate of the CDN cache server. Large-scale cache server clusters are generally deployed in IDC (Internet Data Center) computer rooms.

[0005] However, with the continuous enrichment of business scenarios and the continuous improvement of user experience, users have higher and higher requirements for low latency. For example, for VR videos, if the low latency requirement cannot be met, dizziness will occur during viewing, which greatly affects the user experience. Moreover, the distance between the IDC computer room and the end user is often not the most perfect. To meet the user's needs, it is necessary to sink the cache server to a position closer to the end user, such as sinking to the operator's district and county-level computer rooms, or even further sinking to the positions of bras (broadband access servers) and OLTs (optical line terminals).

[0006] However, due to the limitations of the sinking data center and cost constraints, the number of server clusters in the sinking node is very limited, generally no more than 4. This will cause a sharp drop in the storage space of a single node, and thus greatly reduce the hit rate, causing a large number of requests to fail to hit the cache server and need to go back to the source to pull content, which defeats the purpose of sinking.

[0007] One current optimization method is to improve the hit rate through a global HTTP 302 response. This means that if a cache server fails to hit the source, the server does not return to the source, but instead queries the central 302 cluster for the distribution of the requested URL on other nodes. If a response is found, the client receives an HTTP 302 redirect response to the node corresponding to the request, and the client then makes another request to the redirected node. This optimization method can improve the hit rate, but it has many problems:

[0008] First, the solution requires the client to support HTTP 302;

[0009] Second, HTTP 302 increases the round-trip delay (RTT) of redirection and the round-trip delay (RTT) of querying the 302 cluster, increasing network latency.

[0010] Third, the redirected node may be far away from the user terminal, which defeats the purpose of node sinking;

[0011] Fourth, it relies heavily on the stability and processing performance of the central 302 cluster.

[0012] Therefore, optimizing the hit rate through global HTTP 302 is not the optimal solution for node sinking scenarios at this stage. Summary of the Invention

[0013] In view of this, the purpose of the present invention is to address the problems faced by the existing technology when the node is sunk, and propose a method and system for improving the cache hit rate of the edge CDN sinking node. By merging and counting the user access resource requests of multiple nodes in the area, the single cache server of the sinking node can know the distribution of hot resources in the entire area including other sinking nodes, calculate the hottest resource cache list through the offline algorithm, and pass the resource cache list to the cache server of the sinking node at a lower cost. When the cache server of the sinking node has a new missed access request, the cache server of each sinking node searches for the high-hot resource list to ensure that only high-hot resources are cached, and higher-hot content can be cached in the limited storage space of the sinking node; thereby caching more hot resources and improving the cache hit rate.

[0014] The present invention provides a method for improving the cache hit rate of edge CDN sink nodes, comprising the following steps:

[0015] S1. The sinking node cache server records each URL request in the order of requests and saves it in an independent file.

[0016] S2. Deploy an independent proxy agent on the cache server to collect the file. The compressed file is collected every set time (preferably 5 minutes) and uploaded to the centralized central cache analysis service cluster.

[0017] S3. The central cache analysis service cluster saves the global URL access times within the region, updates the access frequency of the URL in real time, sorts by the access frequency of the URL, and filters out the hottest URL list.

[0018] S4. Obtain the TOP N data records sorted from high to low according to the historical access times from the database at a set time interval (preferably 5 minutes). The size of N represents how many hot resources the sinking node cache server needs to cache. The size of N is calculated by dividing the storage space of the cache server by the average file size. The set information of the MD5 fingerprint information corresponding to the N URLs is converted into resource hot bitmap information using the data structure of the cuckoo filter and sent to each sinking node cache server.

[0019] In order to save space, the present invention uses the data structure of the cuckoo filter to convert the set information of N records into resource hot bitmap information and send it to the sinking node cache server.

[0020] CF (Cuckoo Filter) is a data structure based on a hash table used to determine whether an element exists in a set. The basic principle of CF is to map an element to different positions in the hash array through multiple hash functions. If all hash functions point to the same position, then it is considered that the element exists in the set.

[0021] The present invention uses the cuckoo filtering algorithm to distribute and search for the distribution of hot resources. The cuckoo filter determines whether an element in the set exists through a bitmap. There are two advantages to adopting this solution:

[0022] First, the storage space is greatly compressed through the bitmap method, saving the memory space of the cache server and improving the transmission efficiency of the central cache analysis service cluster to distribute the cache resource heat distribution.

[0023] Second, the query efficiency is improved. For the requested URL, it can be quickly located whether it is a hot resource, and the concurrent processing ability and throughput of the cache server will not be reduced.

[0024] Under the existing conventional circumstances, cache popularity statistics are all based on a single machine as the granularity, and cache replacement is performed through algorithms such as LRU. Even if the resources requested for the first time are cold resources, they need to be cached until they are replaced by hotter resources later. In this way, a large number of long-tail cold resources will occupy storage space before being replaced, which will cause great damage to the hit rate in the case of limited storage space. The present invention merges and counts the resource access requests of multiple nodes in the region, calculates the hottest resource cache list through an offline algorithm, and transfers the resource cache list to the cache server of the sinking node at a relatively low cost. When the cache server of the sinking node has a new unhit access request, by looking up the resource list with high popularity in each cache server of the sinking node, it is ensured that only high-popularity resources are cached, thereby improving the cache hit rate.

[0025] S5. After the cache server of the sinking node receives the resource hot bitmap information, the resource hot bitmap information is saved in the memory and persisted to the disk at the same time to prevent the loss of the resource hot bitmap information due to the restart of the cache server.

[0026] Further, the method of recording each URL request in step S1 and saving it in an independent file includes: performing MD5 operation on each URL request to generate a 128-bit MD5 fingerprint and saving it in the file. Whether the URL participates in the MD5 operation with parameters is determined by the cache configuration. Each record in the file includes the request timestamp + MD5 fingerprint information. The record file is cut and compressed every 5 minutes and saved, and the file name is determined by the time of the corresponding time period.

[0027] In order to save space and transmission time, the present invention performs MD5 operation on each URL request to generate a 128-bit MD5 fingerprint for saving; since the number of requests may be relatively large, resulting in a relatively large file, the present invention cuts and compresses the record file every 5 minutes and saves it.

[0028] Further, the method of updating the access frequency of the URL in step S3 includes:

[0029] The central cache analysis service uses a database to save the access frequency of the URL. The database table contains 3 fields, namely MD5 fingerprint information, the last access time, and the number of accesses, with the MD5 fingerprint information as the index.

[0030] Further, the method of sorting by the access frequency of the URL in step S3 includes:

[0031] The central cache analysis cluster receives the uploaded compressed files from each sinking node cache server, decompresses the compressed files and reads the MD5 fingerprint information and the corresponding timestamp in sequence, and increases the historical access count of the corresponding MD5 fingerprint information in the database update. While updating the access count, it determines the time period from the last access time to the current timestamp, and attenuates the original access count with a large time span to ensure that the access frequency can be reflected in the most recent time period as much as possible. The strategy adopted by the algorithm for attenuating the access count is that the longer the key has not been accessed, the greater the degree of attenuation.

[0032] Furthermore, the algorithm for decaying the number of visits is:

[0033] Subtract the last access timestamp from the current timestamp to calculate the duration T in minutes. Set the decay factor m and decay step n. Divide the duration T by the decay factor m to obtain the decay step n. If the number of historical visits exceeds the decay step n, subtract the decay step n from the number of historical visits to obtain the latest decayed historical visit count, and then add 1 to update the historical visit count. If the number of historical visits is less than or equal to the decay step n, the number of historical visits is directly decayed to 0, and then added 1 to update the historical visit count. The decay factor m is adjustable; the smaller the value of m, the more drastic the decay of the historical visit count. This ensures that the historical visit count can more effectively represent the most recent access frequency of the resource.

[0034] The present invention uses a resource heat decay algorithm in the central cache analysis service cluster to decay the historical number of resource visits according to time. The longer the last access time of the resource is from the current time, the more drastic the decay of the statistical historical number of resource visits. Ultimately, it is ensured that resources with high access frequency in the recent period are defined as hot resources with high heat, rather than defining resources with high access frequency in a long period of time but low access frequency in the recent period as hot resources.

[0035] Furthermore, after the step S5, the following steps are further included:

[0036] When the URL access request for the resource hotspot bitmap information enters the cache server, the MD5 hash algorithm is used to generate MD5 fingerprint information, and the cuckoo filter algorithm is used to find out whether the MD5 fingerprint information exists in the resource hotspot bitmap information. If it exists, it proves that the resource hotspot bitmap information is a hot resource in the area, and the resource hotspot bitmap information is cached; if it does not exist, it is considered that the resource hotspot bitmap information has not reached the standard of popularity in the area, and it is not cached, and only the MD5 fingerprint information is recorded in the file.

[0037] The present invention aggregates and counts the access resource URLs of multiple sinking nodes according to the region, and counts the hot resource distribution of the region, breaking through the following bottleneck that the conventional cache server can only count the access requests at the granularity of a single machine: in the case of a single machine granularity, all requests that do not hit for the first time need to be cached and heated by subsequent requests for the same URL. In the case that the resource is not hot enough, it needs to be overwritten and removed when a disk write loop is made, and storage space will be occupied during this period. The present invention collects statistics on the heat of resources in the region. When a resource is not hit for the first time, it can be determined whether the resource is a hot resource by querying the resource heat bitmap. Only when it is determined to be a hot resource will it be cached. The solution of the present invention can ensure that when the storage space of the sinking node is limited, resources with low access hot spots in the region are discarded, and more and hotter resources are cached, thereby achieving the optimization effect of improving the cache hit rate.

[0038] Furthermore, the resource hotspot bitmap information is periodically updated and synchronously distributed by the central cache analysis service cluster.

[0039] The present invention also provides a system for improving the cache hit rate of edge CDN sink nodes, which executes the method for improving the cache hit rate of edge CDN sink nodes as described above, including:

[0040] URL request recording module: used by the sink node cache server to record each URL request in the order of request and save it in a separate file;

[0041] File collection, compression and upload module: used to deploy an independent agent on the cache server to collect the files, collect compressed files once every set time interval (preferably 5 minutes), and upload them to the centralized central cache analysis service cluster;

[0042] URL access frequency update and sorting module: used for the central cache analysis service cluster to save the global URL access count in the region, update the URL access frequency in real time, and sort the URLs by access frequency to filter out the hottest URL list;

[0043] The module for converting resource hotspot bitmap information is used to obtain the top N data records sorted from the database according to the number of historical visits at a set time interval (preferably 5 minutes). The size of N indicates how many hotspot resources the sinking node cache server needs to cache. The size of N is calculated by dividing the storage space of the cache server by the average file size. The collection of MD5 fingerprint information corresponding to the N URLs obtained is converted into resource hotspot bitmap information using the data structure of the cuckoo filter and sent to each sinking node cache server.

[0044] Resource hotspot bitmap information preservation and persistence module: used to save the resource hotspot bitmap information in memory after the sinking node cache server receives the resource hotspot bitmap information, and persist it to disk to prevent the resource hotspot bitmap information from being lost due to cache server restart.

[0045] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for improving the cache hit rate of edge CDN sink nodes as described above.

[0046] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the steps of the method for improving the cache hit rate of the edge CDN sinking node as described above are implemented.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The method and system for improving the cache hit rate of edge CDN sink nodes provided by the present invention combine and count user access resource requests from multiple nodes in a region, and a single cache server of the sink node can know the distribution of hot resources in the entire region including other sink nodes, calculate the hottest resource cache list through an offline algorithm, and pass the resource cache list to the cache server of the sink node at a relatively low cost. When the cache server of the sink node has a new missed access request, the cache server of each sink node searches for a high-hot resource list to ensure that only high-hot resources are cached, and higher-hot content can be cached within the limited storage space of the sink node; thereby caching more hot resources, effectively improving the cache hit rate, and enhancing user access quality and usage experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Various other advantages and benefits will become apparent to those skilled in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiment and are not to be considered as limiting the present invention.

[0050] In the attached picture:

[0051] Figure 1 This is a CDN network interaction scenario diagram according to an embodiment of the present invention;

[0052] Figure 2 This is a hotspot caching flow chart of a cache server according to an embodiment of the present invention;

[0053] Figure 3 This is a flowchart of a resource hotspot analysis service according to an embodiment of the present invention;

[0054] Figure 4 This is a flow chart of a method for improving the cache hit rate of edge CDN sink nodes in the present invention;

[0055] Figure 5 Schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of systems and products consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0057] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms of "a", "said" and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0058] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0059] The embodiments of the present invention are described in further detail below.

[0060] The embodiment of the present invention proposes a method and system for improving the cache hit rate of edge CDN sink nodes, in order to solve the problem that the cache replacement of cache heat statistics of conventional cache servers occupies a large storage space and significantly reduces the hit rate when the storage space is limited.

[0061] Method Example

[0062] The embodiment of the present invention provides a method for improving the cache hit rate of edge CDN sink nodes, see Figure 4 As shown, the following steps are included:

[0063] S1. The sink node cache server records each URL request in the order of request and saves it in a separate file;

[0064] The method of recording each URL request and saving it in an independent file includes: performing MD5 operation on each URL request, generating a 128-bit MD5 fingerprint and saving it in a file, determining whether the URL includes parameters in the MD5 operation based on the cache configuration, each record in the file including the request timestamp + MD5 fingerprint information, cutting the record file every 5 minutes and compressing and saving it, and determining the file name based on the corresponding time period.

[0065] In order to save space and transmission time, this embodiment performs MD5 operation on each URL request and generates a 128-bit MD5 fingerprint for storage. Since the number of requests may be relatively large, resulting in a relatively large file, the present invention cuts the record file every 5 minutes and compresses it for storage.

[0066] S2. Deploy an independent agent on the cache server to collect the files, collect the compressed files every 5 minutes, and upload them to the centralized central cache analysis service cluster;

[0067] S3. The central cache analysis service cluster stores the global URL access count in the region, updates the URL access frequency in real time, and sorts the URLs by access frequency to select the hottest URL list;

[0068] The method for updating the access frequency of a URL in real time includes:

[0069] The central cache analysis service uses a database to store URL access frequencies. The database table contains three fields: MD5 fingerprint information, last access time, and number of accesses, with MD5 fingerprint information used as an index.

[0070] The method for sorting by URL access frequency includes:

[0071] The central cache analysis cluster receives the uploaded compressed files from each sinking node cache server, decompresses the compressed files and reads the MD5 fingerprint information and the corresponding timestamp in sequence, and increases the historical access count of the corresponding MD5 fingerprint information in the database update. While updating the access count, it determines the time period from the last access time to the current timestamp, and attenuates the original access count with a large time span to ensure that the access frequency can be reflected in the most recent time period as much as possible. The strategy adopted by the algorithm for attenuating the access count is that the longer the key has not been accessed, the greater the degree of attenuation.

[0072] The algorithm for decaying the number of visits is:

[0073] Subtract the last access timestamp from the current timestamp to calculate the duration T in minutes. Set the decay factor m and decay step n. Divide the duration T by the decay factor m to obtain the decay step n. If the number of historical visits exceeds the decay step n, subtract the decay step n from the number of historical visits to obtain the latest decayed historical visit count, and then add 1 to update the historical visit count. If the number of historical visits is less than or equal to the decay step n, the number of historical visits is directly decayed to 0, and then added 1 to update the historical visit count. The decay factor m is adjustable; the smaller the value of m, the more drastic the decay of the historical visit count. This ensures that the historical visit count can more effectively represent the most recent access frequency of the resource.

[0074] S4. Obtain the TOP N data records sorted from the database according to the number of historical visits from high to low at 5-minute intervals. The size of N indicates how many hot resources the sinking node cache server needs to cache. The size of N is calculated by dividing the storage space of the cache server by the average file size. The collection information of the MD5 fingerprint information corresponding to the N URLs obtained is converted into resource hot spot bitmap information using the data structure of the cuckoo filter and sent to each sinking node cache server (see Figure 3 shown);

[0075] In order to save space, this embodiment uses the data structure of the cuckoo filter to convert the collection information of N records into resource hotspot bitmap information and sends it to the sinking node cache server.

[0076] Under conventional circumstances, cache heat statistics are based on a single machine granularity, and cache replacement is performed through algorithms such as LRU. Even if the resource requested for the first time is a cold resource, it needs to be cached until it is replaced by a hotter resource. In this way, a large number of long-tail cold resources will occupy storage space before being replaced. In the case of limited storage space, the hit rate is greatly damaged. This embodiment combines statistics on user access resource requests from multiple nodes in a region, calculates the hottest resource cache list through an offline algorithm, and passes the resource cache list to the cache server of the sinking node at a relatively low cost. When the cache server of the sinking node has a new missed access request, it searches for a high-heat resource list in the cache server of each sinking node to ensure that only high-heat resources are cached, thereby improving the cache hit rate.

[0077] S5. After the sinking node cache server receives the resource hotspot bitmap information, it saves the resource hotspot bitmap information in memory and persists it to disk to prevent the resource hotspot bitmap information from being lost due to cache server restart.

[0078] When a URL access request for resource hot spot bitmap information enters the cache server, the MD5 hash algorithm is used to generate MD5 fingerprint information, and the cuckoo filter algorithm is used to check whether the MD5 fingerprint information exists in the resource hot spot bitmap information (see Figure 2 as shown). If it exists, it proves that the resource hot spot bitmap information is a hot resource in the region, and the resource hot spot bitmap information is cached; if it does not exist, it is considered that the popularity of the resource hot spot bitmap information in the region has not reached the standard, and it is not cached, but only the MD5 fingerprint information is recorded in a file.

[0079] The resource hot spot bitmap information is periodically synchronized and updated by the central cache analysis service cluster.

[0080] Figure 1 shows the CDN network interaction scenario of this embodiment.

[0081] System embodiment

[0082] An embodiment of the present invention further provides a system for improving the cache hit rate of edge CDN sinking nodes, which executes the method for improving the cache hit rate of edge CDN sinking nodes as described above, including:

[0083] URL request recording module: used to record each URL request in the order of requests by the sinking node cache server and save it in an independent file;

[0084] File collection, compression and upload module: used to deploy an independent proxy agent on the cache server to collect the file, collect the compressed file every 5 minutes, and upload it to the centralized central cache analysis service cluster;

[0085] URL access frequency update and sorting module: used to save the global URL access times in the region by the central cache analysis service cluster, update the access frequency of the URL in real time, and sort by the access frequency of the URL to filter out the hottest URL list;

[0086] Resource hot spot bitmap information conversion module: used to obtain the TOP N data records sorted from high to low according to the historical access times from the database at 5-minute time intervals. The size of N represents how many hot resources the sinking node cache server needs to cache. The size of N is calculated by dividing the storage space of the cache server by the average file size; the set information of the MD5 fingerprint information corresponding to the N URLs obtained is converted into resource hot spot bitmap information using the data structure of the cuckoo filter and sent to each sinking node cache server;

[0087] Resource hotspot bitmap information preservation and persistence module: used to save the resource hotspot bitmap information in memory after the sinking node cache server receives the resource hotspot bitmap information, and persist it to disk to prevent the resource hotspot bitmap information from being lost due to cache server restart.

[0088] An embodiment of the present invention further provides a computer device, Figure 5 This is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention; see the accompanying drawings Figure 5 As shown, the computer device includes: an input system 23, an output system 24, a memory 22 and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the method for improving the cache hit rate of the edge CDN sinking node as provided in the above embodiment; wherein the input system 23, the output system 24, the memory 22 and the processor 21 can be connected by a bus or other means, Figure 5 The bus connection is taken as an example.

[0089] The memory 22 is a readable and writable storage medium of a computing device, which can be used to store software programs and computer executable programs, such as the program instructions corresponding to the method for improving the cache hit rate of edge CDN sinking nodes described in the embodiment of the present invention; the memory 22 may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the device, etc.; in addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device; in some instances, the memory 22 may further include a memory remotely located relative to the processor 21, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0090] The input system 23 may be used to receive input digital or character information, and generate key signal input related to user settings and function control of the device; the output system 24 may include a display device such as a display screen.

[0091] The processor 21 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 22, that is, implements the above-mentioned method of improving the cache hit rate of the edge CDN sink node.

[0092] The computer device provided above can be used to execute the method for improving the cache hit rate of edge CDN sink nodes provided in the above embodiment, and has corresponding functions and beneficial effects.

[0093] An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which are used to execute the method for improving the cache hit rate of edge CDN sinking nodes provided in the above embodiment when executed by a computer processor. The storage medium is any of various types of memory devices or storage devices, including: installation media, such as CD-ROMs, floppy disks, or tape systems; computer system memories or random access memories, such as DRAM, DDRRAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memories, such as flash memories, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memories or combinations thereof; additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, and the second computer system is connected to the first computer system through a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that may reside in different locations (such as in different computer systems connected through a network). The storage medium may store program instructions (such as specifically implemented as a computer program) executable by one or more processors.

[0094] Of course, the computer-executable instructions of a storage medium containing computer-executable instructions provided in an embodiment of the present invention are not limited to the method for improving the cache hit rate of edge CDN sinking nodes described in the above embodiment, and may also execute related operations in the method for improving the cache hit rate of edge CDN sinking nodes provided in any embodiment of the present invention.

[0095] The applicant of the present invention has made a detailed description and illustration of the embodiments of the present invention in combination with the accompanying drawings of the specification. However, those skilled in the art should understand that the above embodiments are only the preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, rather than a limitation on the protection scope of the present invention. On the contrary, any improvement or modification made based on the spirit of the present invention should fall within the protection scope of the present invention.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for improving the cache hit rate of edge CDN sinking nodes, characterized in that, It includes the following steps: S1. The sinking node cache server records each URL request in the order of requests and saves it in an independent file; S2. Deploy an independent proxy agent on the cache server to collect the file. The compressed file is collected every set time and uploaded to the centralized central cache analysis service cluster; S3. The central cache analysis service cluster saves the global URL access times within the region, updates the access frequency of the URL in real time, sorts by the access frequency of the URL, and filters out the hottest URL list; S4. Obtain the TOPN data records sorted from high to low according to the historical access times from the database at set time intervals. The size of N represents how many hot resources the sinking node cache server needs to cache. The size of N is calculated by dividing the storage space of the cache server by the average file size; the set of MD5 fingerprint information corresponding to the N URLs obtained is converted into resource hot bitmap information using the data structure of the cuckoo filter and sent to each sinking node cache server; S5. After receiving the resource hot bitmap information at the sinking node cache server, save the resource hot bitmap information in memory and persist it to disk to prevent the loss of resource hot bitmap information due to the restart of the cache server; The method of recording each URL request in step S1 and saving it in an independent file includes: performing MD5 operation on each URL request to generate a 128-bit MD5 fingerprint and saving it in the file. Whether the URL participates in the MD5 operation with parameters is determined by the cache configuration. Each record in the file includes the request timestamp + MD5 fingerprint information. The record file is cut and compressed every 5 minutes and saved, and the file name is determined by the corresponding time period; The method of updating the access frequency of the URL in step S3 includes: The central cache analysis service uses the database to save the access frequency of the URL. The database table contains 3 fields, namely MD5 fingerprint information, the last access time, and the access times, with the MD5 fingerprint information as the index; The method of sorting by the access frequency of the URL in step S3 includes: The central cache analysis cluster receives the uploaded compressed files from each sinking node cache server, decompresses the compressed files and sequentially reads the MD5 fingerprint information and the corresponding timestamps, updates and increases the historical access times of the corresponding MD5 fingerprint information in the database. When updating the access times, judge the time period between the last access time and the current timestamp, and attenuate the access times with a large original time span to ensure that the access frequency can be reflected as much as possible in the recent time period. The strategy adopted by the algorithm for attenuating the access times is that the longer the key has not been accessed, the greater the degree of attenuation.

2. The method for improving the cache hit rate of edge CDN sinking nodes according to claim 1, wherein The algorithm for attenuating the access times is: Subtract the last access timestamp from the current timestamp to calculate the duration T in minutes. Set the attenuation factor m and the attenuation step n, and obtain the value of the attenuation step n by dividing the duration T by the attenuation factor m; If the historical access count > the decay step n, subtract the decay step n from the historical access count to obtain the latest decayed historical access count, and then add 1 to update the historical access count; if the historical access count ≤ the decay step n, directly decay the historical access count to 0, and then add 1 to update the historical access count; the decay factor m is adjustable, and the smaller the value of m, the more drastic the decay of the historical access count.

3. The method for improving the cache hit rate of edge CDN sinking nodes according to claim 2, wherein, After the step S5, it further includes: When a URL access request for the resource hot bitmap information enters the cache server, use the MD5 hash algorithm to generate MD5 fingerprint information, and use the cuckoo filter algorithm to check whether the MD5 fingerprint information exists in the resource hot bitmap information. If it exists, it proves that the resource hot bitmap information is a hot resource in the region, and cache the resource hot bitmap information; if it does not exist, it is considered that the heat of the resource hot bitmap information in the region has not reached the standard, do not cache it, and only record the MD5 fingerprint information in a file.

4. The method for improving the cache hit rate of edge CDN sinking nodes according to claim 3, wherein The resource hot bitmap information is periodically synchronized and updated by the central cache analysis service cluster.

5. A system for improving the cache hit rate of edge CDN sinking nodes, which executes the method for improving the cache hit rate of edge CDN sinking nodes according to any one of claims 1-4, characterized in that, It includes: URL request record module: used to record each URL request in the order of requests by the sink node cache server and save it in an independent file; File collection, compression and upload module: used to deploy an independent proxy agent on the cache server to collect the file, collect the compressed file at a set time interval, and upload it to the centralized central cache analysis service cluster; URL access frequency update and sorting module: used to save the global URL access count in the region by the central cache analysis service cluster, update the access frequency of the URL in real time, and sort by the access frequency of the URL to filter out the hottest URL list; Convert resource hot bitmap information module: used to obtain the TOP N data records sorted from high to low according to the historical access count from the database at a set time interval. The size of N represents how many hot resources the sink node cache server needs to cache, and the size of N is calculated by dividing the storage space of the cache server by the average file size; convert the set information of the MD5 fingerprint information corresponding to the N URLs into resource hot bitmap information using the data structure of the cuckoo filter, and send it to each sink node cache server; Resource hot bitmap information save and persistency module: used to save the resource hot bitmap information in memory when the sink node cache server receives the resource hot bitmap information, and at the same time persist it to the disk to prevent the loss of the resource hot bitmap information due to the restart of the cache server.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method for improving the cache hit rate of the edge CDN sink node according to any one of claims 1-4.

7. A computer device, the computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for improving the cache hit rate of the edge CDN sink node according to any one of claims 1-4.

Images