Data preheating method and apparatus used in CDN, and device and medium

Abstract

Provided in the embodiments of the present disclosure are a data preheating method and apparatus used in a CDN, and a device and a medium. The method comprises: receiving a resource request of a service party, wherein the resource request comprises a network link corresponding to target data to be preheated of the service party; on the basis of a domain name in the network link, determining an IP address set for the network link from a mapping table between regions in a global network range of a CDN and IP segments; on the basis of the IP address set, determining a service node set in the CDN that is used for preheating; and executing a preheating task for the target data in each service node in the service node set.

Description

Data preheating method, device, equipment and medium for CDN

[0001] This application claims priority to the Chinese invention patent application entitled “Data preheating method, device, equipment and medium for CDN” and application number 202410330208.3, filed on March 21, 2024. The entire contents of that application are incorporated herein by reference. Technical Field

[0002] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to a method, apparatus, device, and computer-readable storage medium for preheating data in a content delivery network (CDN). Background Art

[0003] With the development of internet technology, individual users' demand for access to a large number of resources has gradually increased. Content Delivery Network (CDN) pre-warming technology can cache the corresponding resources from the origin site on CDN nodes before the user's first request for the resource. As a result, when the user actually accesses the resource, they can directly obtain the latest, cached data from the CDN node, efficiently and quickly. CDN allows users to access the content they need nearby, thereby alleviating network congestion and improving the response time of users accessing websites. Summary of the Invention

[0004] In a first aspect of the present disclosure, a data preheating method for a content delivery network (CDN) is provided. The method comprises: receiving a resource request from a service provider, the resource request including a network link corresponding to target data to be preheated; determining, based on a domain name in the network link, a set of IP addresses for the network link from a mapping table of regions within the CDN's global network scope to Internet Protocol (IP) segments; determining, based on the set of IP addresses, a set of service nodes in the CDN for preheating; and executing a preheating task for the target data on each service node in the set of service nodes.

[0005] In a second aspect of the present disclosure, a data preheating device for a content delivery network (CDN) is provided. The device includes: a request receiving module configured to receive a resource request from a business party, the resource request including a network link corresponding to target data to be preheated by the business party; an address determining module configured to determine, based on a domain name in the network link, a set of IP addresses for the network link from a mapping table between various regions in the CDN's global network scope and Internet Protocol (IP) segments; a node determining module configured to determine, based on the set of IP addresses, a set of service nodes in the CDN for preheating; and a task executing module configured to execute a preheating task for the target data in each service node in the set of service nodes.

[0006] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method of the first aspect of the present disclosure.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium and can be executed by a processor to perform the method according to the first aspect of the present disclosure.

[0008] It should be understood that the contents described in the Summary of the Invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent hereinafter with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] FIG1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0011] FIG2 shows a flow chart of a process for data pre-warming in a CDN according to some embodiments of the present disclosure;

[0012] FIG3 shows a flowchart of a process for determining an edge node of an IP address corresponding to a domain name according to some embodiments of the present disclosure;

[0013] FIG4 shows an example diagram of creating a warm-up task according to some embodiments of the present disclosure;

[0014] FIG5 shows a flowchart of a process for preheating task scheduling according to some embodiments of the present disclosure;

[0015] FIG6 shows a block diagram of a data preheating device for a CDN according to some embodiments of the present disclosure; and

[0016] FIG7 illustrates a block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0017] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0018] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below.

[0019] Herein, unless explicitly stated otherwise, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0020] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0021] It is understandable that before using the technical solutions disclosed in the various embodiments of the present disclosure, the type, scope of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to relevant users and authorization should be obtained from relevant users in an appropriate manner in accordance with relevant laws and regulations. The relevant users may include any type of right holders, such as individuals, enterprises, and groups.

[0022] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly prompt the relevant user that the operation requested to be performed will require obtaining and using the information of the relevant user, so that the relevant user can independently choose whether to provide information to the software or hardware such as the electronic device, application, server or storage medium that executes the operation of the technical solution of the present disclosure based on the prompt message.

[0023] As an optional but non-limiting implementation, in response to receiving an active request from a relevant user, a prompt message may be sent to the relevant user in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0024] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure. The activation of the digital assistant-related functions of the embodiment of the present disclosure, the data obtained, the processing and storage of the data, etc., shall all obtain the prior authorization of the user and other rights holders associated with the user, and shall comply with the provisions of relevant laws and regulations and the rules of agreement between rights holders.

[0025] 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. In the environment 100 , a preheating system 120 is configured to control the provision of a resource preheating service in a content delivery network (CDN) 110 .

[0026] The preheating system 120 can be any type of device with computing capabilities, including a terminal device or a server device. The terminal device can be any type of mobile terminal, fixed terminal, or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof.

[0027] The server-side device may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like. The server-side device may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The server-side device may include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like. In some embodiments, the preheating system 120 may be implemented based on cloud services.

[0028] The CDN 110 may include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in a cloud environment, and the like. The server-side device may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. The server-side device may include, for example, computing systems / servers, such as mainframes, edge computing nodes, computing devices in a cloud environment, and the like. It should be understood that the structure and functionality of the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0029] CDN preheating technology caches the corresponding resources from the origin server on CDN nodes before the user's first resource request. This allows users to retrieve the latest, cached data directly from the CDN node when they actually access the resource, efficiently and quickly. Due to the differences in the service node data centers of traditional CDN vendors, a single vendor cannot achieve global resource preheating acceleration. This means they cannot provide standardized preheating capabilities and cannot support preheating at the vendor, global, country, or regional levels.

[0030] FIG2 shows a flow chart of a process for data preheating in a CDN according to some embodiments of the present disclosure. Process 200 may be implemented in preheating system 120 of FIG1 . For ease of discussion, process 200 will be described with reference to environment 100 of FIG1 .

[0031] At block 210, the pre-warming system 120 receives a resource request from a business entity. The resource request includes a network link corresponding to the target data to be pre-warmed. The business entity may be an entity with a large data pre-warming requirement, such as an organization or enterprise. Resources can be obtained via a network link (URL, uniform resource location), each of which contains corresponding data content for the business entity to use.

[0032] Preheating resource URLs can reduce the time spent on returning to the source and improve user experience. This is especially true during large-scale business events, new content releases, or traffic peaks. It can effectively handle situations where a large number of users are accessing the source site simultaneously, avoiding problems that may arise from high load on the source site.

[0033] At block 220, preheating system 120 determines a set of IP addresses for the network link based on the domain name in the network link from a mapping table of regions and IP segments within CDN 110's global network scope. In some embodiments, the CDN's global network scope can be the entire world, and the regions can cover various countries or regions around the world. Each country / region has its own IP segment information, and these do not conflict with each other.

[0034] At block 230, preheating system 120 determines a set of service nodes in the CDN for preheating based on the set of IP addresses. Each IP address has a corresponding service node. Service nodes may include edge nodes, data link layer (L2) nodes, network layer (L3) nodes, and the like. In some embodiments, using edge nodes as service nodes is more advantageous for improving the performance of preheating system 120.

[0035] This embodiment helps to achieve CDN node preheating in all regions or even globally, without specifying the user and domain name range, so that the business party does not need to specifically identify the service provider of the preheating service.

[0036] The pre-warming system 120 can extract all domain names from the URL and obtain a domain name list after deduplication. When determining a set of IP (Internet Protocol) addresses from a mapping table based on the domain names in the network link, the Domain Name System (DNS) can be used. DNS is a distributed database that maps domain names to IP addresses, making it easier for people to access the Internet.

[0037] In some embodiments, for global pre-warming, the pre-warming system 120 can obtain edge node query results for the dial-up test point IP address in each country, obtain IP address polling in each country / region, and calculate the corresponding HttpDNS coverage results. In some embodiments, IP segment information for each country can be maintained, and an EDNS (Extended DNS) request can be initiated based on the actual client IP address in the city.

[0038] Table 1 below shows an example of edge IPs determined through domain name resolution and polling:

[0039] Table 1

[0040] At block 240, the preheating system 120 executes the preheating task for the target data on each service node in the set of service nodes. Specifically, the CDN can, based on comprehensive information such as network traffic, connectivity and load status of each node, distance to the user, and response time, direct the target data to the service node closest to the user in real time, thereby executing the preheating task on the nearest service node.

[0041] In some embodiments, the warm-up system 120 determining the IP address set for the network link includes: determining access validity of the network link; and determining the IP address set for the network link in response to determining that the network link is valid.

[0042] Specifically, the access validity of the network link can be verified by executing a HEAD request (a network request that obtains the metadata of a resource without transmitting actual data content) on the IP. For example, assuming that the IP of the CDN edge node obtained is 2.2.2.2 (as shown in the example in Table 1), a HEAD request can be executed on all IPs in the 2.2.2.1 to 2.2.2.254 network segment using the URL. If a 200 status code is returned normally, it means that the access to the URL is valid. In this case, the corresponding IP can be added to the edge node set of the domain name.

[0043] By verifying the access validity of network links, we can ensure the correctness of the network links used by users and avoid wasting service resources by obtaining a large number of invalid network links. In addition, since HEAD requests only return response header information and not the actual response body, the data volume is small, which can reduce the service resource consumption during the verification process.

[0044] Furthermore, in some embodiments, in response to determining that the network link is invalid, access control information corresponding to the network link is obtained; based on the access control information, it is determined whether there is an access control configuration for the network link; and based on the determination of whether there is an access control configuration for the network link, prompt information is provided to the business party. Specifically, after executing the HEAD request, if a non-200 status code is returned (for example, a 403 status code is returned), it indicates that access to the network link is invalid. At this time, the relevant access control information can be pulled through the domain name configuration interface to determine whether there is an access control configuration that causes the preheating to fail, or to determine whether there is an anti-hotlinking feature, thereby determining the reason why the URL cannot be preheated, and providing relevant prompt information to the business party to inform the business party of the reason, thereby facilitating analysis and improvement, and subsequently improving the preheating success rate.

[0045] Further, in some embodiments, the preheating system 120 determines the IP address set for the network link, including: utilizing the network link to perform a network request on each IP address in the IP address set; in response to the first IP address in the IP address set being unable to be successfully requested, deleting the first IP address from the IP address set; and in response to the first IP address in the IP address set being able to be successfully requested, retaining the first IP address in the IP address set.

[0046] Specifically, after executing the above-mentioned HEAD request (i.e., network request) on each IP address in the IP address set, the IP address for which the request is confirmed to be successful (i.e., a 200 status code is returned) is retained in the IP address set; the IP address for which the request is unsuccessful (i.e., no 200 status code is returned) is deleted from the IP address set.

[0047] FIG3 shows a flowchart of a process 300 for determining an edge node of an IP address corresponding to a domain name according to some embodiments of the present disclosure. As shown in FIG3 , the process 300 includes: after submitting the preheating task creation (310), obtaining all domain names (312), and obtaining a set of IP addresses of the preheating area (382), then calculating the service node covered by the domain name through DNS (330), and then storing the preheating URL and the IP of the corresponding service node and the target data size (340), and storing them in a database 350. The created preheating task can also be stored in the database 380. On the other hand, after obtaining all domain names (312), for each domain name, filter the URL with a status code of 200 (320) to obtain a valid URL, and then preheat the valid URL. When filtering the URL, the status code and data size of all IP addresses in the URL are obtained by executing a HEAD request (314). If the status of the IP address of the URL is not 200, the domain name configuration interface is pulled according to the domain name to determine whether there is an anti-hotlink or access control configuration (360); if not, the returned abnormal status code information (362) is marked and stored in the database 370; if so, an anti-hotlink or access control description (364) is added to the abnormal URL and also stored in the database 370.

[0048] In some embodiments, the preheating system 120 executing the preheating task for the target data in each service node in the set of service nodes includes: determining the preheating traffic size based on the size of the target data and the number of IP addresses in the set of IP addresses; and in response to determining that the preheating traffic size does not exceed a traffic threshold, executing the preheating task for the target data in each service node in the set of service nodes. Further, in some embodiments, in response to determining that the preheating traffic size exceeds the traffic threshold, sending a traffic prompt message to the business party; and in response to receiving a preheating confirmation from the business party, executing the preheating task for the target data in each service node in the set of service nodes.

[0049] Specifically, preheat traffic is aggregated traffic based on a domain name. This process traverses all URLs under a domain name to obtain the corresponding set of IP addresses, thereby determining the number of IP addresses corresponding to the domain name. By initiating a HEAD request to the URL and obtaining basic attributes, the target data size (expressed as content-length) can be obtained. The preheat traffic size (content-length × N) is obtained by multiplying the target data size by the number of IP addresses.

[0050] Furthermore, a determination is made as to whether the preheating traffic exceeds a preset traffic threshold. If it is determined that the preheating traffic does not exceed the traffic threshold, the preheating task for the target data is executed on each service node in the service node set. If the preheating traffic exceeds the traffic threshold, the preheating URL for this domain name can be deleted from the task, and a traffic reminder message can be sent to the business party to confirm whether to continue the preheating. If the business party confirms the preheating, the preheating task for the target data is executed on each service node in the service node set.

[0051] In such an embodiment, by calculating the possible preheating traffic before executing the task, it is possible to avoid the situation where abnormal tasks cause traffic bursts.

[0052] In some embodiments, the preheating system 120 performs a preheating task for the target data in each service node in the service node set, including: creating multiple preheating subtasks corresponding to the preheating task, each preheating subtask corresponds to an IP address in the IP address set, and is configured to perform preheating of the target data at the service node corresponding to the IP address; and executing multiple preheating subtasks.

[0053] FIG4 illustrates an example diagram for creating a preheating task according to some embodiments of the present disclosure. As shown in FIG4 , preheating task 400 may include, for example, a main work order, deployment ID (490), and its corresponding preheating subtasks, which may be multiple sub-work orders, such as task ID set 412, task ID set 422, task ID set 432, and task ID set 442. Each sub-work order corresponds to an IP address, and each sub-work order is used to perform preheating of target data at the service node corresponding to its IP address.

[0054] Furthermore, in some embodiments, during the execution of multiple preheating subtasks, the execution states corresponding to the multiple preheating subtasks are maintained, where the execution states are selected from a state set including the following: executing state, waiting state, failed state, and successful state; and the state of the preheating task is determined based on the execution states corresponding to the multiple preheating subtasks. As shown in Figure 4, for the subtask with task ID set 412, it corresponds to the executing state (process) 410; for the subtask with task ID set 422, it corresponds to the waiting state (waiting) 420; for the subtask with task ID set 432, it corresponds to the failed state (fail) 430; and for the subtask with task ID set 442, it corresponds to the successful state (success) 440.

[0055] Furthermore, each sub-work order can be stored in a database. For example, during the execution of each sub-work order, it can be temporarily stored in a REDIS (Remote Dictionary Server) database and then periodically synchronized to a persistent database (such as MySQL). After each sub-work order is executed, it can be stored in a database management system via an MQ (message queue) data channel, and the sub-work order record in the persistent database can be deleted.

[0056] In some embodiments, the preheating system 120 performs multiple preheating subtasks including: for a given preheating subtask among the preheating subtasks, initiating a cache request to the IP address corresponding to the given preheating subtask to cache the target data to the service node corresponding to the IP address; and in response to the caching being completed, setting the status corresponding to the IP address to a cache hit status.

[0057] Specifically, the preheating system 120 initiates a request to the IP address in a given subtask so that the target data can be cached on the corresponding node for preheating. After the request is completed, another request is initiated to the IP address in the given subtask, while recording the header information of the X-Cache (which specifically checks whether a web page is cached in the service node) to determine whether there is a hit field. If the hit field is present in the header information, the status corresponding to the IP address is set to the cache hit status (for example, the Hit Status can be updated to Hit).

[0058] In some embodiments, the preheating system 120 initiates a cache request to the IP address corresponding to a given preheating subtask, including: in response to the size of the target data exceeding a preset threshold, dividing the target data into multiple data segments; and initiating multiple cache requests in parallel to the IP address corresponding to the given preheating subtask, each cache request indicating that one of the multiple data segments is cached to the service node corresponding to the IP address.

[0059] Specifically, if the target data size exceeds a preset threshold, the target data is segmented into multiple data segments (denoted as "ranges"), each of which corresponds to a segment request (i.e., a cache request). For example, if the target data size is 90MB, it can be segmented into two data segments, 0-50MB and 50MB-90MB, each of which corresponds to a segment request. Furthermore, multiple segment requests corresponding to the multiple data segments are initiated in parallel, thereby enabling the corresponding service nodes to perform data caching.

[0060] In some embodiments, the preheating system 120 initiates a cache request to the IP address corresponding to a given preheating subtask, including: in response to the size of the target data exceeding a preset threshold, dividing the target data into multiple data segments; and performing multiple batches of data caching, in which, in each batch of data cache, multiple cache requests are initiated in parallel to the IP address corresponding to the given preheating subtask and the IP address corresponding to at least one other preheating subtask in the multiple preheating subtasks, each cache request indicating that one of the multiple data segments is cached to the service node corresponding to the IP address, wherein each IP address in the multiple batches of data cache is cached with a different data segment from the multiple data segments.

[0061] Specifically, for the multiple data fragments after segmentation, they can be cached in multiple batches. For example, assuming the size of the target data is 150MB, it can be segmented into 3 corresponding fragment requests; when initiating a task, you can select random edge nodes with different IP segments in the same URL for preheating; the first batch of data cache can be: edge node A with IP 1.1.1.1 preheats 0-50MB data fragments, edge node B with IP 2.2.2.2 preheats 50-100MB data fragments, and edge node C with IP 3.3.3.3 preheats 100-150MB data fragments; the second batch of data cache can be: edge node A with IP 1.1.1.1 preheats 0-50MB data fragments, edge node B with IP 2.2.2.2 preheats 50-100MB data fragments, and edge node C with IP 3.3.3.3 preheats 100-150MB data fragments; .1.1 edge node A preheats 100-150MB of data segments, the IP of the edge node B is 2.2.2.2 preheats 0-50MB of data segments, and the IP of the edge node C is 3.3.3.3 preheats 50-100MB of data segments; the third batch of data cache can be: the IP of the edge node A is 1.1.1.1 preheats 50-100MB of data segments, the IP of the edge node B is 2.2.2.2 preheats 100-150MB of data segments, and the IP of the edge node C is 3.3.3.3 preheats 0-50MB of data segments.

[0062] The fragment (range) random acquisition mode ensures that only one file is requested from the origin server simultaneously, effectively controlling the concurrency of back-to-origin requests.

[0063] In addition, during this batch cache warmup, a merged back-to-source mode was used for both L2 and L3 nodes. Furthermore, it was recommended to select random edge nodes with different IP segments for warmup within the same URL to disperse the L2 cache file content.

[0064] FIG5 shows a flowchart of a process 500 for preheating task scheduling according to some embodiments of the present disclosure. The process 500 can be implemented at the preheating system 120. As shown in FIG5, the process 500 includes: after the task engine of the preheating system 120 obtains the main work order to be executed (510), it determines the status of each of its sub-work orders (512). The main work order corresponds to the preheating task, and the sub-work orders correspond to the multiple preheating sub-tasks divided by the preheating task. If all sub-tasks are completed, the status of the main work order is updated to a successful state (success), and the REDIS and MYSQL sub-task information are cleared. If the main work order task is in the initialization state (520), the task engine of the preheating system 120 determines whether it is a scheduled task (522). If it is a scheduled task, the task engine waits for the execution time and determines whether the execution time has arrived (524). If it is not a scheduled task, or if the execution time has arrived, the task engine performs respective concurrent request control on the main work order according to the domain name group (530), and uploads the execution results to the remote dictionary service (540). Supports scheduled preheating capabilities for special tasks such as scheduled activities of some business parties or scheduled updates of installation packages.

[0065] Next, the task engine determines whether the target data size (content-length) of the URL exceeds the segment setting size (for example, 50MB or any other size) (550). If so, the task engine enters the large file preheating mode, including splitting the target data into multiple data segments, corresponding to multiple segment requests, and initiating concurrent execution of multiple segment requests (552); otherwise, direct preheating. After the direct preheating is completed, the task engine requests the marking results of the completed sub-work orders (560). If all sub-work orders under the main work order are completed (570), the task engine synchronizes each sub-work order (580) to the database (586) on the one hand, and stores it in the database management system (584) through the MQ (message queue) data channel (582) on the other hand. In some embodiments, if a sub-task fails to execute, the failed sub-task can also be retried.

[0066] It should be understood that process 500 is an example implementation of performing the warm-up task. In practical applications, the warm-up task of target data can also be completed through other processes as needed.

[0067] In another embodiment, data analysis can be performed on the work order records, including: extracting the task execution rate through the creation time and completion time in the work order records to calculate the preheating efficiency; extracting the URL cache hit rate information if there is a URL hit in the work order record; calculating the quality information of the preheated IP if there is a URL preheating time in the work order record, and then comparing the capabilities of different service providers.

[0068] Embodiments of the present disclosure also provide corresponding apparatuses for implementing the aforementioned methods or processes. Figure 6 shows a block diagram of an apparatus 600 for preheating data in a CDN, according to certain embodiments of the present disclosure. Apparatus 600 can be implemented as or included in preheating system 120. Each module / component in apparatus 600 can be implemented using hardware, software, firmware, or any combination thereof.

[0069] As shown in Figure 6, the device 600 includes a request receiving module 610, which is configured to receive a resource request from a business party, wherein the resource request includes a network link corresponding to the target data to be preheated of the business party. The device 600 also includes an address determination module 620, which is configured to determine a set of IP addresses for the network link from a mapping table of various regions and IP segments in the global network range of the CDN based on the domain name in the network link. The device 600 also includes a node determination module 630, which is configured to determine a set of service nodes for preheating in the CDN based on the IP address set. The device 600 also includes a task execution module 640, which is configured to execute a preheating task for the target data in each service node in the service node set.

[0070] In some embodiments, the address determination module 620 is further configured to determine access validity of the network link; and in response to determining that the network link is valid, determine an IP address set for the network link.

[0071] In some embodiments, the device 600 also includes an access control module, which is configured to obtain access control information corresponding to the network link in response to determining that the network link is invalid; determine whether there is an access control configuration for the network link based on the access control information; and provide prompt information to the business party based on the determination of whether there is an access control configuration for the network link.

[0072] In some embodiments, the address determination module 620 is further configured to utilize the network link to perform a network request on each IP address in the IP address set; in response to the first IP address in the IP address set being unable to be successfully requested, delete the first IP address from the IP address set; and in response to the first IP address in the IP address set being able to be successfully requested, retain the first IP address in the IP address set.

[0073] In some embodiments, the task execution module 640 is further configured to determine the preheating traffic size based on the size of the target data and the number of IPs in the IP address set; and in response to determining that the preheating traffic size does not exceed the traffic threshold, execute the preheating task for the target data in each service node in the service node set.

[0074] In some embodiments, the task execution module 640 is further configured to, in response to determining that the preheating traffic size exceeds the traffic threshold, send traffic prompt information to the business party; and in response to receiving a preheating confirmation from the business party, execute the preheating task for the target data in each service node in the service node set.

[0075] In some embodiments, the task execution module 640 is further configured to create multiple preheating subtasks corresponding to the preheating task, each preheating subtask corresponds to an IP address in the IP address set, and is configured to perform preheating of the target data at the service node corresponding to the IP address; and execute the multiple preheating subtasks.

[0076] In some embodiments, the device 600 also includes a state determination module, which is configured to maintain the execution state corresponding to each of the multiple preheating subtasks during the execution of the multiple preheating subtasks, and the execution state is selected from a state set including the following items: executing state, waiting state, failure state, and success state; and determine the state of the preheating task based on the execution state corresponding to each of the multiple preheating subtasks.

[0077] In some embodiments, the task execution module 640 includes a sub-task execution module, which is configured to initiate a cache request to the IP address corresponding to a given preheating subtask in the preheating subtasks, so as to cache the target data to the service node corresponding to the IP address; and in response to the completion of the cache, set the status corresponding to the IP address to a hit cache status.

[0078] In some embodiments, the subtask execution module includes a request initiation module, which is configured to divide the target data into multiple data fragments in response to the size of the target data exceeding a preset threshold; and initiate multiple cache requests in parallel to the IP address corresponding to the given preheating subtask, each cache request indicating that one of the multiple data fragments is cached to the service node corresponding to the IP address.

[0079] In some embodiments, the request initiation module is further configured to, in response to the size of the target data exceeding a preset threshold, divide the target data into multiple data segments; and perform multi-batch data caching, in which, in each batch of data cache, multiple cache requests are initiated in parallel to the IP address corresponding to the given preheating subtask and the IP address corresponding to at least one other preheating subtask among the multiple preheating subtasks, each cache request indicating that one of the multiple data segments is cached to the service node corresponding to the IP address, wherein each IP address in the multiple batches of data cache is cached with a different data segment among the multiple data segments.

[0080] The units and / or modules included in the device 600 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units and / or modules in the device 600 can be implemented at least in part by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0081] FIG7 shows a block diagram of an electronic device 700 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 700 shown in FIG7 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 700 shown in FIG7 can be used to implement the preheating system 120 of FIG1 or the apparatus 600 of FIG6.

[0082] As shown in FIG7 , electronic device 700 is in the form of a general-purpose computing device. Components of electronic device 700 may include, but are not limited to, one or more processors or processing units 710, memory 720, storage device 730, one or more communication units 740, one or more input devices 750, and one or more output devices 760. Processing unit 710 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 720. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of electronic device 700.

[0083] The electronic device 700 typically includes a plurality of computer storage media. Such media can be any available media accessible to the electronic device 700, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 720 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory) or some combination thereof. The storage device 730 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk or any other medium, which can be used to store information and / or data and can be accessed within the electronic device 700.

[0084] The electronic device 700 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG. 7 , a disk drive for reading from or writing to a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 720 may include a computer program product 725 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0085] The communication unit 740 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 700 can be implemented as a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 700 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0086] Input device 750 may be one or more input devices, such as a mouse, keyboard, or trackball. Output device 760 may be one or more output devices, such as a display, a speaker, or a printer. Electronic device 700 may also communicate with one or more external devices (not shown) via communication unit 740 as needed, such as storage devices, display devices, or the like, with one or more devices that allow a user to interact with electronic device 700, or with any device that allows electronic device 700 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0087] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0088] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0089] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0090] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0091] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0092] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A data preheating method for a content delivery network (CDN), comprising: Receive a resource request from a business party, where the resource request includes a network link corresponding to target data to be preheated by the business party; Based on the domain name in the network link, determining a set of IP addresses for the network link from a mapping table of regions in the CDN's global network scope to Internet Protocol IP segments; Determining a set of service nodes for preheating in the CDN based on the set of IP addresses; as well as A preheating task for the target data is executed in each service node in the service node set.

2. The method of claim 1 , wherein determining a set of IP addresses for the network link comprises: Determining access validity of the network link; In response to determining that the network link is valid, a set of IP addresses for the network link is determined.

3. The method according to claim 1, further comprising: In response to determining that the network link is invalid, obtaining access control information corresponding to the network link; determining whether an access control configuration exists for the network link based on the access control information; as well as Based on the determination of whether the network link has an access control configuration, prompt information is provided to the business party.

4. The method of claim 1 , wherein determining a set of IP addresses for the network link comprises: executing a network request for each IP address in the set of IP addresses using the network link; In response to a first IP address in the set of IP addresses failing to be successfully requested, deleting the first IP address from the set of IP addresses; as well as In response to a first IP address in the set of IP addresses being able to be successfully requested, the first IP address is retained in the set of IP addresses.

5. The method according to claim 1, wherein executing the warm-up task for the target data in each service node in the service node set comprises: Determining a preheating traffic size based on the size of the target data and the number of IP addresses in the IP address set; as well as In response to determining that the preheating traffic size does not exceed a traffic threshold, a preheating task for the target data is executed in each service node in the service node set.

6. The method according to claim 5, wherein executing the warm-up task for the target data in each service node in the service node set comprises: In response to determining that the preheating traffic size exceeds the traffic threshold, sending traffic prompt information to the business party; as well as In response to receiving a preheating confirmation from the business party, a preheating task for the target data is executed in each service node in the service node set.

7. The method according to claim 1, wherein executing the warm-up task for the target data in each service node in the service node set comprises: Creating a plurality of preheating subtasks corresponding to the preheating task, each preheating subtask corresponding to an IP address in the IP address set and configured to perform preheating of the target data at a service node corresponding to the IP address; as well as The plurality of warm-up subtasks are executed.

8. The method according to claim 7, wherein the method further comprises: During the execution of the plurality of preheating subtasks, maintaining the execution status corresponding to each of the plurality of preheating subtasks, wherein the execution status is selected from a state set including the following: executing state, waiting state, failed state, and successful state; as well as The status of the warm-up task is determined based on the execution statuses corresponding to the plurality of warm-up subtasks.

9. The method according to claim 7, wherein executing the plurality of warm-up subtasks comprises: For a given preheating subtask in the preheating subtasks, initiating a cache request to an IP address corresponding to the given preheating subtask, so as to cache the target data to a service node corresponding to the IP address; as well as In response to the cache completion, the state corresponding to the IP address is set to a cache hit state.

10. The method according to claim 9, wherein initiating a cache request to the IP address corresponding to the given preheating subtask comprises: In response to the size of the target data exceeding a preset threshold, dividing the target data into a plurality of data segments; as well as Multiple cache requests are initiated in parallel to the IP address corresponding to the given preheating subtask, each cache request indicating that one of the multiple data fragments is cached to the service node corresponding to the IP address.

11. The method according to claim 9, wherein initiating a cache request to the IP address corresponding to the given warm-up subtask comprises: In response to the size of the target data exceeding a preset threshold, dividing the target data into a plurality of data segments; as well as Perform multiple batches of data caching. In each batch of data caching, initiate multiple cache requests in parallel to the IP address corresponding to the given warm-up subtask and the IP address corresponding to at least one other warm-up subtask among the multiple warm-up subtasks, each cache request instructing to cache one of the multiple data segments to the service node corresponding to the IP address. In the multiple batches of data caches, each IP address is cached with a different data fragment from the multiple data fragments.

12. A data preheating device for a content delivery network (CDN), comprising: A request receiving module is configured to receive a resource request from a business party, wherein the resource request includes a network link corresponding to target data to be preheated of the business party; an address determination module configured to determine, based on the domain name in the network link, a set of IP addresses for the network link from a mapping table of regions in the global network scope of the CDN and Internet Protocol IP segments; a node determination module configured to determine a set of service nodes for preheating in the CDN based on the set of IP addresses; as well as The task execution module is configured to execute a preheating task for the target data in each service node in the service node set.

13. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 11 when executed by the at least one processing unit.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product tangibly stored in a computer storage medium and comprising computer executable instructions which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 11.

Images