Method for adaptive source station to acquire resources and electronic device

Abstract

The application discloses a method for adaptive source station to acquire resources and an electronic device, and belongs to the technical field of computers. The method comprises the following steps: a client initiates a resource request to a CDN node, wherein the resource request is a range request; the CDN node receives the resource request, judges whether the local cache is hit based on the resource request, acquires the corresponding file from the local cache and feeds back the value to the client if the local cache is hit, and executes a source returning step if the local cache is not hit; the source returning step comprises the following steps: determining a source returning strategy based on the state attribute of the target IP corresponding to the resource request, wherein the state attribute of the target IP comprises a change attribute, a timestamp attribute and a range_err attribute, and the source returning strategy comprises a fragment source returning logic and a complete file source returning logic. The application realizes efficient and accurate file searching, and improves the efficiency and stability of source returning.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and specifically relates to a method and electronic device for adaptive source site resource acquisition. Background Technology

[0002] Currently, sharded origin pull, also known as Range request origin pull, uses the Range header in an HTTP request to retrieve files within a specified range. A Range request allows you to request a portion of a file's content from the server. Existing sharded origin pull methods require that the origin server already supports Range requests; otherwise, the origin pull may fail. If the origin server has multiple IPs, some supporting Range requests and others not, enabling sharded origin pull might result in different shards being routed to different IPs, leading to failure. Furthermore, if the origin server has multiple IPs with inconsistent content, enabling sharded origin pull might cause different shards to be routed to different IPs, resulting in CDN-cached content being pieced together and inconsistent with the origin server's MD5 hash.

[0003] To address the aforementioned issues, this application proposes a method and electronic device for adaptive source station resource acquisition. Summary of the Invention

[0004] To address the shortcomings of the prior art, this application provides a method and electronic device for adaptive source server resource acquisition, which solves the problems of existing fragmented source acquisition methods being unable to adaptively adjust and prone to source acquisition failures.

[0005] The technical effect to be achieved in this application is accomplished through the following solution: In a first aspect, this application provides a method for adaptive origin server resource acquisition, the method comprising: The client initiates a resource request to the CDN node; the resource request is a range request. The CDN node receives the resource request, determines whether the local cache is hit based on the resource request, and if the local cache is hit, retrieves the corresponding file from the local cache and returns it to the client; if the local cache is not hit, the origin retrieval step is executed. The source retrieval steps include: The origin pull strategy is determined based on the status attributes of the target IP corresponding to the resource request. The status attributes of the target IP include the change attribute, timestamp attribute, and range_err attribute. The origin pull strategy includes fragmented origin pull logic and complete file origin pull logic.

[0006] In some embodiments, determining the origin pull strategy based on the status attributes of the target IP corresponding to the resource request includes: The `change` attribute is checked. If `change` is false, the fragmented origin-fetching logic is executed. If `change` is true, the difference between the current time and the timestamp is checked. If the difference is greater than 12 hours, it means that the switch from fragmented origin-fetching logic to full file origin-fetching logic has failed. If `range_err` is 0, the timestamp is updated to the current time, and the fragmented origin-fetching logic is executed. If the difference is less than or equal to 12 hours, it means that the switch from fragmented origin-fetching logic to full file origin-fetching logic is still valid, and the full file origin-fetching logic is executed.

[0007] In some embodiments, the execution of the sharding-to-origin logic includes: The CDN node sends a range request to the origin server. After receiving the response from the origin server, the CDN node processes the response based on the response code.

[0008] In some embodiments, processing is performed based on the response code, including: If the response code is 206, it is a normal response, and the content of the corresponding range will be cached in the CDN node without changing the state attribute of the target IP. If the response code is 200, it means that the origin server does not support range requests, and the complete file return logic needs to be executed to obtain the complete file.

[0009] In some embodiments, when the response code is 200, the method further includes: The CDN node determines the difference between the current time and the timestamp. If the difference is greater than 12 hours, it sets range_err=1. If the difference is less than or equal to 12 hours, it increments the value of range_err by 1 and updates the timestamp to the current time.

[0010] In some embodiments, the method further includes: Continue to check if range_err has reached the threshold. If it has, set change=true; otherwise, set change=false. Save the updated status attributes of the target IP.

[0011] In some embodiments, performing full file origin-back logic includes: CDN nodes download the complete file from the origin server. If a partial cache of the complete file already exists in the cache, the partial cache is deleted, and the complete file is cached.

[0012] In some embodiments, the cached full file is edited to be a cached full file.

[0013] Secondly, this application provides an electronic device, the electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any of the foregoing descriptions.

[0014] Thirdly, this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the method described in any of the foregoing claims.

[0015] The method and electronic device for obtaining resources from the adaptive origin server provided in this application improve the efficiency and stability of the origin server by adaptively switching between fragmented origin server logic and complete file origin server logic, and by real-time management of IP status attributes. Attached Figure Description

[0016] To more clearly illustrate the embodiments of this application or the existing technical solutions, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a method for adaptive source site resource acquisition in one embodiment of this application; Figure 2 This is a schematic block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in one or more embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] A CDN (Content Delivery Network) acts as a network distribution server, accelerating user access by caching content at edge nodes. When the content requested by a user is not cached or the cache expires, the edge node sends a request to the origin server to retrieve the latest content, caches it locally, and then returns the content to the user. This process is called origin fetching. CDNs improve origin fetching efficiency through technologies such as load balancing, protocol optimization, and caching layering, ensuring that users can quickly obtain data.

[0021] As a network distribution server, a CDN does not generate content itself; it simply replicates the corresponding resources from the origin server to its nodes. It accepts client requests, and for each different requested URL and its corresponding resource, if the content is cached in the CDN and within the CDN's cache validity period, the CDN directly retrieves the content from its local cache. Otherwise, it initiates a request from the origin server of that URL to retrieve the resource corresponding to that URL and caches it on the CDN server.

[0022] Fragmented origin pull: Range requests retrieve the origin server's data. Range is an HTTP request header used to retrieve a specified range of files. Using a Range request, you can request a portion of a file's content from the server. For example, if the request includes the HTTP header: range: bytes=0-999, the first 1000 bytes of the file will be returned to the user. CDN fragmented origin pulls have a fixed size, determining the range of the origin pull based on the client's requested Range, and aligning the fragments before and after the request.

[0023] With sharded origin pull configured, the CDN will by default include a range request. If a user's requested file is not cached on the node or the cache has expired, the CDN will perform sharded origin pull based on the user's request, fetching only the required portion of the file to the node cache and returning it to the user. If sharded origin pull is disabled, and the user's request does not include a range request, the CDN will still fetch the entire file during the origin pull. Supported Range requests: Response with a 206 status code and the corresponding range offset. Unsupported Range requests: Response with a 200 status code and the complete file. `unordered_map`: An associative container in C++, a hash-based associative container.

[0024] Enabling origin sharding has advantages, and CDNs support Range. Without sharding, the CDN can only respond to clients after fetching the complete file from the origin server. Enabling origin sharding improves distribution and response speeds. However, under certain conditions, it's necessary to actively disable origin sharding. This can only be detected and manually addressed through monitoring, and cannot be adaptively implemented by the program.

[0025] This application designs an adaptive switching method for origin server access. By default, the first step is to use the sharded origin server access logic. If the origin server does not support this, the CDN can be actively switched to a sharded origin server access strategy.

[0026] Previously, CDN programs only had one logic for origin server access: selecting an IP address. The IP address was obtained by resolving DNS, and a request was sent directly to the selected IP address. Now, an unordered_map hash data structure can be added, with each IP address as the key and the value having three corresponding attributes: the number of sharding failures, whether the switchover was completed, and a timestamp (recording the write time).

[0027] For example, the IP address can be 1.2.3.4, with the default change=false (fragmentation enabled). The corresponding attributes are: range_err=1, change=true, timestamp=1700187241 (second-level timestamp).

[0028] These three attributes—the number of range_err occurrences—are used to determine whether the origin pull method corresponding to this IP is fragmented, change enables adaptive switching between fragmented origin pull and full file origin pull, and timestamp controls the timestamp to determine the validity period of range_err statistics and change.

[0029] The various non-limiting embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0030] First, refer to Figure 1This application provides a detailed description of the method for obtaining resources from an adaptive source site.

[0031] This application provides a method for adaptive origin server resource acquisition, the method comprising: S1: The client initiates a resource request to the CDN node, and the resource request is a range request; S2: The CDN node receives the resource request, determines whether the local cache is hit based on the resource request, and if the local cache is hit, retrieves the corresponding file from the local cache and returns it to the client; if the local cache is not hit, the origin retrieval step is executed. S3: The source retrieval step includes: The origin pull strategy is determined based on the status attributes of the target IP corresponding to the resource request. The status attributes of the target IP include the change attribute, timestamp attribute, and range_err attribute. The origin pull strategy includes fragmented origin pull logic and complete file origin pull logic.

[0032] The above method achieves efficient and accurate file retrieval by adaptively switching between fragmented origin retrieval logic and complete file origin retrieval logic, and by real-time management of IP status attributes, thereby improving the efficiency and stability of origin retrieval.

[0033] The source code adaptive mechanism of this application: 1) Sharding back to the source: The default strategy is to split large files into multiple small blocks and push them back to the source in parallel to improve the efficiency of the back-to-source process.

[0034] 2) Adaptive switching: If the origin server does not support sharded origin pull, the CDN node will automatically switch to non-sharded origin pull mode to ensure successful origin pull.

[0035] In some embodiments, determining the origin pull strategy based on the status attributes of the target IP corresponding to the resource request includes: The `change` attribute is checked. If `change` is false, the fragmented origin-fetching logic is executed. If `change` is true, the difference between the current time and the timestamp is checked. If the difference is greater than 12 hours, it means that the switch from fragmented origin-fetching logic to full file origin-fetching logic has failed. If `range_err` is 0, the timestamp is updated to the current time, and the fragmented origin-fetching logic is executed. If the difference is less than or equal to 12 hours, it means that the switch from fragmented origin-fetching logic to full file origin-fetching logic is still valid, and the full file origin-fetching logic is executed.

[0036] The IP selection and status management strategy of this application includes: 1) DNS resolution: Obtain the list of IP addresses of the source site through DNS resolution.

[0037] 2) IP Status Attributes: Introduce the unordered_map data structure, using each IP as the key, to store the following attributes: ① Number of fragmentation failures: Records the number of times this IP failed in fragmentation origin mode.

[0038] ② Whether the switchover is complete: Mark whether the IP has switched from fragmented origin to non-fragmented origin.

[0039] ③ Timestamp: Records the last update time of the IP status information, used to clean up expired data.

[0040] In some embodiments, the execution of the sharding-to-origin logic includes: The CDN node sends a range request to the origin server. After receiving the response from the origin server, the CDN node processes the response based on the response code.

[0041] In some embodiments, processing is performed based on the response code, including: If the response code is 206, it is a normal response, and the content of the corresponding range will be cached in the CDN node without changing the state attribute of the target IP. If the response code is 200, it means that the origin server does not support range requests, and the complete file return logic needs to be executed to obtain the complete file.

[0042] In some embodiments, when the response code is 200, the method further includes: The CDN node determines the difference between the current time and the timestamp. If the difference is greater than 12 hours, it sets range_err=1. If the difference is less than or equal to 12 hours, it increments the value of range_err by 1 and updates the timestamp to the current time.

[0043] The dynamic adjustment strategy for returning to the source in this application includes: 1) Based on the status information in unordered_map, dynamically select the optimal origin return strategy (fragmented origin return logic or complete file origin return logic).

[0044] 2) When the number of fragmentation failures of an IP reaches the threshold, it will be automatically marked as "not supporting fragmentation back to origin" and switched to the full file back to origin logic, i.e. non-fragmentation back to origin mode.

[0045] In some embodiments, the method further includes: Continue to check if range_err has reached the threshold. If it has, set change=true; otherwise, set change=false. Save the updated status attributes of the target IP.

[0046] The advantages of the data structure in this application: 1) Efficient lookup: unordered_map is implemented based on a hash table, and the time complexity of looking up and updating the IP status is O(1), which is suitable for high-frequency operations.

[0047] 2) Dynamic updates: Supports real-time updates of IP status, ensuring the flexibility and accuracy of origin pull policies.

[0048] In some embodiments, performing full file origin-back logic includes: CDN nodes download the complete file from the origin server. If a partial cache of the complete file already exists in the cache, the partial cache is deleted, and the complete file is cached.

[0049] In some embodiments, the cached full file is edited to be a cached full file.

[0050] The advantages of this application compared to existing CDN origin pull solutions are as follows: 1. Dynamic switching: The default is to use sharded origin server. If the origin server does not support it, it will automatically switch to non-sharded origin server to ensure compatibility.

[0051] 2. Real-time status management: Records IP status (failure count, switch status, timestamp) through unordered_map, supporting efficient lookup and update.

[0052] 3. Efficient Origin Query: Sharding the origin improves the efficiency of large file queries, and adaptive switching avoids failures, balancing speed and stability.

[0053] 4. Robustness: Quick failover avoids repeated attempts with invalid IPs, ensuring stable operation even under abnormal conditions.

[0054] It should be noted that the methods of one or more embodiments of this application can be executed by a single device, such as a computer or server. The methods of this embodiment can also be applied in a distributed scenario, where multiple devices cooperate to complete the process. In such a distributed scenario, one of these devices may execute only one or more steps of the methods of one or more embodiments of this application, and the multiple devices will interact with each other to complete the method described.

[0055] It should be noted that the above description describes specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0056] Based on the same inventive concept, and corresponding to the methods of any of the above embodiments, this application also discloses an electronic device; Specifically, Figure 2 This diagram illustrates the hardware structure of an electronic device for an adaptive source station resource acquisition method provided in this embodiment. The device may include: a processor 410, a memory 420, an input / output interface 430, a communication interface 440, and a bus 450. The processor 410, memory 420, input / output interface 430, and communication interface 440 are interconnected internally via the bus 450.

[0057] The processor 410 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0058] The memory 420 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 420 can store the operating system and other applications. When the technical solutions provided in the embodiments of this application are implemented by software or firmware, the relevant program code is stored in the memory 420 and is called and executed by the processor 410.

[0059] Input / output interface 430 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0060] The communication interface 440 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (e.g., USB, Ethernet cable, etc.) or wireless means (e.g., mobile network, WIFI, Bluetooth, etc.).

[0061] Bus 450 includes a pathway for transmitting information between various components of the device, such as processor 410, memory 420, input / output interface 430, and communication interface 440.

[0062] It should be noted that although the above-described device only shows the processor 410, memory 420, input / output interface 430, communication interface 440, and bus 450, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this application, and not necessarily all the components shown in the figures.

[0063] The electronic devices described above are used to implement the corresponding adaptive source station resource acquisition method in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0064] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, one or more embodiments of this application also provide a computer-readable storage medium storing computer instructions for causing the computer to perform the adaptive origin server resource acquisition method as described in any of the above embodiments.

[0065] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0066] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the adaptive source station resource acquisition method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0067] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0068] Additionally, to simplify the description and discussion, and to avoid obscuring one or more embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring one or more embodiments of this application, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which one or more embodiments of this application will be implemented (i.e., these details should be fully within the understanding of those skilled in the art). While specific details (e.g., circuits) are set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that one or more embodiments of this application may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0069] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0070] One or more embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for adaptively acquiring resources from an origin server, characterized in that, The method includes: The client initiates a resource request to the CDN node; the resource request is a range request. The CDN node receives the resource request and determines whether the local cache is hit based on the resource request. If the local cache is hit, the corresponding file is retrieved from the local cache and sent back to the client; if the local cache is not hit, the origin retrieval step is executed. The source retrieval steps include: The origin pull strategy is determined based on the status attributes of the target IP corresponding to the resource request. The status attributes of the target IP include the change attribute, the timestamp attribute, and the range_err attribute. The origin pull strategy includes fragmented origin pull logic and full file origin pull logic. The change attribute is used to mark whether the target IP has switched from fragmented origin pull logic to full file origin pull logic. The range_err attribute is used to record the number of fragmentation failures of the target IP under the fragmented origin pull logic. The timestamp attribute is used to record the update time of the status attributes of the target IP to determine the statistical validity period of range_err and the validity period of change.

2. The method for adaptive source station resource acquisition according to claim 1, characterized in that, Determining the origin retrieval strategy based on the status attributes of the target IP corresponding to the resource request includes: The `change` attribute is checked. If `change` is false, the fragmented origin-fetching logic is executed. If `change` is true, the difference between the current time and the timestamp is checked. If the difference is greater than 12 hours, it means that the switch from fragmented origin-fetching logic to full file origin-fetching logic has failed. If `range_err` is 0, the timestamp is updated to the current time, and the fragmented origin-fetching logic is executed. If the difference is less than or equal to 12 hours, it means that the switch from fragmented origin-fetching logic to full file origin-fetching logic is still valid, and the full file origin-fetching logic is executed.

3. The method for adaptive source station resource acquisition according to claim 2, characterized in that, The execution of the sharding and origin return logic includes: The CDN node sends a range request to the origin server. After receiving the response from the origin server, the CDN node processes the response based on the response code.

4. The method for adaptive source station resource acquisition according to claim 3, characterized in that, Process according to the response code, including: If the response code is 206, it is a normal response, and the content of the corresponding range will be cached in the CDN node without changing the state attribute of the target IP. If the response code is 200, it means that the origin server does not support range requests, and the complete file return logic needs to be executed to obtain the complete file.

5. The method for adaptive source station resource acquisition according to claim 4, characterized in that, When the response code is 200, the method further includes: The CDN node determines the difference between the current time and the timestamp. If the difference is greater than 12 hours, it sets range_err=1. If the difference is less than or equal to 12 hours, it increments the value of range_err by 1 and updates the timestamp to the current time.

6. The method for adaptive source station resource acquisition according to claim 5, characterized in that, The method further includes: Continue to check if range_err has reached the threshold. If it has, set change=true; otherwise, set change=false. Save the updated status attributes of the target IP.

7. The method for adaptive source station resource acquisition according to claim 6, characterized in that, Execute the complete file origin return logic, including: CDN nodes download the complete file from the origin server. If a partial cache of the complete file already exists in the cache, the partial cache is deleted, and the complete file is cached.

8. The method for adaptive source station resource acquisition according to claim 7, characterized in that, Edit the cached full file to show that the full file is cached.

9. An electronic device, the electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that can be executed by one or more processors to implement the method as described in any one of claims 1 to 8.

Images