Request processing method, device, computer equipment, storage medium and program product

Abstract

The present application relates to a request processing method, apparatus, computer equipment, storage medium and program product. The method includes: a CDN node receives a main request initiated by a user and obtains self-protection variable information, and when the self-protection variable information indicates that the self-protection mechanism is turned on, obtains the target file suffix of the main request, and when the target file suffix is ​​different from any file suffix in the file suffix list, initiates a sub-request and obtains the corresponding response metadata, if this data does not include content-length, or the content-length included in this data does not exceed the self-protection byte length threshold, then based on the sub-request, the data blocks corresponding to the actual data content of the response are obtained one by one and added to the response body, and the sub-request is processed according to the judgment result of whether the current byte length of the response body exceeds the self-protection byte length threshold. The use of this method can avoid the generation of core files caused by abnormal termination of request processing, maintain system storage space, ensure the operability of the system, and thus improve the stability of the system.

Description

Technical Field

[0001] The present application relates to the technical field of CDN (Content Delivery Network), and in particular to a request processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In today's rapidly developing internet, CDNs are becoming increasingly important as a key technology for improving website access speeds and reducing the pressure on origin servers. By caching static and dynamic content across multiple nodes distributed globally, CDNs enable users to access the resources they need locally, significantly improving access speed and user experience. Subrequests play a crucial role in CDN architectures. Subrequests are additional requests sent by CDN nodes to origin servers to ensure timely access to content even when there's a cache miss, improving retrieval efficiency and accuracy.

[0003] Nginx (Engine X) is a web server and reverse proxy software widely used in CDN systems. It is often combined with Lua (a lightweight, efficient scripting language) to further expand its functionality and enhance its flexibility. In this combination of Nginx and Lua, subrequests typically receive data in a streaming manner, then aggregate this data into a complete response before passing it to the main request.

[0004] However, this approach has limitations when processing large files. Specifically, because the underlying Lua parsing library (such as luagit) used by Nginx has a maximum length limit for string concatenation, when the file content received by a subrequest exceeds this threshold, the underlying layer cannot handle the oversized string, resulting in a memory overflow or program crash. This situation causes the system to terminate abnormally and generate core files, which occupy a large amount of disk space. If a large number of core files are generated frequently, the server's disk space may be quickly exhausted, resulting in reduced system performance, impaired stability, and even, in extreme cases, a system crash. Although existing CDN technologies provide certain caching and load balancing strategies, they often lack sophisticated control mechanisms when processing large file subrequests, which poses a certain risk of stability and performance degradation to CDN systems. Summary of the Invention

[0005] Based on this, it is necessary to provide a request processing method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve the stability of the CDN system in response to the above technical problems.

[0006] In a first aspect, the present application provides a request processing method, applying any current CDN node in a CDN system, comprising:

[0007] The current CDN node receives the main request initiated by the user and obtains self-protection variable information;

[0008] When the self-protection variable information indicates that the self-protection mechanism is enabled, obtaining target file type information of the main request;

[0009] In a case where the target file type information is different from any file type information in a preset file type information list, initiating a sub-request corresponding to the main request, and obtaining response metadata according to the sub-request;

[0010] If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then sequentially obtain data blocks corresponding to the response data content based on the sub-request and add the data blocks to the response body;

[0011] The sub-request is processed according to a determination result of whether the current byte length of the response body exceeds a preset self-preservation byte length threshold.

[0012] In one embodiment, processing the sub-request based on a determination result of whether the current byte length of the response body exceeds a preset self-preservation byte length threshold includes:

[0013] If the judgment result indicates that the current byte length of the response body is greater than the self-preservation byte length threshold, terminating the processing of the sub-request, and obtaining corresponding response data based on the main request and returning the response data to the user terminal;

[0014] If the judgment result indicates that the current byte length of the response body is less than or equal to the self-preservation byte length threshold, returning to execute sequentially obtaining data blocks corresponding to the response data content according to the sub-request, and adding the data blocks to the response body to obtain a new response body, until all data blocks corresponding to the response data content are obtained;

[0015] Based on the response metadata and the new response body, response data corresponding to the sub-request is obtained, and the response data is processed according to the main request to obtain updated response data and return it to the user terminal.

[0016] In one embodiment, obtaining corresponding response data for the sub-request based on the response metadata and the new response body includes:

[0017] The response metadata and the new response body are concatenated to obtain response data corresponding to the sub-request.

[0018] In one embodiment, the method further comprises:

[0019] If the content-length included in the response metadata is greater than the self-preservation byte length threshold, the processing of the sub-request is terminated, and the corresponding response data is obtained based on the main request and returned to the user terminal.

[0020] In an exemplary embodiment, the method further comprises:

[0021] When the self-protection variable information indicates that the self-protection mechanism is not enabled, obtaining corresponding response data according to the main request and returning it to the user terminal;

[0022] or

[0023] Initiate a sub-request corresponding to the main request, obtain corresponding response data according to the sub-request, and do not perform any length monitoring on the sub-request process.

[0024] In one embodiment, the method further comprises:

[0025] In a case where the target file type information is identical to any file type information in the file type information list, corresponding response data is obtained according to the main request and returned to the user terminal.

[0026] In a second aspect, the present application further provides a request processing device, which is applied to any current CDN node in a CDN system, comprising:

[0027] The request receiving module is used by the current CDN node to receive the main request initiated by the user and obtain the self-protection variable information;

[0028] The file type identification and scheduling module is used to obtain the target file type information of the main request when the self-preservation variable information indicates that the self-preservation mechanism is enabled; if the target file type information is different from any file type information in the pre-set file type information list, initiate a sub-request corresponding to the main request and obtain response metadata based on the sub-request;

[0029] A file size prediction module is used to obtain data blocks corresponding to the response data content based on sub-requests and add the data blocks to the response body if the response metadata does not include content-length, or if the content-length included in the response metadata is less than a preset self-preservation byte length threshold;

[0030] The real-time monitoring and interruption module is used to process the sub-request based on the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold.

[0031] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] The current CDN node receives the main request initiated by the user and obtains the self-protection variable information;

[0033] When the self-preservation variable information indicates that the self-preservation mechanism is enabled, obtaining the target file type information of the main request;

[0034] In the case that the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request;

[0035] If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then the data blocks corresponding to the response data content are obtained one by one based on the sub-requests and added to the response body;

[0036] Process the subrequest based on whether the current byte length of the response body exceeds the preset self-preservation byte length threshold.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0038] The current CDN node receives the main request initiated by the user and obtains the self-protection variable information;

[0039] When the self-preservation variable information indicates that the self-preservation mechanism is enabled, obtaining the target file type information of the main request;

[0040] In the case that the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request;

[0041] If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then the data blocks corresponding to the response data content are obtained one by one based on the sub-requests and added to the response body;

[0042] Process the subrequest based on whether the current byte length of the response body exceeds the preset self-preservation byte length threshold.

[0043] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps:

[0044] The current CDN node receives the main request initiated by the user and obtains the self-protection variable information;

[0045] When the self-preservation variable information indicates that the self-preservation mechanism is enabled, obtaining the target file type information of the main request;

[0046] In the case that the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request;

[0047] If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then the data blocks corresponding to the response data content are obtained one by one based on the sub-requests and added to the response body;

[0048] Process the subrequest based on whether the current byte length of the response body exceeds the preset self-preservation byte length threshold.

[0049] The above-mentioned request processing method, device, computer equipment, computer-readable storage medium and computer program product, the current CDN node receives the main request initiated by the user and obtains self-preservation variable information. When the self-preservation variable information indicates that the self-preservation mechanism is turned on, the target file type information of the main request is obtained. When the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request. If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, the data blocks corresponding to the actual data content of the response are obtained one by one based on the sub-request, and the data blocks are added to the response body. Finally, the sub-request is processed according to the judgment result of whether the current byte length of the response body exceeds the preset self-preservation byte length threshold. By adding a self-protection mechanism to the CDN node, a preliminary judgment is first made on whether to enable the self-protection mechanism. If it is enabled, the target file type information corresponding to the main request is further obtained, and the target file type information is judged again. If the judgment is passed, a sub-request corresponding to the main request is initiated, and a further threshold comparison is made based on the response metadata and response body obtained from the sub-request. Once the conditions for continuing the sub-request are not met, the execution is stopped immediately. By adding multiple judgments, the system is effectively avoided from being abnormally terminated due to excessive response data when directly processing the sub-request, generating a large number of core files that occupy system space, thereby causing system crashes, thereby enhancing the operational stability of the CDN system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A diagram of an application environment of a request processing method in one embodiment;

[0052] Figure 2 A flowchart of a request processing method in one embodiment;

[0053] Figure 3 A schematic diagram of a process for processing a main request and a sub-request in one embodiment;

[0054] Figure 4 is a flowchart of a request processing method in another embodiment;

[0055] Figure 5is a structural block diagram of a request processing device in one embodiment;

[0056] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] The request processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, a browser 102 communicates with a CDN node i104 via a network. The CDN system consists of multiple CDN nodes, which cache static content such as images, videos, CSS, and JavaScript files, enabling rapid responses to file retrieval requests initiated by users through browsers. Any current CDN node i104 receives a subrequest, including a main request and a subrequest, initiated by a user through a browser. Upon receiving the subrequest, it can generate and retrieve corresponding self-preservation variable information, or directly retrieve pre-set self-preservation variable information from the CDN node i104. If the self-preservation variable information indicates that the node's self-preservation mechanism is enabled, it retrieves the target file type information intended for the subrequest. If the target file type information differs from any file type information in a pre-set file type information list, it sequentially retrieves data blocks corresponding to the actual response data content based on the subrequest and adds the data blocks to the response body. Finally, based on a determination of whether the current byte length of the response body exceeds a pre-set self-preservation byte length threshold, it processes the subrequest and the main request. The CDN node 104 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0059] In an exemplary embodiment, Figure 2 As shown, a request processing method is provided, which is applied to Figure 1 Taking the CDN node i104 in FIG. 1 as an example, the method includes the following steps S201 to S205. In which:

[0060] Step S201: The current CDN node receives a main request initiated by a user and obtains self-protection variable information.

[0061] Among them, the main request can be understood as a resource acquisition request initiated by a user to the CDN node, such as clicking a browser request to display a website page, play music or play a video, etc. The self-protection request variable can be understood as the start switch of the self-protection mechanism pre-set in the CDN node. It can be generated according to the request and the start switch after the CDN node receives the request, or it can be variable information pre-set in the node.

[0062] In an exemplary embodiment, CDN node i104 receives a main request initiated by a user after accessing browser 102 and obtains self-preservation variable information pre-set in CDN node i104, or obtains self-preservation variable information generated by a triggering mechanism after CDN node i104 receives the main request. The former is associated with data pre-set by the CDN node itself, while the latter is related to both the CDN node itself and the received main request. After receiving the main request, the CDN node does not immediately respond to the main request, providing buffering to avoid an immediate response that would result in the main request generating a subrequest. The subrequest's intended file size would be too large, resulting in unnecessary core files occupying system space. Acquiring self-preservation variable information provides a basis for determining whether to activate the self-preservation mechanism.

[0063] Step S202 : when the self-protection variable information indicates that the self-protection mechanism is enabled, obtain the target file type information of the main request.

[0064] Among them, the self-protection mechanism can be understood as a mechanism for determining whether the CDN node handles sub-requests normally. It can be understood as a judgment and monitoring mechanism for whether to generate sub-requests and the process of processing sub-requests. Once the conditions for interruption processing are triggered, the sub-request processing will be immediately terminated to protect the system space of the CDN system. The target file type information can be understood as the type of file corresponding to the sub-request to be obtained, such as doc, docx, or mp3.

[0065] For example, when the CDN node i104 determines that the self-protection variable information indicates that the node's self-protection mechanism is enabled, it further obtains the target file type of the main request, laying a data foundation for whether the CDN node i104 generates a sub-request corresponding to the main request. Compared with directly generating a sub-request for processing, the present application has an additional sub-request generation and judgment process, reducing the unnecessary cost of sub-request generation and processing.

[0066] Step S203: When the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request.

[0067] Among them, the file type information list can be understood as the file type information corresponding to large files that do not generate sub-requests. The sub-request can be understood as corresponding to the main request, which is an additional request issued by the CDN node to the source server, and is the operation performed by the CDN node to speed up the request processing; the response metadata can be understood as the resource header information of the resource requested by the sub-request. The resource header information user describes the actual content of the resource and can distinguish different response resources, such as HTTP response data. The response metadata can be understood as additional information related to the HTTP response. This information is not directly included in the response body, but describes or explains the characteristics, status and processing methods of the response body.

[0068] In one exemplary embodiment, if the target file type information differs from any file type information in a pre-set file type information list, meaning the file to be retrieved by the generated subrequest is not a large file that could cause the response to terminate abnormally, the abnormal termination occurs because the CDN node uses Nginx. Because the underlying Lua parsing library (e.g., luagit) used by Nginx has a maximum length limit for string concatenation, when the file content received by the subrequest exceeds this threshold, the underlying library cannot handle the oversized string, causing a memory overflow or program crash. Since the file is not a large file, meaning the file to be retrieved by the current subrequest is unlikely to cause a program crash, a subrequest corresponding to the main request is initiated, and response metadata is first obtained based on the subrequest. Compared to directly obtaining the response data for the subrequest, obtaining the response metadata first avoids the problem of excessive data content in the subrequest, which would generate a large number of core files occupying system space and causing system crashes. This also lays the data foundation for subsequent file size prediction. Furthermore, by obtaining the response metadata first, the CDN node can evaluate and determine the availability and appropriateness of the response without directly processing the large response body. This optimization reduces bandwidth and computing resource consumption.

[0069] Step S204: If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then the data blocks corresponding to the response data content are obtained one by one based on the sub-requests, and the data blocks are added to the response body.

[0070] The CL value (content-length) can be understood as an HTTP header field that indicates the byte length of the message body in a request or response. The self-preservation byte length threshold can be understood as a pre-set value that is smaller than the maximum string concatenation threshold limited by Luagit's underlying layer. Avoiding setting it equal to this value is to allow the system room for error, thereby reducing the risk of terminating the processing of the subrequest and generating a core file when an equal value occurs.

[0071] For example, self-protection mechanisms include:

[0072] 1. When the CL value included in the response metadata is less than or equal to the self-preservation byte length threshold, or when the response metadata does not include a CL value, CDN node i104 continues to respond to the sub-request, sequentially acquiring the data blocks corresponding to the first portion of resources based on the sub-request. This means streaming the data blocks corresponding to the first portion of resources and gradually accumulating the acquired data blocks to form the response body. In the absence of a clear Content-Length value or when the CL value is less than the self-preservation byte threshold, the CDN can better manage unstable network conditions by acquiring data block by block. If the previous request encounters a delay or error, the CDN can make effective adjustments and remediation based on the received data blocks without having to re-acquire the entire response body. This approach also enables real-time monitoring, thereby avoiding system crashes caused by acquiring large numbers of files and improving system stability.

[0073] 2. If the response metadata includes a CL value and that CL value is greater than the self-protection byte length threshold, the step of streaming the data blocks corresponding to the response data content is skipped. Instead, the corresponding response data is retrieved directly from the source based on the main request and fed back to the user. If the CL (Content-Length) value in the response metadata exceeds the custom byte length threshold, this may indicate a potential data content anomaly or a malicious attack risk. By terminating the processing of subrequests, CDN nodes can effectively prevent security risks caused by abnormally large data packets. Retrieving response data directly from the source based on the main request, rather than processing subrequests, reduces unnecessary processing and storage resource consumption. This decision reduces the burden on CDN nodes, improves system operational efficiency, and promptly feeds response data back to users, ensuring timely responses.

[0074] Step S205 : Process the sub-request based on the result of determining whether the current byte length of the response body exceeds a preset self-preservation byte length threshold.

[0075] In an exemplary embodiment, CDN node i104 processes the sub-request based on whether the current byte length of the response body exceeds a preset self-preservation byte length threshold. This includes the following two processing methods:

[0076] When the judgment result indicates that the current byte length of the response body is greater than the preset self-protection byte length threshold, the processing of the sub-request is terminated, the transmission of the data block is stopped, and the corresponding response data is obtained according to the main request and fed back to the user;

[0077] When the judgment result indicates that the current byte length of the response body is less than or equal to the preset self-preservation byte length threshold, the sub-request is processed to obtain the corresponding response data, and then the response is modified based on the main request initiated by the user to the CDN node to adapt to the user's needs, and the modified response data is fed back to the user.

[0078] By setting an appropriate self-preservation byte length threshold, once the size of the response body exceeds this threshold, data block transmission and sub-request processing are immediately stopped to prevent the response body from continuing to grow and avoid system crashes due to resource exhaustion; and sub-requests are continued to be processed if the size does not exceed the threshold, ensuring normal responses to small requests. Regardless of the above situations, the CDN node will promptly feedback the acquired resources to the user, ensuring the user's service experience.

[0079] In the above request processing method, the current CDN node receives the main request initiated by the user and obtains the self-preservation variable information. When the self-preservation variable information indicates that the self-preservation mechanism is turned on, the current CDN node obtains the target file type information of the main request. When the target file type information is different from any file type information in the preset file type information list, the current CDN node initiates a sub-request corresponding to the main request and obtains the response metadata according to the sub-request. If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, the current CDN node obtains the data blocks corresponding to the actual data content of the response one by one based on the sub-request, and adds the data blocks to the response body. Finally, the current sub-request is processed based on the judgment result of whether the current byte length of the response body exceeds the preset self-preservation byte length threshold. By adding a self-protection mechanism to the CDN node, a preliminary judgment is first made on whether to enable the self-protection mechanism. If it is enabled, the target file type information corresponding to the main request is further obtained, and the target file type information is judged again. If the judgment is passed, a sub-request corresponding to the main request is initiated, and a further threshold comparison is made based on the response metadata and response body obtained from the sub-request. Once the conditions for continuing the sub-request are not met, the execution is stopped immediately. By adding multiple judgments, the system is effectively avoided from being abnormally terminated due to excessive response data when directly processing the sub-request, generating a large number of core files that occupy system space, thereby causing system crashes, thereby enhancing the operational stability of the CDN system.

[0080] In an exemplary embodiment, Figure 3 As shown, based on the judgment result of whether the current byte length of the response body exceeds the preset self-preservation byte length threshold, the sub-request is processed, including step S301a, step S301b and step S302b.

[0081] Step S301a: when the judgment result indicates that the current byte length of the response body is greater than the self-preservation byte length threshold, the processing of the sub-request is terminated, and the corresponding response data is obtained based on the main request and returned to the user terminal.

[0082] For example, if the judgment result indicates that the current byte length of the response body is greater than the self-preservation byte length threshold, that is, the response body size obtained by the accumulation of data blocks has exceeded the limit for normal system operation, CDN node i104 can immediately stop transmitting the data blocks and terminate processing of the sub-requests due to the streaming transmission method. Based on the back-source retrieval of the main request, the retrieved resources are returned to the user. When the response body size exceeds the limit, the data block transmission is terminated, avoiding unnecessary resource waste, memory overflow, and system crash. After terminating the sub-requests, the main request processing is immediately switched to ensure timely feedback on user requests and effectively improve the user service experience.

[0083] Step S301b: If the judgment result indicates that the current byte length of the response body is less than or equal to the self-preservation byte length threshold, return to execute the process of obtaining the data blocks corresponding to the response data content one by one according to the sub-requests, and add the data blocks to the response body to obtain a new response body until all the data blocks corresponding to the response data content are obtained.

[0084] Step S302b: Based on the response metadata and the new response body, the response data corresponding to the sub-request is obtained, and the response data is processed according to the main request to obtain updated response data and return it to the user terminal.

[0085] In an exemplary embodiment, if the judgment result indicates that the current byte length of the response body is less than or equal to the self-preservation byte length threshold, the process returns to sequentially acquiring data blocks corresponding to the response data content according to the sub-request, i.e., continuing to acquire data blocks and appending them to the response body to obtain a new response body until the current byte length of the response body exceeds the self-preservation byte length threshold, or all data blocks corresponding to the response data content are acquired. At this point, the data block acquisition loop is exited. If the loop is exited in the former manner, the processing is the same as step S301a. If the loop is exited in the latter manner, the resource corresponding to the sub-request is obtained using the response metadata and the new response body, and the resource is further updated according to the main request and fed back to the user. When the byte length does not exceed the threshold, the response data content can be continuously acquired, ensuring that the user can receive the requested data more quickly. Through streaming monitoring, the size of the response body is determined in real time during the resource acquisition process, effectively avoiding the problem of memory overflow and system crash caused by excessive file size. Furthermore, by splicing the response metadata and the new response body to form a complete response data, and updating the response data according to the main request, the user's feedback experience is improved.

[0086] In one embodiment, obtaining corresponding response data of the sub-request based on the response metadata and the new response body includes: concatenating the response metadata and the new response body to obtain the response data corresponding to the sub-request.

[0087] For example, after obtaining all data blocks corresponding to the response data content to jump out of the loop mode, CDN node i104 splices the response metadata and the new response body to form complete response data, and uses this complete response data as the response data corresponding to the sub-request. Combining the metadata with the response body helps to ensure the consistency and integrity of the data, allowing users to obtain all the information they need in one request, reducing data inconsistencies that may be caused by multiple requests.

[0088] In one embodiment, the method further includes: if the content-length included in the response metadata is greater than the self-preservation byte length threshold, terminating the processing of the sub-request, and obtaining the corresponding response data based on the main request and returning it to the user terminal.

[0089] Optionally, if the response metadata contains a CL value but the CL value exceeds the self-protection byte length threshold, CDN node i104 terminates processing the subrequest and does not proceed with the step of streaming the corresponding data block of the response data content. Instead, it directly retrieves the corresponding response data based on the main request and feeds this response data back to the user. This process is also part of the CDN node's self-protection mechanism. If the CL (Content-Length) value in the response metadata exceeds the custom byte length threshold, this may indicate potential data anomalies or malicious attack risks. By terminating subrequest processing, the CDN node can effectively prevent security risks caused by abnormally large data packets. Retrieving response data directly from the source based on the main request, rather than processing subrequests, reduces unnecessary processing and storage resource consumption. This decision reduces the burden on CDN nodes, improves system efficiency, and ensures timely feedback of response data to users.

[0090] In one embodiment, the method further includes obtaining corresponding response data according to the main request and returning it to the user terminal when the self-protection variable information indicates that the self-protection mechanism is not enabled; or, initiating a sub-request corresponding to the main request, obtaining corresponding response data according to the sub-request, and not performing any length monitoring on the sub-request process.

[0091] For example, when the self-protection variable information indicates that the self-protection mechanism is not enabled, the CDN node i104 directly returns to the source based on the main request to obtain the corresponding response data, and feeds this response data back to the user, or initiates a sub-request corresponding to the main request, and obtains the corresponding response data based on the sub-request. It can be seen that when the self-protection mechanism is not enabled, the CDN node i104 can directly process the main request without initiating a sub-request, or it can initiate a sub-request and process the sub-request, but will not perform any length monitoring on the sub-request process. In other words, under this request, the system may or may not generate redundant core files. Even if the CDN node determines that the self-protection mechanism is not enabled, it will respond to the request initiated by the user in a timely manner and return resources if the main request is successfully responded to. If the sub-request is responded to and the response fails, the system will also promptly feedback to the user, avoiding wasting the user's useless waiting time and improving the user's service experience.

[0092] In one embodiment, the method further includes: when the target file type information is the same as any file type information in the file type information list, obtaining corresponding response data according to the main request and returning the response data to the user terminal.

[0093] For example, when the target file type information is the same as any file type information in the file type information list, that is, when the file to be obtained by the current sub-request is a large file that may cause the system to crash, a sub-request corresponding to the main request is not initiated, and the corresponding response data is directly obtained and fed back to the user based on the main request. Before entering the request processing, the file type of the file to be obtained by the request is identified to avoid the CDN node wasting extra resources initiating a sub-request corresponding to the main request that cannot respond normally when the type of file to be obtained by the main request is a large file, thereby reducing operating costs. Directly responding to the main request and feeding back the response data also ensures the timeliness of the response and improves the user's service experience.

[0094] In an exemplary embodiment, Figure 4 As shown, a specific implementation of a request processing method is provided, wherein:

[0095] A Content Delivery Network (CDN) is a distributed network architecture used to improve the speed and reliability of delivering websites and other online content. By caching static and dynamic content across multiple geographically distributed servers, CDNs enable users to access content from the server closest to them, reducing latency and load, and improving the user experience.

[0096] Upstream rewrite phase: In Nginx's request processing, the upstream rewrite phase occurs before a request is forwarded to the upstream server. It modifies the request URI by applying rewrite directives to align the request with the upstream server's routing requirements. This process ensures that requests are correctly mapped to the target upstream server, adapting to varying service requirements or load balancing strategies.

[0097] Head filter phase: In Nginx's response processing, the head filter phase focuses on processing HTTP response headers. This phase allows headers to be added, removed, or modified to meet client requirements or for security or compatibility adjustments. This phase also enables operations such as setting caching policies and adjusting content types.

[0098] Body filter phase: In Nginx's response processing, the body filter phase involves processing the HTTP response body. This phase allows filtering, modification, or compression of the response body to ensure that the data returned to the client meets expectations. For example, this phase can be used to compress content, replace certain data segments, or add additional information.

[0099] Content-Length: This is an HTTP header field that indicates the byte length of the message body in a request or response. It tells the receiver to stop reading the message body after reading the specified number of bytes. This field helps determine data integrity and boundaries when transmitting content.

[0100] ngx.ctx: ngx.ctx is a table in Nginx's Lua module used to store context data for the current request, which is available throughout the life cycle of the request, including its subrequests.

[0101] Secondly, when officially entering the request processing method, you first need to do the following preparations:

[0102] Step 0: Configuration and Initialization. First, a self-protection threshold is preset at the bottom layer, for example, 10M. This value can be selected based on the actual application scenario. Second, a variable ngx.ctx.sub_protect is set to control whether the self-protection mechanism is enabled. An oversize flag ngx.ctx.oversize=flase is set in ngx.ctx to indicate whether the requested file size exceeds the self-protection threshold. Finally, a list of suffixes that do not require subrequest back-to-source is defined at the bottom layer of nginx. For example, for page rewriting functions, there is no need to rewrite audio, video files, and images, so the following configuration can be configured:

[0103] unsupport_type_default={["png"]=1,["mp4"]=1,["jpg"]=1,["mp3"]=1}

[0104] Formal entry request processing:

[0105] Step 1: Request type check and dispatch. For file requests that reach the CDN node, the upstream rewrite phase determines whether the self-protection mechanism is enabled. If not, the main request is made directly back to the source to obtain the resource. If it is enabled, the suffix is ​​extracted to determine whether the file suffix is ​​in the configuration, that is, whether unsupport_type[suffix] is equal to 1. If it is, it indicates that the suffix is ​​in the list, and the main request is made directly back to the source to obtain the resource. Otherwise, a subrequest is initiated, and the process proceeds to step 3.

[0106] Step 2: Predicting the file size. When the subrequest reaches the head filter stage, extract the value of the content-length in its response header, convert the unit of the self-preservation threshold to byte length, and then compare it with the self-preservation threshold. If the content-length is equal to or greater than the self-preservation threshold, set the value of ngx.ctx.oversize to true and terminate the subrequest. The main request reads the value of ngx.ctx.oversize. If it is true, it will automatically return to the source to obtain the file. If the content-length is less than the self-preservation threshold or the response header does not contain content-length, continue the subrequest and proceed to step 3.

[0107] Step 3: Streaming monitoring. During the body filter phase, the subrequest receives the response body. Each time a data block is received, the response body size is cumulatively updated and a check is performed to determine whether the value exceeds the self-protection threshold. If so, data transmission is interrupted, the transmitted data is cleared, the value of ngx.ctx.oversize is set to true, and the subrequest is exited. The main request reads the value of ngx.ctx.oversize. If it is true, it automatically returns to the source to obtain the file; otherwise, the subrequest process continues and enters step 4.

[0108] Step 4: Data Delivery and Response. Once the subrequest receives all the data and its size does not exceed the threshold, the data is concatenated into a complete response and delivered to the main request. The main request first reads the value of ngx.ctx.oversize. If it is false, indicating that the subrequest response is valid, the data is finally processed and the response is returned to the client.

[0109] Compared with the existing technology, this application has the following advantages:

[0110] 1. Reduce the risk of system failure: Introducing a type identification mechanism in CDN nodes, flexibly scheduling sub-requests and main requests based on file suffixes, avoiding unnecessary sub-request processing, reducing the risk of generating core files, and thus reducing the possibility of system crashes.

[0111] 2. Optimize system performance: Early warning and interruption of sub-requests for overly large files at the beginning of sub-request processing reduce disk space usage and improve overall system performance.

[0112] 3. Improve user experience: For files that are transmitted in chunks or have inaccurate Content-Length, we implement streaming transmission monitoring to provide timely warnings and interrupt sub-requests for overly large files, avoiding the generation of core files. This ensures the stability and rapid response of request processing and improves the user experience.

[0113] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0114] Based on the same inventive concept, embodiments of the present application also provide a request processing device for implementing the aforementioned request processing method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations in one or more request processing device embodiments provided below can be found in the above-mentioned limitations on the request processing method and will not be further elaborated here.

[0115] In an exemplary embodiment, Figure 5 As shown, a request processing device is provided, which is applied to any current CDN node in the CDN system, including: a request receiving module 501, a file type identification and scheduling module 502, a file size prediction module 503 and a real-time monitoring and interruption module 504, wherein:

[0116] The request receiving module 501 is used for the current CDN node to receive the main request initiated by the user and obtain the self-protection variable information;

[0117] The file type identification and dispatching module 502 is configured to obtain target file type information of the main request when the self-preservation variable information indicates that the self-preservation mechanism is enabled; if the target file type information is different from any file type information in a preset file type information list, initiate a sub-request corresponding to the main request and obtain response metadata based on the sub-request;

[0118] The file size prediction module 503 is configured to, if the response metadata does not include a content-length parameter, or if the content-length parameter included in the response metadata is less than a preset self-preservation byte length threshold, sequentially obtain data blocks corresponding to the response data content based on the sub-requests and add the data blocks to the response body;

[0119] The real-time monitoring and interruption module 504 is used to process the sub-request according to the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold.

[0120] In one embodiment, the real-time monitoring and interruption module 504 is also used to terminate the processing of the sub-request and obtain the corresponding response data based on the main request and return it to the user terminal when the judgment result indicates that the current byte length of the response body is greater than the self-protection byte length threshold; if the judgment result indicates that the current byte length of the response body is less than or equal to the self-protection byte length threshold, return to execute the data blocks corresponding to the response data content according to the sub-request one by one, and add the data blocks to the response body to obtain a new response body until all the data blocks corresponding to the response data content are obtained; based on the response metadata and the new response body, obtain the response data corresponding to the sub-request, process the response data according to the main request, obtain the updated response data and return it to the user terminal.

[0121] In one embodiment, the real-time monitoring and interruption module is further used to splice the response metadata and the new response body to obtain response data corresponding to the sub-request.

[0122] In an exemplary embodiment, the request processing device also includes a module for terminating the processing of the sub-request, which is used to terminate the processing of the sub-request if the content-length included in the response metadata is greater than the self-protection byte length threshold, and obtain the corresponding response data based on the main request and return it to the user terminal.

[0123] In an exemplary embodiment, the request processing device further includes a non-self-protection request processing module, which is used to obtain corresponding response data according to the main request and return it to the user terminal when the self-protection variable information indicates that the self-protection mechanism is not enabled;

[0124] or

[0125] Initiate a subrequest corresponding to the main request, obtain the corresponding response data based on the subrequest, and do not monitor the length of the subrequest process.

[0126] In one embodiment, the sub-request processing module is terminated and further configured to obtain corresponding response data according to the main request and return the response data to the user terminal when the target file type information is the same as any file type information in the file type information list.

[0127] Each module in the request processing device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0128] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, memory and I / O interface are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store the main request, file type information, response metadata, response data content and self-protection byte length threshold. The I / O interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a request processing method is implemented.

[0129] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0130] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the request processing method of the above embodiment when executing the computer program.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the request processing method of the above embodiment is implemented.

[0132] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the request processing method of the above embodiment is implemented.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0134] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0135] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0136] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A request processing method, characterized in that: Using any current CDN node in the CDN system, the method includes: The current CDN node receives the main request initiated by the user and obtains self-protection variable information; When the self-protection variable information indicates that the self-protection mechanism is enabled, obtaining target file type information of the main request; In a case where the target file type information is different from any file type information in a preset file type information list, initiating a sub-request corresponding to the main request, and obtaining response metadata according to the sub-request; If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then sequentially obtain data blocks corresponding to the response data content based on the sub-request and add the data blocks to the response body; The sub-request is processed according to a determination result of whether the current byte length of the response body exceeds a preset self-preservation byte length threshold.

2. The method according to claim 1, characterized in that The processing of the sub-request based on the result of determining whether the current byte length of the response body exceeds a preset self-preservation byte length threshold includes: If the judgment result indicates that the current byte length of the response body is greater than the self-preservation byte length threshold, terminating the processing of the sub-request, and obtaining corresponding response data based on the main request and returning the response data to the user terminal; If the judgment result indicates that the current byte length of the response body is less than or equal to the self-preservation byte length threshold, returning to execute sequentially obtaining data blocks corresponding to the response data content according to the sub-request, and adding the data blocks to the response body to obtain a new response body, until all data blocks corresponding to the response data content are obtained; Based on the response metadata and the new response body, response data corresponding to the sub-request is obtained, and the response data is processed according to the main request to obtain updated response data and return it to the user terminal.

3. The method according to claim 2, characterized in that Obtaining corresponding response data for the sub-request based on the response metadata and the new response body includes: The response metadata and the new response body are concatenated to obtain response data corresponding to the sub-request.

4. The method according to claim 1, wherein The method further comprises: If the content-length included in the response metadata is greater than the self-preservation byte length threshold, the processing of the sub-request is terminated, and the corresponding response data is obtained based on the main request and returned to the user terminal.

5. The method according to claim 1, wherein The method further includes When the self-protection variable information indicates that the self-protection mechanism is not enabled, obtaining corresponding response data according to the main request and returning it to the user terminal; or Initiate a sub-request corresponding to the main request, obtain corresponding response data according to the sub-request, and do not perform any length monitoring on the sub-request process.

6. The method according to claim 1, characterized in that The method further comprises: In a case where the target file type information is identical to any file type information in the file type information list, corresponding response data is obtained according to the main request and returned to the user terminal.

7. A request processing device, characterized in that: Applied to any current CDN node in the CDN system, the device includes: A request receiving module, configured for the current CDN node to receive a main request initiated by a user and obtain self-protection variable information; A file type identification and scheduling module is configured to obtain target file type information of the main request when the self-preservation variable information indicates that the self-preservation mechanism is enabled; initiate a sub-request corresponding to the main request when the target file type information is different from any file type information in a preset file type information list, and obtain response metadata based on the sub-request; A file size prediction module is configured to, if the response metadata does not include content-length, or if the content-length included in the response metadata is less than a preset self-preservation byte length threshold, successively obtain data blocks corresponding to the response data content based on the sub-request and add the data blocks to the response body; The real-time monitoring and interruption module is used to process the sub-request according to the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images