Request processing method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a request processing method and device, computer equipment, a storage medium and a program product. The method comprises the steps that a CDN node receives a main request initiated by a user and obtains self-protection variable information, under the condition that the self-protection variable information represents that a self-protection mechanism is started, a target file suffix of the main request is obtained, and under the condition that the target file suffix is different from any file suffix in a file suffix list, the self-protection variable information represents that the self-protection mechanism is started; initiating a sub-request and obtaining corresponding response metadata, and if the data does not comprise content-length or the content-length contained in the data does not exceed a self-protection byte length threshold value, gradually obtaining data blocks corresponding to the actual data content based on the sub-request and adding the data blocks into a response body, and if the data does not comprise the content-length contained in the data does not exceed the self-protection byte length threshold value, gradually obtaining the data blocks corresponding to the actual data content based on the sub-request and adding the data blocks into the response body. And processing the sub-request according to a judgment result of whether the current byte length of the response body exceeds the self-protection byte length threshold value or not. By adopting the method, the core file which requests to be abnormally terminated in processing can be prevented from being generated, the storage space of the system is maintained, the operability of the system is ensured, and the stability of the system is improved.

Description

Technical Field

[0001] The present application relates to the technical field of CDN (Content Delivery Network), and in particular to a request processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In today's context of rapid Internet development, CDN is becoming increasingly important as a key technology to improve website access speed and reduce the pressure on source servers. CDN caches static and dynamic content on multiple nodes distributed around the world, allowing users to obtain the required resources nearby, thereby significantly improving access speed and user experience. In the CDN architecture, subrequests play a vital role. Subrequests are additional requests sent by CDN nodes to source servers to ensure that users can obtain content in a timely manner when the cache misses, improving acquisition efficiency and accuracy.

[0003] Nginx (Engine X) is a web server and reverse proxy software widely used in CDN systems. It is often used in combination with Lua (a lightweight and efficient scripting language) to further expand its functionality and enhance flexibility. In the combined environment of Nginx and Lua, subrequests usually receive data in a streaming manner, then integrate the data into a complete response and pass it to the main request.

[0004] However, this method has certain limitations when processing large files. Specifically, because the underlying Lua parsing library (such as luagit) used by Nginx has a maximum length limit for string concatenation, when the file content received by the subrequest exceeds this threshold, the underlying layer cannot handle the oversized string, which will cause a memory overflow or program crash. This situation will cause the system to terminate abnormally and generate core files, which will take up a lot of disk space. If a large number of core files are generated frequently, the server's disk space may be quickly exhausted, resulting in system performance degradation, impaired stability, and even in extreme cases, system crashes. Although existing CDN technologies provide certain caching and load balancing strategies, they often lack sophisticated control mechanisms when processing large file subrequests, which makes the CDN system have certain risks of stability and performance degradation. Summary of the invention

[0005] Based on this, it is necessary to provide a request processing method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve the stability of the CDN system in response to the above technical problems.

[0006] In a first aspect, the present application provides a request processing method, using any current CDN node in a CDN system, including:

[0007] The current CDN node receives the main request initiated by the user and obtains self-protection variable information;

[0008] When the self-preservation variable information indicates that the self-preservation mechanism is enabled, obtaining target file type information of the main request;

[0009] In the case that the target file type information is different from any file type information in the preset file type information list, initiating a sub-request corresponding to the main request, and obtaining response metadata according to the sub-request;

[0010] If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then based on the sub-request, the data blocks corresponding to the response data content are obtained one by one, and the data blocks are added to the response body;

[0011] The sub-request is processed according to a determination result of whether the current byte length of the response body exceeds a preset self-preservation byte length threshold.

[0012] In one of the embodiments, the processing of the sub-request based on the result of determining whether the current byte length of the response body exceeds a preset self-preservation byte length threshold includes:

[0013] If the judgment result indicates that the current byte length of the response body is greater than the self-preservation byte length threshold, terminating the processing of the sub-request, and obtaining corresponding response data based on the main request and returning it to the user terminal;

[0014] If the judgment result indicates that the current byte length of the response body is less than or equal to the self-preservation byte length threshold, returning to execute sequentially obtaining data blocks corresponding to the response data content according to the sub-request, and adding the data blocks to the response body to obtain a new response body, until all data blocks corresponding to the response data content are obtained;

[0015] Based on the response metadata and the new response body, the response data corresponding to the sub-request is obtained, and the response data is processed according to the main request to obtain updated response data and return it to the user terminal.

[0016] In one embodiment, obtaining the corresponding response data of the sub-request based on the response metadata and the new response body includes:

[0017] The response metadata and the new response body are concatenated to obtain response data corresponding to the sub-request.

[0018] In one embodiment, the method further comprises:

[0019] If the content-length included in the response metadata is greater than the self-preservation byte length threshold, the processing of the sub-request is terminated, and the corresponding response data is obtained based on the main request and returned to the user terminal.

[0020] In an exemplary embodiment, the method further comprises:

[0021] When the self-protection variable information indicates that the self-protection mechanism is not enabled, corresponding response data is obtained according to the main request and returned to the user terminal;

[0022] or

[0023] A sub-request corresponding to the main request is initiated, and corresponding response data is obtained according to the sub-request, without any length monitoring of the sub-request process.

[0024] In one embodiment, the method further comprises:

[0025] In the case where the target file type information is the same as any file type information in the file type information list, corresponding response data is acquired according to the main request and returned to the user terminal.

[0026] In a second aspect, the present application further provides a request processing device, which is applied to any current CDN node in a CDN system, including:

[0027] The request receiving module is used for the current CDN node to receive the main request initiated by the user and obtain the self-protection variable information;

[0028] The file type identification and scheduling module is used to obtain the target file type information of the main request when the self-preservation variable information indicates that the self-preservation mechanism is turned on; when the target file type information is different from any file type information in the pre-set file type information list, initiate a sub-request corresponding to the main request, and obtain response metadata according to the sub-request;

[0029] A file size prediction module is used to obtain data blocks corresponding to the response data content one by one based on the sub-requests and add the data blocks to the response body if the response metadata does not include content-length, or the content-length included in the response metadata is less than the preset self-preservation byte length threshold;

[0030] The real-time monitoring and interruption module is used to process the sub-request according to the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold.

[0031] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0032] The current CDN node receives the main request initiated by the user and obtains the self-protection variable information;

[0033] When the self-preservation variable information indicates that the self-preservation mechanism is enabled, the target file type information of the main request is obtained;

[0034] In the case that the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request;

[0035] If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then the data blocks corresponding to the response data content are obtained one by one based on the sub-requests, and the data blocks are added to the response body;

[0036] The sub-request is processed based on the result of determining whether the current byte length of the response body exceeds the preset self-preservation byte length threshold.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0038] The current CDN node receives the main request initiated by the user and obtains the self-protection variable information;

[0039] When the self-preservation variable information indicates that the self-preservation mechanism is enabled, the target file type information of the main request is obtained;

[0040] In the case that the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request;

[0041] If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then the data blocks corresponding to the response data content are obtained one by one based on the sub-requests, and the data blocks are added to the response body;

[0042] The sub-request is processed based on the result of determining whether the current byte length of the response body exceeds the preset self-preservation byte length threshold.

[0043] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0044] The current CDN node receives the main request initiated by the user and obtains the self-protection variable information;

[0045] When the self-preservation variable information indicates that the self-preservation mechanism is enabled, the target file type information of the main request is obtained;

[0046] In the case that the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request;

[0047] If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then the data blocks corresponding to the response data content are obtained one by one based on the sub-requests, and the data blocks are added to the response body;

[0048] The sub-request is processed based on the result of determining whether the current byte length of the response body exceeds the preset self-preservation byte length threshold.

[0049] The above-mentioned request processing method, device, computer equipment, computer-readable storage medium and computer program product, the current CDN node receives the main request initiated by the user and obtains the self-preservation variable information. When the self-preservation variable information indicates that the self-preservation mechanism is turned on, the target file type information of the main request is obtained. When the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request. If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, data blocks corresponding to the actual data content of the response are obtained one by one based on the sub-request, and the data blocks are added to the response body. Finally, the sub-request is processed according to the judgment result of whether the current byte length of the response body exceeds the preset self-preservation byte length threshold. By adding a self-protection mechanism in the CDN node, a preliminary judgment is first made on whether to enable the self-protection mechanism. If it is enabled, the target file type information corresponding to the main request is further obtained, and the target file type information is judged again. If the judgment is passed, a sub-request corresponding to the main request is initiated, and a further threshold comparison is made based on the response metadata and response body obtained from the sub-request. Once the conditions for continuing the sub-request are not met, the execution is stopped immediately. By adding several layers of judgment, the system is effectively avoided from being abnormally terminated due to excessive response data when directly processing sub-requests, and a large number of core files are generated to occupy system space, thereby causing system crashes, thereby enhancing the operational stability of the CDN system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 An application environment diagram of a request processing method in an embodiment;

[0052] Figure 2 A schematic diagram of a request processing method in one embodiment;

[0053] Figure 3 A schematic diagram of a process for processing a main request and a sub-request in one embodiment;

[0054] Figure 4 A flowchart of a request processing method in another embodiment;

[0055] Figure 5is a structural block diagram of a request processing device in one embodiment;

[0056] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] The request processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the browser 102 communicates with the CDN node i104 through the network. The CDN system is composed of multiple CDN nodes, and the CDN nodes cache static content, such as images, videos, CSS and JavaScript files, etc., which can quickly respond to the file acquisition request initiated by the user through the browser. Any current CDN node i104 receives a sub-request including a main request and a sub-request initiated by the user through the browser, and can generate and obtain the corresponding self-protection variable information when the sub-request enters the CDN node i104, or directly obtain the preset self-protection variable information from the CDN node i104. In the case where the self-protection variable information represents the self-protection mechanism of the enabled node, the target file type information to be obtained by the sub-request is obtained. In the case where the target file type information is different from any file type information in the preset file type information list, the data blocks corresponding to the actual data content of the response are obtained one by one according to the sub-request, and the data blocks are added to the response body. Finally, based on the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold, the sub-request and the main request are processed. The CDN node 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0059] In an exemplary embodiment, Figure 2 As shown, a request processing method is provided, which is applied to Figure 1 Taking the CDN node i104 in the example as an example, the method includes the following steps S201 to S205. Among them:

[0060] Step S201, the current CDN node receives a main request initiated by a user, and obtains self-protection variable information.

[0061] Among them, the main request can be understood as a resource acquisition request initiated by a user to a CDN node, such as clicking a browser request to display a website page, play music or play a video, etc. The self-protection request variable can be understood as a start switch of the self-protection mechanism pre-set in the CDN node, which can be generated according to the request and the start switch after the CDN node receives the request, or it can be variable information pre-set in the node.

[0062] In an exemplary embodiment, CDN node i104 receives the main request initiated by the user after accessing the browser 102, obtains the self-preservation variable information pre-set in CDN node i104, or obtains the self-preservation variable information currently generated by the trigger mechanism after CDN node i104 receives the main request. The former is associated with the data pre-set by the CDN node itself, and the latter is related to both the CDN node itself and the received main request. After receiving the main request, it does not respond to the received main request immediately, providing buffering space to avoid immediate response resulting in the main request generating a sub-request, and the sub-request to obtain a file that is too large and generates unnecessary core files occupying system space. At the same time, the self-preservation variable information is obtained, which provides a judgment basis for determining whether to start the self-preservation mechanism in the future.

[0063] Step S202: when the self-preservation variable information indicates that the self-preservation mechanism is enabled, obtain the target file type information of the main request.

[0064] Among them, the self-protection mechanism can be understood as a mechanism for whether the CDN node processes sub-requests normally, and can be understood as a judgment and monitoring mechanism for whether to generate sub-requests and during the processing of sub-requests. Once the conditions for interruption processing are triggered, the processing of sub-requests will be immediately terminated to protect the system space of the CDN system; the target file type information can be understood as the type of file that the sub-request wants to obtain, such as doc, docx or mp3.

[0065] For example, when the CDN node i104 determines that the self-preservation variable information represents the activation of the node's self-preservation mechanism, it further obtains the target file type of the main request, laying a data foundation for whether the CDN node i104 generates a sub-request corresponding to the main request. Compared with directly generating a sub-request for processing, the present application has an additional sub-request generation and judgment process, reducing the unnecessary cost of sub-request generation and processing.

[0066] Step S203: When the target file type information is different from any file type information in the preset file type information list, a sub-request corresponding to the main request is initiated, and response metadata is obtained according to the sub-request.

[0067] Among them, the file type information list can be understood as file type information corresponding to large files that do not generate sub-requests. The sub-request can be understood as corresponding to the main request, which is an additional request issued by the CDN node to the source server, and is an operation performed by the CDN node to speed up the request processing. The response metadata can be understood as the resource header information of the resource requested by the sub-request. The resource header information user describes the actual content of the resource and can distinguish different response resources, such as HTTP response data. The response metadata can be understood as additional information related to the HTTP response. This information is not directly included in the response body, but describes or explains the characteristics, status and processing method of the response body.

[0068] In an exemplary embodiment, when the target file type information is different from any file type information in the preset file type information list, that is, the file to be obtained by the generated sub-request is not a large file that may cause the response to terminate abnormally, the abnormal termination occurs because the Nginx used by the CDN node has a maximum length limit for string concatenation because the underlying Lua parsing library (such as luagit) used by Nginx has a maximum length limit for string concatenation. When the file content received by the sub-request exceeds this threshold, the underlying layer cannot process the oversized string, which will cause memory overflow or program crash. It is not a large file, that is, the file to be obtained by the current sub-request may not cause the program to crash. A sub-request corresponding to the main request is initiated, and response metadata is first obtained according to the sub-request. Compared with directly obtaining the response data of the sub-request, the response metadata is selected to be obtained first, which avoids the problem that the data content requested by the sub-request is too much and a large number of core files are generated to occupy system space, causing the system to crash, and the subsequent file size pre-judgment lays a data foundation. At the same time, by obtaining the response metadata first, the CDN node can evaluate and judge the availability and appropriateness of the response without directly processing the large response body. This optimization reduces the consumption of bandwidth and computing resources.

[0069] Step S204: If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then the data blocks corresponding to the response data content are obtained one by one based on the sub-requests, and the data blocks are added to the response body.

[0070] The CL value (content-length) can be understood as an HTTP header field, which indicates the length of the message body in the request or response in bytes. The self-protection byte length threshold can be understood as a pre-set value that is less than the maximum string concatenation threshold of the underlying limit of Luagit. Avoiding setting it equal to this is to allow the system to have room for errors, thereby reducing the risk of terminating the processing of the subrequest and generating a core file when an equal value occurs.

[0071] For example, the self-protection mechanism includes:

[0072] 1. When the CL value included in the response metadata is less than or equal to the self-preservation byte length threshold, or the response metadata does not include the CL value, the CDN node i104 continues to respond to the sub-request, and successively obtains the data blocks corresponding to the first part of the resources according to the sub-request, that is, streams the data blocks corresponding to the first part of the resources, and gradually accumulates the obtained data blocks to form a response body. In the absence of a clear Content-Length value or the CL value is less than the self-preservation byte threshold, by obtaining data block by block, the CDN can perform better management under unstable network conditions. If the previous request encounters a delay or error, the CDN can make effective adjustments and remedies based on the received data blocks without having to re-obtain the entire response body, and this method can achieve real-time monitoring, thereby avoiding system crashes caused by obtaining a large number of files, thereby improving system stability.

[0073] 2. If the response metadata includes a CL value and the CL value is greater than the self-protection byte length threshold, the step of streaming to obtain the data block corresponding to the response data content is not entered, and the corresponding response data is directly returned to the source according to the main request, and the response data is fed back to the user. If the CL (Content-Length) value in the response metadata exceeds the custom byte length threshold, this may indicate a potential data content anomaly or malicious attack risk. By terminating the processing of sub-requests, CDN nodes can effectively prevent security risks caused by abnormally large data packets. Directly returning to the source to obtain response data based on the main request instead of processing sub-requests can reduce unnecessary processing and storage resource consumption. This decision can reduce the burden on CDN nodes, improve the operating efficiency of the system, and promptly feed back response data to users, ensuring the timeliness of the response.

[0074] Step S205 , processing the sub-request according to the result of determining whether the current byte length of the response body exceeds a preset self-preservation byte length threshold.

[0075] In an exemplary embodiment, the CDN node i104 processes the sub-request based on the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold. There are two processing methods:

[0076] When the judgment result indicates that the current byte length of the response body is greater than the preset self-protection byte length threshold, the processing of the sub-request is terminated, the transmission of the data block is stopped, and the corresponding response data is obtained according to the main request and fed back to the user;

[0077] When the judgment result indicates that the current byte length of the response body is less than or equal to the preset self-protection byte length threshold, the sub-request continues to be processed to obtain the corresponding response data, and then the response is modified based on the main request initiated by the user to the CDN node to adapt to the user's needs, and the modified response data is fed back to the user.

[0078] By setting an appropriate self-protection byte length threshold, once the size of the response body exceeds this threshold, the data block transmission and sub-request processing are stopped immediately to prevent the response body from continuing to grow and avoid the system crash due to resource exhaustion; and sub-requests will continue to be processed if it does not exceed the threshold, ensuring the normal response of small requests. In either of the above situations, the CDN node will promptly feed back the acquired resources to the user, ensuring the user's service experience.

[0079] In the above request processing method, the current CDN node receives the main request initiated by the user and obtains the self-preservation variable information. When the self-preservation variable information indicates that the self-preservation mechanism is turned on, the current CDN node obtains the target file type information of the main request. When the target file type information is different from any file type information in the preset file type information list, the current CDN node initiates a sub-request corresponding to the main request, and obtains the response metadata according to the sub-request. If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, the current CDN node obtains the data blocks corresponding to the actual data content of the response one by one based on the sub-request, and adds the data blocks to the response body. Finally, the current sub-request is processed according to the judgment result of whether the current byte length of the response body exceeds the preset self-preservation byte length threshold. By adding a self-protection mechanism in the CDN node, a preliminary judgment is first made on whether to enable the self-protection mechanism. If it is enabled, the target file type information corresponding to the main request is further obtained, and the target file type information is judged again. If the judgment is passed, a sub-request corresponding to the main request is initiated, and a further threshold comparison is made based on the response metadata and response body obtained from the sub-request. Once the conditions for continuing the sub-request are not met, the execution is stopped immediately. By adding several layers of judgment, the system is effectively avoided from being abnormally terminated due to excessive response data when directly processing sub-requests, and a large number of core files are generated to occupy system space, thereby causing system crashes, thereby enhancing the operational stability of the CDN system.

[0080] In an exemplary embodiment, Figure 3 As shown, based on the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold, the sub-request is processed, including step S301a, step S301b and step S302b. Among them:

[0081] Step S301a, when the judgment result indicates that the current byte length of the response body is greater than the self-protection byte length threshold, the processing of the sub-request is terminated, and the corresponding response data is obtained based on the main request and returned to the user terminal.

[0082] For example, when the judgment result indicates that the current byte length of the response body is greater than the self-protection byte length threshold, that is, the response body size obtained by the accumulation of data blocks has exceeded the limit for the normal operation of the system, since it is a streaming transmission method, CDN node i104 can immediately stop the transmission of data blocks, terminate the processing of sub-requests, and return the resources obtained by returning to the source to the user based on the return to the source of the main request. When the size of the response body exceeds the limit, the data block transmission is terminated to avoid unnecessary resource waste, memory overflow and system crash. After terminating the sub-request, it immediately turns to the processing of the main request, ensuring timely feedback of user requests and effectively improving the user's service experience.

[0083] Step S301b, if the judgment result indicates that the current byte length of the response body is less than or equal to the self-protection byte length threshold, return to execute to obtain the data blocks corresponding to the response data content one by one according to the sub-requests, and add the data blocks to the response body to obtain a new response body, until all the data blocks corresponding to the response data content are obtained.

[0084] Step S302b, based on the response metadata and the new response body, obtain the response data corresponding to the sub-request, and process the response data according to the main request to obtain the updated response data and return it to the user terminal.

[0085] In an exemplary embodiment, if the judgment result indicates that the current byte length of the response body is less than or equal to the self-protection byte length threshold, the process returns to execute the process of obtaining the data blocks corresponding to the response data content one by one according to the sub-request, that is, continuing to obtain the data blocks, and adding the data blocks to the response body to obtain a new response body, until the current byte length of the response body is greater than the self-protection byte length threshold, or the data blocks corresponding to the response data content are all obtained, then the loop of data block acquisition is jumped out, if the loop is jumped out in the former way, the processing method is the same as step S301a, if the loop is jumped out in the latter way, the resource corresponding to the sub-request is obtained by means of the response metadata and the new response body, and the resource is further updated according to the main request and fed back to the user. When the byte length does not exceed the threshold, the response data content can be continuously obtained to ensure that the user can receive the requested data faster, and the size of the response body can be judged in real time during the resource acquisition process by means of streaming monitoring, effectively avoiding the problem of memory overflow and system crash caused by excessive file size, and forming complete response data by splicing the response metadata and the new response body, and updating the response data according to the main request, thereby improving the user's feedback experience.

[0086] In one embodiment, obtaining corresponding response data of the sub-request based on the response metadata and the new response body includes: concatenating the response metadata and the new response body to obtain the response data corresponding to the sub-request.

[0087] For example, on the premise of obtaining all data blocks corresponding to the response data content to jump out of the loop mode, CDN node i104 correspondingly splices the response metadata and the new response body to form complete response data, and uses this complete response data as the response data corresponding to the sub-request, combining the metadata with the response body, which helps to ensure the consistency and integrity of the data, allowing users to obtain all the information they need in one request, reducing data inconsistencies that may be caused by multiple requests.

[0088] In one of the embodiments, the method further includes: if the content-length included in the response metadata is greater than the self-preservation byte length threshold, terminating the processing of the sub-request, and obtaining the corresponding response data based on the main request and returning it to the user terminal.

[0089] Optionally, if the response metadata contains a CL value but the CL value is greater than the self-protection byte length threshold, the CDN node i104 terminates processing the sub-request and does not enter the step of streaming to obtain the data block corresponding to the response data content, and directly returns to the source according to the main request to obtain the corresponding response data, and feeds back the response data to the user. The above process also belongs to the content processing part of the CDN node self-protection mechanism. If the CL (Content-Length) value in the response metadata exceeds the custom byte length threshold, this may indicate a potential data content anomaly or malicious attack risk. By terminating the processing of sub-requests, CDN nodes can effectively prevent security risks caused by abnormally large data packets. Directly returning to the source to obtain response data according to the main request instead of processing sub-requests can reduce unnecessary processing and storage resource consumption. This decision can reduce the burden on CDN nodes, improve the operating efficiency of the system, and feed back response data to users in a timely manner, ensuring the timeliness of the response.

[0090] In one embodiment, the method also includes obtaining corresponding response data according to the main request and returning it to the user terminal when the self-protection variable information indicates that the self-protection mechanism is not enabled; or, initiating a sub-request corresponding to the main request, obtaining corresponding response data according to the sub-request, and not performing any length monitoring on the sub-request process.

[0091] For example, when the self-protection variable information indicates that the self-protection mechanism is not enabled, the CDN node i104 directly returns to the source according to the main request to obtain the corresponding response data, and feeds back the response data to the user, or initiates a sub-request corresponding to the main request, and obtains the corresponding response data according to the sub-request. It can be known that when the self-protection mechanism is not enabled, the CDN node i104 can directly process the main request without initiating a sub-request, or it can initiate a sub-request and process the sub-request, but it will not perform any length monitoring on the sub-request process, that is, under this request, the system may generate redundant core files, or it may not generate redundant core files. Even if the CDN node determines that the self-protection mechanism is not enabled, it will respond to the request initiated by the user in a timely manner, and return resources when the main request is successfully responded. If the sub-request is responded to and the response fails, the system crashes and feedback is given to the user in a timely manner, avoiding wasting the user's useless waiting time and improving the user's service experience.

[0092] In one of the embodiments, the method further includes: when the target file type information is the same as any file type information in the file type information list, obtaining corresponding response data according to the main request and returning it to the user terminal.

[0093] Exemplarily, when the target file type information is the same as any file type information in the file type information list, that is, when the file to be obtained by the current sub-request is a large file that may cause the system to crash, the sub-request corresponding to the main request is not initiated, and the corresponding response data is directly obtained and fed back to the user according to the main request. Before entering the request processing, the file type of the file to be obtained by the request is identified to avoid the CDN node wasting extra resources to initiate a sub-request corresponding to the main request that cannot respond normally when the type of file to be obtained by the main request is a large file, thereby reducing the operating cost. Directly responding to the main request and feeding back the response data also ensures the timeliness of the response and improves the user's service experience.

[0094] In an exemplary embodiment, Figure 4 As shown, a specific implementation of a request processing method is provided, wherein:

[0095] CDN (Content Delivery Network) is a distributed network architecture used to improve the delivery speed and reliability of websites and other online content. CDN caches static and dynamic content on servers distributed in multiple geographical locations, allowing users to obtain this content from the server closest to them, thereby reducing latency and load and improving user experience.

[0096] Upstream rewrite phase: In Nginx's request processing flow, the upstreamrewrite phase is performed before the request is forwarded to the upstream server. It modifies the request URI by applying the rewrite directive so that the request conforms to the routing requirements of the upstream server. This process ensures that the request is correctly mapped to the target upstream server and can adapt to different service requirements or load balancing strategies.

[0097] Head filter stage: In Nginx's response processing flow, the head filter stage focuses on processing HTTP response headers. This stage allows response headers to be added, deleted, or modified to meet client requirements or to make security and compatibility adjustments. Through this stage, operations such as setting cache policies and adjusting content types can be achieved.

[0098] Body filter phase: In Nginx's response processing flow, the body filter phase involves processing the HTTP response body content. This phase allows the response body to be filtered, modified, or compressed to ensure that the data returned to the client meets the expected requirements. For example, this phase can be used for content compression, replacing certain data segments, or adding additional information.

[0099] Content-Length: is an HTTP header field that indicates the length of the byte length of the message body in a request or response. It tells the receiver to stop reading the message body after reading the specified number of bytes. This field helps determine the integrity and boundaries of the data when transmitting content.

[0100] ngx.ctx: ngx.ctx is a table in Nginx's Lua module that stores context data for the current request, which is available throughout the life cycle of the request, including its subrequests.

[0101] Secondly, when you formally enter the request processing method, you first need to do the following preparations:

[0102] Step 0: Configuration and initialization. First, preset a self-protection threshold at the bottom layer, such as 10M. This value can be selected according to the actual application scenario; second, set a variable ngx.ctx.sub_protect to control whether to enable the self-protection mechanism, and set an over-bounds flag ngx.ctx.oversize=flase in ngx.ctx to indicate whether the size of the requested file exceeds the self-protection threshold; finally, define a suffix list that does not require sub-request back to the source at the bottom layer of nginx. For example, for page rewriting functions, there is no need to rewrite audio and video files and pictures, so you can configure:

[0103] unsupport_type_default={["png"]=1,["mp4"]=1,["jpg"]=1,["mp3"]=1}

[0104] Formal entry request processing:

[0105] Step 1: Request type check and scheduling. For file requests that reach the CDN node, determine whether the self-protection mechanism is enabled in the upstream rewrite phase. If not, the main request is made directly to the source to obtain resources. If enabled, extract the suffix suffix to determine whether the file suffix is ​​in the configuration, that is, whether unsupport_type[suffix] is equal to 1. If it is equal, it means that the suffix is ​​in the list, and the main request is made directly to the source to obtain it. Otherwise, a sub-request is initiated and the process goes to step 3.

[0106] Step 2: Predict the file size. When the sub-request reaches the head filter stage, extract the value of content-length in its response header, convert the unit of the self-protection threshold into byte length and compare it with the self-protection threshold. If the content-length is equal to or greater than the self-protection threshold, set the value of ngx.ctx.oversize to true and interrupt the sub-request. The main request reads the value of ngx.ctx.oversize. If it is true, it will automatically go back to the source to obtain the file. If the content-length is less than the self-protection threshold or there is no content-length in the response header, continue the sub-request and go to step 3.

[0107] Step 3: Streaming monitoring. When the subrequest receives the response body in the body filter phase, each time a data block is received, the response body size is cumulatively updated and a check is made to see if the value exceeds the self-protection threshold. If so, data transmission is interrupted, the transmitted data is cleared, the value of ngx.ctx.oversize is set to true, and then the subrequest is exited. The main request reads the value of ngx.ctx.oversize, and if it is true, it goes back to the source to obtain the file; otherwise, the subrequest process continues and enters step 4.

[0108] Step 4: Data delivery and response. After the sub-request receives the data and its size does not exceed the threshold, the data is spliced ​​into a complete response and delivered to the main request. The main request first reads the value of ngx.ctx.oversize and finds that it is false, indicating that the sub-request response is valid. Then the data is finally processed and responded to the client.

[0109] Compared with the prior art, this application has the following advantages:

[0110] 1. Reduce the risk of system failure: Introduce a type identification mechanism in the CDN node, flexibly schedule sub-requests and main requests based on file suffixes, avoid unnecessary sub-request processing, reduce the risk of generating core files, and thus reduce the possibility of system crashes.

[0111] 2. Optimize system performance: In the early stage of sub-request processing, early warning is given and sub-requests for processing overly large files are interrupted, which reduces disk space usage and improves the overall system performance.

[0112] 3. Improve user experience: For files that are transmitted in chunks or have inaccurate Content-Length, implement streaming transmission monitoring to promptly warn and interrupt sub-requests for files that are too large, avoid the generation of core files, ensure the stability and rapid response of request processing, and improve user experience.

[0113] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0114] Based on the same inventive concept, the embodiment of the present application also provides a request processing device for implementing the request processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more request processing device embodiments provided below can refer to the limitations on the request processing method above, and will not be repeated here.

[0115] In an exemplary embodiment, Figure 5 As shown, a request processing device is provided, which is applied to any current CDN node in the CDN system, including: a request receiving module 501, a file type identification and scheduling module 502, a file size prediction module 503 and a real-time monitoring and interruption module 504, wherein:

[0116] The request receiving module 501 is used for the current CDN node to receive the main request initiated by the user and obtain the self-protection variable information;

[0117] The file type identification and scheduling module 502 is used to obtain the target file type information of the main request when the self-preservation variable information indicates that the self-preservation mechanism is turned on; when the target file type information is different from any file type information in the preset file type information list, initiate a sub-request corresponding to the main request, and obtain response metadata according to the sub-request;

[0118] The file size prediction module 503 is used to obtain data blocks corresponding to the response data content one by one based on the sub-requests and add the data blocks to the response body if the response metadata does not include content-length, or the content-length included in the response metadata is less than a preset self-preservation byte length threshold;

[0119] The real-time monitoring and interruption module 504 is used to process the sub-request according to the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold.

[0120] In one embodiment, the real-time monitoring and interruption module 504 is also used to terminate the processing of the sub-request and obtain the corresponding response data based on the main request and return it to the user terminal when the judgment result indicates that the current byte length of the response body is greater than the self-protection byte length threshold; if the judgment result indicates that the current byte length of the response body is less than or equal to the self-protection byte length threshold, return to execute the acquisition of data blocks corresponding to the response data content according to the sub-request one by one, and add the data blocks to the response body to obtain a new response body until all the data blocks corresponding to the response data content are obtained; based on the response metadata and the new response body, obtain the response data corresponding to the sub-request, process the response data according to the main request, obtain the updated response data and return it to the user terminal.

[0121] In one of the embodiments, the real-time monitoring and interruption module is further used to splice the response metadata and the new response body to obtain the response data corresponding to the sub-request.

[0122] In an exemplary embodiment, the request processing device also includes a module for terminating processing a sub-request, which is used to terminate processing the sub-request if the content-length included in the response metadata is greater than a self-preservation byte length threshold, and obtain the corresponding response data based on the main request and return it to the user terminal.

[0123] In an exemplary embodiment, the request processing device further includes a non-self-protection request processing module, which is used to obtain corresponding response data according to the main request and return it to the user terminal when the self-protection variable information indicates that the self-protection mechanism is not enabled;

[0124] or

[0125] Initiate a sub-request corresponding to the main request, obtain the corresponding response data based on the sub-request, and do not perform any length monitoring on the sub-request process.

[0126] In one of the embodiments, the sub-request processing module is terminated, and is further used to obtain corresponding response data according to the main request and return it to the user terminal when the target file type information is the same as any file type information in the file type information list.

[0127] Each module in the above request processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0128] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the main request, file type information, response metadata, response data content and self-protection byte length threshold. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a request processing method is implemented.

[0129] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0130] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the request processing method of the above embodiment is implemented.

[0131] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the request processing method of the above embodiment is implemented.

[0132] In one embodiment, a computer program product is provided, including a computer program, which implements the request processing method of the above embodiment when executed by a processor.

[0133] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0134] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0135] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0136] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A request processing method, characterized in that: Using any current CDN node in the CDN system, the method includes: The current CDN node receives the main request initiated by the user and obtains self-protection variable information; When the self-preservation variable information indicates that the self-preservation mechanism is enabled, obtaining target file type information of the main request; In the case that the target file type information is different from any file type information in the preset file type information list, initiating a sub-request corresponding to the main request, and obtaining response metadata according to the sub-request; If the response metadata does not include content-length, or the content-length included in the response metadata is less than or equal to the preset self-preservation byte length threshold, then based on the sub-request, the data blocks corresponding to the response data content are obtained one by one, and the data blocks are added to the response body; The sub-request is processed according to a determination result of whether the current byte length of the response body exceeds a preset self-preservation byte length threshold.

2. The method according to claim 1, characterized in that The processing of the sub-request based on the judgment result of whether the current byte length of the response body exceeds the preset self-preservation byte length threshold includes: If the judgment result indicates that the current byte length of the response body is greater than the self-preservation byte length threshold, terminating the processing of the sub-request, and obtaining corresponding response data based on the main request and returning it to the user terminal; If the judgment result indicates that the current byte length of the response body is less than or equal to the self-preservation byte length threshold, returning to execute sequentially obtaining data blocks corresponding to the response data content according to the sub-request, and adding the data blocks to the response body to obtain a new response body, until all data blocks corresponding to the response data content are obtained; Based on the response metadata and the new response body, the response data corresponding to the sub-request is obtained, and the response data is processed according to the main request to obtain updated response data and return it to the user terminal.

3. The method according to claim 2, characterized in that The obtaining corresponding response data of the sub-request based on the response metadata and the new response body includes: The response metadata and the new response body are concatenated to obtain response data corresponding to the sub-request.

4. The method according to claim 1, characterized in that: The method further comprises: If the content-length included in the response metadata is greater than the self-preservation byte length threshold, the processing of the sub-request is terminated, and the corresponding response data is obtained based on the main request and returned to the user terminal.

5. The method according to claim 1, characterized in that The method further comprises When the self-protection variable information indicates that the self-protection mechanism is not enabled, corresponding response data is obtained according to the main request and returned to the user terminal; or A sub-request corresponding to the main request is initiated, and corresponding response data is obtained according to the sub-request, without any length monitoring of the sub-request process.

6. The method according to claim 1, characterized in that The method further comprises: In the case where the target file type information is the same as any file type information in the file type information list, corresponding response data is obtained according to the main request and returned to the user terminal.

7. A request processing device, characterized in that: Applied to any current CDN node in the CDN system, the device includes: A request receiving module, used for the current CDN node to receive a main request initiated by a user and obtain self-protection variable information; A file type identification and scheduling module is used to obtain the target file type information of the main request when the self-preservation variable information indicates that the self-preservation mechanism is turned on; when the target file type information is different from any file type information in a preset file type information list, initiate a sub-request corresponding to the main request, and obtain response metadata according to the sub-request; A file size prediction module, configured to obtain data blocks corresponding to the response data content one by one based on the sub-request if the response metadata does not include content-length, or the content-length included in the response metadata is less than a preset self-preservation byte length threshold, and add the data blocks to the response body; The real-time monitoring and interruption module is used to process the sub-request according to the judgment result of whether the current byte length of the response body exceeds the preset self-protection byte length threshold.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images