Network request processing method and apparatus, and computing device

Abstract

The embodiments of the present application relate to the technical field of cloud computing. Provided are a network request processing method and apparatus, and a computing device. The method comprises: receiving an asynchronous request, which is initiated by a client and is used for updating a first HTML page, wherein the first HTML page includes an SDK; a proxy end performing legitimacy verification on the asynchronous request; the client parsing challenge data in the legitimacy verification by means of the SDK, and generating a verification result for the challenge data; on the basis of the verification result, the proxy end determining that the asynchronous request passes the verification, and forwarding the asynchronous request to a cloud server end; and on the basis of the verification result, determining that the asynchronous request fails to pass the verification, and blocking the asynchronous request. In the embodiments of the present application, the data format of challenge data matches the data format which an SDK can parse, such that in a single-page application scenario, legitimacy verification can be initiated to identify a crawler program. Thus, the problem in the related art of it being impossible to apply a challenge response mechanism to a single-page application is solved, thereby guaranteeing data security in a single-page application scenario.

Description

Network request processing method and apparatus, and computing device

[0001] The present application claims priority to a Chinese patent application No. 202410410796.1, filed on April 7, 2024, entitled “A data processing method, device and other equipment”, and to a Chinese patent application No. 202410708026.5, filed on May 31, 2024, entitled “Network request processing method and apparatus, and computing device”, both of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of cloud computing, and more particularly to a network request processing method, device and computing device. BACKGROUND

[0003] Web crawling refers to the behavior of using an automated program (or called crawler program) to access and collect website data, which is an attack behavior on the website, and which can cause problems such as performance degradation, server load increase, and data leakage of the website. To protect the security of website data, a large number of anti-crawling technologies have emerged to prevent crawler programs from automatically crawling website data.

[0004] Currently, the anti-crawling technology commonly used in the related art includes challenge-response mechanism technology. This technology detects crawler programs, and when a suspicious request is detected, a verification code is returned to challenge the client that initiates the request, and the client's response information is returned to determine whether the client is a crawler program. For example, the challenge-response mechanism can be a JavaScript challenge (abbreviated as JS challenge).

[0005] However, in the single-page application (SPA) scenario, there is only one Hyper Text Markup Language (HTML) page in the entire website, and the content and resources of the website are all dynamically loaded without page refresh. However, when the challenge-response mechanism is used for dynamic content, the client cannot parse the returned verification code. Therefore, in the SPA scenario, the challenge-response mechanism cannot be used. This causes a website data security risk in the SPA scenario. Therefore, how to apply the challenge-response mechanism to identify crawler programs in the SPA scenario has become a technical problem to be solved. SUMMARY

[0006] The embodiments of the present application provide a network request processing method, device and computing device to apply the challenge-response mechanism to identify crawler programs in the SPA scenario.

[0007] In a first aspect, an embodiment of the present application provides a network request processing method, which is applied to a proxy end in a cloud service system. The cloud service system further includes a cloud service end. The cloud service end is configured to provide a web application based on a single-page application to a client. In the network request processing, the proxy end receives an asynchronous request sent by the client. The asynchronous request is configured to update a first HyperText Markup Language (HTML) page based on the single-page application. Optionally, the first HTML page includes a Software Development Kit (SDK). The proxy end performs legality verification on the asynchronous request. The legality verification includes challenge data, and a data format of the challenge data matches a data format that can be parsed by the SDK. The SDK is configured to parse the challenge data in the legality verification initiated by the asynchronous request by the client, and generate a verification result of the legality verification challenge data. The proxy end determines that the asynchronous request passes the verification based on the verification result, forwards the asynchronous request to the cloud service end, determines that the asynchronous request fails the verification based on the verification result, and blocks the asynchronous request.

[0008] In the related art, the challenge data format received by the client does not match the expected data format under the single-page application, so that it is difficult to identify a web crawler through the legality verification mechanism under the single-page application, and thus network data security problems are caused. Therefore, based on the first aspect, by providing the HTML page including the SDK to the client, since the data format that can be parsed by the SDK matches the challenge data format received by the client, when performing the legality verification on the asynchronous request, the client can parse the challenge data in the legality verification through the SDK, and verify the result. Then, the proxy end can determine that the asynchronous request fails the verification or determines that the asynchronous request passes the verification. Therefore, in the single-page application scenario, the legality verification can be initiated to identify the crawler program. In this way, the problem that the challenge and response mechanism in the related art cannot be applied to the single-page application is solved, so as to ensure the data security in the single-page application scenario.

[0009] In a possible implementation, the proxy end receives a page access request sent by the client before receiving the asynchronous request sent by the client. The page access request is configured to request the first HTML page. The proxy end forwards the page access request to the cloud service end. The cloud service end sends a second HTML page not including the SDK to the proxy end in response to the page access request. The proxy end receives the second HTML page sent by the cloud service end, injects the SDK into the second HTML page to generate the first HTML page, and sends the first HTML page to the client.

[0010] Based on the possible implementation, the first HTML page is generated by injecting the SDK in the second HTML page, so that the client can run the SDK when receiving the first HTML page, thereby analyzing the challenge data in the legality verification initiated for the asynchronous request by the SDK in the legality verification, and generating a verification result of the legality verification challenge data. In the single-page application scenario, the legality verification can be initiated to identify the crawler program.

[0011] In a possible implementation, the SDK is injected into the second HTML page to generate the first HTML page, specifically, a resource access address of the SDK is added in the second HTML page to generate the first HTML page.

[0012] Since directly adding the source code of the SDK in the second HTML page may increase the amount of data transmitted from the cloud server to the client, the time length for the client to load the first HTML page is increased. Therefore, in the possible implementation, the resource access address of the SDK is injected into the second HTML page to generate the first HTML page, thereby reducing the amount of data of the first HTML page.

[0013] In a possible implementation, the verification result includes response information. In the legality verification process of the asynchronous request, the proxy end sends a legality verification request containing challenge data to the client. The legality verification request is used to instruct the client to analyze the challenge data in the legality verification initiated for the asynchronous request by the SDK, and generate response information of the challenge data. The client returns the response information to the proxy end. The proxy end checks the response information. If the response information is checked successfully, it is determined that the asynchronous request verification is passed. If the response information fails to be checked, it is determined that the asynchronous request verification fails.

[0014] In this way, the response of the legality verification request is changed to the SDK, and the challenge data carried in the legality verification request is analyzed by the SDK, thereby solving the problem that the client cannot analyze the challenge data in the challenge response mechanism in the related art due to the mismatch between the data format of the challenge data in the challenge response mechanism and the data format that the client can analyze.

[0015] In a possible implementation, specifically, before sending the legality verification request to the client, the proxy end identifies whether the asynchronous request is a suspicious request according to at least one of user agent information carried in the asynchronous request or a request frequency of the client. The suspicious request indicates an asynchronous request initiated by a crawler program. When it is determined that the asynchronous request is a suspicious request, the proxy end sends the legality verification request to the client.

[0016] In this way, the agent end filters the received asynchronous request through suspicious request identification, and only sends a legitimacy verification request to the client sending the asynchronous request identified as a suspicious request, thereby reducing the amount of response information that the agent end needs to verify, reducing the time length of the agent end responding to the asynchronous request, and reducing the refresh latency of the client.

[0017] In a possible implementation, the challenge data includes a set of JavaScript codes, and the response information includes an access token, which is obtained by the client by parsing the JavaScript codes through the SDK.

[0018] In this way, the SDK parses the JavaScript codes to ensure that the JS challenge can be initiated in the single-page application to identify the crawler program. In this way, the problem that the challenge and response mechanism in the related art cannot be applied to the single-page application is solved, thereby ensuring the data security in the single-page application scenario.

[0019] In a possible implementation, the challenge data includes a set of codes for generating a graphic verification code, and the response information includes at least one of mouse movement information or verification information of the graphic verification code. The SDK is configured to parse the codes for generating the graphic verification code to generate the graphic verification code, so that the client generates the response information based on the image verification code.

[0020] Based on this possible implementation, the graphic verification code is generated through the SDK to ensure that the graphic verification can be initiated in the single-page application to identify the crawler program. In this way, the problem that the challenge and response mechanism in the related art cannot be applied to the single-page application is solved, thereby ensuring the data security in the single-page application scenario.

[0021] In a possible implementation, the asynchronous request includes at least one of a request initiated by the client based on a Fetch interface or a request initiated by the client based on an XHR (XMLHttpRequest) interface.

[0022] In a second aspect, the embodiments of the present application provide a network request processing method. The method is applied to a cloud server. The cloud server is configured to provide a single-page application based web application to a client. In network request processing, the cloud server receives an asynchronous request sent by the client. The asynchronous request is configured to update a first hypertext markup language (HTML) page based on a single-page application. Optionally, the first HTML page includes a software development kit (SDK). The client initiates the asynchronous request to the cloud server. The cloud server performs legality verification on the asynchronous request. The legality verification includes challenge data, and the data format of the challenge data matches a data format that can be parsed by the SDK. The SDK is configured to parse the challenge data in the legality verification initiated by the asynchronous request, generate a verification result of the challenge data, and send the verification result to the cloud server. The cloud server determines that the asynchronous request passes the verification based on the verification result, and allows the asynchronous request to be responded. Alternatively, the cloud server determines that the asynchronous request fails the verification based on the verification result, and rejects the asynchronous request.

[0023] In a single-page application scenario, the challenge data format received by the client does not match the expected data format, so it is difficult to identify a web crawler through a legality verification mechanism in the related art, which causes a network data security problem. Therefore, based on the second aspect, by providing the client with an HTML page including an SDK, since the data format that can be parsed by the SDK matches the challenge data format received by the client, when performing legality verification on the asynchronous request, the client can parse the challenge data in the legality verification through the SDK and verify the result. Then, the cloud server can determine that the asynchronous request fails the verification and that the asynchronous request passes the verification based on the verification result. In a single-page application scenario, legality verification can be initiated to identify a crawler program. In this way, the challenge and response mechanism in the related art cannot be applied to a single-page application, thereby ensuring data security in a single-page application scenario.

[0024] In a possible implementation, before receiving the asynchronous request sent by the client, the cloud server receives a page access request sent by the client for requesting a first HTML page, and in response to the page access request, obtains a second HTML page that does not include an SDK. The cloud server injects the SDK into the second HTML page to generate the first HTML page, and sends the first HTML page to the client.

[0025] In a possible implementation, the SDK is injected into the second HTML page to generate the first HTML page, and the implementation is as follows: the cloud server adds a resource access address of the SDK in the second HTML page to generate the first HTML page.

[0026] In a possible implementation, the verification result includes response information. In the legality verification of the asynchronous request, the cloud server sends a legality verification request including challenge data to the client. The legality verification request is used to instruct the client to parse the challenge data in the legality verification initiated for the asynchronous request through the SDK, to generate response information of the challenge data. The client returns the response information to the cloud server. The cloud server checks the response information. If the response information is checked successfully, it is determined that the asynchronous request is verified. If the response information fails to be checked, it is determined that the asynchronous request fails to be verified.

[0027] In a possible implementation, before the legality verification request is sent to the client, the cloud server identifies whether the asynchronous request is a suspicious request according to at least one of user agent information carried in the asynchronous request or a request frequency of the client. The suspicious request indicates an asynchronous request initiated by a crawler program. When it is determined that the asynchronous request is a suspicious request, the cloud server sends the legality verification request to the client.

[0028] In a possible implementation, the challenge data includes a set of JavaScript code, and the response information includes an access token. The access token is obtained by the client by parsing the JavaScript code through the SDK.

[0029] In a possible implementation, the challenge data includes a set of code for generating a graphical verification code, and the response information includes at least one of mouse movement information or verification information of the graphical verification code. The SDK is used to parse the code for generating the graphical verification code by the client, to generate the graphical verification code, so that the client generates the response information based on the graphical verification code.

[0030] In a possible implementation, the asynchronous request includes at least one of a request initiated by the client based on a Fetch interface or a request initiated by the client based on an XHR interface.

[0031] In a third aspect, the embodiments of the present application provide a network request processing method. The method is applied to a client. The client is configured to request a single-page application based web page web application from a cloud service system. The cloud service system includes a cloud server and a proxy server. In the network request processing, the client sends a page access request to the cloud server, where the page access request is configured to request a first hypertext markup language (HTML) page based on a single-page application. The client receives the first HTML page, and the first HTML page includes a software development kit (SDK). The client initiates an asynchronous request to the proxy server based on the first HTML page, so that the proxy server performs legality verification on the asynchronous request. The asynchronous request is configured to update the first HTML page. The legality verification includes challenge data, and the data format of the challenge data matches a data format that can be parsed by the SDK. The SDK is configured to parse the challenge data in the legality verification initiated by the client for the asynchronous request, and generate a verification result of the challenge data.

[0032] In a fourth aspect, the embodiments of the present application provide a network request processing apparatus. The network request processing apparatus is arranged in a cloud service system, and includes a communication module and a processing module.

[0033] The communication module is configured to receive an asynchronous request sent by a client. The asynchronous request is configured to update a first hypertext markup language (HTML) page based on a single-page application. The first HTML page includes a software development kit (SDK).

[0034] The processing module is configured to perform legality verification on the asynchronous request. The legality verification includes challenge data, and the data format of the challenge data matches a data format that can be parsed by the SDK. The SDK is configured to parse the challenge data in the legality verification initiated by the client for the asynchronous request, and generate a verification result of the challenge data.

[0035] The processing module is further configured to determine, based on the verification result, whether the asynchronous request passes the verification. If the asynchronous request passes the verification, the processing module is configured to allow a response to the asynchronous request. If the asynchronous request fails the verification, the processing module is configured to reject the response to the asynchronous request.

[0036] In a possible implementation, the communication module is further configured to receive, before receiving the asynchronous request sent by the client, a page access request sent by the client and configured to request the first HTML page.

[0037] The processing module is configured to obtain a second HTML page. The second HTML page does not include the SDK.

[0038] The processing module is further configured to inject the SDK into the second HTML page, and generate the first HTML page.

[0039] The communication module is further configured to send the first HTML page to the client.

[0040] In a possible implementation, the verification result includes response information. The communication module is configured to send a legality verification request to the client, and the legality verification request includes challenge data. The legality verification request is configured to instruct the client to parse the challenge data in the legality verification initiated for the asynchronous request by using the SDK, and generate response information of the challenge data.

[0041] The processing module is configured to check the response information returned by the client. If the response information is checked successfully, it is determined that the asynchronous request passes the verification. If the response information fails the check, it is determined that the asynchronous request fails the verification.

[0042] In a possible implementation, before the communication module sends the legality verification request to the client, the processing module is further configured to identify whether the asynchronous request is a suspicious request according to at least one of user agent information carried in the asynchronous request or a request frequency of the client. The suspicious request indicates an asynchronous request initiated by a crawler program. If it is determined that the asynchronous request is a suspicious request, the communication module is controlled to send the legality verification request to the client.

[0043] In a possible implementation, the challenge data includes a set of JavaScript code, and the response information includes an access token. The access token is obtained by the client by parsing the JavaScript code by using the SDK.

[0044] In a possible implementation, the challenge data includes a set of code for generating a graphical verification code, and the response information includes at least one of mouse movement information or verification information of the graphical verification code. The SDK is configured to parse the code for generating the graphical verification code, generate the graphical verification code, and enable the client to generate the response information based on the graphical verification code.

[0045] In a possible implementation, the asynchronous request includes at least one of a request initiated by the client based on a Fetch interface or a request initiated by the client based on an XHR interface.

[0046] In a fifth aspect, an embodiment of the present application provides a proxy device. The proxy device is applied to a cloud service system, and the cloud service system further includes a cloud server. The cloud server is configured to provide a web application based on a single-page application to a client. The proxy device includes a processing module and a communication module.

[0047] The communication module is configured to receive an asynchronous request sent by the client, and the asynchronous request is configured to update a first HyperText Markup Language (HTML) page based on a single-page application. The first HTML page includes a Software Development Kit (SDK).

[0048] The processing module is configured to perform legitimacy verification on the asynchronous request, wherein the legitimacy verification contains challenge data, and a data format of the challenge data matches a data format that can be parsed by the SDK; and the SDK is configured to parse the challenge data in the legitimacy verification initiated by the asynchronous request by the client, and generate a verification result of the challenge data.

[0049] The communication module is further configured to, based on the verification result, determine that the asynchronous request passes the verification, and forward the asynchronous request to the cloud server.

[0050] The communication module is further configured to, based on the verification result, determine that the asynchronous request fails the verification, and block the asynchronous request.

[0051] In a possible implementation, the communication module is further configured to, before receiving the asynchronous request sent by the client, receive a page access request sent by the client for requesting the first HTML page, forward the page access request to the cloud server, and receive a second HTML page sent by the cloud server, wherein the second HTML page does not contain the SDK.

[0052] The processing module is further configured to inject the SDK into the second HTML page, and generate the first HTML page.

[0053] The communication module is further configured to send the first HTML page to the client.

[0054] In a possible implementation, the verification result includes response information. The communication module is configured to send a legitimacy verification request to the client, wherein the legitimacy verification request contains challenge data, and the legitimacy verification request is configured to instruct the client to parse the challenge data in the legitimacy verification initiated by the asynchronous request by using the SDK, and generate response information of the challenge data.

[0055] The processing module is configured to check the response information returned by the client. If the response information passes the check, it is determined that the asynchronous request passes the verification. If the response information fails the check, it is determined that the asynchronous request fails the verification.

[0056] In a possible implementation, before sending the legitimacy verification request to the client, the processing module is further configured to identify whether the asynchronous request is a suspicious request according to at least one of user agent information carried by the asynchronous request or a request frequency of the client; the suspicious request indicates an asynchronous request initiated by a crawler program; and if it is determined that the asynchronous request is a suspicious request, the communication module is controlled to send the legitimacy verification request to the client.

[0057] In a possible implementation, the challenge data includes a set of JavaScript codes; the response information includes an access token; and the access token is obtained by the client by parsing the JavaScript codes through the SDK.

[0058] In a possible implementation, the challenge data includes a set of codes for generating a graphical verification code; the response information includes at least one of mouse movement information or verification information of the graphical verification code; and the SDK is configured to parse the codes for generating the graphical verification code, and generate the graphical verification code, so that the client generates the response information based on the graphical verification code.

[0059] In a possible implementation, the asynchronous request includes at least one of a request initiated by the client based on a Fetch interface or a request initiated by the client based on an XHR interface.

[0060] In a sixth aspect, an embodiment of the present application provides a computing device, including a processor and a memory. The processor is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device performs the method in the first aspect or any possible implementation of the first aspect. Or so that the computing device performs the method in the second aspect or any possible implementation of the second aspect. Or so that the computing device performs the method in the third aspect.

[0061] In a seventh aspect, an embodiment of the present application provides a computing device cluster. The computing device cluster includes at least one computing device. Each computing device includes a processor and a memory; and the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method in the first aspect or any possible implementation of the first aspect. Or so that the computing device cluster performs the method in the second aspect or any possible implementation of the second aspect. Or so that the computing device cluster performs the method in the third aspect.

[0062] In an eighth aspect, an embodiment of the present application provides a computer program product of instructions. When the instructions are executed by the computing device cluster or the computing device, the computing device cluster or the computing device performs the method in the first aspect or any possible implementation of the first aspect. Or the computing device cluster or the computing device performs the method in the second aspect or any possible implementation of the second aspect. Or the computing device cluster or the computing device performs the method in the third aspect.

[0063] In a ninth aspect, an embodiment of the present application provides a computer readable storage medium, including computer program instructions. When the computer program instructions are executed by a computing device cluster or a computing device, the computing device cluster or the computing device executes instructions in the computer program stored in the computer readable storage medium to perform the method in the first aspect or any possible implementation manner of the first aspect. Or to perform the method in the second aspect or any possible implementation manner of the second aspect. Or to perform the method in the third aspect.

[0064] The technical effects brought by any implementation manner of the second aspect to the ninth aspect can refer to the technical effects brought by the first aspect and different implementation manners of the first aspect. Details are not described herein.

[0065] On the basis of the implementation manners of the aspects described above, the present application can be further combined to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0066] FIG. 1 is a schematic diagram of a system architecture of a network request processing method provided by an embodiment of the present application in a cloud service scenario;

[0067] FIG. 2 is a schematic diagram of another system architecture of a network request processing method provided by an embodiment of the present application in a cloud service scenario;

[0068] FIG. 3 is a schematic diagram of a system architecture of a network request processing method provided by an embodiment of the present application in a local service scenario;

[0069] FIG. 4 is a schematic diagram of a flow of a network request processing method provided by an embodiment of the present application;

[0070] FIG. 5 is a schematic diagram of another flow of a network request processing method provided by an embodiment of the present application;

[0071] FIG. 6 is a schematic diagram of a flow of generating a first HTML page by a cloud service end provided by an embodiment of the present application;

[0072] FIG. 7 is a schematic diagram of a flow of generating a first HTML page by a cloud service end in another structure provided by an embodiment of the present application;

[0073] FIG. 8 is a schematic diagram of a flow of evaluating the legality of an asynchronous request by a cloud service end provided by an embodiment of the present application;

[0074] FIG. 9 is a schematic diagram of a flow of evaluating whether an asynchronous request is a suspicious request by a cloud service end provided by an embodiment of the present application;

[0075] FIG. 10 is a schematic diagram of a flow of evaluating the legality of an asynchronous request by a cloud service end in another structure provided by an embodiment of the present application;

[0076] Fig. 11 is a schematic diagram of a page access request processing flow in an XHR interface scenario according to an embodiment of the present application;

[0077] Fig. 12 is a schematic diagram of a page access request processing flow in a Fetch interface scenario according to an embodiment of the present application;

[0078] Fig. 13 is a schematic diagram of a structure of a network request processing apparatus 13 according to an embodiment of the present application;

[0079] Fig. 14 is a schematic diagram of a structure of a proxy apparatus 14 according to an embodiment of the present application;

[0080] Fig. 15 is a schematic diagram of a structure of a computing device 15 according to an embodiment of the present application;

[0081] Fig. 16 is a schematic diagram of a structure of a computing device cluster 16 according to an embodiment of the present application;

[0082] Fig. 17 is a schematic diagram of a structure of a network connection between computing devices in the computing device cluster 16 according to an embodiment of the present application. DETAILED DESCRIPTION

[0083] First, the terms related to the embodiments of the present application are introduced.

[0084] Web crawler: refers to the behavior of using an automated program to access and collect website data.

[0085] Optionally, the web crawler includes a search engine crawler and a malicious crawler.

[0086] The search engine crawler refers to a web crawler that complies with the robots protocol. This technology captures website data according to the content specified in the robots protocol, and implements search engine optimization (SEO).

[0087] The malicious crawler refers to a web crawler that does not comply with the robots protocol. This technology can cause problems such as performance degradation, increased server load, and data leakage of the website.

[0088] Anti-crawler technology: refers to a technology that identifies and blocks malicious crawlers. This technology usually identifies whether an asynchronous request is initiated by a malicious crawler through the request frequency of asynchronous requests, user agent information (User-Agent), or adding a verification code.

[0089] The user agent information is used to identify the visitor information initiating the asynchronous request, for example, the visitor can be a client, and the visitor information can include but is not limited to the network address of the client, the operating system used, the version information, the processor information, the browser information, etc.

[0090] The network address of the client can refer to the Internet Protocol Address (IP) of the device where the client is located.

[0091] JavaScript challenge (JS challenge): also known as challenge response mechanism. JS challenge is a common anti-crawler technology. By using JavaScript code in the webpage, the crawler program is prevented from automatically crawling the website data. JS challenge is usually used in static page scenarios.

[0092] According to the refreshing mode of the applied web page, the application can be divided into single-page application (SPA) and multi-page application (MPA).

[0093] The multi-page application includes multiple independent HTML pages, and each HTML page corresponds to a different function or view. When the user switches between different pages, the new page needs to be loaded through the browser refresh. In the multi-page application scenario, each HTML page is a static page.

[0094] Single-page application: a web application developed with Vue.js framework, which only has one HTML file. In single-page application, all pages and components are dynamically loaded, and the page in single-page application is also called dynamic page. By responding to different asynchronous requests initiated by the user, the content in the page is updated instead of refreshing the entire page. In single-page application, there is a Document Object Model (DOM) node.

[0095] The DOM node is used to present an application programming interface (API) that interacts with other HTML documents or extensible markup language documents. DOM nodes represent HTML documents in the form of a node tree, and each node represents a component of the HTML document (such as page elements, strings, or comments, etc.). DOM nodes are one of the most widely used APIs on the World Wide Web, and code running in the client can access nodes in the file to interact with DOM nodes. DOM nodes can be created, moved or modified. Event listeners can be added to DOM nodes and triggered at a specific time. For example, the DOM node is bound with a trigger event, and the event listener listens to the trigger event bound by the DOM node, and an asynchronous request occurs.

[0096] The trigger event bound by the DOM node can include clicking, mouse moving, mouse hovering, etc.

[0097] In a single-page application scenario, a client loads an HTML page only once. When a DOM node is triggered to enter a second page, the single-page application switches the view without completely refreshing the entire page through a routing mechanism. That is, the uniform resource locator (URL) of the page changes, but a new HTML page is not reloaded, but the content in the HTML page is updated. The routing describes the mapping relationship between the URL and the user interface (UI), and this mapping is one-way, that is, the URL change causes the UI update.

[0098] Since the page is dynamically loaded, in the case of dynamic updating of the page, the data type of the JavaScript code returned by the server to the client is text / html, and the data type expected to be returned by the dynamic interface of the client is application / json, application / xml, and the like. Since the returned data type does not match the data type expected to be returned by the dynamic interface of the client, the client cannot parse the JavaScript code in the challenge-response mechanism, which makes it difficult to identify a web crawler through the challenge-response mechanism in a single-page application, and thus causes a network data security problem.

[0099] Based on this, in order to apply the challenge-response mechanism in a single-page application scenario and protect network data security, an embodiment of the present application provides a network request processing method. The method provides an HTML page containing an SDK to a client in a single-page application scenario. Since the data format that can be parsed by the SDK matches the challenge data format received by the client, when the legality of the asynchronous request is verified, the client can parse the challenge data in the legality verification through the SDK and generate a verification result. Then, the proxy end can determine that the asynchronous request verification fails or that the asynchronous request verification passes based on the verification result. This enables the legality verification to be initiated in a single-page application scenario, and the crawler program to be identified. In this way, the problem that the challenge-response mechanism cannot be applied to a single-page application in the related art is solved, thereby protecting the data security in a single-page application scenario.

[0100] The network request processing method provided by the embodiment of the present application can be applied in a cloud service scenario to prevent a single-page application program deployed in the cloud from being attacked by a network crawler. Alternatively, the network request processing method provided by the embodiment of the present application can also be applied in a non-cloud service scenario to prevent a single-page application program deployed locally from being attacked by a network crawler.

[0101] Exemplarily, a system architecture of the network request processing method provided by the embodiments of the present application is introduced taking a cloud service scenario as an example. As shown in FIG. 1, FIG. 1 is a system architecture schematic diagram of the network request processing method provided by the embodiments of the present application in a cloud service scenario. The system architecture shown includes a cloud service system 100 and a client 200. The cloud service system 100 includes a cloud server 10. The cloud server 10 performs data interaction with the client 200 through a network 30.

[0102] The cloud server 10 is configured to provide an application service based on a single-page application to the client 200. The client 200 is configured to request the application service from the cloud server 10. The application service refers to a web application service provided by the cloud server 10 to the client 200. The application service includes but is not limited to a data query service, a video playing service, a data analysis service, etc.

[0103] The client 200 can be an application program, for example, a cloud application type application program. Alternatively, the client 200 can be a web client 200. Alternatively, the client 200 can be a browser. The client 200 runs on a terminal device. The terminal device includes but is not limited to a mobile phone terminal, a tablet computer, a personal computer or a notebook computer, etc.

[0104] The cloud server provides a web application to the client 200 in the form of a single-page application. That is, the client 200 sends a request for a single-page application to the cloud server 10, the cloud server responds to the request for the single-page application, and sends an HTML page to the client 200. The client 200 displays the HTML page. Based on the HTML page, the client 200 initiates an asynchronous request to the cloud server, the cloud server responds to the asynchronous request, and sends data corresponding to the asynchronous request to the client 200. The client 200 processes the returned data, and updates the corresponding page content in the HTML page based on the processing result.

[0105] In the embodiments of the present application, when the cloud server 10 receives an asynchronous request, the cloud server 10 performs the network request processing method provided by the embodiments of the present application to perform legality verification. When it is determined that the legality verification of the asynchronous request is passed, the cloud server 10 is allowed to respond to the asynchronous request.

[0106] The legality verification is configured to identify whether the asynchronous request is generated by a crawler program. It can be understood that when the legality verification of the asynchronous request is passed, the asynchronous request is not generated by a crawler program. When the legality verification of the asynchronous request fails, the asynchronous request can be generated by a crawler program.

[0107] In a possible implementation manner, as shown in FIG. 1, the cloud server 10 includes a hardware resource layer 103, a virtual service layer 102, and an access layer 101.

[0108] The hardware resource layer 103 is configured to provide hardware resources. For example, the hardware resources include computing resources, storage resources, and network resources.

[0109] The virtual service layer 102 is configured to allocate virtual resources of a service system based on the hardware resource layer 103, and provide service to the client 200, and respond to an asynchronous request sent by the client 200. The virtual resources include virtual computing resources, virtual storage resources, and virtual network resources.

[0110] The access layer 101 is configured to receive a request for a single-page application sent by the client 200, and an asynchronous request.

[0111] It should be noted that the cloud server 10 shown in FIG. 1 is only an example and does not limit the network request processing method provided by the embodiments of the present application. The naming and grouping of the cloud server 10 in FIG. 1 are illustrative and are only a logical grouping. In actual implementation, other grouping manners can be used. For example, the cloud server 10 can also be divided into a proxy module 11 and a processing module 12. For another example, refer to the example corresponding to FIG. 7.

[0112] In addition, the cloud server 10 can also be named as a network request processing apparatus 13, which can include modules different from those of the cloud server 10 shown in FIG. 1. The network request processing apparatus 13 can refer to the embodiments corresponding to FIG. 13, and the embodiments of the present application will not be described here.

[0113] In the first example, the access layer 101 is further configured to perform the network request processing method provided by the embodiments of the present application, and perform legality verification on the asynchronous request initiated by the client 200.

[0114] As shown in FIG. 1, the access layer 101 is deployed with a Web application firewall component 1012 and an access gateway 1011.

[0115] The access gateway 1011 in the access layer 101 receives the asynchronous request sent by the client 200. The Web application firewall component 1012 in the access layer 101 performs the network request processing method provided by the embodiments of the present application, and performs legality verification on the asynchronous request initiated by the client 200. When it is determined that the legality verification of the asynchronous request is passed, the asynchronous request is forwarded to the virtual service layer 102. When it is determined that the legality verification of the asynchronous request fails, the asynchronous request is discarded.

[0116] In the second example, the virtual service layer 102 is further configured to perform the network request processing method provided by the embodiments of the present application, and perform legality verification on the asynchronous request initiated by the client 200, and respond to the asynchronous request when it is determined that the legality verification of the asynchronous request is passed.

[0117] A Web application firewall component 1012 (not shown in FIG. 1) is deployed in the virtual service layer 102. The Web application firewall component 1012 in the virtual service layer 102 performs the network request processing method provided by the embodiments of the present application to perform legitimacy verification, and when it is determined that the legitimacy verification of the asynchronous request is passed, the asynchronous request is responded to. When it is determined that the legitimacy verification of the asynchronous request fails, the asynchronous request is rejected.

[0118] It should be noted that FIG. 1 is only an exemplary drawing and does not constitute a limitation on the network request processing method provided by the embodiments of the present application. The division of the cloud service system 100 in FIG. 1 is illustrative and is only a logical functional grouping. In actual implementation, there can be another grouping manner. For example, the cloud service system 100 can also be divided into a cloud service end 10 and a proxy end 40.

[0119] For example, as shown in FIG. 2, FIG. 2 is another system architecture diagram of the network request processing method provided by the embodiments of the present application in a cloud service scenario. The system architecture shown includes a cloud service system 100 and a client 200. The cloud service system 100 includes a cloud service end 10 and a proxy end 40. The cloud service end 10 and the proxy end 40 perform data interaction through a network 30. The proxy end 40 and the client 200 perform data interaction through the network 30.

[0120] As shown in FIG. 2, compared with the cloud service end 10 provided in FIG. 1, only an access gateway 1011 is deployed in the access layer 101 in the cloud service end 10 provided in FIG. 2. The components included in the other layers in the cloud service end 10 shown in FIG. 2 are the same as the components included in the other layers in the cloud service end 10 shown in FIG. 1, and are not described here.

[0121] In the embodiments of the present application, when the proxy end 40 receives an asynchronous request sent by the client 200, the proxy end 40 performs the network request processing method provided by the embodiments of the present application to perform legitimacy verification on the asynchronous request, and when it is determined that the legitimacy verification of the asynchronous request is passed, the asynchronous request is forwarded to the cloud service end 10. When it is determined that the legitimacy verification of the asynchronous request fails, the asynchronous request is blocked.

[0122] In a possible implementation manner, the proxy end 40 can be a Web application firewall. Alternatively, the proxy end 40 can be a gateway device.

[0123] It should be noted that FIG. 2 is only an exemplary diagram, and does not constitute a limitation on the network request processing method provided by the embodiments of the present application. The naming of the cloud server 10, the client 200 and the proxy 40 in FIG. 2 is illustrative, and in actual application, the proxy 40 can also be named as the proxy device 14. The proxy device 14 can include modules different from those shown in the proxy 40 in FIG. 2. The proxy device 14 can refer to the corresponding embodiments of FIG. 14, and the embodiments of the present application will not be described here.

[0124] It should be noted that FIG. 1 and FIG. 2 are only exemplary diagrams, and do not constitute a limitation on the network request processing method provided by the embodiments of the present application. The network request processing method can also be applied to other non-cloud service scenarios, for example, the network processing method can also be applied to a local service scenario.

[0125] As shown in FIG. 3, FIG. 3 is a system architecture schematic diagram of the network request processing method provided by the embodiments of the present application in a local service scenario. The system architecture shown includes a browser 31, a Web application firewall 32 and a server 33.

[0126] Among them, the server 33 is used to provide web services to the browser 31 in the form of a single-page application. The client 200 displays web content to the user in the form of a single-page application.

[0127] Among them, the Web application firewall 32 and the server 33 are coupled. For example, the Web application firewall 32 can be deployed in the server 33. Or the Web application firewall 32 is deployed in a proxy server, and the proxy server is in communication connection with the server 33.

[0128] In one possible implementation, the Web application firewall 32 performs the network request processing method provided by the embodiments of the present application, and performs legality verification on the asynchronous request. When it is determined that the legality verification of the asynchronous request is passed, the asynchronous request is forwarded to the server 33. When it is determined that the legality verification of the asynchronous request fails, the asynchronous request is prevented.

[0129] The network request processing method provided by the embodiments of the present application will be introduced below taking the cloud service scenario as an example. The system architecture in the cloud service scenario is shown in FIG. 1. Correspondingly, the network request processing method in the local service scenario can refer to the network request processing method in the cloud service scenario, and the embodiments of the present application will not be described here.

[0130] As shown in FIG. 4, FIG. 4 is a flow diagram of the network request processing method provided by the embodiments of the present application. The network request processing method shown includes steps S410 to S460.

[0131] Step S410, the client sends a page access request to the cloud server.

[0132] The page access request is used to request a first HTML page of the single-page application from the cloud server.

[0133] In a possible implementation, the page access request contains a network link. The network link is used to indicate an access address of the HTML page requested by the client. For example, the client is a browser, the browser sends a page access request to the cloud server based on a network link input by a user. For another example, the browser acquires a search term input by a user, finds a network link corresponding to the search term, and provides the network link and summary information of each network link to the user through a search result interface of the browser. The user selects the network link for access in the search result interface. The browser sends a page access request to the cloud server based on the network link selected by the user.

[0134] In step S420, the cloud server returns the first HTML page to the client based on the page access request.

[0135] The first HTML page contains a software development kit (SDK).

[0136] In the embodiment of the application, the single-page application loads the HTML page only once, and subsequent page switching is performed through page content dynamic updating. In the dynamic page scenario, the verification information returned by the cloud server is of a text / html data type, which causes the client to be unable to parse challenge data in the challenge-response mechanism. Based on this, the first HTML page returned by the cloud server contains an SDK, which is used to hook and intercept an asynchronous request initiated by the client, so that challenge data returned by the cloud server is acquired by the SDK before entering the kernel of the client, and the challenge data in the challenge-response mechanism is parsed. In this way, in the dynamic page scenario, the problem that the client cannot parse in the related art due to the verification information returned by the cloud server being of the text / html data type is solved, and the cloud server can perform legality verification on the asynchronous request through the challenge-response mechanism.

[0137] In an example, the first HTML page can be a website homepage. Alternatively, the first HTML page can also be a certain secondary page of a website. The first HTML page is provided with a DOM node. Related descriptions of the DOM node can be referred to the foregoing, and will not be described herein.

[0138] In a possible implementation, the cloud server obtains the storage address of the source code of the page content corresponding to the page access request based on the network link contained in the page access request from the network links of the plurality of HTML pages and the storage addresses of the source codes of the HTML pages, and obtains the source code from the storage space indicated by the storage address of the source code. The first HTML page is returned to the client based on the source code.

[0139] The page content corresponding to the page access request can refer to the page content of the HTML page of the single-page application requested by the page access request.

[0140] For example, the cloud server returns the source code information to the client. The client generates the first HTML page based on the source code information. For another example, the cloud server generates the first HTML page based on the source code information, and sends the data of the generated first HTML page to the client.

[0141] In a possible implementation, the cloud server obtains the SDK in the page content based on the page access request. That is, the cloud server adds the SDK in the page content of the single-page application in advance.

[0142] In another possible implementation, the cloud server obtains the HTML page of the single-page application, injects the SDK in the HTML page, and generates the first HTML page.

[0143] In step S430, the client initiates an asynchronous request to the cloud server based on the first HTML page.

[0144] In a possible implementation, the asynchronous request can be triggered by the user by clicking the DOM node in the first HTML page. Alternatively, the asynchronous request can be triggered by a crawler program.

[0145] In the embodiment of the application, the asynchronous request can refer to the interaction request initiated by the client to the cloud server based on the interface in the first HTML page, for example, a query request, a login request, a view request, an input request, and the like.

[0146] In an example, the asynchronous request carries the URL of the requested page content, a request method, a network address of the cloud server, a network address of the client, and user agent information. The related description of the user agent information can be referred to the above, and will not be described here. The request method includes obtaining data, writing data, responding to operation, returning data, and the like.

[0147] The interface can be a JavaScript API interface. The JavaScript API includes at least one of an XMLHttpRequest (XHR) interface and a Fetch interface.

[0148] In a possible implementation, when the SDK in the single-page application monitors that the client initiates an asynchronous request, the asynchronous request is intercepted by hook, and the challenge data returned by the cloud server to the client is obtained before the data returned by the cloud server to the client reaches the kernel of the client. For example, the client executes the challenge data of the challenge response mechanism returned by the SDK of the cloud server.

[0149] In another possible implementation, when the SDK in the single-page application monitors that the client initiates an asynchronous request, the asynchronous request is intercepted by hook, and the information of the asynchronous request is sent to the cloud server, so that the cloud server verifies the legality of the asynchronous request initiated by the client based on the information of the asynchronous request, and when receiving the prompt information returned by the cloud server that the asynchronous request is legal, the client sends the operation of sending the asynchronous request to the cloud server to the cloud server through the SDK. When the prompt information returned by the cloud server that the asynchronous request is legal is not received, the SDK performs the operation of preventing the asynchronous request, that is, does not send the asynchronous request to the cloud server.

[0150] In the present application, the information of the asynchronous request can include user agent information carried by the asynchronous request, a network address of the client initiating the asynchronous request, and the like. The description of the user agent information and the network address of the client can be referred to the above, and will not be repeated here.

[0151] The client sending the asynchronous request to the cloud server through the SDK can include: the operation of sending the asynchronous request to the cloud server can be adding a legality label to the asynchronous request, and sending the asynchronous request with the added legality label to the cloud server. The legality label can be used to indicate that the asynchronous request is legal.

[0152] In step S440, the cloud server receives the asynchronous request initiated by the client, and verifies the legality of the asynchronous request.

[0153] In the present application, the legality verification is used to verify whether the asynchronous request initiated by the client is legal.

[0154] In the embodiments of the present application, the legality verification contains challenge data, and a data format of the challenge data matches a data format that can be parsed by the SDK. For example, the data format of the challenge data can be text / html, etc. The data format that can be parsed by the SDK includes, but is not limited to, application / javascript, application / json, application / xml, text / html, etc.

[0155] In an example, the client parses the challenge data in the legality verification through the SDK, generates a verification result of the challenge data in the legality verification, and the cloud server checks the verification result. If the verification result is checked successfully, it is determined that the legality verification of the asynchronous request is passed. If the verification result is checked unsuccessfully, it is determined that the legality verification of the asynchronous request fails.

[0156] In a possible implementation, the legality verification can be a challenge response, and the verification result can be response information generated by the client in response to the challenge. For example, when the cloud server receives the asynchronous request initiated by the client, the cloud server initiates a challenge response to the client that sends the asynchronous request. The client responds to the challenge, generates the response information through the SDK, and returns the response information to the cloud server.

[0157] For example, taking a JS challenge as an example, the response information can be an access token. The client parses a set of JS codes of the access token sent by the cloud server through the SDK, and generates the access token. The client sends the generated access token to the cloud server. The cloud server compares the access token sent by the client with a reference access token. If the access token sent by the client is consistent with the reference access token, it is determined that the verification of the response information is successful. If the access token sent by the client is inconsistent with the reference access token, it is determined that the verification of the response information fails.

[0158] In an example, the reference access token can be generated by the cloud server in advance. Alternatively, the cloud server parses a set of JS codes of the access token sent to the client to generate. The comparison in the embodiments of the present application is not specifically limited.

[0159] For example, taking a graphical verification as an example, the response information can be at least one of verification information of a graphical verification code, or mouse movement information or mouse click information. The cloud server sends a set of JS codes of the graphical verification code to the client, the client parses the JS code of the graphical verification code through the SDK, and displays the graphical verification code. The client generates the response information based on an input operation of the graphical verification code.

[0160] The input operation includes, but is not limited to, one or more of a character string input operation, a calculation result input operation, a mouse click operation or a mouse movement operation.

[0161] In a case where the response information includes mouse movement information, the cloud server determines whether the client has a mouse movement operation according to the mouse movement information. In a case where the client has a mouse movement operation, the cloud server obtains a mouse movement distance based on the mouse movement information. In a case where the mouse movement distance matches a reference movement distance, it is determined that the response information is verified successfully. In a case where the mouse movement distance does not match the reference movement distance, or the client does not have a mouse movement operation, it is determined that the response information is verified unsuccessfully.

[0162] In a case where the response information includes verification information of the CAPTCHA, the cloud server compares the verification information of the CAPTCHA with reference verification information of the CAPTCHA. In a case where the verification information of the CAPTCHA is consistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified successfully. In a case where the verification information of the CAPTCHA is inconsistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified unsuccessfully.

[0163] In a case where the response information includes both the verification information of the CAPTCHA and the mouse movement information, in a case where the client has a mouse movement operation, the mouse movement distance matches the reference movement distance, and the verification information of the CAPTCHA is consistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified successfully. In a case where the client does not have a mouse movement operation, or the client has a mouse movement operation, but the mouse movement distance does not match the reference movement distance, or the verification information of the CAPTCHA is inconsistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified unsuccessfully.

[0164] In a case where the response information includes both the verification information of the CAPTCHA and the mouse movement information, in a case where the client has a mouse movement operation, the mouse movement distance matches the reference movement distance, and the verification information of the CAPTCHA is consistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified successfully. In a case where the client does not have a mouse movement operation, or the client has a mouse movement operation, but the mouse movement distance does not match the reference movement distance, or the verification information of the CAPTCHA is inconsistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified unsuccessfully.

[0165] In a case where the response information includes both the verification information of the CAPTCHA and the mouse movement information, in a case where the client has a mouse movement operation, the mouse movement distance matches the reference movement distance, and the verification information of the CAPTCHA is consistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified successfully. In a case where the client does not have a mouse movement operation, or the client has a mouse movement operation, but the mouse movement distance does not match the reference movement distance, or the verification information of the CAPTCHA is inconsistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified unsuccessfully.

[0166] In a case where the response information includes both the verification information of the CAPTCHA and the mouse movement information, in a case where the client has a mouse movement operation, the mouse movement distance matches the reference movement distance, and the verification information of the CAPTCHA is consistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified successfully. In a case where the client does not have a mouse movement operation, or the client has a mouse movement operation, but the mouse movement distance does not match the reference movement distance, or the verification information of the CAPTCHA is inconsistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified unsuccessfully.

[0167] In a case where the response information includes both the verification information of the CAPTCHA and the mouse movement information, in a case where the client has a mouse movement operation, the mouse movement distance matches the reference movement distance, and the verification information of the CAPTCHA is consistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified successfully. In a case where the client does not have a mouse movement operation, or the client has a mouse movement operation, but the mouse movement distance does not match the reference movement distance, or the verification information of the CAPTCHA is inconsistent with the reference verification information of the CAPTCHA, it is determined that the response information is verified unsuccessfully.

[0168] In the case that the client intercepts the asynchronous request and sends the information of the asynchronous request to the cloud server first in S430, the cloud server returns prompt information that the asynchronous request is verified to be legal to the client when determining that the asynchronous request is verified to be legal. The client sends the asynchronous request to the cloud server in response to the prompt information, such as sending the asynchronous request carrying the legality mark to the cloud server. The cloud server responds to the received asynchronous request carrying the legality mark.

[0169] In the present application, the cloud server responding to the asynchronous request can include: obtaining the page content information corresponding to the asynchronous request according to the page URL carried by the asynchronous request, and the cloud server sending the page content information to the client. The client renders the received page content information and displays the page content by refreshing the first HTML page. That is, the page content corresponding to the asynchronous request is displayed in the first HTML page.

[0170] In step S460, based on the verification result, it is determined that the asynchronous request legality verification fails, and then the cloud server refuses to respond to the asynchronous request.

[0171] In the case that the client changes the sending logic of the asynchronous request through the SDK and sends the asynchronous request to the cloud server in S430, the cloud server refusing to respond to the asynchronous request can be that the cloud server discards the asynchronous request. Or the cloud server does not process the asynchronous request.

[0172] In the case that the client intercepts the asynchronous request and sends the information of the asynchronous request to the cloud server first in S430, the cloud server refusing to respond to the asynchronous request can be that the cloud server sends information indicating that the asynchronous request legality verification fails to the client.

[0173] In a possible implementation, when the asynchronous request legality verification fails, biological information verification is initiated to the client. The biological information verification is used to verify whether the visitor initiating the asynchronous request is a real user. When it is determined that the visitor initiating the asynchronous request is a real user, the cloud server responds to the asynchronous request. When it is determined that the visitor initiating the asynchronous request is not a real user, the cloud server refuses to respond to the asynchronous request.

[0174] The biological information verification can be live detection initiated to the client to identify whether the visitor of the asynchronous request is a real user.

[0175] In a possible implementation, after the cloud server refuses to respond to the asynchronous request, the visitor information of the client is written into a risk client list. The risk client list includes at least one visitor with a crawler risk. The visitor information includes device information, user agent information, network address, etc. of the client.

[0176] Based on the embodiment provided in FIG. 4, by providing the client with the HTML page containing the SDK, since the data format that can be parsed by the SDK matches the challenge data format received by the client, the client can parse the challenge data in the legality verification through the SDK and verify the result when performing legality verification on the asynchronous request. In turn, the cloud server can determine that the asynchronous request verification fails and that the asynchronous request verification passes based on the verification result. This enables the legality verification to be initiated in the single-page application scenario to identify the crawler program. In this way, the problem that the challenge and response mechanism in the related art cannot be applied to the single-page application is solved, thereby ensuring data security in the single-page application scenario.

[0177] The network request processing method provided in the embodiments of the present application will be introduced below based on another system architecture in the cloud service scenario shown in FIG. 2. As shown in FIG. 5, FIG. 5 is another flowchart of the network request processing method provided in the embodiments of the present application. The network request processing method shown includes steps S510 to S560.

[0178] Step S510, the client sends a page access request to the proxy end.

[0179] In a possible implementation, the client sends a page access request to the proxy end according to the above step S410, and the embodiments of the present application will not be repeated here.

[0180] Step S520, the proxy end returns a first HTML page to the client.

[0181] The first HTML page is the same as the description of the first HTML page in the above step S420, and the embodiments of the present application will not be repeated here.

[0182] In a possible implementation, the proxy end forwards the page access request to the cloud server. The cloud server returns a first HTML page to the proxy end in response to the page access request. The proxy end forwards the first HTML page returned by the cloud server to the client.

[0183] In another possible implementation, the proxy end forwards the page access request to the cloud server. The cloud server returns a second HTML page to the proxy end in response to the page access request. The proxy end injects an SDK into the received second HTML page to generate a first HTML page. The proxy end returns the first HTML page to the client.

[0184] The second HTML page can refer to the HTML page of the single-page application requested by the client. The second HTML page does not contain an SDK.

[0185] Step S530, the client sends an asynchronous request to the proxy end based on the first HTML page.

[0186] The asynchronous request is the same as the description of the asynchronous request in S430, and details are not repeated herein.

[0187] In a possible implementation, similar to the sending of the information of the asynchronous request to the cloud server by the client in S430, the client executes the SDK, and the SDK forwards the information of the asynchronous request to the proxy when the client initiates the asynchronous request. Details are not repeated herein.

[0188] In another possible implementation, similar to the sending of the asynchronous request to the cloud server by the client through the SDK in S430, the SDK in the single-page application monitors the client to initiate the asynchronous request, hooks the asynchronous request, and obtains the data returned by the proxy to the client before the data returned by the proxy to the client reaches the kernel of the client. For example, the client executes the SDK to obtain the verification information of the challenge-response mechanism returned by the proxy. Details are not repeated herein.

[0189] In S540, the proxy receives the asynchronous request initiated by the client, and performs legality verification on the asynchronous request.

[0190] In the embodiment of the application, the legality verification can be performed with reference to S440, and details are not repeated herein.

[0191] In S550, when it is determined that the legality verification of the asynchronous request is passed, the proxy forwards the asynchronous request to the cloud server.

[0192] In a possible implementation, the proxy forwards the asynchronous request to the cloud server, and the cloud server responds to the asynchronous request with reference to S450. Details are not repeated herein.

[0193] For example, the proxy adds a legality label to the asynchronous request, and forwards the asynchronous request with the added legality label to the cloud server.

[0194] In S560, when it is determined that the legality verification of the asynchronous request fails, the proxy blocks the asynchronous request.

[0195] In a possible implementation, when the client changes the sending logic of the asynchronous request through the SDK in S530, and sends the asynchronous request to the proxy, when the legality verification of the asynchronous request fails, the proxy refuses to forward the asynchronous request, and discards the asynchronous request, thereby avoiding the collection of the data of the cloud server by the web crawler.

[0196] In another possible implementation, when the client intercepts the asynchronous request in step S530, the proxy end sends information indicating that the legality verification of the asynchronous request fails to the client.

[0197] Based on the embodiment provided in FIG. 5, in the single-page application scenario, the client is provided with an HTML page containing the SDK. Since the data format that can be parsed by the SDK matches the challenge data format received by the client, when the legality of the asynchronous request is verified, the client can parse the challenge data in the legality verification through the SDK and verify the result. Then, the proxy end can determine the asynchronous request verification failure or determine that the asynchronous request verification is passed based on the verification result. In the single-page application scenario, the proxy end can initiate the legality verification and identify the crawler program. In this way, the problem that the challenge and response mechanism in the related art cannot be applied to the single-page application is solved, thereby ensuring the data security in the single-page application scenario.

[0198] Next, based on the embodiment provided in FIG. 4, the implementation of the cloud server generating the first HTML page is introduced. It can be understood that the implementation of the proxy end generating the first HTML page can refer to the implementation of the cloud server generating the first HTML page, and the embodiments of the present application will not be described here.

[0199] As shown in FIG. 6, FIG. 6 is a flowchart of the cloud server generating the first HTML page provided by the embodiments of the present application. The flowchart of the cloud server generating the first HTML page includes steps S421 to S423.

[0200] Step S421, the cloud server obtains the second HTML page corresponding to the page access request based on the page access request sent by the client.

[0201] In a possible implementation, the cloud server can refer to the above S420 to obtain the second HTML page corresponding to the page access request according to the network link carried by the page access request, and the embodiments of the present application will not be described here.

[0202] Step S422, the cloud server injects the SDK into the second HTML page to generate the first HTML page.

[0203] In a possible implementation, the cloud server can increase the source code of the SDK in the source code information of the second HTML page to form the source code information of the first HTML page. The first HTML page is generated through the source code information of the first HTML page.

[0204] For example, the source code of the SDK is embedded in the source code of the second HTML page.

[0205] In another possible implementation, since the source code of the SDK is directly added in the second HTML page, the amount of data transmitted from the cloud server to the client increases, and the time for the client to load the first HTML page increases. Therefore, in order to reduce the amount of data of the first HTML page, the resource access address of the SDK can be injected in the second HTML page to generate the first HTML page.

[0206] For example, the cloud server can embed the source code of the SDK in the source code of the second HTML page, and the source code of the SDK can be as follows: <script>标签，将该<script>标签的资源属性(src属性)指向SDK脚本文件的资源地址(URL)。其中，src属性用于指向外部资源的存储位置。如此，可以在不修改第二HTML页面的源代码的情况下，通过嵌入标签的形式，将SDK注入第二HTML页面中。

[0207] 步骤S423，云服务端向客户端返回第一HTML页面。

[0208] 在一种可能的实现方式中，如图7所示，云服务端10中部署有处理模块12和代理模块11。处理模块12和代理模块11耦合。

[0209] 如图7所示，代理模块11接收客户端200发送的页面访问请求(S71)。并将页面访问请求转发至处理模块12(S72)。处理模块12响应页面访问请求，获取页面访问请求对应的第二HTML页面(S73)。处理模块12将第二HTML页面转发至代理模块11(S74)。代理模块11执行S422，对第二HTML页面注入SDK，生成第一HTML页面(S75)。代理模块11向客户端200返回第一HTML页面(S76)。

[0210] 在一种可能的实现方式中，单页面应用对应的管理人员可以在云服务端中配置单页面应用的合法性验证功能。云服务端在接收到客户端发送的页面访问请求后，若该页面访问请求所请求的页面开启了合法性验证功能，则云服务端执行上述步骤S422至S423，向客户端返回第一HTML页面。若该页面访问请求所请求的页面未开启合法性验证功能，则云服务端向客户端返回第二HTML页面。

[0211] 在一种可能的实现方式中，客户端在接收到第一HTML页面时，显示该第一HTML页面。当第一HTML页面中包含SDK的源代码时，客户端运行SDK的源代码，执行SDK。当第一HTML页面中按SDK的资源访问地址时，客户端通过该资源访问地址获取SDK的脚本文件，通过运行该SDK的脚本文件执行SDK。

[0212] 在本申请实施例中，在SDK监测到客户端发起异步请求时，代理端或者云服务端根据客户端发起的异步请求，发起合法性验证，评估客户端发起的异步请求的合法性。

[0213] 接下来，以图4提供的实施例为例，对云服务端评估异步请求的合法性的实现方式进行介绍。可理解的，代理端评估异步请求的合法性的实现方式可以参考云服务端评估异步请求的合法性的实现方式。

[0214] 如图8所示，图8为本申请实施例提供的云服务端评估异步请求的合法性的流程示意图。云服务端向客户端发送合法性验证请求(S441)。客户端通过SDK解析合法性验证请求包含的挑战数据，生成挑战数据的应答信息(S442)。客户端向云服务端返回应答信息(S443)。云服务端对客户端返回的应答信息进行校验(S444)。若应答信息校验成功，则确定异步请求合法性验证通过(S445)。若应答信息校验失败，则确定异步请求合法性验证失败(S446)。

[0215] 其中，合法性验证请求用于指示客户端通过SDK解析合法性验证请求包含的挑战数据，生成挑战数据的应答信息。

[0216] 在本申请中，客户端通过SDK监测到云服务端返回合法性验证请求时，在合法性验证请求传递到客户端的内核之前，对合法性验证请求进行拦截，并响应合法性验证请求，获取合法性验证请求包含的挑战数据，对挑战数据进行解析，并执行合法性验证，生成应答信息。如此，将合法性验证请求的响应由客户端的内核变为SDK，通过SDK解析合法性验证请求携带的JS代码，从而解决相关技术中由于挑战应答机制中的JS代码的数据类型与客户端能够解析的数据类型不匹配，导致客户端不能解析挑战应答机制中的JS代码的问题。

[0217] 在一种可能的实现方式中，挑战数据包括一组访问令牌的JS代码。在另一种可能的实现方式中，挑战数据包括一组用于生成图形验证码的JS代码。

[0218] 在客户端通过SDK解析合法性验证请求得到一组访问令牌的JS代码的情况下，客户端通过SDK执行上述S440中的JS挑战，生成应答信息。

[0219] 例如，以访问令牌为"verify”为例。云服务端向客户端返回用于生成"verify”的JS代码。当客户端中执行SDK时，SDK对用于生成"verify”的JS代码进行解析，生成"verify”访问令牌发送云服务端。当客户端中未执行SDK时，由于云服务端返回的用于生成"verify”的JS代码的数据格式与客户端可解析的数据格式不匹配，使得客户端解析生成"verify”的JS代码时报错，或者解析出的访问令牌不是"verify”。

[0220] 在客户端通过SDK解析合法性验证请求得到一组图形验证码的JS代码的情况下，客户端通过SDK执行上述S440中的图形验证，生成应答信息。

[0221] 在一种可能的实现方式中，云服务端参考上述S440中应答信息的校验方式，对客户端返回的应答信息进行校验。

[0222] 在一种可能的实现方式中，当云服务端返回合法性验证请求中携带有指示合法性验证的字段时，客户端通过SDK解析合法性验证请求包含的挑战数据。当云服务端返回合法性验证请求中未携带有指示合法性验证的字段时，客户端不通过SDK处理合法性验证请求。

[0223] 例如，当合法性验证请求的报文中包含"X-Header:WAF-JS-Challenge”时，确定合法性验证请求中携带有指示合法性验证的字段。

[0224] 在一种可能的实现方式中，如果云服务端对于客户端的每个异步请求，都向发送该异步请求的客户端发送合法性验证请求，将增加云服务端处理数据量，增加云服务端响应异步请求的时长，增加客户端刷新页面的时延。基于此，为降低客户端刷新的时延，以及减少云服务端响应异步请求的时长，云服务端对于接收到的每个异步请求，识别该异步请求是否为可疑请求。在该异步请求为可疑请求的情况下，云服务端向发送异步请求的客户端发送合法性验证请求。如此，云服务端通过可疑请求识别对接收到的异步请求进行筛选，仅对发送识别为可疑请求的异步请求的客户端发送合法性验证请求，减少云服务端需要校验的应答信息的数据量，从而减少云服务端响应异步请求的时长，降低客户端刷新的时延。

[0225] 基于图8提供的实施例，如图9所示，在S441之前，客户端向云服务端发送异步请求(S91)。云服务端根据异步请求携带的用户代理信息或客户端的请求频率中至少一项，识别异步请求是否为可疑请求(S92)。若异步请求为可疑请求，则云服务端向客户端发送合法性验证请求(S93)。若异步请求为非可疑请求，则云服务端允许响应异步请求(S94)。

[0226] 在第一种可能的实现方式中，异步请求中携带有用户代理信息。云服务端根据用户代理信息获取异步请求对应的访问者信息。

[0227] 其中，用户代理信息可以指一串字符串，用于标识发起异步请求的访问者信息。例如，以客户端为Windows系统中Chrome浏览器版本为88.0.4324.182为例，客户端发起的异步请求中携带的用户代理信息为"Mozilla / 5.0(Windows NT 10.0；Win64；x64)AppleWebKit / 537.36(KHTML,like Gecko)Chrome / 88.0.4324.182Safari / 537.36”。

[0228] 在异步请求是由用户触发的情况下，或者异步请求是由搜索引擎爬虫触发的情况下，异步请求携带的用户代理信息包含所使用的浏览器信息。而当异步请求是由恶意爬虫程序触发时，异步请求携带的用户代理信息中可能不包含浏览器信息。此外，当异步请求是由恶意爬虫程序触发时，异步请求携带的用户代理信息中可能携带有特定的字段。因此在本申请实施例中，可以根据异步请求中携带的用户代理信息识别异步请求是否为爬虫程序触发的。

[0229] 其中，特定的字段用于标识该异步请求为爬虫程序触发的。特定的字段例如为：爬虫程序的开发软件信息。例如，当用户代理信息包含"Python-urllib / 3.7”字段时，确定异步请求为爬虫程序触发。

[0230] 例如，可以通过识别异步请求携带的用户代理信息中是否包含浏览器信息。若包含浏览器信息，则确定该异步请求为非可疑请求。若不包含浏览器信息，则确定该异步请求为可疑请求。

[0231] 又例如，可以通过识别异步请求携带的用户代理信息中是否包含特定的字段。若异步请求携带的用户代理信息中包含特定的字段，则确定该异步请求为可疑请求。若异步请求携带的用户代理信息中不包含特定的字段，则确定该异步请求为非可疑请求。

[0232] 又例如，可以根据异步请求携带的用户代理信息进行概率预测，得到该异步请求为可疑请求的预测概率。当异步请求为可疑请求的预测概率大于或等于概率阈值时，确定该异步请求为可疑请求。当会话信息为可疑请求的预测概率小于概率阈值时，确定该异步请求为非可疑请求。

[0233] 在一示例中，可以将异步请求携带的用户代理信息输入预测模型进行概率预测。其中，预测模型可以是基于机器学习的预测模型。或者预测模型可以是基于神经网络的预测模型。

[0234] 在第二种可能的实现方式中，通过爬虫程序触发异步请求时，往往会在一段时间内发送多个异步请求，以进行数据抓取。即爬虫的请求频率往往比用户触发的异步请求的请求频率高。因此，可以根据客户端的请求频率识别异步请求是否为可疑请求。例如，当客户端的请求频率大于或等于频率阈值时，确定该异步请求会可疑请求。当客户端的请求频率小于频率阈值时，确定该异步请求会非可疑请求。

[0235] 在一示例中，异步请求中携带有访问者的网络地址。访问者的网络地址可以用于识别客户端。即不同的客户端具有不同的访问者的网络地址。云服务端根据访问者的网络地址，统计预设时长内接收到的多个异步请求中与该访问者的网络地址相同的异步请求的数量，得到发送该异步请求的客户端的请求频次。根据客户端的请求频次以及预设时长，得到客户端的请求频率。

[0236] 其中，预设时长可以是1毫秒、30毫秒、1秒钟、10秒钟、或者1分钟中的任一种。本申请实施例对此不进行具体限定。

[0237] 在第三种可能的实现方式中，由于爬虫攻击者可能会搭建IP池，通过IP池中多个IP向云服务端发起爬虫攻击。并且，爬虫程序触发的异步请求中可能为伪造浏览器信息。如果仅从用户代理信息，或者客户端的请求频率来识别可疑请求，可能会造成可疑请求的误识别或漏识别，从而降低云服务端的数据安全性。基于此，为提高可疑请求的识别可靠性，可以结合异步请求携带的用户代理信息以及客户端的请求频率，识别异步请求是否为可疑请求。

[0238] 在一种示例中，可以对异步请求携带的用户代理信息以及客户端的请求频率进行特征提取，得到异步请求对应的联合特征。将联合特征与爬虫程序触发的异步请求对应的模板特征进行比对。若联合特征与模板特征之间的特征相似度大于或等于预设相似度阈值，则确定该异步请求为可疑请求。若联合特征与模板特征之间的特征相似度小于预设相似度阈值，则确定该异步请求为非可疑请求。

[0239] 其中，可以通过特征提取模型对异步请求携带的用户代理信息以及客户端的请求频率进行特征提取，得到异步请求对应的联合特征。其中，特征提取模块可以是基于transformer的模型。

[0240] 其中，可以通过联合特征与模板特征的之间的距离得到联合特征与模板特征之间的特征相似度。例如，计算联合特征与模板特征的之间的欧式距离、马氏距离、绝对值或二范数，得到联合特征与模板特征之间的特征相似度。

[0241] 在又一示例中，可以根据预测模型、异步请求携带的用户代理信息以及客户端的请求频率进行概率预测，得到该异步请求为可疑请求的预测概率。

[0242] 在又一示例中，当异步请求携带的用户代理信息中不包含浏览器信息，或者包含特定的字段，并且客户端的请求频率大于或等于频率阈值时，确定异步请求为可疑请求。当异步请求携带的用户代理信息中包含浏览器信息，且不包含特定的字段，并且客户端的请求频率小于频率阈值时，确定该异步请求会非可疑请求。

[0243] 在本申请实施例中，为进一步降低云服务端对爬虫攻击的风险，云服务端在识别到异步请求为可疑请求的情况下，根据异步请求的可疑程度，发起不同的挑战应答。

[0244] 其中，异步请求的可疑程度可以根据异步请求为可疑请求的预测概率得到。预测概率越大，相应的异步请求的可疑程度越大。

[0245] 在第一种可能的实现方式中，当异步请求的可疑程度小于预设程度阈值时，发起JS挑战。当异步请求的可疑程度大于或等于预设程度阈值时，发起图形验证。

[0246] 在第二种可能的实现方式中，当异步请求的可疑程度小于预设程度阈值时，发起JS挑战或发起图形验证。当异步请求的可疑程度大于或等于预设程度阈值时，发起JS挑战和图形验证。

[0247] 在第三种可能的实现方式中，可以预先设置不同可疑程度对应的挑战应答方式。在确定异步请求的可疑程度之后，按照异步请求的可疑程度从多个不同可疑程度对应的挑战应答方式中，选取挑战应答方式的可疑程度与异步请求的可疑程度匹配的挑战应答方式。

[0248] 示例性的，以JS挑战为例，预先按照不同可疑程度设置相应的访问令牌的JS代码。在确定异步请求的可疑程度之后，按照异步请求的可疑程度从多个不同可疑程度对应的访问令牌的JS代码，选取与异步请求的可疑程度匹配的访问令牌的JS代码。

[0249] 在本申请实施例中，云服务端确定异步请求合法性验证通过时，云服务端按照上述S460对异步请求进行响应。

[0250] 在一种可能的实现方式中，以S430中客户端通过SDK更改异步请求的发送逻辑，将异步请求发送给云服务端为例，基于图7提供的实施例，在云服务端中部署有代理模块11和处理模块12的情况下，如图10所示，客户端执行SDK，异步请求发送云服务端的代理模块11(S101)。代理模块11识别接收到的异步请求是否为可疑请求(S102)。若异步请求为可疑请求，则代理模块11向客户端返回合法性验证请求(S1031)。若异步请求为非可疑请求，则代理模块11向处理模块12转发异步请求(S1032)。客户端中的SDK接收合法性验证请求，并通过SDK执行合法性验证，生成应答信息(S104)。客户端向代理模块11返回应答信息(S105)。代理模块11对客户端返回的应答信息进行校验(S106)。若应答信息校验成功，则代理模块11向处理模块12转发异步请求(S1071)。若应答信息校验失败，则代理模块11阻止异步请求(S1072)。

[0251] 为更好的说明本申请实施例提供的网络请求处理方法，以云服务系统100包括代理端40和云服务端10为例，接下来分别以客户端通过XHR接口发起异步请求和客户端通过Fetch接口发起异步请求为例，对本申请实施例提供的网络请求处理方法的应用场景进行介绍。

[0252] 示例性的，以客户端为浏览器、代理端40为Web应用程序防火墙(WAF)、云服务端为浏览器31的源服务器为例，如图11所示，图11为本申请实施例提供的XHR接口场景下的页面访问请求处理流程示意图。浏览器31向WAF发送页面访问请求(S111)。WAF向源服务器发送页面访问请求(S112)。源服务器向WAF返回第二HTML页面(S113)。WAF对第二HTML页面进行SDK注入，生成第一HTML页面(S114)。WAF向客户端返回第一HTML页面(S115)。客户端执行SDK(S116)。客户端向源服务器发起XHR异步请求(S117)。WAF识别XHR异步请求是否为可疑请求(S118)。若XHR异步请求为非可疑请求，WAF将XHR异步请求转发源服务器(S119.2)。若XHR异步请求为可疑请求，WAF向浏览器发起挑战应答(S119.1)。浏览器通过SDK执行挑战，生成应答信息(S1110)。浏览器向WAF返回应答信息(S1111)。WAF对应答信息进行校验(S1112)，若应答信息校验成功，则WAF将XHR异步请求转发源服务器(S1113.2)。若应答信息校验失败，则WAF将阻止XHR异步请求发送源服务器(S1113.1)。

[0253] 在图11的基础上，当客户端通过Fetch接口发起异步请求时，如图12所示。图12为本申请实施例提供的Fetch接口场景下的页面访问请求处理流程示意图。图12所示的页面访问请求处理流程中除客户端发起Fetch异步请求以及Fetch异步请求的合法性验证流程之外，其余流程与图11中所示的页面访问请求处理流程中除XHR异步请求以及XHR异步请求的合法性验证流程之外的流程相同，在此不予赘述。

[0254] 客户端向WAF发送Fetch异步请求(S127)。WAF识别Fetch异步请求是否为可疑请求(S128)。若Fetch异步请求为非可疑请求，WAF将Fetch异步请求转发源服务器(S129.2)。若Fetch异步请求为可疑请求，WAF向浏览器发起挑战应答(S129.1)。浏览器通过SDK执行挑战，生成应答信息(S1210)。浏览器向WAF返回应答信息(S1211)。WAF对应答信息进行校验(S1212)，若应答信息校验成功，则WAF将Fetch异步请求转发源服务器(S1213.2)。若应答信息校验失败，则WAF将阻止Fetch异步请求发送源服务器(S1213.1)。

[0255] 上述主要以云服务系统100中各模块之间的交互为例，对本申请实施例提供的网络请求处理方法进行了介绍。可以理解的是，上述云服务系统100为了实现上述功能，其包含了执行各个功能相应的硬件结构和 / 或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法操作，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。

[0256] 本申请可以根据上述方法示例对上述云服务系统100进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块12中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。可以理解的是，本申请实施例对装置、模块的命名、分组是示意性的，仅仅为一种逻辑功能分组，实际实现时可以有另外的分组方式。

[0257] 例如，在云服务系统100包括云服务端10时，云服务端10还可以命名为网络请求处理装置13。该网络请求处理装置13可以包括不同于图1中云服务端所示的功能层的功能模块。如图13所示，图13为本申请实施例提供的网络请求处理装置13的结构示意图。所示的网络请求处理装置13包括通信模块131、处理模块132。

[0258] 其中，通信模块131用于接收客户端发送的异步请求。该异步请求用于更新基于单页面应用的第一HTML页面。其中，第一HTML页面包含SDK。例如，通信模块执行上述图4中的S440。

[0259] 处理模块132，用于对异步请求进行合法性验证。其中，合法性验证包含挑战数据，挑战数据的数据格式与SDK可解析的数据格式匹配；SDK用于客户端解析针对异步请求发起的合法性验证中挑战数据，生成挑战数据的验证结果。例如，处理模块132用于执行上述图4中的S440。

[0260] 处理模块133，用于基于验证结果确定异步请求验证通过，则允许响应异步请求。以及基于验证结果确定异步请求验证失败，则拒绝响应异步请求。例如，处理模块133用于执行上述图4中的S450和S460。

[0261] 其中，通信模块131、处理模块132均可以通过软件实现，或者可以通过硬件实现。示例性，接下来以通信模块131为例，介绍通信模块131的实现方式。类似的，处理模块132的实现方式可以参考通信模块131的实现方式。

[0262] 模块作为软件功能单元的一种举例，通信模块131可以包括运行在计算实例上的代码。其中，计算实例可以包括物理主机(计算设备)、虚拟机、容器中的至少一种。进一步地，上述计算实例可以是一台或者多台。例如，通信模块131可以包括运行在多个主机 / 虚拟机 / 容器上的代码。需要说明的是，用于运行该代码的多个主机 / 虚拟机 / 容器可以分布在相同的区域(region)中，也可以分布在不同的region中。进一步地，用于运行该代码的多个主机 / 虚拟机 / 容器可以分布在相同的可用区(availability zone，AZ)中，也可以分布在不同的AZ中，每个AZ包括一个数据中心或多个地理位置相近的数据中心。其中，通常一个region可以包括多个AZ。

[0263] 同样，用于运行该代码的多个主机 / 虚拟机 / 容器可以分布在同一个虚拟私有云(virtual private cloud，VPC)中，也可以分布在多个VPC中。其中，通常一个VPC设置在一个region内，同一region内两个VPC之间，以及不同region的VPC之间跨区通信需在每个VPC内设置通信网关，经通信网关实现VPC之间的互连。

[0264] 模块作为硬件功能单元的一种举例，通信模块131可以包括至少一个计算设备，如服务器等。或者，通信模块131也可以是利用专用集成电路(application-specific integrated circuit，ASIC)实现、或可编程逻辑器件(programmable logic device，PLD)实现的设备等。其中，上述PLD可以是复杂程序逻辑器件(complex programmable logical device，CPLD)、现场可编程门阵列(field-programmable gate array，FPGA)、通用阵列逻辑(generic array logic，GAL)或其任意组合实现。

[0265] 通信模块131包括的多个计算设备可以分布在相同的region中，也可以分布在不同的region中。通信模块131包括的多个计算设备可以分布在相同的AZ中，也可以分布在不同的AZ中。同样，通信模块131包括的多个计算设备可以分布在同一个VPC中，也可以分布在多个VPC中。其中，多个计算设备可以是服务器、ASIC、PLD、CPLD、FPGA和GAL等计算设备的任意组合。

[0266] 需要说明的是，在其他实施例中，通信模块131可以用于执行网络请求处理方法中的任意步骤，处理模块132可以用于执行网络请求处理方法中的任意步骤。通信模块131、处理模块132负责实现的步骤可根据需要指定，通过通信模块131、处理模块132分别实现网络请求处理方法中不同的步骤来实现网络请求处理装置的全部功能。

[0267] 又例如，在云服务系统包括代理端40和云服务端10的情况下，代理端40还可以命名为代理装置14。如图14所示，图14为本申请实施例提供的代理装置14的结构示意图。所示的代理装置14包括通信模块141和处理模块142。

[0268] 其中，通信模块141，用于接收客户端发送的异步请求，异步请求用于更新基于单页面应用的第一超文本标记语言HTML页面；其中，第一HTML页面包含软件开发工具包SDK。例如，通信模块141执行上述图5中的S540。

[0269] 处理模块142，用于对异步请求进行合法性验证，其中，合法性验证包含挑战数据，挑战数据的数据格式与SDK可解析的数据格式匹配；SDK用于客户端解析针对异步请求发起的合法性验证中挑战数据，生成挑战数据的验证结果。例如，处理模块142用于执行上述图5中的S540。

[0270] 通信模块141，还用于基于验证结果确定异步请求验证通过，则向云服务端转发异步请求。以及用于基于验证结果确定异步请求验证失败，则阻止异步请求。例如，通信模块141执行上述图5中的S550和S560。

[0271] 其中，通信模块141和处理模块142均可以通过软件实现，或者通过硬件实现。通信模块141和处理模块142的实现方式可以参考上述通信模块131的实现方式。

[0272] 本申请实施例还提供一种用于执行上述网络请求处理方法的计算设备15。

[0273] 在一示例中，该计算设备15包括上述图13所示的网络请求处理装置13。该网络请求处理装置13包括通信模块131、处理模块132。

[0274] 在又一示例中，该计算设备15包括上述图14提供的代理装置14。该代理装置14包括通信模块141和处理模块142。

[0275] 在又一示例中，如图15所示，该计算设备15包括：总线152、处理器154、存储器156和通信接口158。处理器154、存储器156和通信接口158之间通过总线152通信。计算设备15可以是服务器或终端设备。应理解，本申请不限定计算设备15中的处理器、存储器的个数。

[0276] 总线152可以是外设部件互连标准(peripheral component interconnect，PCI)总线或扩展工业标准结构(extended industry standard architecture，EISA)总线等。总线可以分为地址总线、数据总线、控制总线等。为便于表示，图15中仅用一条线表示，但并不表示仅有一根总线或一种类型的总线。总线152可包括在计算设备15各个部件(例如，存储器156、处理器154、通信接口158)之间传送信息的通路。

[0277] 处理器154可以包括中央处理器(central processing unit，CPU)、图形处理器(graphics processing unit，GPU)、微处理器(micro processor，MP)或者数字信号处理器(digital signal processor，DSP)等处理器中的任意一种或多种。

[0278] 存储器156可以包括易失性存储器(volatile memory)，例如随机存取存储器(random access memory，RAM)。处理器154还可以包括非易失性存储器(non-volatile memory)，例如只读存储器(read-only memory，ROM)，快闪存储器，机械硬盘(hard disk drive，HDD)或固态硬盘(solid state drive，SSD)。

[0279] 如图15所示，存储器156中存储有可执行的程序代码，处理器154执行该可执行的程序代码以分别实现前述通信模块131、处理模块132的功能，从而实现网络请求处理方法。也即，存储器156上存有用于执行网络请求处理方法的指令。

[0280] 或者，存储器156中存储有可执行的程序代码，处理器154执行该可执行的程序代码以分别实现前述通信模块141和处理模块142的功能(图15中未示出)，从而实现网络请求处理方法。也即，存储器156上存有用于执行网络请求处理方法的指令。

[0281] 或者，存储器156中存储有可执行的代码，处理器154执行该可执行的代码以实现前述网络请求处理装置13、或者代理装置14的功能，从而实现网络请求处理方法。也即，存储器156上存有用于执行网络请求处理方法的指令。

[0282] 通信接口158使用例如但不限于网络接口卡、收发器一类的收发模块，来实现计算设备15与其他设备或通信网络之间的通信。

[0283] 上述方法实施例揭示的网络请求处理方法可以应用于处理器154中，或者由处理器154实现。处理器154可以是一种集成电路芯片，具有信号处理器能力。

[0284] 在实现过程中，上述方法的各步骤可以通过处理器154中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器154可以是通用处理器，包括中央处理器(central processing unit，CPU)、网络处理器(network processor，NP)等；还可以是数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其他可编程逻辑器件、分立电子管或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器156，处理器154读取存储器156中的信息，结合其硬件完成上述方法的步骤。

[0285] 在一种可能的实现方式中，处理器154还可用于执行网络请求处理方法，具体实现可参考上述网络请求处理方法提供的实施例，本申请实施例在此不再赘述。

[0286] 本申请实施例中，芯片系统可以由芯片构成，也可以包括芯片和其他分立器件。

[0287] 本申请实施例还提供一种计算设备集群16。该计算设备集群16包括至少一台如图15所示的计算设备。该计算设备可以是服务器，例如是中心服务器、边缘服务器，或者是本地数据中心中的本地服务器。在一些实施例中，计算设备也可以是台式机、笔记本电脑或者智能手机等终端设备。

[0288] 在一示例中，该计算设备集群16包括上述图13所示的网络请求处理装置13。该网络请求处理装置13包括通信模块131、处理模块132。

[0289] 在又一示例中，该计算设备集群16包括上述图14提供的代理装置14。该代理装置14包括通信模块141和处理模块142。

[0290] 在又一示例中，如图16所示，计算设备集群16包括至少一个计算设备15。该计算设备15包括：总线152、处理器154、存储器156和通信接口158。处理器154、存储器156和通信接口158之间通过总线152通信。计算设备15可以是服务器或终端设备。

[0291] 计算设备集群16中的一个或多个计算设备15中的存储器156中可以存有相同的用于执行网络请求处理方法的指令。

[0292] 在一些可能的实现方式中，该计算设备集群16中的一个或多个计算设备15的存储器156中也可以分别存有用于执行网络请求处理方法的部分指令。换言之，一个或多个计算设备15的组合可以共同执行用于执行网络请求处理方法的指令。

[0293] 需要说明的是，计算设备集群16中的不同的计算设备15中的存储器156可以存储不同的指令，分别用于执行网络请求处理装置13的部分功能。也即，如图16所示，不同的计算设备15中的存储器156存储的指令可以实现通信模块131、处理模块132中的一个或多个模块的功能。或者计算设备集群16中的不同的计算设备15中的存储器156可以存储不同的指令，分别用于执行代理装置14的部分功能。也即，不同的计算设备15中的存储器156存储的指令可以实现通信模块141和处理模块142中的一个或多个模块的功能(图16中未示出)。

[0294] 在一些可能的实现方式中，计算设备集群16中的一个或多个计算设备可以通过网络连接。其中，网络可以是广域网或局域网等等。

[0295] 示例性的，以计算设备集群16包括如图13所示的网络请求处理装置13为例，图16示出了一种可能的实现方式。如图17所示，两个计算设备15A和15B之间通过网络进行连接。具体地，通过各个计算设备中的通信接口与网络进行连接。在一种可能的实现方式中，计算设备15A中的存储器156中存有执行处理模块132的功能的指令。同时，计算设备15B中的存储器156中存有执行通信模块131的功能的指令。

[0296] 图17所示的计算设备集群16之间的连接方式可以是考虑到本申请提供的网络请求处理方法需要接收异步请求，在评估异步请求的合法性之后，当异步请求的合法性验证通过的时候，才允许响应异步请求。而且接收异步请求的功能和评估异步请求的合法性的功能可以由不同的计算设备实现。因此，考虑将处

[0297] 应理解，图17中示出的计算设备15A的功能也可以由多个计算设备15完成。同样，计算设备15B的功能也可以由多个计算设备15完成。

[0298] 本申请实施例还提供了一种包含指令的计算机程序产品。计算机程序产品可以是包含指令的，能够运行在计算设备上或被储存在任何可用介质中的软件或程序产品。当计算机程序产品在至少一个计算设备上运行时，使得至少一个计算设备执行网络请求处理方法。

[0299] 本申请实施例还提供了一种计算机可读存储介质。计算机可读存储介质可以是计算设备能够存储的任何可用介质或者是包含一个或多个可用介质的数据中心等数据存储设备。可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘)等。该计算机可读存储介质包括指令，指令指示计算设备执行网络请求处理方法。

[0300] 最后应说明的是：以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的保护范围。< / script>

Claims

1. A network request processing method, characterized in that: The method is applied to the proxy end in the cloud service system; The cloud service system further includes a cloud service end; the cloud service end is used to provide a web application based on a single page application to the client; the method includes: Receiving an asynchronous request sent by the client, wherein the asynchronous request is used to update a first hypertext markup language HTML page based on a single page application; wherein the first HTML page includes a software development kit SDK; Performing a validity verification on the asynchronous request, wherein the validity verification includes challenge data, and the data format of the challenge data matches the data format that can be parsed by the SDK; the SDK is used by the client to parse the challenge data in the validity verification initiated for the asynchronous request and generate a verification result of the challenge data; Determining that the asynchronous request passes verification based on the verification result, and forwarding the asynchronous request to the cloud service end; Determining based on the verification result that the asynchronous request verification fails, and blocking the asynchronous request.

2. The method according to claim 1, characterized in that Before receiving the asynchronous request sent by the client, the method includes: receiving a page access request sent by the client, wherein the page access request is used to request a first HTML page; Forwarding the page access request to the cloud service end; Receive a second HTML page sent by the cloud server; the second HTML page does not include the SDK; Injecting the SDK into the second HTML page to generate the first HTML page; The first HTML page is sent to the client, where the first HTML page includes the SDK.

3. The method according to claim 2, characterized in that The step of injecting the SDK into the second HTML page to generate the first HTML page includes: The resource access address of the SDK is added to the second HTML page to generate the first HTML page.

4. The method according to any one of claims 1 to 3, characterized in that The verification result includes response information; the legitimacy verification of the asynchronous request includes: Sending a legitimacy verification request to the client; the legitimacy verification request includes challenge data; the legitimacy verification request is used to instruct the client to parse the challenge data in the legitimacy verification initiated for the asynchronous request through the SDK and generate response information for the challenge data; Verifying the response information returned by the client; After verifying the legitimacy of the asynchronous request, the method further includes: If the response information is successfully verified, it is determined that the asynchronous request verification is passed; If the response information verification fails, it is determined that the asynchronous request verification has failed.

5. The method according to claim 4, characterized in that Before sending the legitimacy verification request to the client, the method further includes: Identify whether the asynchronous request is a suspicious request based on at least one of user agent information carried by the asynchronous request or the request frequency of the client; the suspicious request indicates an asynchronous request initiated by a crawler program; Determine that the asynchronous request is the suspicious request, and send a legitimacy verification request to the client.

6. The method according to claim 4 or 5, characterized in that The challenge data includes a set of JavaScript codes; the response information includes an access token; and the access token is obtained by the client parsing the JavaScript code through the SDK.

7. The method according to any one of claims 1 to 3, characterized in that The challenge data includes a set of codes for generating a graphic verification code; the response information includes at least one of mouse movement information or verification information of the graphic verification code; the SDK is used by the client to parse the code for generating the graphic verification code and generate the graphic verification code, so that the client generates the response information based on the image verification code.

8. The method according to any one of claims 1 to 7, characterized in that The asynchronous request includes at least one of a request initiated by the client based on a Fetch interface and a request initiated based on an XHR interface.

9. A network request processing method, characterized in that: The method is applied to a cloud service end in a cloud service system, and the cloud service end is used to provide a web application based on a single-page application to a client; the method includes: Receive an asynchronous request sent by the client; the asynchronous request is used to update a first hypertext markup language HTML page based on a single page application; wherein the first HTML page includes a software development kit SDK; Performing a validity verification on the asynchronous request, wherein the validity verification includes challenge data, and the data format of the challenge data matches the data format that can be parsed by the SDK; the SDK is used by the client to parse the challenge data in the validity verification initiated for the asynchronous request and generate a verification result of the challenge data; If it is determined based on the verification result that the asynchronous request verification is passed, then the asynchronous request is allowed to be responded to; If it is determined based on the verification result that the asynchronous request verification fails, the asynchronous request is refused to be responded to.

10. The method according to claim 9, characterized in that Before receiving the asynchronous request sent by the client, the method further includes: receiving a page access request sent by the client, wherein the page access request is used to request a first HTML page; Obtain a second HTML page; the second HTML page does not include the SDK; Injecting the SDK into the second HTML page to generate the first HTML page; The first HTML page is sent to the client, where the first HTML page includes the SDK.

11. A network request processing method, characterized in that: The method is applied to a client; the client is used to request a web application based on a single-page application from a cloud service system; the cloud service end includes a cloud service end and a proxy end; the method includes: Sending a page access request to the cloud server, wherein the page access request is used to request a first hypertext markup language HTML page based on a single page application; Receive the first HTML page, where the first HTML page includes a software development kit (SDK); An asynchronous request is initiated to the agent so that the agent verifies the legitimacy of the asynchronous request; the asynchronous request is used to update the first HTML page; the legitimacy verification includes challenge data, and the data format of the challenge data matches the data format that can be parsed by the SDK; the SDK is used by the client to parse the challenge data in the legitimacy verification initiated for the asynchronous request and generate a verification result of the challenge data.

12. A network request processing device, characterized in that: The network request processing device is applied to a cloud service system; the device includes: A communication module, configured to receive an asynchronous request sent by the client, wherein the asynchronous request is used to update a first hypertext markup language HTML page based on a single page application; wherein the first HTML page includes a software development kit SDK; a processing module configured to perform a validity verification on the asynchronous request, wherein the validity verification includes challenge data, and the data format of the challenge data matches a data format parseable by the SDK; the SDK is configured to parse the challenge data in the validity verification initiated for the asynchronous request by the client and generate a verification result of the challenge data; The processing module is further configured to determine, based on the verification request, that the asynchronous request passes verification, and then allow the response to the asynchronous request; and to determine, based on the verification result, that the asynchronous request fails verification, and then refuse to respond to the asynchronous request.

13. An agent device, characterized in that: The device is applied to a cloud service system, which also includes a cloud service end; the cloud service end is used to provide a web application based on a single-page application to a client; the device includes: A communication module, configured to receive an asynchronous request sent by the client, wherein the asynchronous request is used to update a first hypertext markup language HTML page based on a single page application; wherein the first HTML page includes a software development kit SDK; a processing module configured to perform a validity verification on the asynchronous request, wherein the validity verification includes challenge data, and the data format of the challenge data matches a data format parseable by the SDK; the SDK is configured to parse the challenge data in the validity verification initiated for the asynchronous request by the client and generate a verification result of the challenge data; The communication module is further configured to determine, based on the verification result, that the asynchronous request has passed verification, and then forward the asynchronous request to the cloud service end; and to determine, based on the verification result, that the asynchronous request has failed verification, and then block the asynchronous request.

14. A computing device, characterized in that The computing device includes a processor and a memory; The processor is configured to instruct the instructions stored in the memory so that the computing device executes the method according to any one of claims 1 to 8; or so that the computing device executes the method according to claim 9 or 10, or so that the computing device executes the method according to claim 11.

15. A computing device cluster, characterized in that: comprising at least one computing device, each computing device including a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method as described in any one of claims 1 to 8; or so that the computing device cluster performs the method as described in any one of claims 9 or 10, or so that the computing device cluster performs the method as described in claim 11.

16. A computer program product comprising instructions, characterized in that When the instruction is executed by a computing device or a computing device cluster, the computing device or computing device cluster executes the method as described in any one of claims 1 to 8; or causes the computing device or computing device cluster to execute the method as claimed in claim 9 or 10; or causes the computing device or computing device cluster to execute the method as claimed in claim 11.

17. A computer-readable storage medium, characterized in that comprising computer program instructions, which, when executed by a computing device cluster or a computing device, cause the computing device cluster or the computing device to perform the method according to any one of claims 1 to 8; or the computing device cluster or the computing device performs the method according to claim 9 or 10; Alternatively, the computing device cluster or the computing device executes the method according to claim 11.

Images