Crawler prevention method and system based on dynamic request frequency modulation and pseudo signal interference

Abstract

The invention discloses an anti-crawler method and system based on dynamic request frequency modulation and pseudo signal interference, and relates to the technical field of network security. According to the method, the request time interval is adjusted in real time through the nonlinear function, and the dynamic performance of protection is enhanced; a pseudo signal is injected to induce crawler response, and automatic behaviors are actively recognized; the crawler detection precision is greatly improved by combining dual verification of the time interval and the pseudo signal response; a protection strategy is dynamically adjusted according to network-server data, so that high adaptability of the system is ensured, and the protection effect is improved; through a lightweight mathematical model and pseudo signal injection, an efficient anti-crawler capability is provided, and the method is suitable for a high-concurrency scene.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to an anti-crawler method and system based on dynamic request frequency modulation and pseudo-signal interference. Background Art

[0002] In existing web applications, web crawlers place a significant burden on website servers through their high-frequency, large-scale requests. Traditional protection methods (such as IP blocking and verification code verification) are no longer able to cope with the automated behavior of advanced crawlers. Existing technologies suffer from fixed protection mechanisms and an inability to respond to dynamic attacks. Summary of the Invention

[0003] The purpose of the present invention is to provide an anti-crawler method and system based on dynamic request frequency modulation and false signal interference to improve the above technical problems.

[0004] In order to achieve the above-mentioned object of the invention, the embodiment of the present invention provides the following technical solutions:

[0005] An anti-crawler method based on dynamic request frequency modulation and false signal interference, comprising:

[0006] Based on the user's historical request activity data and the network-server data at the current timestamp, the frequency modulation parameters and the pseudo signal generation parameters are adjusted to determine the protection strategy;

[0007] Based on the frequency modulation parameter, the allowed request time interval of the current request is generated by a request interval nonlinear function;

[0008] Based on the allowed request time interval, the client makes the current request and calculates the actual request time interval between the previous request and the current request;

[0009] Based on the pseudo signal generation parameters, a pseudo signal is randomly generated to determine whether the current request is responded to and obtain the response result of the current request;

[0010] Based on the actual request time interval and response result of the current request, the request type of the current request is determined and the protection strategy is executed.

[0011] The adjusting of the frequency modulation parameters and the pseudo signal generation parameters comprises:

[0012] Screen historical user behavior data and historical request frequencies to determine historical suspected abnormal behavior data and historical suspected abnormal request frequencies; use the historical access time periods corresponding to the historical abnormal behavior data and historical abnormal request frequencies as the historical suspected abnormal access time periods;

[0013] Analyze historical suspected abnormal behavior data, historical suspected abnormal request frequencies, and historical suspected abnormal access time periods to obtain comprehensive correlation analysis results;

[0014] Based on the comprehensive correlation analysis results, the frequency modulation parameters in the previous round of requests are dynamically adjusted through an adaptive adjustment algorithm;

[0015] Based on the activity data of the current request, the pseudo signal generation parameters in the previous round of requests are dynamically adjusted; the pseudo signal generation parameters include the injection frequency and complexity of the pseudo signal.

[0016] Furthermore, the historical user behavior data includes historical access logs, historical request time intervals, historical request header information, and historical access paths; historical access logs include historical IP addresses and historical request URL parameters; historical access paths include deep pages and resource requests; and historical request frequencies include the number or speed of user requests.

[0017] Furthermore, the network-server data includes the user's network traffic situation and server load data;

[0018] Determining the protection strategy for the current request includes:

[0019] Based on the user's network traffic, the overall bandwidth usage is calculated. Based on the overall bandwidth usage, it is determined whether the current request is at a peak period. If so, the protection strategy of the previous request is adjusted to obtain the initial protection strategy. Otherwise, the protection strategy of the previous request is used as the initial protection strategy.

[0020] Based on the server load data, the resource utilization of the server is calculated; based on the resource utilization, the initial protection strategy is adjusted again to obtain the protection strategy of the current request.

[0021] Furthermore, the screening of historical user behavior data and historical request frequencies to determine historical suspected abnormal behavior data and historical suspected abnormal request frequencies includes:

[0022] Identify whether the historical request frequency in different historical access time periods exceeds the request threshold condition; if so, the historical request frequency is regarded as the historical suspected abnormal request frequency; otherwise, the historical request frequency is regarded as the historical normal request frequency;

[0023] Identify whether the historical request time intervals in different historical access time periods are the same; if so, treat the historical request time intervals as historical suspected abnormal behavior data; otherwise, treat the historical request time intervals as historical normal behavior data;

[0024] Identify whether the historical access logs within different historical access time periods meet any one of the following conditions: the repetition rate of the historical IP addresses reaches the repetition rate threshold, the geographical distribution of the historical IP addresses changes, or the historical request URL parameters are exactly the same; if so, treat the historical access log as historical suspected abnormal behavior data; otherwise, treat the historical access log as historical normal behavior data;

[0025] Identify whether the historical access paths within different historical access time periods meet the conditions for a fixed historical access path; if so, treat the historical access paths as historical suspected abnormal behavior data; otherwise, treat the historical access paths as historical normal behavior data;

[0026] Identify whether the historical request header information in different historical access time periods is abnormal or lacks browser identification; if so, treat the historical request header information as historical suspected abnormal behavior data; otherwise, treat the historical request header information as historical normal behavior data;

[0027] Identify whether the number of historical application pages in different historical access time periods reaches the page number threshold; if so, use the historical application page number as historical suspected abnormal behavior data; otherwise, use the historical application page number as historical normal behavior data.

[0028] Furthermore, the step of making a current request through the client based on the allowed request time interval and calculating the actual request time interval between the previous request and the current request includes:

[0029] The client determines the allowed request time interval for receiving the current request and makes the current request;

[0030] Determine whether the client sends the current request after the allowed request time interval; if so, calculate the actual request time interval between the previous request and the current request; otherwise, directly determine that the application type of the current request is an illegal application and the applicant is a crawler activity, reject the current request, enable data protection, and trigger the anti-crawler mechanism.

[0031] Furthermore, the determining of the request type of the current request and executing the protection strategy based on the actual request time interval and the response result of the current request includes:

[0032] Determine whether the actual request time interval of the current request satisfies any one of the following conditions: the nonlinear function of the request interval, or the response result is no response; if so, determine that the request type of the current request is a legal request, the applicant is a real user, and agree to the current request; otherwise, determine that the application type of the current request is an illegal application, the applicant is a crawler activity, reject the current request, enable data protection, and trigger the anti-crawler mechanism.

[0033] An anti-crawler system based on dynamic request frequency modulation and false signal interference, comprising:

[0034] Request activity data collection module, used to collect the user's historical request activity data and network-server data at the current timestamp;

[0035] a parameter setting module for adjusting frequency modulation parameters and pseudo signal generation parameters based on historical request activity data of the user and network-server data at a current timestamp;

[0036] The protection strategy determination module is used to calculate the overall bandwidth usage based on the network-server data at the current timestamp and determine the protection strategy for the current request;

[0037] An allowed request time interval calculation module is used to generate an allowed request time interval of a current request based on a frequency modulation parameter and a request interval nonlinear function;

[0038] An actual request time interval calculation module, configured to calculate an actual request time interval between a previous request and a current request by making a current request through a client based on an allowed request time interval;

[0039] The pseudo signal generation and response module is used to randomly generate a pseudo signal based on pseudo signal generation parameters, determine whether the current request is responded to, and obtain the response result of the current request;

[0040] The data protection module is used to determine the request type of the current request and execute the protection strategy based on the actual request time interval and response result of the current request.

[0041] The beneficial effects of the present invention are:

[0042] The present invention adjusts the request time interval in real time through a nonlinear function, thereby enhancing the dynamic nature of protection; induces crawler responses by injecting pseudo-signals and actively identifies automated behaviors; combines the dual verification of time intervals and pseudo-signal responses to greatly improve the accuracy of crawler detection; dynamically adjusts the protection strategy based on network-server data to ensure strong system adaptability and increase protection effectiveness; and provides efficient anti-crawler capabilities through lightweight mathematical models and pseudo-signal injection, which is suitable for high-concurrency scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A flow chart of a method in an embodiment of the present invention;

[0045] Figure 2 2 is a system structure diagram in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0047] See also Figure 1 This embodiment provides an anti-crawler method based on dynamic request frequency modulation and false signal interference, which includes:

[0048] S1. Based on the user's historical request activity data and the network-server data at the current timestamp, adjust the frequency modulation parameters and the pseudo signal generation parameters to determine the protection strategy.

[0049] Said S1 comprises:

[0050] S1-1. Collect the user's historical request activity data and network-server data at the current timestamp;

[0051] The historical request activity data includes historical user behavior data, historical request frequency and historical number of application pages of historical requests in different historical access time periods; historical user behavior data includes historical access logs, historical request time intervals, historical request header information (such as User-Agent, Referer, Accept, etc.), and historical access paths; historical access logs include historical IP addresses and historical request URL parameters; historical access paths include deep pages and resource requests; historical request frequency includes the number or speed of user requests; historical access time periods include peak periods and trough periods.

[0052] Network-server data includes user network traffic and server load data; server load data includes server CPU load, memory usage, and response time.

[0053] S1-2. Calculate the overall bandwidth usage based on network-server data and determine the protection strategy for the current request;

[0054] Based on the user's network traffic, the overall bandwidth utilization rate is calculated. Based on the overall bandwidth utilization rate, it is determined whether the current request is at a peak period. If so, the protection strategy of the previous request is adjusted to obtain the initial protection strategy. The basic request interval C0 and modulation amplitude A of the modulation function are increased, the request interval is extended, high-frequency requests are restricted, the modulation phase φ is adjusted, and the difference in user request intervals is increased. Otherwise, the protection strategy of the previous request is used as the initial protection strategy.

[0055] Based on the server load data, the resource utilization of the server is calculated; based on the resource utilization, the initial protection strategy is adjusted again to obtain the protection strategy of the current request.

[0056] According to actual needs, set the first resource utilization threshold, the second resource utilization threshold and the resource utilization threshold;

[0057] If the resource utilization is greater than or equal to the first resource utilization threshold, the request interval and page access path in the protection strategy are increased, the complexity of crawler identification is increased, the crawler's crawling process is further interfered with, and the flexibility and robustness of data protection are improved.

[0058] If the resource utilization is less than the first resource utilization threshold and greater than or equal to the second load threshold range, multiple virtual users will be simulated and added to the protection strategy for parallel operation, which increases the parsing difficulty of the crawler and interferes with its normal crawling.

[0059] If the resource utilization is less than the third resource utilization threshold, the operation corresponding to setting the IP proxy pool is added to the protection policy, and the request source is dispersed through the proxy pool to avoid frequent requests from a single IP address.

[0060] By using the network-server data at the current timestamp, the protection strategy for the current request can be adjusted in real time. This not only helps to avoid the impact of strict server protection measures on the normal user access experience, but also can formulate a protection strategy in real time that adapts to the network conditions of the current request, which is conducive to rapid response and timely processing of crawler activities.

[0061] S1-3. Screen historical user behavior data and historical request frequencies to determine historical suspected abnormal behavior data and historical suspected abnormal request frequencies; and use the historical access time periods corresponding to the historical abnormal behavior data and historical abnormal request frequencies as the historical suspected abnormal access time periods;

[0062] Identify whether the historical request frequency in different historical access time periods exceeds the request threshold condition; if so, use the historical request frequency as the historical suspected abnormal request frequency; otherwise, use the historical request frequency as the historical normal request frequency; in this embodiment, the request threshold condition is that the request frequency for accessing the same resource in a short period of time reaches the frequency threshold; formulate the frequency threshold according to actual specific requirements.

[0063] Identify whether the historical request time intervals in different historical access time periods are the same; if so, use the historical request time interval as historical suspected abnormal behavior data; otherwise, use the historical request time interval as historical normal behavior data; because fixed request time intervals are typical characteristics of crawler activities, historical user behavior data can be analyzed from the perspective of fixed request time intervals to determine whether they are abnormal data.

[0064] Identify whether the historical access logs within different historical access time periods meet any one of the following conditions: the repetition rate of the historical IP addresses reaches the repetition rate threshold, the geographical distribution of the historical IP addresses changes, or the historical request URL parameters are exactly the same; if so, treat the historical access log as historical suspected abnormal behavior data; otherwise, treat the historical access log as historical normal behavior data;

[0065] Identify whether the historical access paths within different historical access time periods meet the conditions for a fixed historical access path; if so, treat the historical access paths as historical suspected abnormal behavior data; otherwise, treat the historical access paths as historical normal behavior data;

[0066] Identify whether the historical request header information in different historical access time periods is abnormal or lacks common browser identifiers; if so, treat the historical request header information as historical suspected abnormal behavior data; otherwise, treat the historical request header information as historical normal behavior data;

[0067] Identify whether the number of historical request pages within different historical access time periods reaches the page count threshold. If so, consider this historical request page count as historical suspected abnormal behavior data; otherwise, consider this historical request page count as historical normal behavior data. Generally, crawlers often request more pages than real users, so the number of request pages can be used to distinguish crawler activity.

[0068] S1-4. Analyze historical suspected abnormal behavior data, historical suspected abnormal request frequencies, and historical suspected abnormal access time periods to obtain comprehensive correlation analysis results;

[0069] A machine learning algorithm is used to analyze the association between crawler activities and historical suspected abnormal behavior data, historical suspected abnormal request frequency, and historical suspected abnormal access time periods. The historical activity data (historical user behavior data, historical request frequency, and historical access time period) corresponding to the request type of historical requests for crawler activities, historical suspected abnormal behavior data, historical suspected abnormal request frequency, and historical suspected abnormal access time periods are input into the machine learning algorithm, and the association between crawler activity and user behavior, the association between crawler activity and request frequency, and the association between crawler activity and access time period are output, that is, the comprehensive association analysis results.

[0070] S1-5. Based on the comprehensive correlation analysis results, the frequency modulation parameters in the previous round of requests are dynamically adjusted through the adaptive adjustment algorithm. The corresponding process is as follows:

[0071] The correlation between crawler activity and request frequency is: the higher the request frequency, the more active the crawler activity is, so adjustments need to be made: increase the modulation amplitude A and the modulation frequency ω ， Increase the fluctuation of the request interval and change the request interval faster to prevent crawlers from sending requests at a fixed frequency. In addition, you can increase the frequency of false signal injection to limit high-frequency crawlers.

[0072] The correlation between crawler activity and user behavior is as follows: fixed time intervals, abnormal request headers, or fixed paths are highly correlated with crawlers. Adjusting the modulation phase φ is conducive to generating complex pseudo signals, which increases the difficulty of crawler parsing.

[0073] The correlation between crawler activity and access time periods is as follows: crawlers are more active during high-load periods. When access times are at their peak, the base request interval C0 is reduced to shorten the request intervals of normal users. At the same time, a new value is randomly assigned to the modulation phase φ and updated. This ensures that the request intervals of different users vary significantly at the same time, preventing crawlers from exploiting group patterns and launching attacks.

[0074] S1-6. Based on the activity data of the current request (IP request frequency), dynamically adjust the pseudo signal generation parameters in the previous round of requests; the pseudo signal generation parameters include the injection frequency and complexity of the pseudo signal.

[0075] When an abnormal IP request frequency is detected for the current request, the frequency and complexity of the pseudo signal injection are increased. Increasing the pseudo signal injection frequency can more frequently interfere with the crawler's identification and parsing process, making it more difficult for the crawler to distinguish between legitimate requests and pseudo signals. For example, by randomly delaying and inserting pseudo requests to simulate the browsing intervals of a normal user, the randomness between requests is increased, thereby increasing the pseudo signal injection frequency.

[0076] Increasing the complexity of the pseudo signal means adding dynamic page content of the pseudo signal or replacing the disguised request header of the pseudo signal.

[0077] In dynamic page content that adds false signals, the page content is dynamically generated or modified through JavaScript, or the page structure is changed according to user behavior (such as hiding elements, loading new content); since crawlers usually cannot handle dynamically loaded pages, it increases the difficulty of crawler parsing.

[0078] In the disguised request headers of the replaced pseudo-signal, the User-Agent, Referer and other request headers are regularly changed to simulate access from different browsers and devices; since crawlers often use a fixed User-Agent (request header), by changing the request header, the diverse access methods of normal users can be simulated, increasing the difficulty of crawler parsing.

[0079] The present invention dynamically adjusts frequency modulation parameters and pseudo signal generation parameters in real time based on the comprehensive correlation analysis results, increases the injection frequency or complexity of pseudo signals, further confuses advanced crawlers, and makes them more likely to expose automated behaviors.

[0080] S2. Generate an allowed request time interval of the current request based on frequency modulation parameters and through a request interval nonlinear function; the frequency modulation parameters include modulation amplitude, modulation frequency, modulation phase and basic request interval.

[0081] The S2 includes:

[0082] Determine the current timestamp of the client; based on the current timestamp and the request interval nonlinear function, generate the allowed request time interval for the current request. The request interval nonlinear function is the nonlinear function f(t) corresponding to the formula:

[0083] f(t)=A·sin(ω·t+φ)+C0;

[0084] sin(·) represents the sine function; A represents the modulation amplitude; φ represents the modulation phase, which is used to control the offset of the initial request interval; t represents the timestamp; ω represents the modulation frequency, which is used to determine the fluctuation speed of the allowed request interval; C0 represents the basic request interval, which is the average value of the historical request intervals.

[0085] The present invention is based on frequency modulation parameters and can generate different allowed request time intervals in different time periods, thereby improving the flexibility and robustness of anti-crawler.

[0086] S3. Making a current request through the client based on the allowed request time interval, and calculating the actual request time interval between the previous request and the current request;

[0087] The S3 includes:

[0088] The client determines the allowed request time interval for receiving the current request and makes the current request;

[0089] Determine whether the client sends the current request after the allowed request time interval; if so, calculate the actual request time interval between the previous request and the current request; otherwise, directly determine that the application type of the current request is an illegal application and the applicant is a crawler activity, reject the current request, enable data protection, and trigger the anti-crawler mechanism.

[0090] S4. Based on the pseudo signal generation parameters, randomly generate a pseudo signal, determine whether the current request is responded to, and obtain a response result for the current request; the pseudo signal is used to induce a response to the current request, and includes random data, a pseudo HTTP header, pseudo content, forged URL parameters, and a forged HTTP status code. The response result is either a response or a non-response.

[0091] The S4 includes:

[0092] S4-1. Use Random to generate random data; generate fake HTTP headers and fake content; fake content includes invalid or forged JavaScript comments or images or other media resources.

[0093] S4-2. Construct a fake URL and use it as a forged URL parameter; generate an error status code, such as 403, 404, or 500, and use it as a forged HTTP status code.

[0094] S4-3. Embed random data, fake HTTP header, fake content, fake URL parameters and fake HTTP status code into the normal HTTP response of the current request, and determine whether the client responds to the fake signal, for example, whether it attempts to access the address in the fake signal; if so, the response result of the current request is determined to be a response; otherwise, the response result of the current request is determined to be a non-response.

[0095] S5. Based on the actual request time interval and response result of the current request, determine the request type of the current request and execute the protection strategy;

[0096] Determine whether the actual request time interval of the current request satisfies any one of the following conditions: a nonlinear function of the request interval, or a non-response response. If so, determine that the request type of the current request is a legitimate request, the applicant is a real user, and agree to the current request. Otherwise, determine that the application type of the current request is an illegal application, the applicant is a crawler activity, reject the current request, enable data protection, and trigger the anti-crawler mechanism, such as banning the IP address of the application or restricting access.

[0097] like Figure 2 As shown, an anti-crawler system based on dynamic request frequency modulation and pseudo signal interference includes:

[0098] Request activity data collection module, used to collect the user's historical request activity data and network-server data at the current timestamp;

[0099] a parameter setting module for adjusting frequency modulation parameters and pseudo signal generation parameters based on historical request activity data of the user and network-server data at a current timestamp;

[0100] The protection strategy determination module is used to calculate the overall bandwidth usage based on the network-server data at the current timestamp and determine the protection strategy for the current request;

[0101] An allowed request time interval calculation module is used to generate an allowed request time interval of a current request based on a frequency modulation parameter and a request interval nonlinear function;

[0102] An actual request time interval calculation module, configured to calculate an actual request time interval between a previous request and a current request by making a current request through a client based on an allowed request time interval;

[0103] The pseudo signal generation and response module is used to randomly generate a pseudo signal based on pseudo signal generation parameters, determine whether the current request is responded to, and obtain the response result of the current request;

[0104] The data protection module is used to determine the request type of the current request and execute the protection strategy based on the actual request time interval and response result of the current request.

[0105] In summary, the present invention adjusts the request time interval in real time through a nonlinear function, thereby enhancing the dynamic nature of protection; induces crawler responses by injecting pseudo-signals, and actively identifies automated behaviors; combines the dual verification of time intervals and pseudo-signal responses to greatly improve the accuracy of crawler detection; dynamically adjusts protection strategies based on network-server data to ensure strong system adaptability and increase protection effects; and provides efficient anti-crawler capabilities through lightweight mathematical models and pseudo-signal injection, which is suitable for high-concurrency scenarios.

[0106] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. An anti-crawler method based on dynamic request frequency modulation and false signal interference, characterized in that: include: Based on the user's historical request activity data and network-server data at the current timestamp, the frequency modulation parameters and the pseudo signal generation parameters are adjusted to determine the protection strategy; the historical request activity data includes historical user behavior data, historical request frequency, and historical number of application pages for historical requests in different historical access time periods; Based on the frequency modulation parameter, the allowed request time interval of the current request is generated by a request interval nonlinear function; Based on the allowed request time interval, the client makes the current request and calculates the actual request time interval between the previous request and the current request; Based on the pseudo signal generation parameters, a pseudo signal is randomly generated to determine whether the current request is responded to and obtain the response result of the current request; Based on the actual request time interval and response result of the current request, determine the request type of the current request and execute the protection strategy; The adjusting of the frequency modulation parameters and the pseudo signal generation parameters comprises: Screen historical user behavior data and historical request frequencies to determine historical suspected abnormal behavior data and historical suspected abnormal request frequencies; use the historical access time periods corresponding to the historical abnormal behavior data and historical abnormal request frequencies as the historical suspected abnormal access time periods; Analyze historical suspected abnormal behavior data, historical suspected abnormal request frequencies, and historical suspected abnormal access time periods to obtain comprehensive correlation analysis results; Based on the comprehensive correlation analysis results, the frequency modulation parameters in the previous round of requests are dynamically adjusted through an adaptive adjustment algorithm; Based on the activity data of the current request, the pseudo signal generation parameters in the previous round of requests are dynamically adjusted; the pseudo signal generation parameters include the injection frequency and complexity of the pseudo signal.

2. The anti-crawler method based on dynamic request frequency modulation and false signal interference according to claim 1, characterized in that: The historical user behavior data includes historical access logs, historical request time intervals, historical request header information, and historical access paths; historical access logs include historical IP addresses and historical request URL parameters; historical access paths include deep pages and resource requests; and historical request frequency includes the number or speed of user requests.

3. The anti-crawler method based on dynamic request frequency modulation and false signal interference according to claim 1, characterized in that: The network-server data includes the user's network traffic situation and server load data; Determining the protection strategy for the current request includes: Based on the user's network traffic, the overall bandwidth usage is calculated. Based on the overall bandwidth usage, it is determined whether the current request is at a peak period. If so, the protection strategy of the previous request is adjusted to obtain the initial protection strategy. Otherwise, the protection strategy of the previous request is used as the initial protection strategy. Based on the server load data, the resource utilization of the server is calculated; based on the resource utilization, the initial protection strategy is adjusted again to obtain the protection strategy of the current request.

4. The anti-crawler method based on dynamic request frequency modulation and false signal interference according to claim 2, characterized in that: The screening of historical user behavior data and historical request frequencies to determine historical suspected abnormal behavior data and historical suspected abnormal request frequencies includes: Identify whether the historical request frequency in different historical access time periods exceeds the request threshold condition; if so, the historical request frequency is regarded as the historical suspected abnormal request frequency; otherwise, the historical request frequency is regarded as the historical normal request frequency; Identify whether the historical request time intervals in different historical access time periods are the same; if so, treat the historical request time intervals as historical suspected abnormal behavior data; otherwise, treat the historical request time intervals as historical normal behavior data; Identify whether the historical access logs within different historical access time periods meet any one of the following conditions: the repetition rate of the historical IP addresses reaches the repetition rate threshold, the geographical distribution of the historical IP addresses changes, or the historical request URL parameters are exactly the same; if so, treat the historical access log as historical suspected abnormal behavior data; otherwise, treat the historical access log as historical normal behavior data; Identify whether the historical access paths within different historical access time periods meet the conditions for a fixed historical access path; if so, treat the historical access paths as historical suspected abnormal behavior data; otherwise, treat the historical access paths as historical normal behavior data; Identify whether the historical request header information in different historical access time periods is abnormal or lacks browser identification; if so, treat the historical request header information as historical suspected abnormal behavior data; otherwise, treat the historical request header information as historical normal behavior data; Identify whether the number of historical application pages in different historical access time periods reaches the page number threshold; if so, use the historical application page number as historical suspected abnormal behavior data; otherwise, use the historical application page number as historical normal behavior data.

5. The anti-crawler method based on dynamic request frequency modulation and false signal interference according to claim 1, characterized in that: The step of making a current request through the client based on the allowed request time interval and calculating the actual request time interval between the previous request and the current request includes: The client determines the allowed request time interval for receiving the current request and makes the current request; Determine whether the client sends the current request after the allowed request time interval; if so, calculate the actual request time interval between the previous request and the current request; otherwise, directly determine that the application type of the current request is an illegal application and the applicant is a crawler activity, reject the current request, enable data protection, and trigger the anti-crawler mechanism.

6. The anti-crawler method based on dynamic request frequency modulation and false signal interference according to claim 1, characterized in that: The step of determining the request type of the current request and executing the protection strategy based on the actual request time interval and the response result of the current request includes: Determine whether the actual request time interval of the current request satisfies any one of the following conditions: the nonlinear function of the request interval, or the response result is no response; if so, determine that the request type of the current request is a legal request, the applicant is a real user, and agree to the current request; otherwise, determine that the application type of the current request is an illegal application, the applicant is a crawler activity, reject the current request, enable data protection, and trigger the anti-crawler mechanism.

7. An anti-crawler system based on dynamic request frequency modulation and pseudo signal interference, used to implement the anti-crawler method based on dynamic request frequency modulation and pseudo signal interference according to any one of claims 1 to 6, characterized in that: include: Request activity data collection module, used to collect the user's historical request activity data and network-server data at the current timestamp; a parameter setting module for adjusting frequency modulation parameters and pseudo signal generation parameters based on historical request activity data of the user and network-server data at a current timestamp; The protection strategy determination module is used to calculate the overall bandwidth usage based on the network-server data at the current timestamp and determine the protection strategy for the current request; An allowed request time interval calculation module is used to generate an allowed request time interval of a current request based on a frequency modulation parameter and a request interval nonlinear function; An actual request time interval calculation module, configured to calculate an actual request time interval between a previous request and a current request by making a current request through a client based on an allowed request time interval; The pseudo signal generation and response module is used to randomly generate a pseudo signal based on pseudo signal generation parameters, determine whether the current request is responded to, and obtain the response result of the current request; The data protection module is used to determine the request type of the current request and execute the protection strategy based on the actual request time interval and response result of the current request.

Images