IP (Internet Protocol) sealing control method and system based on website data

Abstract

The invention provides an IP (Internet Protocol) sealing control method and system based on website data. The method comprises the following steps: acquiring user access track data; monitoring the IP address of each access website according to the user access track data; inputting the IP address of each access website into a supervised learning model to obtain an abnormal IP address; adding the abnormal IP address into a sealing control IP list; determining access authority limits of different degrees according to the sealing control times of the abnormal IP address; by identifying the abnormal IP address, the access authority of the abnormal IP address is limited, so that malicious data acquisition is prevented, and the data security and integrity of a website are protected.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an IP blocking method and system based on website data. Background Art

[0002] In the Internet age, website data collection has become an important business activity. Many websites need to collect a large amount of data for analysis and research. However, some malicious data collection behaviors have also begun to appear, which may threaten the normal operation and data security of the website. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide an IP blocking method and system based on website data, which can limit the access rights of abnormal IP addresses by identifying abnormal IP addresses, thereby preventing malicious data collection and protecting the data security and integrity of the website.

[0004] In a first aspect, an embodiment of the present invention provides an IP blocking method based on website data, the method comprising:

[0005] Get user access track data;

[0006] Monitor the IP address of each visited website according to the user access trajectory data;

[0007] Inputting the IP address of each visited website into a supervised learning model to obtain abnormal IP addresses;

[0008] Add the abnormal IP address to the blocked IP list;

[0009] Different degrees of access permission restriction are determined based on the number of times the abnormal IP address is blocked.

[0010] Furthermore, different degrees of access rights restrictions are determined according to the number of times the abnormal IP address is blocked, including:

[0011] When the number of blocking times of the abnormal IP address is less than or equal to the first preset threshold, limiting the access speed of the website;

[0012] When the number of blocking times of the abnormal IP address is greater than or equal to the second preset threshold and less than or equal to the third preset threshold, restricting access to important pages of the website;

[0013] When the number of blocking times of the abnormal IP address is greater than or equal to a fourth preset threshold, restricting access to the website within a preset time period;

[0014] The first preset threshold is smaller than the second preset threshold, the second preset threshold is smaller than the third preset threshold, and the third preset threshold is smaller than the fourth preset threshold.

[0015] Furthermore, the supervised learning model is constructed in the following way:

[0016] Preprocessing the user access trajectory data to obtain preprocessed data;

[0017] Performing feature extraction on the preprocessed data to obtain feature data;

[0018] Labeling the normal IP addresses and the abnormal IP addresses in the feature data and historical data to construct a labeled data set;

[0019] Using the labeled data set as a training data set, wherein the training data set includes a plurality of training samples;

[0020] The association relationship between the input feature and the corresponding label in each of the training samples is learned to construct the supervised learning model.

[0021] Furthermore, the characteristic data includes IP access frequency, average request interval time, number of URL types visited, geographical location changes and user agent changes.

[0022] Furthermore, the user access trajectory data includes page URL, access time, IP address, geographic location, user agent and Token identity.

[0023] Furthermore, the method further comprises:

[0024] Share the blocked IP list with a third-party website.

[0025] In a second aspect, an embodiment of the present invention provides an IP blocking system based on website data, the system comprising:

[0026] An access trajectory data acquisition module is used to acquire user access trajectory data;

[0027] A monitoring module, used to monitor the IP address of each visited website according to the user access trajectory data;

[0028] An abnormal IP address acquisition module is used to input the IP address of each visited website into a supervised learning model to obtain an abnormal IP address;

[0029] An adding module is used to add the abnormal IP address to the blocked IP list;

[0030] The determination module is used to determine different degrees of access permission restrictions according to the number of times the abnormal IP address is blocked.

[0031] Furthermore, the determination module is specifically used for:

[0032] When the number of blocking times of the abnormal IP address is less than or equal to the first preset threshold, limiting the access speed of the website;

[0033] When the number of blocking times of the abnormal IP address is greater than or equal to the second preset threshold and less than or equal to the third preset threshold, restricting access to important pages of the website;

[0034] When the number of blocking times of the abnormal IP address is greater than or equal to a fourth preset threshold, restricting access to the website within a preset time period;

[0035] The first preset threshold is smaller than the second preset threshold, the second preset threshold is smaller than the third preset threshold, and the third preset threshold is smaller than the fourth preset threshold.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method described above when executing the computer program.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the method as described above.

[0038] The embodiment of the present invention provides an IP blocking method and system based on website data, including: obtaining user access trajectory data; monitoring the IP address of each visited website according to the user access trajectory data; inputting the IP address of each visited website into a supervised learning model to obtain abnormal IP addresses; adding the abnormal IP addresses to a blocked IP list; determining different degrees of access permission restrictions according to the number of blocking times of the abnormal IP addresses; and limiting the access permission of the abnormal IP addresses by identifying the abnormal IP addresses, thereby preventing malicious data collection and protecting the data security and integrity of the website.

[0039] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 A flow chart of an IP blocking method based on website data provided in Embodiment 1 of the present invention;

[0043] Figure 2 This is a schematic diagram of an IP blocking system based on website data provided in Embodiment 2 of the present invention.

[0044] icon:

[0045] 1-Access trajectory data acquisition module; 2-Monitoring module; 3-Abnormal IP address acquisition module; 4-Joining module; 5-Confirmation module. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] To facilitate understanding of this embodiment, the embodiment of the present invention is described in detail below.

[0048] Embodiment 1:

[0049] Figure 1 This is a flow chart of the IP blocking method based on website data provided in Example 1 of the present invention.

[0050] Reference Figure 1 , the method comprises the following steps:

[0051] Step S101, obtaining user access trajectory data;

[0052] Here, the website can monitor and record user access trajectory data through LUA scripts.

[0053] Step S102, monitoring the IP address of each visited website according to the user access track data;

[0054] Step S103, inputting the IP address of each visited website into a supervised learning model to obtain abnormal IP addresses;

[0055] Step S104, adding the abnormal IP address to the blocked IP list;

[0056] Step S105, determining different degrees of access permission restrictions according to the number of times the abnormal IP address is blocked.

[0057] Further, step S105 includes the following steps:

[0058] Step S201, when the number of blocking times of the abnormal IP address is less than or equal to the first preset threshold, limiting the access speed of the website;

[0059] Step S202, when the number of blocking times of the abnormal IP address is greater than or equal to the second preset threshold and less than or equal to the third preset threshold, restricting access to important pages of the website;

[0060] Step S203, when the number of blocking times of the abnormal IP address is greater than or equal to a fourth preset threshold, restricting access to the website within a preset time period;

[0061] The first preset threshold is less than the second preset threshold, the second preset threshold is less than the third preset threshold, and the third preset threshold is less than the fourth preset threshold. For example, the first preset threshold may be 3 times, the second preset threshold may be 4 times, the third preset threshold may be 6 times, and the fourth preset threshold may be 7 times. The above values ​​are set according to requirements.

[0062] Furthermore, the supervised learning model is constructed in the following way:

[0063] Step S301, preprocessing the user access trajectory data to obtain preprocessed data;

[0064] Step S302, extracting features from the preprocessed data to obtain feature data;

[0065] Specifically, during the preprocessing process, the user access trajectory data needs to be cleaned, such as removing duplicate, erroneous and irrelevant access records; the cleaned data is then standardized or normalized, unified into JSON format, and the preprocessed data is obtained.

[0066] Step S303, labeling the normal IP addresses and the abnormal IP addresses in the feature data and historical data to construct a labeled data set;

[0067] Step S304, using the labeled data set as a training data set, where the training data set includes a plurality of training samples;

[0068] Step S305, learning the association between the input features and the corresponding labels in each training sample, thereby constructing a supervised learning model.

[0069] Specifically, a supervised learning model is a method of predicting a target variable using a training data set (containing input features and corresponding labels). Each training sample has a known label or output value, and a supervised learning model is constructed by learning the relationship between the label and the input features.

[0070] After obtaining the supervised learning model, it is necessary to train the supervised learning model, that is, use a labeled data set to train the supervised learning model. The performance of the supervised learning model can be improved by adjusting the parameters of the supervised learning model and optimizing the algorithm. Use cross-validation and other techniques to evaluate the generalization ability of the supervised learning model and prevent overfitting.

[0071] During the evaluation process of the supervised learning model, an independent test set is used to evaluate the performance of the supervised learning model. The evaluation indicators include accuracy, recall, F1 score, and AUC-ROC curve. The supervised learning model is then adjusted based on the evaluation results.

[0072] In the process of deployment, monitoring, updating and optimization of supervised learning models, the trained supervised learning models are deployed to the actual environment, the access of IP addresses is monitored in real time, and anomaly detection is performed. The labeled data sets and supervised learning models are updated regularly to ensure that the supervised learning models can adapt to new anomaly patterns.

[0073] Furthermore, the characteristic data includes IP access frequency, average request interval, number of URL types visited, geographic location changes, and user agent changes.

[0074] Furthermore, user access trajectory data includes page URL, access time, IP address, geographic location, user agent (User-Agent) and token identity.

[0075] Furthermore, the method further comprises the following steps:

[0076] Step S401, sharing the blocked IP list with a third-party website, where the third-party website may be a social learning website.

[0077] Specifically, some malicious users or competitors may use automated programs or crawlers to collect data on a large scale, which may threaten the normal operation and data security of the website. By blocking abnormal IP addresses, malicious data collection can be restricted and the data security and integrity of the website can be protected.

[0078] Large-scale data collection activities may put a huge load on the website's server, causing the website to respond slowly or even crash. By limiting the abnormal IP addresses of data collection, the server load can be reduced and the stability and performance of the website can be improved.

[0079] Some industries or regions have specific laws and regulations on data collection and privacy protection. Websites can ensure that they meet relevant compliance requirements by using IP blocking methods based on website data.

[0080] The embodiment of the present invention provides an IP blocking method based on website data, including: obtaining user access trajectory data; monitoring the IP address of each visited website according to the user access trajectory data; inputting the IP address of each visited website into a supervised learning model to obtain abnormal IP addresses; adding the abnormal IP addresses to a blocked IP list; determining different degrees of access permission restrictions according to the number of blocking times of the abnormal IP addresses; and limiting the access permission of the abnormal IP addresses by identifying the abnormal IP addresses, thereby preventing malicious data collection and protecting the data security and integrity of the website.

[0081] Embodiment 2:

[0082] Figure 2 This is a schematic diagram of an IP blocking system based on website data provided in Embodiment 2 of the present invention.

[0083] Reference Figure 2 , the system comprises:

[0084] Access track data acquisition module 1, used to acquire user access track data;

[0085] Monitoring module 2, used to monitor the IP address of each visited website based on user access trajectory data;

[0086] The abnormal IP address acquisition module 3 is used to input the IP address of each visited website into the supervised learning model to obtain the abnormal IP address;

[0087] Add module 4, used to add abnormal IP addresses to the blocked IP list;

[0088] The determination module 5 is used to determine different degrees of access permission restrictions according to the number of times the abnormal IP address is blocked.

[0089] Furthermore, the determination module 5 is specifically used for:

[0090] When the number of blocking times of the abnormal IP address is less than or equal to the first preset threshold, the access speed of the website is restricted;

[0091] When the number of blocking times of the abnormal IP address is greater than or equal to the second preset threshold and less than or equal to the third preset threshold, restrict access to important pages of the website;

[0092] When the number of blockings of an abnormal IP address is greater than or equal to a fourth preset threshold, restricting access to the website within a preset time period;

[0093] Among them, the first preset threshold is smaller than the second preset threshold, the second preset threshold is smaller than the third preset threshold, and the third preset threshold is smaller than the fourth preset threshold.

[0094] The beneficial effect of this application is that by blocking the IP addresses of illegal data collection, malicious crawlers can be effectively prevented and the data security and user privacy of the website can be protected.

[0095] The IP addresses of malicious data collection have a high access frequency, which may have a negative impact on the performance and response speed of the website. By blocking these abnormal IP addresses, the load of the website can be reduced and its operation efficiency can be improved.

[0096] Blocking abnormal IP addresses can reduce the risk of theft and infringement and protect the legitimate rights and interests of the website.

[0097] The collection of malicious data may cause the content of the website to be distorted or tampered with, affecting the user's browsing and usage experience. By blocking these abnormal IP addresses, we can ensure that users can access and use the website normally and improve their experience quality.

[0098] A large number of visits from IP addresses with malicious data collection may consume a large amount of network traffic and increase the operating costs of the website. By blocking these abnormal IP addresses, unnecessary network traffic consumption can be reduced and costs can be reduced.

[0099] In addition, in order to protect the security and integrity of website data, many websites also use other anti-crawler technologies. These technologies include: verification code verification and User-Agent detection when access is frequent, which can effectively prevent crawlers from maliciously collecting website data.

[0100] The embodiment of the present invention provides an IP blocking system based on website data, including: obtaining user access trajectory data; monitoring the IP address of each visited website according to the user access trajectory data; inputting the IP address of each visited website into a supervised learning model to obtain abnormal IP addresses; adding the abnormal IP addresses to a blocked IP list; determining different degrees of access permission restrictions according to the number of blocking times of the abnormal IP addresses; and limiting the access permission of the abnormal IP addresses by identifying the abnormal IP addresses, thereby preventing malicious data collection and protecting the data security and integrity of the website.

[0101] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the IP blocking method based on website data provided in the above embodiment are implemented.

[0102] An embodiment of the present invention also provides a computer-readable medium having a non-volatile program code executable by a processor, wherein a computer program is stored on the computer-readable medium, and when the computer program is executed by the processor, the steps of the IP blocking method based on website data of the above embodiment are executed.

[0103] The computer program product provided in the embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment and will not be repeated here.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0105] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0106] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0107] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0108] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An IP blocking method based on website data, characterized in that: The method comprises: Get user access track data; Monitor the IP address of each visited website according to the user access trajectory data; Inputting the IP address of each visited website into a supervised learning model to obtain abnormal IP addresses; Add the abnormal IP address to the blocked IP list; Different degrees of access permission restriction are determined based on the number of times the abnormal IP address is blocked.

2. The IP blocking method based on website data according to claim 1 is characterized in that: Determine different degrees of access rights restrictions based on the number of times the abnormal IP address is blocked, including: When the number of blocking times of the abnormal IP address is less than or equal to the first preset threshold, limiting the access speed of the website; When the number of blocking times of the abnormal IP address is greater than or equal to the second preset threshold and less than or equal to the third preset threshold, restricting access to important pages of the website; When the number of blocking times of the abnormal IP address is greater than or equal to a fourth preset threshold, restricting access to the website within a preset time period; The first preset threshold is smaller than the second preset threshold, the second preset threshold is smaller than the third preset threshold, and the third preset threshold is smaller than the fourth preset threshold.

3. The IP blocking method based on website data according to claim 1 is characterized in that: The supervised learning model is constructed in the following way: Preprocessing the user access trajectory data to obtain preprocessed data; Performing feature extraction on the preprocessed data to obtain feature data; Labeling the normal IP addresses and the abnormal IP addresses in the feature data and historical data to construct a labeled data set; Using the labeled data set as a training data set, wherein the training data set includes a plurality of training samples; The association relationship between the input feature and the corresponding label in each of the training samples is learned to construct the supervised learning model.

4. The IP blocking method based on website data according to claim 3 is characterized in that: The characteristic data includes IP access frequency, average request interval time, number of URL types visited, geographical location changes and user agent changes.

5. The IP blocking method based on website data according to claim 1 is characterized in that: The user access track data includes page URL, access time, IP address, geographic location, user agent and Token identity.

6. The IP blocking method based on website data according to claim 1 is characterized in that: The method further comprises: Share the blocked IP list with a third-party website.

7. An IP blocking system based on website data, characterized in that: The system comprises: An access trajectory data acquisition module is used to acquire user access trajectory data; A monitoring module, used to monitor the IP address of each visited website according to the user access trajectory data; An abnormal IP address acquisition module is used to input the IP address of each visited website into a supervised learning model to obtain an abnormal IP address; An adding module is used to add the abnormal IP address to the blocked IP list; The determination module is used to determine different degrees of access permission restrictions according to the number of times the abnormal IP address is blocked.

8. The IP blocking system based on website data according to claim 7 is characterized in that: The determination module is specifically used for: When the number of blocking times of the abnormal IP address is less than or equal to the first preset threshold, limiting the access speed of the website; When the number of blocking times of the abnormal IP address is greater than or equal to the second preset threshold and less than or equal to the third preset threshold, restricting access to important pages of the website; When the number of blocking times of the abnormal IP address is greater than or equal to a fourth preset threshold, restricting access to the website within a preset time period; The first preset threshold is smaller than the second preset threshold, the second preset threshold is smaller than the third preset threshold, and the third preset threshold is smaller than the fourth preset threshold.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A computer readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the method according to any one of claims 1 to 6.

Images