Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Website anti-banning method and system capable of limiting access times

A technology of number of visits and websites, which is applied in the field of computer networks and can solve problems such as limited number of visits

Inactive Publication Date: 2021-06-25
CHINSESALL DIGITAL PUBLISHING GRP CO LTD
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The object of the present invention is to provide a method and system for preventing website banning with a limit on the number of visits, so as to solve the aforementioned problems in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Website anti-banning method and system capable of limiting access times

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 2

[0028] Such as figure 1 As shown, this embodiment provides a website anti-blocking method with a limit on the number of visits, and the method includes the following steps,

[0029] S1. The target browser uses selenium to access the login page of the website;

[0030] S2. After the website login page is loaded, obtain the prepared user name and password cyclically, and fill the obtained user name and password into the website login page;

[0031] S3. Log in to the website, obtain the cookie of the login result, and return the response body;

[0032] S4, parse the response body, and obtain the content in the set cookie;

[0033] S5. Determine whether the obtained login result cookie is valid, and remove the expired cookie, and save the remaining valid cookie;

[0034] S6. The stored effective cookie is called to directly collect website data.

[0035] In this embodiment, the login page is provided with a login form, and the login form is provided with a user name line and a...

Embodiment 1

[0039] This embodiment provides a website anti-blocking system with a limited number of visits, the system is used to implement the method, and the system includes:

[0040] A crawler tool; the crawler tool is used to automatically control the target browser to use selenium to access the website login page;

[0041] A login information acquisition module; the login information acquisition module is used to cyclically acquire the prepared user name and password, and fill the acquired user name and password into the website login page;

[0042] Result acquisition module; the result acquisition module is used to obtain the cookie of the login result, and returns responsebody;

[0043] Parsing module; the parsing module is used to parse the response body and obtain the content in the set cookie;

[0044] Cookie control module; The Cookie control module is used to judge whether the cookie of the login result is valid, and the expired cookie is removed, and the remaining valid cook...

Embodiment 3

[0049] In this embodiment, an example is used to illustrate the implementation process of the method.

[0050] The target website imposes login verification restrictions on the access data, and limits the number of visits to the target data to 300 times per day, and an error page will be returned if the number of visits exceeds.

[0051] 1. Write the java-selenium driver, call selenium, and use webdriver to access the website login page;

[0052] 2. When the page is loaded, the user name and password in the database are obtained cyclically, and filled into the login form;

[0053] 3. Log in to the website and get the cookie of the login result;

[0054] 4. Parse the response body and submit the content in the set cookie to the cookie control module;

[0055] 5. The cookie control module judges the validity of the current cookie, removes the expired cookie, saves the cookie to the mogodb database, and dynamically adds the cookie to the cookie pool. When the number of cookie r...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a website anti-forbidding method and system with limited access times. The method comprises the steps of: S1, enabling a target browser to use selenium to access a website login page; S2, after the website login page is loaded, circularly acquiring the prepared user name and password, and filling the website login page with the acquired user name and password; S3, logging in a website, obtaining a cookie of a login result, and returning to a response body; S4, analyzing the response body, and obtaining the content in the set cookie; S5, judging whether the cookie of the obtained login result is valid or not, removing the expired cookies, and storing the remaining valid cookies; and S6, calling the stored effective cookie, and directly collecting the website data. The method has the advantages that the effect of rapid login is achieved, the threshold value used by the cookie is effectively controlled by managing the cookie, and the purpose of preventing access limitation by a website is achieved.

Description

technical field [0001] The invention relates to the technical field of computer networks, in particular to a website anti-blocking method and system with limited access times. Background technique [0002] When crawlers collect directional data from directional websites, sometimes we encounter websites that have a daily limit on the number of times a logged-in user can access specific data. When a certain threshold is reached, the current logged-in user’s access will be restricted, and the user will even be banned. Traditionally, through manual logging Code, the way to obtain the login cookie is inefficient and cannot meet the requirements of big data collection. Contents of the invention [0003] The purpose of the present invention is to provide a website anti-blocking method and system with a limited number of visits, so as to solve the aforementioned problems in the prior art. [0004] In order to achieve the above object, the technical scheme adopted in the present i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L29/06
CPCH04L63/08H04L63/083H04L63/10
Inventor 孙香娟李鑫周彦其姜波
Owner CHINSESALL DIGITAL PUBLISHING GRP CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com