Information monitoring system based on data crawler technology
A crawler technology and monitoring system technology, applied in the field of web crawler applications, can solve problems such as low efficiency
Inactive Publication Date: 2018-08-24
四川久久合创信息技术有限公司
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005] At present, most candidates collect postgraduate entrance examination information through manual search to obtain fragmented school information. They also need to calculate the scores of previous years, reference materials of professional courses, and enrollment ratio, etc., which is very inefficient.
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0014] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, the specific implementation manners of the present invention will now be described with reference to the accompanying drawings.
[0015] like figure 1 The shown a kind of information monitoring system based on data crawler technology includes: data grabbing module, crawler detection module, data processing module, VPN proxy server, school enrollment webpage, social media webpage, client, user, described crawler detection The module and the data processing module are respectively connected to the data grabbing module, and the data grabbing module accesses the school enrollment webpage and the social media webpage through the VPN proxy server, and the data processing module processes the webpage data and sends it to the cloud server, and the user uses the client The client accesses the cloud server through the Internet.
[0016] Further, the data grabbing mo...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses an information monitoring system based on a data crawler technology. The information monitoring system based on the data crawler technology comprises a data capture module, a crawler detection module, a data processing module, a VPN proxy server, a school enrolment page, a social media page, a client side and a user; the crawler detection module and the data processing module are separately connected with the data capture module; the data capture module accesses the school enrolment page and the social media page through the VPN proxy server; the data processing moduleprocesses page data and sends the page data to a cloud server; and the user accesses the cloud server through the Internet by using the client side. According to the information monitoring system based on the data crawler technology disclosed by the invention, data on the school enrolment page and the social media page is obtained through the crawler technology; simultaneously, when a web crawleroperates abnormally, alarm information is sent at the client side, so that the user is notified; and thus, the crawler reliability is improved.
Description
technical field [0001] The invention relates to the field of web crawler applications, in particular to an information monitoring system based on data crawler technology. Background technique [0002] With the vigorous development of network technology, obtaining data through the Internet has become an important way for people to obtain information resources. Using web crawlers to obtain data has become the mainstream acquisition method. According to the geographical location and network topology of web crawlers, it can be Web crawlers are divided into single-domain distributed crawlers (Single-domain Distributed crawlers, also known as LAN crawlers) deployed in the same local area network and multi-domain distributed crawlers (Multi-domain Distributed Crawler, also known as WAN crawlers) deployed in a wide area network. Whether it is a single-domain crawler or a wide-area network crawler, the most basic function is web page data capture, and its soul is the scheduling strat...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/06H04L29/08H04L12/24G06F17/30
CPCH04L41/0631H04L63/0272H04L63/0281H04L63/1416H04L63/145G06F16/951H04L67/56
Inventor 孟秀惠
Owner 四川久久合创信息技术有限公司