Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and system for collecting web data

A data acquisition system and data acquisition technology, applied in the direction of digital transmission system, transmission system, data exchange network, etc., can solve the problem of high pressure on the front end of the webpage, achieve the effect of simple structure and avoid excessive pressure

Active Publication Date: 2014-07-16
BEIJING 58 INFORMATION TECH
View PDF3 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, according to the current web data collection method, there is a possibility that the pressure on the front end of the web page will be too large

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for collecting web data
  • Method and system for collecting web data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] The objects and functions of the present invention and methods for achieving the objects and functions will be clarified by referring to the exemplary embodiments. However, the present invention is not limited to the exemplary embodiments disclosed below; it can be implemented in various forms. The essence of the description is only to help those skilled in the relevant art comprehensively understand the specific details of the present invention.

[0023] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0024] figure 1 A system block diagram of web data collection according to an embodiment of the present invention is schematically shown.

[0025] like figure 1 As shown, the web data collection system 100 according to an embodiment of the present invention includes a data providing devi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a method and system for collecting web data. The system comprises a data providing device and a data collecting device. The data providing device is used for collecting the web data. The data collecting device comprises a load detecting module, a data collecting module and a data analyzing and processing module. The load detecting module is used for detecting the load situation of the data providing device and judging whether the load exceeds the threshold value or not, if yes, the load situation of the data providing device is detected again in a period of time, and if not, the data collecting module is notified to collect the web data of the data providing device. The data collecting module is used for collecting the web data stored in the data providing device and storing the collected web data to a database. The data analyzing and processing module is used for processing the web data stored in the database. By means of the method and system for collecting web data, the web data can be collected accurately and efficiently with a simple structure, and the situation that the pressure of the front end of the webpage is too large is avoided.

Description

technical field [0001] The invention relates to the technical field of computer data processing, in particular to a web data collection method and system. Background technique [0002] At present, with the rapid development of Internet technology and the explosive growth of information, web data analysis is of great significance. By analyzing a large amount of unstructured data in web pages, valuable reports can be obtained, which can be provided to website operators and managers for them to make decisions and operations. Wherein, the Web data includes, for example: PV (page view, page views) log, click (or called event) log, effect log (including login, order, etc.) and other data. [0003] In order to analyze web data, it is necessary to collect web data first. The methods currently used for web data collection mainly include: [0004] Method 1: Web crawler. According to certain rules, starting from the URL of one or several initial web pages, the URL on the initial we...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): H04L12/26H04L12/24
Inventor 赵建华谢刚
Owner BEIJING 58 INFORMATION TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com