Unlock instant, AI-driven research and patent intelligence for your innovation.

Method for obtaining website data in user-defined and real-time manner

A real-time acquisition and self-definition technology, applied in the field of network information, can solve problems such as high difficulty and difficulty in obtaining data at will, and achieve the effects of reducing work burden, wide applicability, and reducing work difficulty and workload

Inactive Publication Date: 2019-08-16
鼎复数据科技(北京)有限公司
View PDF6 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, the current web crawler technology is only suitable for network technicians to collect data in a targeted manner, and it is more difficult for non-network technicians to use web crawler technology to collect data
[0005] In addition, a web crawler program is often set up specifically for a specific website for data collection, and it is difficult to customize and freely obtain data from any website

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for obtaining website data in user-defined and real-time manner
  • Method for obtaining website data in user-defined and real-time manner
  • Method for obtaining website data in user-defined and real-time manner

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0084] Establish monitoring of the gold price in the Alipay-Cunjinbao webpage, wherein the URL of the Alipay-Cunjinbao webpage is: https: / / cjb.alipay.com / gold / guide.html; wherein, the monitoring period for the gold price is 120 minutes .

[0085] Such as Figure 4 As shown, enter the URL of the Alipay-Cunjinbao webpage, and the server will obtain the html file of the Alipay-Cunjinbao webpage and the CSS file referenced in the html file according to the input URL, and return the file to the browser. The interface of the Alipay-Cunjinbao webpage is displayed in the form of an embedded webpage, such as Figure 5 shown. Display the interface of Alipay-Cunjinbao webpage in the left window of the monitoring webpage; set the monitoring time period, file name, and monitoring content in the right window of the monitoring webpage.

[0086] At the same time, an XPath path is also provided in the monitoring web page, and the XPath path is: / / *[@id="container"] / div[4] / div[1] / div / div / div...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for obtaining website data in a user-defined and real-time manner. The method comprises the following steps: (1) a browser displays an interface of a target website ina monitoring webpage according to an input target website, and sets a collection content and a data collection period; (2) an XPath path is set in the monitoring webpage, and a server collects information of the target website through the XPath path according to the set data collection period; and (3) when the data is checked, according to the selected data collection time, the browser displays the information of the target website, collected at the selected data collection time, in an interface for checking the webpage. The method for obtaining website data in a user-defined and real-time manner does not need to specially compile and deploy programs for different websites, and is wide in applicability.

Description

technical field [0001] The invention relates to a network information technology, in particular to a method for customizing and obtaining website data in real time. Background technique [0002] With the rapid development of the network, the data on the Internet grows explosively and is constantly updated. For data collectors, it is necessary to frequently log in to relevant websites to collect the required data and organize it. If all these data are collected and organized manually, data companies will consume a lot of labor and time costs in data collection every day. [0003] Under such circumstances, web crawler technology emerged as the times require, liberating people's hands, people don't have to collect and organize data repeatedly every day, these tasks can be done by computers. [0004] Web crawler technology requires a programming foundation, and most practitioners in data collection work do not have deep research in the direction of network technology. In the p...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/951G06F16/957H04L29/08
CPCH04L67/02G06F16/951G06F16/9577
Inventor 淡强强刘炬光王博远吴雪军
Owner 鼎复数据科技(北京)有限公司