Commodity dynamic prediction method and system based on web crawler

A web crawler and dynamic prediction technology, applied in business, instrumentation, data processing applications, etc., can solve problems such as small improvement effect, crawling failure, lack of business scenarios, etc., to achieve work efficiency, maximize, and improve data quality Effect

Inactive Publication Date: 2018-11-23
湖北国网华中科技开发有限责任公司
View PDF4 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The above crawling technology lacks the support of business scenarios, and the improvement effect on actual work is small, which is not conducive to promotion
In fact, with the popularization of web crawler technology, more and more websites will formulate various anti-crawling strategies based on the protection of dat

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Commodity dynamic prediction method and system based on web crawler
  • Commodity dynamic prediction method and system based on web crawler
  • Commodity dynamic prediction method and system based on web crawler

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0050] like figure 1 As shown, a kind of commodity dynamic prediction method based on web crawler of the present invention, comprises the following steps:

[0051] S1. Crawling the key fields or files of the target website;

[0052] S2. Process the crawled data, obtain structured data and store it;

[0053] S3. Perform time series analysis on the data to obtain a prediction model for the relationship between specific fields and time periods;

[0054] S4. Obtain the required prediction results according to different data queries through the prediction model.

[0055] During specific implementati...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a commodity dynamic prediction method and system based on a web crawler. A user can customize an analysis operation flow according to a specific service scene through a complete network data analysis method of data crawling, cleaning, processing, querying, modelling and visually presenting, the maximization of the working efficiency is realized, the data of multiple websites can be crawled by using various targeted anti-crawling technologies, the automatic acquisition of the back-end work data is supported, and the data leaning, the normalization, the smooth processingand like method can be performed by providing multiple statistics method; and then the actual service scene is combined to perform modelling and analysis so as to display in various visual diagrams;the data quality is improved, and the system can help user making decision.

Description

technical field [0001] The invention relates to the technical field of network information processing, in particular to a method and system for commodity dynamic prediction based on a web crawler. Background technique [0002] In systems that obtain information from massive commodity information data sources on the Internet, most of them are to improve query and retrieval efficiency. Their respective characteristics often form their own update modes, and dynamically predict the bandwidth required by each crawler as a basis for dynamic bandwidth allocation. That is, the web crawler systems on the market are all technical research aimed at improving crawling efficiency or retrieval efficiency, and lack data-based statistical processing, presentation, and mining of deep value of data. The above crawling technologies lack the support of business scenarios, and have little improvement in actual work, which is not conducive to promotion. In fact, with the popularization of web c...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06Q30/02
CPCG06Q30/0202
Inventor 周鑫施险峰褚金一莫婷
Owner 湖北国网华中科技开发有限责任公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap