Urban internet open data acquisition, processing and analysis method

A technology of open data and analysis methods, applied in data processing applications, electrical digital data processing, special data processing applications, etc., can solve the problems of low efficiency of urban Internet open data, to overcome low efficiency problems, improve efficiency, and simplify work The effect of the process

Inactive Publication Date: 2018-06-19
SOUTH CHINA UNIV OF TECH
View PDF2 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to overcome the low efficiency of relying on manual acquisition of urban Internet open data in the existing urban planning research and analysis, and proposes an urban Internet open data acquisition, processing and analysis method, which considers the urban planning vertical industry Internet The characteristics of data analysis, such as the spatial information of building space location, overcome the repetitive writing of data capture scripts for specific research questions and rely on spreadsheet software such as excel to process data, under the premise that the data display methods and styles of specific websites are consistent , to extract key data based on the template method from the data in the webpage of the website, such as the address information of the auctioned land, the total number of user reviews of the shops, and the average rating of the shops

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Urban internet open data acquisition, processing and analysis method
  • Urban internet open data acquisition, processing and analysis method
  • Urban internet open data acquisition, processing and analysis method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] The present invention will be further described below in conjunction with specific examples.

[0033] like figure 1 As shown, the urban Internet open data acquisition processing analysis method provided in this embodiment, its specific circumstances are as follows:

[0034] 1) City Internet open data directory page processing

[0035] Initiate network access according to the initial website address defined in the specific website template file given by the user, and provide a variety of anti-crawler strategies. After obtaining the html source code of the webpage, according to the strategy defined in the template, that is, the directory page extraction rules and content page URL extraction rule, and pass the network address of the content page to step 2) for processing. Continue to the next directory page until all directory pages for the site have been processed.

[0036] like figure 2 As shown, first, analyze the website template information to obtain the anti-cra...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an urban internet open data acquisition, processing and analysis method. The method comprises the steps that 1) an urban internet open data directory page is processed; 2) thekey value information of a website content page based on a website template is extracted; and 3) post-processing and analysis of the information are carried out. According to the invention, data withanalysis value are extracted based on the template according to the characteristic of diversity of an internet website, and at the same time simple statistical analysis is carried out; the applicationscope of a system is improved; the problem of low efficiency of manual urban internet open data acquisition is overcome; the process of urban internet open data acquisition and analysis is effectively simplified; the novel method is provided for urban internet open data acquisition; and the method has great scalability.

Description

technical field [0001] The invention relates to the technical field of Internet open data processing and analysis, in particular to a method for acquiring, processing and analyzing urban Internet open data. Background technique [0002] The Internet is a channel for the transmission of various value data. The Internet uses electronic communication technology to reduce the cost of people communicating with each other. With the vigorous development of information and electronic technology, Internet applications such as 58.com, Dianping, Ctrip, and government publicity websites have emerged that combine office and life application scenarios. People rely more and more on such websites to consume and evaluate life services and understand city planning, and due to the gradual increase in users, such websites increasingly represent information on human activities in the city, that is, such websites contain information about Valuable data for urban planning analysis and research. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06Q10/04G06Q30/02G06Q50/26G06F17/30
CPCG06Q10/04G06Q30/0201G06Q50/26G06F16/951
Inventor 黄铎赵立华皮思远方小山王成芳金蓉
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap