Data processing method and system

A technology of data processing and preset data, which is applied in the field of data processing, can solve the problems of actual value interference of data, lack of data processing methods for network data, and affect the value of data, so as to achieve the effect of improving the value of use

Inactive Publication Date: 2017-12-15
GUOXIN YOUE DATA CO LTD
View PDF4 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

If these "dirty data" are not processed, they will interfere with the actual value of the data, thereby affecting the value of the data
The data processing methods in the prior art are mainly aimed at structured data from databases, and with the rapid development of computer network technology, a large amount of valuable network data has been generated, and most of the network data are semi-structured and unstructured data, and there is no effective data processing method for network data in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and system
  • Data processing method and system
  • Data processing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will describe in detail with reference to the drawings and specific embodiments.

[0022] figure 1 It is a schematic flowchart of a data processing method provided by an embodiment of the present invention. Such as figure 1 As shown, the data processing method provided by the embodiment of the present invention includes the following steps:

[0023] S101. Collect web pages from a preset data source.

[0024] S102. Determine the webpage category to which the collected webpage belongs; wherein, the webpage category is divided according to different objects described by the webpage included in the preset data source.

[0025] S103. Using a wrapper corresponding to the webpage category to extract valid information from the collected webpage; wherein, the wrapper is generated according to the attributes of the object described in the web...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data processing method. The method comprises the following steps that a web page is collected from a preset data source; a web page category to which the collected web page belongs is determined; the web page categorization is based on different objects described by the web page included in the preset data source; a wrapper corresponding to the web page category is adopted to extract valid information from the collected web page; the wrapper is generated according to attributes of the objects described by the web page corresponding to the web page category; the extracted valid information is converted into a preset standard format and stored. According to the data processing method, redundant network data can be effectively processed into data required by people, and the use value of the network data is improved.

Description

technical field [0001] The invention belongs to the field of data processing, and in particular relates to a data processing method and system. Background technique [0002] The basic purpose of data processing is to extract and deduce valuable and meaningful data for certain people from a large amount of messy and difficult-to-understand data. Data processing is the basic link of system engineering and automatic control. Data processing runs through all fields of social production and social life. The development of data processing technology and the breadth and depth of its application have greatly affected the progress of human society. [0003] Data processing can detect and correct identifiable errors in data files in a timely manner and correct the errors, mainly including checking data consistency, dealing with invalid and missing values, etc. Because the data in the data warehouse is a collection of data oriented to a certain topic, these data are extracted from m...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/27
CPCG06F40/205G06F40/279G06F40/30
Inventor 陈进宝刘希唐妍
Owner GUOXIN YOUE DATA CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap