Product database forming method based on Internet data and system

Abstract

The invention discloses a product database forming method based on Internet data and a system. The method includes the steps of capturing webpage data with the theme relevance higher than a preset threshold value by the adoption of the focused crawler technology, performing structuralized storage on the captured webpage data, automatically classifying the structuralized storage webpage data according to the categories which products belong to, performing statistics on the frequency and the time of occurrence of attributes of the products in the webpage data after the automatic classification, performing weighting calculation on the frequency and the time of occurrence of the attributes of the products according to preset weighting, acquiring the decision value of the attributes of the products, and determining the sort order of the attributes of the products according to the decision value of the attributes of the products. The system comprises a data capturing module, a structuralized storage module, a data classifying module and an attribute deciding module. According to the product database forming method based on the Internet data and the system, a user can acquire comprehensive and summarized information without needing to collect and sort product information on the Internet, real-time performance of data is ensured, and real-time requirements of the user are met.

Description

technical field [0001] The invention relates to the technical field of Internet data processing, in particular to a method and system for forming a product database based on Internet data. Background technique [0002] At present, the product catalogs of some mainstream websites are formed by using fixed product release templates for various industries to form a product description. Moreover, each website adopts different standards for describing the same product. In this way, due to the non-uniform format of product release standards, for the product demand side, the demand standards are various, and because the product description formats of major websites are not uniform, it is difficult to comprehensively sort out product information, and it is impossible to know the products that meet the demand standards. For more comprehensive product information, if product selection is carried out according to demand standards, it is often necessary to read a large number of web pa...

AI Technical Summary

Problems solved by technology

In this way, due to the non-uniform format of product release standards, for the product demand side, the demand standards are various, and because the product description formats of major websites are not uniform, it is difficult to comprehensively sort out product information, and it is impossible to know the products that meet the demand standards. For more comprehensive product information, if product selection is carried out according to demand standards, for the case of large-volume multi-model product selection, it is often necessary to read a large number of web pages, which is inefficient

[0003] To sum up, due to the lack of a unified product description standard in related technologies, there is a technical problem that it is difficult to sort out product information

Images