Network table column type detection method based on probabilistic graph model

Abstract

The invention provides a network table column type detection method based on a probabilistic graph model, and belongs to the field of table interpretation in a semantic network. The method comprises the following steps: splicing tables belonging to the same mode from the same website into a table; performing single-column classification on the spliced table: firstly, dividing columns in the spliced table into numeric type columns and character type columns, and then classifying the numeric type columns and the character type columns respectively; and on the basis of a single-column classification result, a probabilistic graph model is constructed to mine a semantic relationship implied between columns, so that the column type sequence of the whole table is detected. According to the method, the semantic type of the column in the network table can be detected, a good effect is achieved, and compared with other column type detection methods, the accuracy is improved by 10% or above.

Description

technical field [0001] The invention belongs to the field of table interpretation in the semantic web, and mainly relates to a method for detecting column types of a network table based on a probability graph model. Background technique [0002] Web tables use a fixed structure to present their content, provide a compact representation of entities described by attributes and relationships between entities, and are distinct from other types of tables such as layout tables used primarily for formatting purposes or for display in a grid format Unlike the matrix table of digital summaries), its structure contains very valuable relational knowledge. At the same time, compared with unstructured data, its research reduces the workload of extracting and interpreting data. For the above reasons, web forms have received increasing attention from the research community. There is a wealth of knowledge embedded in web forms, and there are many practical use cases that leverage web form...

AI Technical Summary

Problems solved by technology

[0005] The existing network table column type detection method has the following limitations: (1) The traditional method is based on cell ontology matching to find the entity corresponding to the cell data contained in the target column in the knowledge base. There are a large part of the cells in the network table in the data set that cannot find the corresponding entities in the knowledge base. On the other hand, the semantic association between the cells is ignored, which leads to the lack of robustness to dirty data; (2 ) Most of the feature engineering-based methods require a variety of manually defined features, the scalability is not strong, and the calculation is time-consuming and labor-intensive. At the same time, because only the vocabulary comparison method is used without considering the similarity at the semantic level, it is prone to ambiguity and reduces the matching (3) Most of the models only consider the characteristics and semantic relationships inside the cells, ignoring the possible associations between cells, and only model the features of a single column, without taking advantage of the special organizational structure of the table and potentially complex semantic relationships between cells

Images