Entity matching method and computer program based on non-principal attribute outlier detection

Abstract

The invention discloses an entity matching method based on non-main attribute outlier detection, which belongs to the technical field of the Internet, and is characterized in that: the entity matching method based on non-main attribute outlier detection is embodied in two aspects. On the one hand, it utilizes Non-key attribute values ​​eliminate the ambiguity caused by the diversity of main attribute values. On the other hand, according to the outlier model, the data is quickly screened to extract matching pairs; The characteristics of the main attribute use the corresponding rules to roughly screen the data to reduce the data size of the record pair. On this basis, use the five steps in the outlier model to do further screening to obtain a preliminary entity pair set, and then according to the generated The entity pair set samples the data set, and finally uses machine learning to select an appropriate matcher and train it. The present invention overcomes to a certain extent the drawback that outlier point matching cannot be applied to large-scale data in traditional singular value decomposition.

Description

technical field [0001] The invention belongs to the technical field of the Internet, and in particular relates to an entity matching method and a computer program based on non-main attribute outlier detection. Background technique [0002] In the next 30 years, data applications will become more and more prominent, which will definitely affect the construction and development of civil aviation informatization. With the promotion of the mobile Internet, some convenient applications can be pushed to smart terminals, and big data technology can be used to analyze the behavior of passengers and understand their concerns, so as to improve the aviation experience of users. [0003] From the perspective of global civil aviation development, due to the intensified market competition, the civil aviation industry has long been operating at a meager profit level. With the continuous worsening of the global financial crisis in recent years, the survival pressure of airlines is increasi...

AI Technical Summary

Problems solved by technology

[0007] To sum up, the shortcomings of the existing technology are: the supervised classifier model introduced in this paper needs to be trained, and label labeling requires a lot of work. In the future, we can try to use weak supervision or crowdsourcing to make the system automatically Finding matches and reducing the workload of manual labeling is the focus of future research

Images