Patent data cleaning method and system based on AdaBoost algorithm

Abstract

The invention belongs to the technical field of data processing, and more specifically relates to a patent data cleaning method and a patent data cleaning system based on an AdaBoost algorithm. The method comprises the following steps of S1, collecting patent data from a patent database, and putting the collected patent data into a database to be cleaned; S2, performing data analysis on the patentdata source in the database to be cleaned, and determining attribute information of the patent data; S3, defining a cleaning rule, and making different cleaning rules according to the different errortypes of the patent data source; S4, primarily cleaning the patent data source according to the cleaning rule; S5, using the AdaBoost algorithm to deeply clean the primarily cleaned patent data; S6,verifying a cleaning result, judging whether a cleaning requirement is met, if yes, turning to the step S6, and otherwise, turning to the step S2; and S7, flowing the clean data back, and replacing the original patent data with the cleaned patent data.

Description

technical field [0001] The invention belongs to the technical field of data processing, and more specifically relates to a patented data cleaning method and system based on the AdaBoost algorithm. Background technique [0002] In today's society, with the advent of the information age, the demand for data has increased, making data processing more and more complex. The most important step for mining and analyzing massive amounts of data is data cleaning. Different types of errors need to be identified during the data cleaning process. Manual operations alone are half the effort. Combining the AdaBoosting algorithm to identify and classify data for analysis and detection can greatly improve efficiency and complete large-capacity data cleaning tasks. [0003] At present, the main problems faced by patent data cleaning are: [0004] (1) As my country has gradually become a patent powerhouse, there are more and more patent data, and the demand of various enterprises is also gra...

AI Technical Summary

Problems solved by technology

Due to the explosive growth of data volume, the task of data cleaning has also become more and more difficult. With the rapid development of national conditions, data collection, different data sources, real-time data updates, and data aggregation are constantly following. Scale, so it can easily lead to higher error rates from different aspects of the data, which in turn leads to lower data quality

[0005] (2) The performance of traditional data cleaning methods cannot meet the requirements when dealing with today's large data volumes, and thus cannot effectively process huge data sets

Images