Method and device for realizing attribute normalization

Abstract

The invention discloses a method and a device for realizing attribute normalization, and relates to the technical field of computers. A specific embodiment of the method includes: taking the word of the data to be normalized as KEY and the data to be normalized as VALUE to obtain the first data; aggregating the first data with the same KEY into a first data group, in the first data group The VALUE of each piece of first data is aggregated into the second data, and one of the words in the first data group is selected as the normalized word of the second data; each original word in the second data is KEY, and the second data is VALUE Obtain the third data; aggregate the third data with the same KEY into the second data group, aggregate the VALUE of each third data in the second data group into the fourth data, from all the normalized words in the second data group Select a normalized word as the fourth data and add it to the extended word set to form the result data; determine the normalized attribute value according to the result data. This implementation mode can realize attribute normalization without manual maintenance, has high accuracy, good real-time performance, and saves manpower.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a method and device for realizing attribute normalization. Background technique [0002] The basis of various data mining algorithms is the characteristics of data. However, in today's Internet, in order to increase the exposure rate in various search engines, users often reflect the various ways of writing the same attribute in the text as much as possible. That is, a certain attribute value of a piece of data is often represented by splicing multiple redundant words that can represent the same meaning. Using different writing methods to write the attribute value of the same attribute will bring a lot of trouble to data processing. Taking the e-commerce field as an example, if a certain brand A has four writing styles: A1, A2, A3, and A4, these four writing styles all represent brand A. The number, sequence, etc. are not necessarily the same. If "A1 A2 A3" is the o...

AI Technical Summary

Problems solved by technology

[0005] 1) Manpower consumption: Maintaining the corresponding table requires continuous manpower, because whenever the original words of the same attribute appear in different ways, they must be manually maintained and added to this corresponding table

[0006] 2) Low accuracy and real-time performance: In massive data, it is difficult to manually accurately, comprehensively, and quickly find the original words of a certain attribute that appear in different ways. Classification errors caused by untimely and inaccurate updates, or situations that do not correspond at all

Images