Automatic template mining system and method based on cross support degree evaluation

Abstract

The invention discloses an automatic template mining system and method based on cross support degree evaluation. The system comprises an intention recognition module, a category word replacement module, a frequent item set mining module and a template sorting module. The intention recognition module is used for carrying out intention recognition on historical records of a user and sending the records subjected to intention recognition to the category word replacement module; the category word replacement module is used for carrying out word segmentation on the record subjected to consciousnessidentification, replacing category words and sending the record after the category words are replaced to the frequent item set mining module; the frequent item set mining module is used for mining the records after the category words are replaced by utilizing an association rule mining algorithm and screening frequent items to obtain a preliminary template; and the template sorting module is usedfor sorting the preliminary templates according to the entropy and the similarity with the existing word list. According to the automatic template mining method, a frequent item set mining method isimproved, and evaluation based on the cross support degree has higher quality than evaluation based on confidence.

Description

technical field [0001] The invention relates to the field of automatic mining of search templates, in particular to an automatic template mining system and method based on cross-support evaluation. Background technique [0002] In vertical search, when the user's search keywords match the regular words in the database, relevant data in the database will be returned. In practical applications, users search for various keywords, and it is difficult to manually configure all the matching words. With the increase of the number of search types, manual configuration is obviously an unrealistic approach. Therefore, the design algorithm automatically mines the frequently used search words of users. Templates are necessary. The current research is mainly to mine search templates from the user's historical data. A typical representative is Baidu's search technology patent "Automatic Mining Method for Requirement Recognition Template, Requirement Recognition Method and Corresponding D...

AI Technical Summary

Problems solved by technology

In the process of template automatic mining, the records after intention recognition have the problem of uneven distribution of item set support, which leads to the occurrence frequency of category words far exceeding the frequency of other words

[0005] Another traditional confidence It only considers the influence of A on B, and ignores the occurrence of B. Assuming that the setting of reliability is unreasonable, when When the situation occurs, it indicates that the two items of A and B are independent, but because the confidence level is set too low, such a record is also retained

The records after intention recognition will have unbalanced support distribution, so the quality of mining depends on the setting of confidence, and it is often difficult to find the most suitable confidence threshold

Images