Information processing method and device, equipment and storage medium

Abstract

The invention provides an information processing method and device, equipment and a storage medium. The information processing method comprises the steps: obtaining a to-be-processed word bank which comprises at least one keyword; searching each keyword in a first preset corpus to obtain a target search result corresponding to each keyword; obtaining a target statement corresponding to each keyword from a second preset corpus associated with a target category to which the to-be-processed word bank belongs; inputting the target search result and the target statement corresponding to each keyword into a neural network model for keyword classification to obtain target keywords which do not belong to a target category, wherein the neural network model is obtained by performing machine learningtraining based on a sample search result, a sample statement and a classification label corresponding to each sample keyword in a sample word bank; and filtering the target keywords from the to-be-processed word bank. The invention relates to a natural language processing and machine learning technology of artificial intelligence. By using the scheme of the invention, the accuracy of word bank denoising can be improved.

Description

technical field [0001] The present application belongs to the field of computer technology, and specifically relates to an information processing method, device, equipment and storage medium. Background technique [0002] Existing lexicon denoising methods mainly include the following: (1) use statistical methods, such as degrees of freedom (Degreeof Freedom, DF), term frequency-inverse text frequency index (Term Frequency–Inverse Document Frequency, TF-IDF ) and other methods; (2) use the word vector similarity method, for example, for the category of "3C digital", there are already words such as "computer", "notebook", "camera" in this category, and the newly added words if If the overall similarity with these words is relatively high, it is considered that the newly added words belong to the category of "3C digital"; (3) denoise the thesaurus through a classifier. [0003] However, the existing technology has the following defects: (1) the semantic information of words c...

AI Technical Summary

Problems solved by technology

[0003] However, the existing technology has the following defects: (1) the semantic information of words cannot be obtained simply by using statistical methods, and the denoising accuracy of the thesaurus is low; (2) the method of using word vector similarity cannot calculate and obtain a large number of long-tailed words Word vector; (3) Use a classifier to denoise the thesaurus, the accuracy of the corresponding text of the obtained words is not high, and it is also impossible to obtain a large number of long-tailed word vectors, resulting in the final denoising result of the thesaurus not well known to users , which does not conform to common sense

[0004] Due to the above defects in the existing technology, the accuracy of denoising thesaurus is not high, so that it is impossible to provide users with services such as accurate word classification or sentence recognition through the denoised thesaurus

Images