Bag-of-words model text representation method and system based on fault-tolerant rough set, and medium

Abstract

The invention discloses a bag-of-words model text representation method based on a fault-tolerant rough set, which comprises the following steps: 101, acquiring a corpus by a computer, and performingpreprocessing operation on the corpus; 102, determining a base word set of a corpus by the computer; 103, acquiring an uncertainty function (fault-tolerant class) of each base word by the computer, and constructing an uncertainty function matrix; 104, calculating the fuzzy membership degree of each base word based on each document by the computer, and constructing a fuzzy membership matrix; 105, acquiring upper approximation and lower approximation of each document by the computer based on the fuzzy membership matrix, and constructing upper and lower approximation matrixes; 106, constructing atext representation matrix of the corpus by the computer based on the two weight calculation schemes; according to the method, the fuzzy membership degree of words in a corpus to each document is calculated mainly by applying a fault-tolerant rough set model, the upper approximation and the lower approximation of the documents are obtained, a corresponding weight calculation method is constructedfor document representation, and the current technical development requirements are practically combined.

Description

technical field [0001] The invention belongs to the fields of machine learning, artificial intelligence technology and natural language processing, and is a text representation method in natural language processing. Background technique [0002] Text representation is the basis of Natural Language Processing (NLP). With the explosive growth of the Internet, countless text data continue to accumulate. However, unlike numeric data, which is a structured data type, document or text data is unstructured data. Unstructured data is not suitable for direct application in machine learning or deep learning algorithms. As the basis of natural language processing and text mining tasks, efficient text or document representation is particularly important. A major challenge in document representation is how to transform unstructured text data into structured data. For an effective document representation, it not only needs to truly reflect the content of the document, but also has the...

AI Technical Summary

Problems solved by technology

However, the bag-of-words model has the disadvantages of sparsity and lack of semantic association

Images