Chinese statement simplification method and device

Abstract

The invention discloses a Chinese statement simplification method and device. The method comprises the steps of carrying out the word segmentation processing on a received text sequence to obtain a word segmentation sequence; performing encoding processing on the word segmentation sequence through a trained encoder to generate a semantic vector; and decoding the semantic vector through the traineddecoder to generate a simplified output sequence, wherein the encoder and the decoder are both LSTM models. According to the scheme provided by the embodiment of the invention, the combination of LSTM is selected for encoding and decoding in the model training process because the LSTM can effectively solve the problem of long-term dependence in sequence prediction.

Description

technical field [0001] The invention belongs to the technical field of sentence simplification, and in particular relates to a method and a device for simplifying Chinese sentences. Background technique [0002] Among related technologies, there is currently no technology or method that can simplify and normalize Chinese sentences. Most of the methods are to artificially normalize some words with the same or similar meanings, such as Stop word removal, synonym replacement, statistical based methods, etc. [0003] Among them, the deletion of stop words is based on the sorted out stop words list, delete some common function words, such as help, please, can, etc.; synonym replacement is to artificially sort out some possible synonym lists, and replace the sentences with synonyms. To achieve the purpose of expanding the statement; the statistical method is to count some common prefixes and suffixes in the sentence, as well as some possible redundant phrases, which are directly ...

AI Technical Summary

Problems solved by technology

[0006] The synonym replacement method is too violent, which will lead to excessive expansion of analysis resources; and synonyms are difficult to define and are easily affected by human subjective factors

Its biggest disadvantage is time-consuming, high cost, poor quality of thesaurus and prone to wrong synonyms

The application field is very limited, and the ability to simplify sentences is very weak

[0007] Statistical-based methods only rely on statistical methods and human experience and knowledge, without considering the variability of Chinese semantics. Even a common sentence suffix is ​​different in different sentences. If you simply delete or rewrite the sentence, It is very easy to affect or change the original meaning of the sentence, and the loss outweighs the gain.

Images