Machine writing system based on natural language processing technology

A technology of natural language processing and technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as learning to recognize characters, how to judge the emotion of a sentence, low specificity, high maintenance cost, etc., to achieve The effect of simple word vector encoding, strong specificity, and short training period

CN110287478AActive Publication Date: 2019-09-27GUANGDONG UNIV OF TECH
3 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Publication Date
2019-09-27

Smart Images

  • Figure 1
    Figure 1
Patent Text Reader

Abstract

The invention discloses a machine writing system based on a natural language processing technology, which comprises a preprocessing module, a data loading module, a language model and a main module; the preprocessing module is mainly used for preprocessing text data and comprises a vocabulary segmentation process and a vocabulary numbering process; the data loading module is mainly used for further packaging the preprocessed text data and comprises a data type conversion step and a batch data generation step; the packaged data of each batch can be input into a language model for training; the language model is mainly used for learning language rules in the text data and generating codes of a next prediction word; the main work of the main module is to integrate the functions of all the modules, design a loss function and an optimization method and complete the training work of the model. The system has the advantages of few used technologies, simplicity, high specificity, low maintenance cost and the like.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of natural language processing, in particular to a machine writing system based on natural language processing technology. Background technique

[0002] Since DeepMind's excellent work AlphaGo defeated South Korean professional Go player Lee Sedol with a huge advantage of 4:1 in 2017, the term artificial intelligence has rapidly become popular all over the world. For a time, various achievements relying on artificial intelligence technology continued to emerge, such as image recognition, speech recognition, machine writing, etc.; so that 2017 was called "the first year of AI (artificial intelligence)" by the Wall Street Journal, Forbes and Fortune magazine, Artificial intelligence is getting closer and closer to our life, and I believe it will bring more convenience to our life in the future.

[0003] Machine writing, as the name suggests, is to make machines write articles like humans, which involves many ...

Examples

Embodiment Construction

[0022] The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0023] Such as figure 1 As shown, a machine writing system based on natural language processing technology, including a preprocessing module, a data loading module, a language model and a main module, in which:

[0024] The preprocessing module mainly preprocesses text data, including vocabulary segmentation and vocabulary numbering process;

[0025] In the preprocessing module, the main work is to separate the text in the text data and carry out preliminary numbering to each text; specifically, use conditional expressions to filter out meaningless punctuation marks, space characters, and carriage returns. , and number the segmented vocabulary in sequence. In this process, two dictionaries will be generated, namely (word_to_id) vocabulary to number dictionary and (id_to...