Natural language problem generation method in civil construction information field based on Transformer

Abstract

The invention discloses a Transformer-based natural language problem generation method in the civil construction information field, which comprises the following steps of: analyzing information distribution learned by each layer in a Bert model, training different corpora of different modules of a Transformer in combination with finite characteristics of training data, and proposing a low-layer network for training syntax grammatical characteristics, the high-level network is used for training a mechanism for obtaining semantic characteristics. Then, a UniLM thought is adopted to finely adjusta downstream task of the Bert, so that the natural language text generation capability of the model in the field of civil construction information is improved; the model provided by the invention hasrelatively high feasibility and effectiveness in problem generation in the field of civil construction information, and reaches a relatively high natural language problem generation level.

Description

technical field [0001] The invention belongs to the technical field of computer artificial neural network natural language processing, and in particular relates to a method for generating natural language questions in the field of civil engineering and construction information based on a Transformer encoding and decoding structure. Background technique [0002] With the continuous development of artificial intelligence and big data technology, intelligent question answering systems are rapidly changing people's lifestyles, and greatly improving work efficiency and user experience. Question Generation QG (Question Generation) is the basic task of a question answering system. Its goal is to automatically generate natural language questions given a sentence or paragraph. This task has been widely used in many fields, such as education, medical treatment, Internet of Things, etc. In the field of education, learning and teaching tasks are assisted by reading comprehension-style ...

AI Technical Summary

Problems solved by technology

[0003] The purpose of the present invention is to provide a Transformer-based method for generating natural language questions in the field of civil engineering and construction information, which reduces the workload of manual labeling and solves the problem that the generation of natural language questions in the field of civil engineering and construction information in the prior art is difficult to automate

Images