Machine reading understanding method based on pre-training model

A technology of reading comprehension and pre-training, applied in the direction of reasoning methods, neural learning methods, biological neural network models, etc., can solve problems such as ignoring the improvement of machine reading ability of pre-training models, achieve consistency and adequacy improvement, and improve the machine The effect of reducing reading ability and performance loss

Pending Publication Date: 2021-03-23
ZHEJIANG UNIV CITY COLLEGE
View PDF2 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the above methods all build a system or a method through deep learning, which pay more attention to practicality and ignore the improvement of the machine reading ability of the pre-training model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Machine reading understanding method based on pre-training model
  • Machine reading understanding method based on pre-training model
  • Machine reading understanding method based on pre-training model

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] The present invention will be further described below in conjunction with the examples. The description of the following examples is provided only to aid the understanding of the present invention. It should be pointed out that for those skilled in the art, some modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0054] The present invention improves and optimizes the pre-training model appearing in natural language processing in the era of deep learning, and improves the reasoning speed and reading ability of the pre-training model. The present invention is based on the pre-training model (BERT-base model proposed by Google), and supplements it for the deficiency of the pre-training model; in order to make the model have both fast reasoning speed and good machine reading ability, the pres...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a machine reading understanding method based on a pre-training model. The machine reading understanding method comprises the following steps: 1, preprocessing data, 2, performing advanced semantic fusion through an advanced semantic fusion network layer according to the output of the pre-training model, 3, further performing capability learning on the machine reading modelafter semantic fusion, and 4, calculating the mean square error loss of the named entity, and training the machine reading model. The method has the advantages that high-level semantic information isextracted from the text, higher-dimension information is provided for the model, and meanwhile, the high accuracy of the method has more reference significance compared with the mode that the model tries to extract the information in the training process, according to the method, through capability learning, under the condition that the scale of the model can be kept unchanged, the machine reading capability is improved, so that the model can quickly complete an inference task on the premise of relatively high performance.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to the content related to machine reading in the field of natural language processing, specifically a machine reading comprehension method based on a pre-trained model. Background technique [0002] It has been observed that many people spend a lot of time on the Internet every day, obtaining a large amount of information from all over the world through the screen, including text information, picture information and video information. Among these information, text information accounts for a huge proportion; however, through a large number of browsing, it is found that although text information accounts for a huge proportion, the quality of the text is uneven; many information are produced through topical word eyes in order to attract traffic and gain attention. False titles occupy the reader's reading time, but once read, they will find that for the r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/279G06F40/242G06F40/30G06N3/04G06N3/08G06N5/04
CPCG06F40/279G06F40/30G06F40/242G06N3/084G06N5/04G06N3/045G06N3/044
Inventor 陈观林姚茹韬杨武剑翁文勇李甜
Owner ZHEJIANG UNIV CITY COLLEGE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap