Natural language understanding method based on pre-training model

A natural language understanding and pre-training technology, applied in the field of natural language understanding based on pre-training models, can solve the problems that pre-training models cannot capture semantic information, and it is difficult to build multi-layer network structures, so as to improve quality and user experience. Effect

Active Publication Date: 2020-01-10
识因智能科技有限公司
View PDF4 Cites 30 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, it is difficult to build a multi-layer network structure under this architecture, making the pre-training model unable to capture deep semantic information

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Natural language understanding method based on pre-training model
  • Natural language understanding method based on pre-training model
  • Natural language understanding method based on pre-training model

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0017] The present invention will be described in further detail below in conjunction with the accompanying drawings.

[0018] In the embodiment of the present invention, a method for understanding natural language based on a pre-trained model, the flow chart is as follows figure 1 As shown, the method includes the following steps:

[0019] S101. Establish a pre-training model based on a two-way depth Transformer. The input of the pre-training model is a text vector obtained after word segmentation processing of a sentence and adding special tags at the beginning and end of the sentence, and the output is the text semantic vector of the sentence. ;

[0020] S102. Perform word segmentation processing on the sentence to be understood, and respectively add the special tags at the beginning and end of the sentence to be understood, to obtain a text vector of the sentence to be understood;

[0021] S103. Using the text vector of the sentence to be understood as input, call the pr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a natural language understanding method based on a pre-training model. The method comprises the following steps: establishing a pre-training model based on a bidirectional depth Transformer; carrying out word segmentation processing on the to-be-understood statement, and respectively adding special tags at the beginning and the end of the to-be-understood statement to obtain a text vector of the to-be-understood statement; taking the text vector of the to-be-understood statement as input, and calling the pre-training model to obtain a text semantic vector of the to-be-understood statement; performing intention recognition; and performing entity identification. According to the method, intentions can be accurately and comprehensively understood, entities can be recognized, and a solid foundation is provided for subsequent conversations. The quality and the user experience of the man-machine conversation system can be remarkably improved.

Description

technical field [0001] The invention belongs to the technical field of natural language understanding, and in particular relates to a natural language understanding method based on a pre-trained model. Background technique [0002] In recent years, natural language, as the most convenient and natural way for human beings to express their thoughts, has gradually become the most mainstream way in human-computer interaction. Due to the diversity and complexity of natural language, realizing its accurate machine understanding has always been a research hotspot and difficulty in the field of artificial intelligence. [0003] The first step in human-computer dialogue is natural language understanding. Only by accurately and comprehensively understanding the user's language can a reasonable answer be given. Natural language understanding specifically includes two tasks: intent recognition and entity recognition. Intent recognition and entity recognition can be realized by buildin...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/295G06F40/30G06F16/332G06N3/04G06N3/08
CPCG06F16/3329G06N3/08G06N3/045
Inventor 王春辉胡勇
Owner 识因智能科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap