Unlock instant, AI-driven research and patent intelligence for your innovation.

Model improvement method and device based on pre-trained semantic model

A semantic model and pre-training technology, applied in the computer field, can solve the problems of the semantic model compression ratio and processing speed need to be improved, and achieve the effect of high compression ratio, improved processing speed, and fewer model parameters.

Active Publication Date: 2020-09-25
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF9 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The compression ratio and processing speed of the compressed semantic model based on model distillation technology or quantization and clipping technology need to be improved

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Model improvement method and device based on pre-trained semantic model
  • Model improvement method and device based on pre-trained semantic model
  • Model improvement method and device based on pre-trained semantic model

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0017] Exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

[0018] figure 1 An exemplary architecture 100 to which the pretrained semantic model-based model improvement method and apparatus of the present application may be applied is shown.

[0019] like figure 1 As shown, the system architecture 100 may include terminal devices 101 , 102 , 103 , a network 104 and a server 105 . The network 104 is a medium used to pr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a model improvement method and device based on a pre-trained semantic model, and relates to the technical field of natural language processing and deep learning. According to the specific implementation scheme, an initial improved model is obtained on the basis of a pre-trained semantic model, and in the initial improved model, semantic result information of an input vectoris determined on the basis of a hash search method; and based on a model distillation method, the initial improved model is trained to obtain an improved model. According to the scheme, the semanticresult information of the input vector is obtained on the basis of hash search on the input vector, the complex iterative computation process of an original semantic model is replaced, the improved model with few model parameters and high compression ratio is obtained, and the processing speed of the improved model is increased.

Description

technical field [0001] The embodiments of the present disclosure relate to the field of computer technology, in particular to natural language processing and deep learning technologies, and are a model improvement method and device based on a pre-trained semantic model. Background technique [0002] The use of pre-trained semantic models is a trend in the field of natural language processing. However, the current pre-trained semantic models are generally too large in parameter size and computationally complex, making it difficult to deploy them in production environments. At present, model distillation technology, quantitative clipping technology, etc. are generally used to compress the model to improve the processing speed of the model. The compression ratio and processing speed of the compressed semantic model obtained based on the model distillation technology or the quantitative clipping technology need to be improved. SUMMARY OF THE INVENTION [0003] The present app...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/30G06F16/31G06N20/00
CPCG06F40/30G06F16/325G06N20/00G06N3/08G06N5/02G06F16/36G06N3/045
Inventor 陈徐屹黄世维
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD