Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and device for training model, equipment, medium and program product

A training model and model technology, applied in the field of deep learning, can solve problems such as complex models, and achieve the effect of improving generalization ability and robustness

Active Publication Date: 2022-02-25
BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, as the model becomes more and more complex, the problems of model training overfitting and poor robustness have gradually become prominent

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for training model, equipment, medium and program product
  • Method and device for training model, equipment, medium and program product
  • Method and device for training model, equipment, medium and program product

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0025] As mentioned above, the problems of model training overfitting and poor robustness began to emerge. Traditional perturbation-based model training methods mainly include the following three types: 1) word replacement, randomly replacing some words in the parallel corpus with any word in the vocabulary; 2) word discarding, randomly using all-zero word vectors inste...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a method and device for training a model, equipment, a medium and a program product, and relates to the field of deep learning. According to the specific implementation scheme, the method comprises the steps of generating first disturbance data, wherein the first disturbance data is used for disturbing a first word feature representation set associated with a training text of a model; generating first mask data and first complementary mask data, wherein the first mask data are used for masking a first part of data in the first disturbance data, and the first complementary mask data are used for masking data except the first part of data in the first disturbance data; generating first masking disturbance data based on the first mask data and the first disturbance data; generating second masking disturbance data based on the first complementary mask data and the first disturbance data; and generating a second word feature representation set for training the model based on the first masking perturbation data, the second masking perturbation data and the first word feature representation set. Therefore, the generalization ability and robustness of the model can be improved.

Description

technical field [0001] The present disclosure relates to the field of deep learning, and in particular to a method, device, electronic device, storage medium and computer program product for training a model for natural language processing. Background technique [0002] Models for natural language processing, such as text classification models, translation models, etc., are commonly used services on the Internet. In recent years, especially after the use of neural networks, models for natural language processing have made significant progress in the above tasks. However, as the model becomes more and more complex, the problems of model training overfitting and poor robustness have gradually become prominent. Contents of the invention [0003] The present disclosure provides a method, apparatus, electronic device, storage medium and computer program product for training a model for natural language processing. [0004] According to a first aspect of the present disclosure...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/62G06F40/284G06F16/35G06F40/40G06N3/08
CPCG06F40/284G06F16/35G06F40/40G06N3/08G06F18/214
Inventor 高鹏至何中军吴华王海峰
Owner BEIJING BAIDU NETCOM SCI & TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com