Machine translation model training method and related device

A technology of machine translation and training methods, applied in neural learning methods, biological neural network models, instruments, etc., can solve problems such as large amount of calculation, inefficient time, slow training speed, etc., and achieve the goal of improving training speed and shortening update speed. Effect

Active Publication Date: 2021-06-18
PING AN TECH (SHENZHEN) CO LTD
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, in the process of training the model, when the length of the input and output sequences is very long, the amount of calculation will be very large, and the training speed is not fast, which will cause time inefficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Machine translation model training method and related device
  • Machine translation model training method and related device
  • Machine translation model training method and related device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] Embodiments of the present application are described below with reference to the drawings in the embodiments of the present application.

[0028] The terms "first", "second", "third" and "fourth" in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order . Furthermore, the terms "include" and "have", as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes unlisted steps or units, or optionally further includes For other steps or units inherent in these processes, methods, products or apparatuses.

[0029] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least on...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a machine translation model training method and a related device, and the method comprises the steps of calculating the similarity between a to-be-coded word and each word in a preset first sequence through a self-attention layer, wherein the to-be-coded word is a word input at the ith moment in a preset second sequence, the second sequence is a preset word sequence needing to be input at k moments, the first sequence is a word sequence input before the ith moment in words of the second sequence, both i and k are positive integers, and i is smaller than k; calculating the self-attention of the word to be coded according to the similarity; inputting the self-attention into the feedforward neural network to obtain an output result; calculating a loss value between the output result and self-attention; adjusting network parameters of the machine translation model according to the loss value. According to the embodiment of the invention, the training speed of the model can be improved.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, in particular to a training method and related devices for a machine translation model. Background technique [0002] In the tasks of natural speech generation, most of them are implemented based on the Seq2Seq model, such as generative dialogue, machine translation, text summarization and so on. Seq2Seq is a network of encoder Encoder-decoder Decoder structure, its input is a sequence, and the output is also a sequence. In the Encoder, a variable-length signal sequence is converted into a fixed-length vector expression, and the Decoder converts this fixed-length vector into a variable-length target signal sequence. Among them, the Encoder and Decoder can be composed of a Transformer structure. The attention mechanism in the Transformer structure enables the Seq2Seq model to focus on all important input information for the next target word, which greatly improves the ef...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08G06F40/284G06K9/62
CPCG06N3/04G06N3/08G06F40/284G06F18/22
Inventor 魏文琦王健宗张之勇程宁
Owner PING AN TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap