Non-Recurrent Neural Network Skip Model for Parallel Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural machine translation methods using recurrent neural networks (RNNs) are limited by sequential processing, which slows down translation and increases processing time due to the need for maintaining and sharing hidden states, preventing parallel computation.
Innovation Solution
The implementation of a non-recurrent neural network (NRNN) with a skip model that generates token sequences in parallel, allowing for bulk decoding and reducing the number of decoding steps by using a dictionary or neural network-based sub-decoder to predict subsequent tokens based on a prefix token, thereby accelerating the translation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If recurrent neural networks are used for machine translation, then translation capability is achieved, but processing time increases due to sequential processing requirements
Solution Approach 1:
The patent segments the translation process into two distinct phases: an encoding phase that processes the entire source sentence in parallel to generate hidden states, and a decoding phase that sequentially generates target tokens. This segmentation allows the computationally intensive encoding to be parallelized while maintaining the sequential nature only where necessary for generation, thereby reducing overall processing time while preserving translation capability.
Solution Approach 2:
The patent performs preliminary encoding of the entire source sentence before decoding begins. The encoder processes all source tokens in parallel to generate comprehensive hidden states that capture the full context, which are then reused during decoding. This preliminary parallel processing eliminates the need for repeated sequential analysis of the same source information during each decoding step, significantly reducing processing time.
2Measurement precision
If recurrent neural networks maintain hidden states for sequential processing, then accurate translation is achieved, but computational efficiency decreases
Solution Approach 1:
The patent extracts the computationally burdensome sequential processing component from the overall translation system by dedicating it solely to the decoding phase. The encoding phase uses parallel processing to extract and store comprehensive contextual information in hidden states, which are then efficiently reused during sequential decoding. This extraction separates the functions to optimize each for its appropriate processing mode.
Solution Approach 2:
The patent changes the processing parameter from strictly sequential to a hybrid approach where the encoding phase operates in parallel mode and the decoding phase operates in sequential mode. This parameter change allows the system to leverage parallel computation for context extraction while maintaining sequential generation for accurate token prediction, thereby improving computational efficiency without sacrificing translation accuracy.
3Measurement precision
If sequential decoding is used in neural machine translation, then token generation accuracy is maintained, but processing speed decreases
Solution Approach 1:
The patent segments the translation pipeline into parallel encoding and sequential decoding stages, allowing speed optimization in the encoding phase while preserving accuracy in the decoding phase. The encoder processes all source tokens simultaneously to generate comprehensive hidden states, then the decoder uses these pre-computed states for accurate sequential token generation, achieving both speed and accuracy goals.
Solution Approach 2:
The patent performs preliminary parallel encoding of the entire source sentence before decoding begins. By pre-computing all necessary contextual information in parallel during the encoding phase, the system eliminates redundant processing during decoding, thereby increasing processing speed while the sequential decoding phase maintains token generation accuracy through careful use of the pre-computed hidden states.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A language processing method and apparatus is disclosed. A language processing apparatus using a neural network may obtain context information from a source text using a neural network-based encoder, generate a prefix token from the context information using a neural network-based main decoder, generate a token sequence including at least two successive tokens sequentially following the prefix token using a skip model in response to the prefix token satisfying a preset condition, and indicate a target text in which the prefix token and the token sequence are combined as an inference result with respect to the source text.