Method for integrating trained transformer into other neural networks, and apparatus therefor

Abstract

The present invention proposes a method for integrating a trained transformer model into other neural networks and operating same without an error, so as to enable harmonious operation with other neural networks by inheriting and expanding training that was already performed without having to retrain a transformer model from the beginning for processing an unknown token that has not been learned. The present invention relates to a method performed in a computing device for processing the unknown token, which has not been learned by a neural network, by means of the trained neural network including n stacked encoder or decoder blocks (n is an integer of 1 or more), the method comprising the steps of: inputting an input sequence including the unknown token to an i-th token position; and performing a self-attention operation, wherein the step of performing the self-attention operation may comprise the steps of: generating a query matrix, a key matrix, and a value matrix; generating a query-key matrix on the basis of the query matrix and the key matrix; and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value with respect to the query-key matrix.

Description

Method for integrating a learned transformer into another neural network and device therefor

[0001] The present invention relates to a neural network architecture, and more specifically, to a Transformer, which is an attention-based sequence transforming neural network model.

[0002] Transformer, an attention-based sequence transforming neural network model proposed by Google in a 2017 paper titled "Attention Is All You Need," dramatically improved the problems of the previous RNN (Recurrent Neural Network) model, and since then, neural network technology using the attention mechanism has been widely used in various applications such as translation, text generation, chatbots, image generation, and music composition.

[0003] Recently, various language models have been announced, such as OpenAI's ChatGPT, Google's Bard, Meta's LLaMA, Stanford University's Alpaca, and LMSYS.org's Vicuna, and they are showing potential as artificial general intelligence that can be utilized in various fields as well as text generation.

[0004] Figure 1 illustrates an example of a prior art transformer neural network model. Transformer models come in various types, including those with both an encoder and decoder, as presented in a 2017 Google paper; those with only an encoder, like BERT; and those with only a decoder, like GPT and LLaMA. Figure 1 illustrates a structure with only an encoder or decoder.

[0005] The transformer model (100) illustrated in FIG. 1 may include various layers such as an input transducer (105), n stacked decoder (or encoder) blocks (110, 155, 160), a transposed embedding (165), and a softmax (170).

[0006] Here, each decoder / encoder block (110, 155, 160) may include layers such as a first Layer Norm (115), a Masked Multi-Headed Self-Attention (130), a first Add (135), a second Layer Norm (140), a Pointwise Feed Forward Network (145), and a second Add (150). Depending on the implementation, the functions of the Add and Norm layers may be combined to represent an Add & Norm layer, and the detailed configuration may be changed, such as omitting or adding some layers or changing their positions, as needed. For example, depending on the implementation, dropout layers (not shown) may be added to prevent over-fitting at necessary locations.

[0007] In Fig. 1, the input sequence Input X initmay be a sequence of input tokens, and the input transducer (105) generates an input token ID sequence by representing each token constituting the input sequence as a token ID, performs token embedding to convert it into an embedded vector corresponding to each token ID, and performs positional embedding to indicate the position of each token within the input sequence. Therefore, the input sequence Input X init Through this process, it is converted into an embedded vector sequence, and the embedded vector sequence is input to the first encoder / decoder block (110) as Input X in the form of a matrix.

[0008] Within each encoder / decoder block (110, 155, 160), self-attention operations are performed through a Masked Multi-Headed Self-Attention (130) layer, and a transformation process is performed through a Pointwise Feed Forward Network. In this process, the first Layer Norm (115) and the second Layer Norm (140) layers can perform normalization on the output of the previous layer at appropriate locations, and the first Add (135) and the second Add (150) layers can be employed at appropriate locations for residual connection.

[0009] The output generated by the final encoder / decoder block (160) through the stacked (the number of stacks n is appropriately determined according to the model) encoder / decoder blocks (110, 155, 160) is converted into a logit through an operation with the transposed matrix of the matrix used in the embedding performed by the input transducer (105) in the transposed embedding (165) layer, and based on this, probability is calculated in the softmax (170) layer to obtain a probability vector (175) for a token, and an output token (during inference) or loss (during training) is obtained.

[0010] Typically, natural language processing technologies use large-scale corpora, typically consisting of tens of gigabytes and hundreds of millions of tokens, for learning. A vocabulary is a set of tokens obtained by tokenizing all sentences and documents in the corpus and removing duplicates. In large language models such as transformer models, a dictionary for the encoder / decoder is constructed and referenced in the token embedding process performed in the input transducer (105) and the logit generation process performed in the transposed embedding (165) layer.

[0011] In the case of a transformer neural network model trained based on a specific corpus, if an unknown token that is not defined in the dictionary (vocabulary) of the encoder / decoder is input, it may cause an error in operation, or during the attention operation process, the unknown token may affect other tokens, negatively affecting the quality or representation of the context vector, which may be a cause of generating incorrect responses such as hallucination.

[0012] In these conventional neural network models, it is difficult to increase the embedding dimension to increase the power of representation for the input after learning has started, and in order to process unknown tokens normally, a dictionary containing the unknown tokens had to be rebuilt, input tokens had to be newly defined, and learning had to be performed again from the beginning.

[0013] In addition, a method has not yet been proposed to integrate neural networks trained individually, such as a neural network trained using a Korean-based corpus and a neural network trained using an English-based corpus, to succeed the existing learning while operating without errors.

[0014] The present invention proposes a method for integrating a learned transformer model into another neural network to operate without errors, thereby enabling harmonious operation with another neural network by inheriting and expanding the learning already accomplished without having to retrain the transformer model from scratch to process unknown tokens that have not been learned.

[0015] In addition, the present invention is to prevent an error from occurring in operation even when a transformer model receives an input including an unknown token that has not been learned, and to prevent the unknown token from affecting other tokens during the attention operation process, thereby negatively affecting the quality or representational power of a context vector.

[0016] In addition, the present invention aims to present a system structure and learning method that can minimize the generation of unintended incorrect responses to unlearned inputs by integrating and operating individually learned neural network systems and allowing the neural network system most suitable for processing input content among the neural network systems to contribute more to the generation of results.

[0017] The technical problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention pertains from the contents described in this specification.

[0018] In a first aspect of the present invention, a method is provided for processing an unknown token that has not been learned by a trained neural network including n stacked encoder or decoder blocks (wherein n is an integer greater than or equal to 1), performed on a computing device, the method comprising: inputting an input sequence including the unknown token at an i-th token position; and performing a self-attention operation, wherein the performing the self-attention operation may include: generating a query matrix, a key matrix, and a value matrix; generating a query-key matrix based on the query matrix and the key matrix; and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value in the query-key matrix.

[0019] Here, the step of replacing the matrix elements with predetermined values ​​may be replacing one or more matrix elements of a column or row corresponding to the i-th token position in the query-key matrix with the predetermined values.

[0020] Here, the input sequence input step may represent the unknown token as a predetermined special token ID, embed it into an embedded vector corresponding to the special token ID, and store the position i of the unknown token.

[0021] Additionally, the step of replacing the above matrix elements with predetermined values ​​may be to replace them with negative values ​​so that the result of applying the softmax function can be ignored.

[0022] Additionally, the step of replacing the above matrix elements with predetermined values ​​may utilize the position i of the stored unknown token.

[0023] Additionally, one or more of the n (where n is an integer greater than or equal to 1) stacked encoder or decoder blocks may be initialized by replacing, for each output sequence, the output vector corresponding to the position i of the unknown token with an embedded vector of a predetermined special token.

[0024] In a second aspect of the present invention, a method is provided for integrating a trained first neural network including n stacked encoder or decoder blocks (n is an integer greater than or equal to 1) into another neural network to inherit and extend the training, performed on a computing device, the method comprising: inputting a first input sequence generated by a first input transducer and a second input sequence generated by a second input transducer into the first neural network and the second neural network, respectively, based on an input sequence including an unknown token that the first neural network has not trained at an i-th token position; generating a first output sequence by the first neural network based on the first input sequence, and generating a second output sequence by the second neural network based on the second input sequence; generating a third input sequence of a third neural network based on the first output sequence and the second output sequence, and inputting the third input sequence into the third neural network; And it may include a step of backpropagation learning at least the second neural network and the third neural network based on the output sequence generated by the third neural network.

[0025] Here, the generation of the third input sequence may include a step of replacing the output vector corresponding to the i-th token position in the first output sequence with the output vector corresponding to the i-th token position in the second output sequence.

[0026] In addition, the replaced first output sequence and the second output sequence may have an embedding dimension of 1d, and the generation of the third input sequence may further include a step of concatenating and linearly transforming the replaced first output sequence and the second output sequence to have an embedding dimension of 1d.

[0027] Additionally, the backpropagation learning may target a token or token ID indicated by an index obtained by adding 1 to the index of the token or token ID within the input sequence (a value indicating the position or order of each token within the input sequence).

[0028] In addition, the backpropagation learning may be backpropagation learning of the second neural network, the third neural network, the second input transformer, and the first layer based on the output sequence generated by the third neural network.

[0029] In addition, the first neural network may perform a self-attention operation, and the self-attention operation may include generating a query matrix, a key matrix, and a value matrix, generating a query-key matrix based on the query matrix and the key matrix, and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value in the query-key matrix.

[0030] Additionally, the step of replacing the matrix elements with predetermined values ​​may be to replace one or more matrix elements of a column or row corresponding to the i-th token position in the query-key matrix with the predetermined values.

[0031] Additionally, the first input converter may represent the unknown token as a predetermined special token ID, embed it into an embedded vector corresponding to the special token ID, and store the position i of the unknown token.

[0032] Additionally, replacing the above matrix elements with predetermined values ​​may be done by replacing them with negative values ​​so that the result of applying the softmax function can be ignored.

[0033] Additionally, replacing the matrix elements with predetermined values ​​may be done by using the position i of the unknown token stored in the first input converter.

[0034] Additionally, one or more of the n (n is an integer greater than or equal to 1) stacked encoder or decoder blocks of the first neural network may be initialized by replacing, for each output sequence, the output vector corresponding to the position i of the unknown token with an embedded vector of a predetermined special token.

[0035] In a third aspect of the present invention, a device is provided, comprising: a processor; and a memory, wherein the memory comprises instructions configured to cause the device, when executed by the processor, to implement a specific operation for processing an unknown token that has not been learned by a trained neural network, the specific operation comprising: inputting an input sequence including the unknown token at an i-th token position; and performing a self-attention operation, wherein performing the self-attention operation may include generating a query matrix, a key matrix, and a value matrix, generating a query-key matrix based on the query matrix and the key matrix, and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value in the query-key matrix.

[0036] In a fourth aspect of the present invention, a device is provided, comprising: a processor; and a memory, wherein the memory comprises instructions configured to cause the device to implement a specific operation for inheriting and extending the learning of a learned first neural network including n (where n is an integer greater than or equal to 1) stacked encoder or decoder blocks by integrating the learned first neural network into another neural network, the specific operation comprising: inputting a first input sequence generated by a first input transducer and a second input sequence generated by a second input transducer into the first neural network and the second neural network, respectively, based on an input sequence including an unknown token that the first neural network has not learned at an i-th token position; the first neural network deriving a first output sequence based on the first input sequence, and the second neural network deriving a second output sequence based on the second input sequence; The first layer may include generating a third input sequence of a third neural network based on the first output sequence and the second output sequence and inputting the third input sequence to the third neural network; and performing backpropagation training on at least the second neural network and the third neural network based on the output sequence generated by the third neural network.

[0037] In a fifth aspect of the present invention, a computer-readable non-transitory storage medium is provided, storing instructions configured to cause a device including the processor to implement a specific operation for processing an unknown token that has not been learned by a trained neural network including n stacked encoder or decoder blocks (wherein n is an integer greater than or equal to 1), the specific operation comprising: inputting an input sequence including the unknown token at an i-th token position; and performing a self-attention operation, wherein performing the self-attention operation may include generating a query matrix, a key matrix, and a value matrix, generating a query-key matrix based on the query matrix and the key matrix, and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value in the query-key matrix.

[0038] In a sixth aspect of the present invention, a computer-readable non-transitory storage medium storing instructions configured to cause a device including the processor to implement a specific operation for processing an unknown token not learned by a neural network through a trained neural network including n stacked encoder or decoder blocks (wherein n is an integer greater than or equal to 1), the specific operation comprising: inputting a first input sequence generated by a first input transducer and a second input sequence generated by a second input transducer into the first neural network and the second neural network, respectively, based on an input sequence including an unknown token not learned by the first neural network at an i-th token position; deriving a first output sequence by the first neural network based on the first input sequence, and deriving a second output sequence by the second neural network based on the second input sequence; The first layer may include generating a third input sequence of a third neural network based on the first output sequence and the second output sequence and inputting the third input sequence to the third neural network; and performing backpropagation training on at least the second neural network and the third neural network based on the output sequence generated by the third neural network.

[0039] According to the present invention, a learned transformer model can be integrated into another neural network to operate without error, and the learning already accomplished can be inherited and expanded without having to retrain the transformer model from scratch to process unknown tokens that have not been learned, thereby enabling harmonious operation with another neural network.

[0040] In addition, according to the present invention, even when a transformer model receives an input including an unknown token that has not been learned, it is possible to prevent an error from occurring in the operation and to prevent the unknown token from affecting other tokens during the attention operation process, thereby negatively affecting the quality or representational power of the context vector.

[0041] In addition, according to the present invention, it is possible to integrate and operate individually learned neural network systems, and to achieve a system structure and learning method that minimize the generation of unintended incorrect responses to unlearned inputs by allowing the neural network system most suitable for processing input content among the neural network systems to contribute more to the generation of results.

[0042] The effects that can be obtained from the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the contents described in this specification.

[0043] The accompanying drawings are incorporated into and constitute a part of the detailed description to aid in the understanding of the present invention and together with the detailed description, illustrate embodiments and technical features of the present invention.

[0044] Figure 1 is a schematic diagram for exemplarily explaining the configuration and operation of a transformer neural network model of the prior art.

[0045] FIG. 2 is a conceptual diagram for explaining the configuration of an input transducer according to one embodiment of the present invention.

[0046] FIG. 3 is a flowchart illustrating a process of an input transducer according to one embodiment of the present invention.

[0047] FIG. 4 is a conceptual diagram illustrating the configuration of a masked multi-head self-attention layer according to one embodiment of the present invention.

[0048] FIG. 5 is a flowchart illustrating a process of a masked multi-head self-attention layer according to one embodiment of the present invention.

[0049] FIG. 6 is a flowchart illustrating a process of an encoder / decoder block according to one embodiment of the present invention.

[0050] FIG. 7A and FIG. 7B are diagrams for explaining subsequent token masking and unknown token masking processing according to one embodiment of the present invention.

[0051] FIG. 8 is a diagram illustrating a configuration for integrating a learned transformer according to one embodiment of the present invention into another neural network.

[0052] FIG. 9 is a diagram illustrating a process for integrating a learned transformer into another neural network according to one embodiment of the present invention.

[0053] Figure 10 illustrates an apparatus capable of performing the proposed method of the present invention.

[0054] The present invention can be modified in various ways and has various embodiments. Hereinafter, specific embodiments will be described in detail based on the attached drawings.

[0055] The following examples are provided to provide a comprehensive understanding of the methods, devices, systems, and / or storage media described herein. However, these are merely examples and the scope of the present invention is not limited thereto.

[0056] In describing embodiments of the present invention, if a detailed description of a known technology related to the present invention is judged to unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the terms described below are terms defined in consideration of their functions in the present invention, and this may vary depending on the intention or custom of the user or operator. Therefore, the definitions should be made based on the contents throughout this specification. The terms used in the detailed description are only for the purpose of describing embodiments of the present invention and should never be construed as limiting. Unless clearly used otherwise, the singular form includes the plural form. In this description, expressions such as "comprises" or "having" are intended to indicate certain features, numbers, steps, operations, elements, parts or combinations thereof, and should not be construed to exclude the presence or possibility of one or more other features, numbers, steps, operations, elements, parts or combinations thereof other than those described.

[0057] Additionally, although terms such as first, second, etc. may be used to describe various components, the components are not limited by the terms, and the terms are used only for the purpose of distinguishing one component from another.

[0058] Unknown Token Replacement

[0059] If a token (unknown token) that is not included in the training data of a pre-trained encoder / decoder is input, the encoder / decoder will experience an error. Types of errors include, for example, if an unknown token that is not defined in the dictionary (vocabulary) of the encoder / decoder is input, an error may occur during operation, or during the attention operation, the unknown token (or its corresponding vector) may affect other tokens (or its corresponding vector), negatively affecting the quality or representational power of the context vector, which may lead to incorrect responses such as hallucination.

[0060] Fig. 2 is an example of an improved input transducer (200) configuration to solve these problems, and Fig. 3 is a flowchart for explaining the process of the improved input transducer (200) according to one embodiment of the present invention. In Fig. 2, the input sequence Input X init may be a sequence of input tokens (S10), and the improved input transducer (200) illustrated generates an input token ID sequence by representing each token constituting the input sequence as a token ID, wherein the Replacing (210) module generates an input token ID sequence Input X initIf an unknown token that is not in the dictionary (vocabulary) of the encoder / decoder is included, a special token ID that is predetermined is assigned to the unknown token (S20). Usually, each token is assigned a unique integer ID value, which is called a token ID, and a separate integer that is not assigned to any token is used as the ID value of the special token. In addition, the input transducer (200) separately stores the position of the unknown token (e.g., the position of the i-th token) in the input sequence (S30) and transmits it for the unknown token masking process described later.

[0061] The embedding (220) module performs token embedding (S40) to convert each token ID into its corresponding embedded vector, wherein a special token ID can be converted into the embedded vector of its corresponding special token. In addition, the embedding (220) module, similar to the prior art, performs positional embedding to indicate the position of each token in the input sequence (S40). Input sequence Input X init Through this process, it is converted into an embedded vector sequence, and the embedded vector sequence is input to the first encoder / decoder block as input X (embedded) in the form of a matrix.

[0062] FIG. 4 is a conceptual diagram for explaining the configuration of a masked multi-headed self-attention (300) layer according to one embodiment of the present invention, and FIG. 5 is a flowchart for explaining the process of a masked multi-headed self-attention (300) layer according to one embodiment of the present invention.

[0063] The improved masked multi-headed self-attention (300) layer of the embodiment performs self-attention operations similar to the prior art, and may include h attention heads (350, 360, 370) each operating in parallel. Each attention head (350, 360, 370) obtains a query matrix Q, a key matrix K, and a value matrix V based on the input matrix X (normalized) obtained by performing normalization processing, etc., as needed, based on the input matrix X (embedded) transmitted from the input transducer (200), and then multiplies the weight matrices W through respective matrix multiplication processes (305, 310, 315). Q , W K , W V Through a process such as performing matrix multiplication operations on each of the attention heads (350, 360, 370), a query matrix, a key matrix, and a value matrix for processing are generated (S110).

[0064] Each attention head (350, 360, 370) performs a matrix multiplication process (320) on the query matrix and key matrix obtained as above to generate a query-key matrix (S120), and performs a masking for following tokens process (325) (S130).

[0065] Figure 7a is a diagram for explaining the Masking for Followings (325) process. Here, Figure 7a shows that the input sequence is L tokens (X1, X2, X3, …, X L-2 , X L-1 , X L) is exemplified. In general, the Masking for Followings (325) process is to prevent the embedded vectors of tokens that appear thereafter in the input sequence from affecting the self-attention value for the embedded vector of the i-th token when calculating the self-attention value for the embedded vector of the i-th token for causality, etc., and since it is a known technology in itself, a detailed description is omitted. In Fig. 7a, the matrix elements subjected to the masking for following tokens correspond to the area (420) indicated in white in the upper right. For the masking for following tokens, the values ​​of the matrix elements in the area (420) indicated in white in the upper right are replaced with -∞ (i.e., a relatively very large negative value, and since it is impossible to process an infinite value on a computer, a relatively large negative value such that the result of applying the softmax function can be ignored).

[0066] In an embodiment of the present invention, in addition to the known subsequent token masking (325) process described above, an unknown token masking (330) process may be performed. FIG. 7b is a diagram illustrating the unknown token masking (330) process according to one embodiment of the present invention. The unknown token masking (330) process of the present embodiment masks the matrix element value corresponding to the position of the unknown token among the column elements of the query-key matrix using the position information (e.g., i) of the unknown token transmitted from the input transducer (200) described above.

[0067] The area (530) indicated by a diagonal line in Fig. 7b is an example of an unknown token masking (Masking for Unknown) applied, and this example is an example of L tokens (X1, X2, X3, …, X L-2 , X L-1, X L ) within the input sequence containing X L-1 This represents the case where the unknown token is present. At this time, as shown, among the column elements of the query-key matrix, the unknown token X L-1 The values ​​of the elements indicated by the diagonal lines in Fig. 7b, which are components corresponding to the positions of , can be replaced with -∞ (i.e., a relatively large negative value, and since it is impossible to process infinite values ​​on a computer, a relatively large negative value so that the result of applying the softmax function can be ignored) (i.e., masked to be the same as the values ​​in the area 420 in Fig. 7a or 520 in Fig. 7b) so that other tokens (or vectors corresponding to them) are not affected in the subsequent processing. The area (520) indicated in white in Fig. 7b represents matrix elements that have been subjected to known subsequent token masking, as in the case of Fig. 7a.

[0068] Depending on the implementation, each token of the input sequence may be represented as a row vector rather than a column vector. In this case, the query-key matrix of FIG. 7b has a structure in which the rows and columns are swapped, and the unknown token masking (330) process of this embodiment masks the matrix element value corresponding to the position of the unknown token among the row components of the query-key matrix using the position information (e.g., i) of the unknown token transmitted from the input transducer (200) described above. In addition, the known Masking for Followings (325) process is also applied differently to match the swapped rows and columns.

[0069] Thereafter, each attention head (350, 360, 370) of the embodiment performs a known SoftMax (335) process on the masked query-key matrix (S150), and performs a matrix multiplication operation (340) on the output and value matrix to generate an output sequence of each attention head (S160).

[0070] Fig. 6 is a flowchart for explaining the process of an encoder / decoder block (not shown) according to an embodiment of the present invention. Each of h attention heads (350, 360, 370) of the masked multi-headed self-attention (300) layer of the embodiment shown in Fig. 4 generates each output through a similar process, and the generated h outputs (Head1, Head2, ..., Head h ) are concatenated with each other through a concatenation (380) process (S210), and the final output of the Masked Multi-Headed Self-Attention (300) layer is obtained through linear transformation (S220).

[0071] Afterwards, the output of the decoder / encoder block is obtained (S270) through known layers such as the first Add (S230), the second Layer Norm (S240), the Pointwise Feed Forward Network (S250), and the second Add (S260) (see Fig. 1), and can be passed on to another stacked (the number of stacks n is appropriately determined according to the model) encoder / decoder block.

[0072] Processing cases where an unknown token is located at the beginning of the input sequence, etc.

[0073] Meanwhile, when applying unknown token masking (Masking for Unknown) as illustrated in Fig. 7b, for example, when an unknown token exists in the first token position in the input sequence, problems related to the softmax function processing process of the subsequent softmax (335) layer may occur due to unknown token masking (S330). When X1 located at the first position in the input sequence is an unknown token, the matrix element values ​​of the first column, which are elements of the position corresponding to the unknown token X1 among the column elements of the query-key matrix, are replaced with -∞ (i.e., a relatively very large negative value, and since it is actually impossible for a computer to process infinite values, a relatively large negative value such that the result of applying the softmax function can be ignored).

[0074] However, the softmax function applied in the softmax function processing process is generally expressed as follows.

[0075]

[0076] The above equation represents the value of the ith output element when there are a total of n output elements. Each element of the softmax function's output is a real number between 0 and 1, and since the sum of all elements is 1, the value can be interpreted as the probability for each input element. This characteristic of the softmax function is widely used in neural networks such as transformers.

[0077] However, when an unknown token exists in the first token position in the input sequence, all matrix elements of the first column of the query-key matrix corresponding to that position are filled with the value -∞. Consequently, when considered together with the Masking for Followings, all elements of the first row of the query-key matrix are filled with the value -∞, so that the softmax function loses its original meaning of representing the probability of each input element, and thus may cause errors in the subsequent processing. Furthermore, even if the first and second tokens are both unknown tokens, the same problem may occur in the rows corresponding to tokens X1 and X2.

[0078] In order to solve this problem and not affect the performance of the existing trained transformer even when applying Masking for Unknown, the vector corresponding to the position of the unknown token in the output sequence obtained as the output of the encoder / decoder block can be replaced (initialized) with the embedded vector for the special token used by the first input transducer (200). This initialization process can be performed for the output of the encoder / decoder block of every layer in the case of stacked encoder / decoder blocks.

[0079] According to this initialization, the context vector corresponding to the unknown token output from the final encoder / decoder block becomes the same as the embedding vector for the unknown token generated by the initial input transformer. According to the present embodiment, the context vector corresponding to the already learned tokens other than the unknown token output from the final encoder / decoder block becomes a vector in which the influence of the unknown token is excluded and the influence of the remaining tokens is reflected. Therefore, when fine-tuning an already learned neural network model, after obtaining the final context vector for each token vector of the input token sequence using the unknown token processing technology of the present embodiment, the existing pre-trained neural network can be utilized as is through a technique of replacing only the vector of the unknown token with another desired vector. In addition, according to the embodiment described above, even when an unknown token is input, an existing learned transformer can be operated without error even without increasing the embedding dimension.

[0080] Integrating the learned transformer into another neural network

[0081] FIG. 8 is a diagram illustrating a configuration for integrating a learned transformer (620) according to one embodiment of the present invention into another neural network (630, 650). FIG. 9 is a diagram illustrating a process for integrating a learned transformer (620) according to one embodiment into another neural network (630, 650).

[0082] In Fig. 8, G0 (620) represents a neural network that has already been trained, and G1 (630) represents a separate neural network added for learning tokens that are not used in training in G0 (620) (i.e., not included in the dictionary of G0), that is, unknown tokens from the perspective of G0 (620), and additional training materials that include the unknown tokens. In addition, G2 (650) represents a separate neural network for integration and cumulative learning of G0 (620) and G1 (630). It is preferable that G0 (620) be a transformer neural network model having a typical stacked encoder / decoder block as described above in relation to the Unknown Token Replacement technique, but G1 (630) and G2 (650) may be any neural networks that generate an output sequence from an input sequence, and there is no particular limitation on their types. The configuration of this embodiment is intended to implement cumulative learning, which inherits the learning content of the existing G0 (620) and allows subsequent additional learning to be further enhanced based on the existing learning. To this end, an unknown token masking technique can be applied, which allows the aforementioned trained neural network to operate error-free even when inputting unknown tokens.

[0083] First input transducer and second input transducer

[0084] As shown in Fig. 8, input transducers T0 (605) and T1 (606) are placed for G0 (620) and G1 (630), respectively. Here, input transducers T0 (605) and T1 (606) are input sequences (Input X init ) are input together, and output matrices Output X0 and Output X1 are generated to be input to G0 (620) and G1 (630), respectively. The input sequence (Xinit ) may be a sequence of input tokens. Here, T0 (605) is basically the same as the configuration of the input transducer already described above with reference to FIGS. 2 and 3, and G0 (620) is also basically the same as the configuration of the stacked encoder / decoder block already described with reference to FIGS. 4, 5, and 6.

[0085] T0 (605) and T1 (610) each have their own neural networks for embedding. Learning is not performed on the neural network parameters of T0 (605) and G0 (610), which have already been learned. Instead, the neural network parameters of T1 (610) are initialized with the neural network parameter values ​​of T0 (605) before learning, enabling more efficient learning.

[0086] T0(605) is the input sequence for further learning (Input X init ) includes an unknown token that was not used when training G0 (620) (i.e., not in the dictionary of G0), a special token ID is assigned to the unknown token according to the unknown token replacement process described above, and each embedded vector corresponding to the special token ID and other token IDs is converted to generate an input sequence Input X0 in the form of a matrix. T1 (610) builds its own dictionary (vocabulary) based on additional learning materials (corpus, documents, etc.) and generates an input sequence (Input X) for additional learning. init ) is converted into an embedded vector corresponding to each token ID, thereby generating an input sequence Input X1 in the form of a matrix. T0 (605) and T1 (610) can perform embedding with embedded vectors of the same size with an embedding dimension of 1d. As described above, the parameters of the embedding matrix of T0 (605), which has already been learned, do not change during the additional learning process.

[0087] Structure of the first and second neural networks

[0088] In the case of G0 (620), the stacked encoder / decoder block structure of the transformer that has already been learned is mostly used as is and integrated into other neural networks (630, 650), and the neural network parameter values ​​are also applied as inherited. That is, the neural network parameters of G0 (620) are not relearned during the additional learning process. However, as described above with reference to FIGS. 4, 5, and 7b, an unknown token masking (330) process for processing cases of unknown token input is added to the Masked Multi-Headed Self-Attention (300) layer in the encoder / decoder blocks of G0 (620), and a process (not shown) for initializing the final output of each encoder / decoder block is added.

[0089] Additionally, as shown in Fig. 8, Output X0, which is the output of the final encoder / decoder block of G0 (620), can be normalized once more, if necessary, and then merged with Output X1, which is the output of G1 (630), in the L1 (640) layer.

[0090] Meanwhile, G1(630) is built on the basis of additional learning materials, G O (620) has a vocabulary containing new tokens that have not been previously learned, and can be formed into an arbitrary structure such as a stacked encoder / decoder. Output X1, which is the output of G1 (630), can have the same sequence length as Output X0, which is the output of G0 (620), and Output X1 is also normalized once more and then merged with Output X0, which is the output of G0 (620), in the L1 (640) layer.

[0091] Generating input for the third neural network (Replacing Vectors and Weighted Summation)

[0092] The L1(640) layer, among the output vectors included in Output X0, which is the output of G0(620), corresponds to the unknown tokens (i.e., unknown tokens that G0 has not learned), and among the vectors included in Output X1, which is the output of G1(630), corresponds to the input sequence Input X. init A vector replacement process can be performed (S310) to replace vectors in positions corresponding to the above unknown tokens in order.

[0093] In some embodiments, the L1 (640) layer may further perform a weighted summation process. For example, if the vectors included in the result of applying the above-described vector replacement process to Output X0, which is the output of G0 (620), and the vectors included in Output X1, the output of G1, have a 1d-dimensional embedding, the vectors included in the result of applying the vector replacement process to Output X0 and the vectors included in Output X1 corresponding to each of them are concatenated in the embedding dimension direction to generate vectors having a 2d-dimensional embedding (S320), and then the vectors are linearly transformed back into a 1d-dimensional vector by multiplying them by a 2d x 1d weight matrix to be used as the input of G2 (650). If necessary, layer normalization may further be performed on the 1d-dimensional vector obtained as described above. The 1d-dimensional input sequence Input X2 obtained through this process is input to G2 (S330). In the additional learning process, each element of the 2d x 1d weight matrix described above is trained to reduce the loss. Through this process, Output X0, which is the output of G0 (620), and Output X1, which is the output of G1 (630), can be integrated into a single matrix having the same size as Input X0 and Input X1 through the L1 (640) layer before being input to G2 (650).

[0094] Generating the output of the third neural network

[0095] G2 (650) may be a general stacked encoder / decoder model or another neural network model. G2 (650) processes the input sequence Input X2 based on, for example, stacked encoder / decoder blocks as shown in Fig. 1, and then, for example, in a transposed embedding layer, converts it into a corresponding logit through an operation with the transposed matrix of the embedding matrix used in embedding, and then performs probability calculation in a softmax layer based on the transposed matrix, thereby obtaining the final output, Output X2, through a known processing process. At this time, the transposed embedding layer for generating the final output of G2 (650) may use the transposed matrix of the embedding matrix used by the input transducer T1 (610) of G1 (610) in embedding.

[0096] Loss propagation and backpropagation learning

[0097] The loss is calculated by comparing Output X2, the final output of G2 (650), with the target, and the neural network parameters such as G1 (630), G2 (650), L1 (640), and T1 (610) are learned through back propagation (S340). For example, similar to the learning process of a general transformer, the input sequence Input X initA sequence in which each token (or token ID) is shifted to the left by 1 token position offset can be used as a target. In this case, the index of each token (or token ID) is reduced by 1 compared to before the shift. Here, the index refers to a value indicating the position or order of each token ID within the input sequence (for example, if the input sequence consists of 4 tokens, index = 1, ..., 4). However, for efficient use of memory, the input sequence Input X that is already stored in the memory init In order to enable the target to be used in its current state without performing operations such as shift, the input sequence Input X init It is also possible to use a token (or token ID) as a target, which is indicated by the index of the value that adds 1 (i.e., +1) to the index of a specific token (or token ID). For example, for the input sequence Input X init For token ID X1 that appears first (i.e. index = 1) in the input sequence Input X, the target is init The token ID X2 appears second (i.e. index = 1 + 1 = 2).

[0098] The sequence obtained from Output X2, which is the final output of G2 (650), is compared with the target and used to calculate a loss value. The calculated loss value is transferred to G2 (650), G1 (630), L1 (640), and T1 (610), and backpropagation learning is performed (S340).

[0099] Through the above-described embodiment, it was confirmed that the vocabulary of the already learned G0 (620) can be substantially expanded, and that G0 (620) can simultaneously inherit the concepts it has already learned, acquire concepts learned from newly learned data, and harmonize them with existing concepts. Therefore, according to this embodiment, it is possible to achieve inheritance, expansion, and harmonization of existing learning by integrating an already learned transformer with another neural network. In addition, it is particularly noteworthy that it was confirmed that learning is performed without causing overfitting even when the amount of newly learned data is small compared to the model size. This embodiment of the present invention allows for the integration and operation of individually trained systems, such as a neural network trained with a Korean corpus and a neural network trained with an English corpus, and allows the system most suitable for processing the input content among the systems to play a greater role in generating results, thereby minimizing the generation of unintended incorrect responses to untrained inputs (i.e., hallucination).

[0100] Device to which the proposed method of the present invention can be applied

[0101] Fig. 10 illustrates a device (120) capable of performing the proposed method of the present invention.

[0102] Referring to FIG. 10, the device (120) may be configured to implement a process for processing unknown tokens not learned by the neural network through the proposed method of the present invention and the learned neural network according to the method, or a process for inheriting and expanding the learning by integrating the learned first neural network into another neural network. For example, the device (120) may be a computing device, a server device, a terminal device, a network device, etc. for performing the process of the present invention.

[0103] For example, the device (120) to which the proposed method of the present invention can be applied may include network devices such as repeaters, hubs, bridges, switches, routers, gateways, etc., computer devices such as desktop computers, workstations, etc., mobile terminals such as smartphones, portable devices such as laptop computers, etc., home appliances such as digital TVs, etc., and transportation means such as automobiles, etc. As another example, the device (120) to which the present invention can be applied may be included as a part of an ASIC (Application Specific Integrated Circuit) implemented in the form of a SoC (System On Chip).

[0104] The memory (20) can be connected to the processor (10) when it operates, and can store programs and / or commands for processing and controlling the processor (10), and can store data and information used in the present invention, control information required for data and information processing according to the present invention, temporary data generated during data and information processing, etc. The memory (20) can be implemented as a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, an SRAM (Static RAM), an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc.

[0105] The processor (10) can be operatively connected to the memory (20) and the network interface (30), and controls the operation of each module within the device (120). In particular, the processor (120) can perform various control functions for performing the proposed method of the present invention. The processor (120) may also be called a controller, a microcontroller, a microprocessor, a microcomputer, etc. The proposed method of the present invention can be implemented by hardware, firmware, software, or a combination thereof. When the present invention is implemented using hardware, an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. configured to perform the present invention can be provided in the processor (10). Meanwhile, when implementing the proposed method of the present invention using firmware or software, the firmware or software may include instructions related to modules, procedures, or functions that perform functions or operations necessary to implement the proposed method of the present invention, and the instructions may be stored in the memory (20) or stored in a computer-readable recording medium (not shown) separate from the memory (20) and, when executed by the processor (10), the device (120) may be configured to implement the proposed method of the present invention.

[0106] In addition, the device (120) may include a network interface device (30). The network interface device (30) is connected to the processor (10) when in operation, and the processor (10) may control the network interface device (30) to transmit or receive wireless / wired signals carrying information and / or data, signals, messages, etc. through a wireless / wired network. The network interface device (30) supports various communication standards, such as, for example, IEEE 802 series, 3GPP LTE(-A), 3GPP 5G, etc., and may transmit and receive control information and / or data signals according to the communication standards. The network interface device (30) may also be implemented outside the device (120) as needed.

[0107] The embodiments described above are combinations of components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented without being combined with other components or features. Furthermore, it is also possible to form an embodiment of the present invention by combining some components and / or features. The order of operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment. It is self-evident that claims that do not have an explicit citation relationship in the patent claims may be combined to form an embodiment or incorporated as a new claim through a post-application amendment.

[0108] The present invention can be applied to various devices such as computing devices, server devices, terminal devices, network devices, etc. for performing learning and inference of attention-based neural networks.

Claims

1. A method performed on a computing device for processing unknown tokens that have not been learned by a learned neural network, wherein the learned neural network comprises n stacked encoder or decoder blocks (where n is an integer greater than or equal to 1), a step of inputting an input sequence including the unknown token at the i-th token position; and comprising a step of performing a self-attention operation, The above self-attention operation execution step is: A method comprising the steps of generating a query matrix, a key matrix, and a value matrix, generating a query-key matrix based on the query matrix and the key matrix, and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value in the query-key matrix.

2. In claim 1, The step of replacing the above matrix elements with predetermined values ​​is: A method for replacing one or more matrix elements of a column or row corresponding to the i-th token position in the above query-key matrix with a predetermined value.

3. In claim 1, The above input sequence input step is, A method of representing the above-mentioned unknown token with a predetermined special token ID, embedding it with an embedded vector corresponding to the special token ID, and storing the position i of the above-mentioned unknown token.

4. In claim 1, A method in which the step of replacing the above matrix elements with predetermined values ​​is to replace them with negative values ​​so that the result of applying the softmax function can be ignored.

5. In claim 3, A method in which the step of replacing the above matrix elements with predetermined values ​​uses the position i of the stored unknown token.

6. In claim 1, A method wherein at least one of the n stacked encoder or decoder blocks (where n is an integer greater than or equal to 1) is initialized by replacing, for each output sequence, the output vector corresponding to the position i of the unknown token with an embedded vector of a predetermined special token.

7. A method performed on a computing device for inheriting and extending the learning by integrating a learned first neural network including n stacked encoder or decoder blocks (where n is an integer greater than or equal to 1) into another neural network, A step of inputting a first input sequence generated by a first input transducer and a second input sequence generated by a second input transducer into the first neural network and the second neural network, respectively, based on an input sequence including an unknown token that the first neural network has not learned at the i-th token position; A step in which the first neural network derives a first output sequence based on the first input sequence, and the second neural network derives a second output sequence based on the second input sequence; A step of the first layer generating a third input sequence of the third neural network based on the first output sequence and the second output sequence and inputting the third input sequence to the third neural network; and A method comprising a step of backpropagation learning at least the second neural network and the third neural network based on the output sequence generated by the third neural network.

8. In claim 7, The above third input sequence generation is, A method comprising the step of replacing an output vector corresponding to the i-th token position in the first output sequence with an output vector corresponding to the i-th token position in the second output sequence.

9. In claim 8, The above-described first output sequence and the above-described second output sequence have an embedding dimension of 1d, A method wherein the generation of the third input sequence comprises a step of concatenating and linearly transforming the replaced first output sequence and the second output sequence to have an embedding dimension of 1d.

10. In claim 7, The above backpropagation learning is, A method for targeting a token or token ID indicated by an index that adds 1 to the index of the token or token ID within the above input sequence.

11. In claim 7, The above backpropagation learning is, A method for backpropagation learning the second neural network, the third neural network, the second input transformer, and the first layer based on the output sequence generated by the third neural network.

12. In claim 7, The above first neural network performs self-attention operations, A method wherein the self-attention operation comprises generating a query matrix, a key matrix, and a value matrix, generating a query-key matrix based on the query matrix and the key matrix, and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value in the query-key matrix.

13. In claim 12, The step of replacing the above matrix elements with predetermined values ​​is: A method for replacing one or more matrix elements of a column or row corresponding to the i-th token position in the above query-key matrix with a predetermined value.

14. In claim 12, The above first input converter, A method of representing the above-mentioned unknown token with a predetermined special token ID, embedding it with an embedded vector corresponding to the special token ID, and storing the position i of the above-mentioned unknown token.

15. In claim 12, A method of replacing the above matrix elements with predetermined values, wherein the result of applying the softmax function is replaced with a negative value so that the result can be ignored.

16. In claim 14, A method of replacing the above matrix elements with predetermined values, using the position i of the unknown token stored in the first input transformer.

17. In claim 7, A method wherein at least one of the n (where n is an integer greater than or equal to 1) stacked encoder or decoder blocks of the first neural network is initialized by replacing, for each output sequence, the output vector corresponding to the position i of the unknown token with an embedded vector of a predetermined special token.

18. A device including a processor and a memory, The memory comprises instructions configured to cause the device, when executed by the processor, to perform a specific operation for processing an unknown token that has not been learned by a trained neural network, the specific operation comprising n stacked encoder or decoder blocks, where n is an integer greater than or equal to 1. Inputting an input sequence containing the unknown token at the i-th token position; and Includes performing self-attention operations, Performing the above self-attention operation is: A device comprising: generating a query matrix, a key matrix, and a value matrix; generating a query-key matrix based on the query matrix and the key matrix; and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value in the query-key matrix.

19. A device including a processor and a memory, The memory comprises instructions configured to cause the device, when executed by the processor, to implement a specific operation for inheriting and extending the learning of a first neural network comprising n (where n is an integer greater than or equal to 1) stacked encoder or decoder blocks by integrating the learned first neural network into another neural network, the specific operation comprising: Inputting a first input sequence generated by a first input transducer and a second input sequence generated by a second input transducer into the first neural network and the second neural network, respectively, based on an input sequence including an unknown token that the first neural network has not learned at the i-th token position; The first neural network derives a first output sequence based on the first input sequence, and the second neural network derives a second output sequence based on the second input sequence; The first layer generates a third input sequence of the third neural network based on the first output sequence and the second output sequence and inputs the third input sequence to the third neural network; and A device comprising backpropagation learning of at least the second neural network and the third neural network based on an output sequence generated by the third neural network.

20. A computer-readable non-transitory storage medium storing instructions configured to cause a device including the processor to implement a specific operation for processing an unknown token that has not been learned by a trained neural network including n stacked encoder or decoder blocks (where n is an integer greater than or equal to 1), when executed by a processor, the specific operation comprising: Inputting an input sequence containing the unknown token at the i-th token position; and Includes performing self-attention operations, Performing the above self-attention operation is: A computer-readable non-transitory storage medium comprising: generating a query matrix, a key matrix, and a value matrix, generating a query-key matrix based on the query matrix and the key matrix, and replacing one or more matrix elements corresponding to the i-th token position with a predetermined value in the query-key matrix.

21. A computer-readable non-transitory storage medium storing instructions configured to cause a device including the processor to implement a specific operation for processing an unknown token that has not been learned by a trained neural network including n stacked encoder or decoder blocks (where n is an integer greater than or equal to 1), when executed by a processor, the specific operation comprising: Inputting a first input sequence generated by a first input transducer and a second input sequence generated by a second input transducer into the first neural network and the second neural network, respectively, based on an input sequence including an unknown token that the first neural network has not learned at the i-th token position; The first neural network derives a first output sequence based on the first input sequence, and the second neural network derives a second output sequence based on the second input sequence; The first layer generates a third input sequence of the third neural network based on the first output sequence and the second output sequence and inputs the third input sequence to the third neural network; and A computer-readable non-transitory storage medium, comprising backpropagation learning of at least the second neural network and the third neural network based on the output sequence generated by the third neural network.

Images