Learning device, inference device, learning method, and program

Abstract

A learning device according to one aspect of the present disclosure comprises: a trained model including one or more intermediate layers, a final layer disposed at the subsequent stage of the one or more intermediate layers, a first output layer disposed at the subsequent stage of the final layer, and a second output layer added to at least one of the one or more intermediate layers; an inference unit that outputs an inference result from the first output layer and the second output layer on the basis of data to be input to the trained model; and an update unit that updates the parameter of at least predetermined one of the second output layers and the parameter of the intermediate layer to which the predetermined second output layer is added, on the basis of the inference result output from the first output layer and the second output layer and information corresponding to the data.

Description

Learning device, inference device, learning method, and program

[0001] This disclosure relates to a learning device, an inference device, a learning method, and a program.

[0002] A technique called Early Exit is known for reducing the time required for inference processing in machine learning models (hereinafter also referred to as "inference time"). Early Exit uses a model (hereinafter also referred to as the "Early Exit model") in which an output layer is added to at least one intermediate layer of an existing machine learning model, thereby obtaining inference results with the desired accuracy without processing up to the final layer. Hereinafter, the output layer added to the intermediate layer will also be referred to as the "intermediate output layer".

[0003] As a method for training an Early Exit model, the method described in Non-Patent Document 1 is known. In the method described in Non-Patent Document 1, the accuracy of the inference results output from the intermediate output layer for intermediate layers closer to the final layer can be improved by increasing the weight of the error function for intermediate output layers closer to the final layer.

[0004] Z. Liu, Z. Xu, H.-J. Wang, T. Darrell, and E. Shelhamer, "Anytime dense prediction with confidence adaptivity," in Proc. International Conference on Learning Representations (ICLR), 2022.

[0005] However, the method described in Non-Patent Document 1 does not necessarily improve the accuracy of inference results output from intermediate output layers for intermediate layers that are not close to the final layer, and therefore may not always reduce inference time.

[0006] This disclosure is made in view of the above points and aims to reduce the inference time of machine learning models.

[0007] A learning device according to one aspect of the present disclosure includes a trained model which includes one or more intermediate layers, a final layer located after the one or more intermediate layers, a first output layer located after the final layer, and a second output layer added to at least one of the intermediate layers which is one or more intermediate layers; an inference unit which outputs inference results from the first output layer and the second output layer based on data which is input to the trained model; and an update unit which updates the parameters of at least one predetermined second output layer among the second output layers and the parameters of the intermediate layer to which the predetermined second output layer is added, based on the inference results output from the first output layer and the second output layer and information corresponding to the data.

[0008] This can reduce the inference time of machine learning models.

[0009] This figure shows an example of an Early Exit model. This figure shows an example of creating an Early Exit model from an encoder-type Backbone model. This figure shows an example of creating an Early Exit model from a decoder-type Backbone model. This figure shows an example of creating an Early Exit model from an encoder / decoder-type Backbone model. This figure shows an example of retraining related to the proposed method. This figure shows an example of the hardware configuration of the speech recognition device according to this embodiment. This figure shows an example of the functional configuration of the speech recognition device in the learning phase. This flowchart shows an example of the learning process according to this embodiment. This figure shows an example of the functional configuration of the speech recognition device in the inference phase. This flowchart shows an example of the speech recognition process according to this embodiment.

[0010] One embodiment of the present invention will be described in detail below with reference to the drawings.

[0011] <Early Exit> One well-known task that utilizes machine learning models is speech recognition. Speech recognition is a task that takes speech as input and outputs a transcribed text. When speech recognition is used in practice, real-time performance is often required, so it is important to reduce the inference time (i.e., the time required for inference processing of the machine learning model), which accounts for a large portion of the processing time.

[0012] One known technique for reducing the inference time of machine learning models is called Early Exit (Reference 1). With Early Exit, by using an Early Exit model, it is possible to obtain inference results with the desired accuracy without processing up to the final layer. An Early Exit model is a machine learning model that adds an output layer called an intermediate output layer to at least one intermediate layer of an existing machine learning model (hereinafter also referred to as a "Backbone model").

[0013] An example of an Early Exit model is shown in Figure 1. The Early Exit model 100 shown in Figure 1 consists of hidden layers 110-1 to 110-N, a final layer 120, an output layer 130, and intermediate output layers 140-1 to 140-N. In the Early Exit model 100 shown in Figure 1, the machine learning model consisting of hidden layers 110-1 to 110-N, the final layer 120, and the output layer 130 is the Backbone model. Also, for n=1, ..., N, the output layer added to the hidden layer 110-n is the intermediate output layer 140-n.

[0014] Hereafter, we will refer to intermediate layers near the final layer of the Backbone model as "deep layers," and intermediate layers far from the final layer as "shallow layers." We will also refer to the output layer of the Backbone model as the "final output layer." The Backbone model has a hierarchical structure, and the layer following a certain layer is called the "subsequent layer," and the layer preceding a certain layer is called the "preceding layer," and so on.

[0015] Early Exit allows the system to output inference results at intermediate output layers if the desired accuracy is achieved, even without processing to the final layer. For example, if the inference difficulty is high, the system can perform inference processing to deeper layers, while if the inference difficulty is low, it can terminate the inference processing early at shallower layers, thereby reducing inference time while maintaining the accuracy of the inference results.

[0016] Here, Early Exit can be divided into two phases: a "training phase" in which a trained Early Exit model is created, and an "inference phase" in which inference processing is performed using the trained Early Exit model.

[0017] ≪Early Exit Learning Phase≫ In the Early Exit learning phase, an Early Exit model is created by adding intermediate output layers to one or more intermediate layers of the Backbone model. The entire Early Exit model is then trained to improve the accuracy of the inference results output from the final output layer and each intermediate output layer. As a method for training the Early Exit model, for example, the method described in Non-Patent Document 1 is known. In the method described in Non-Patent Document 1, the accuracy of the inference results output from intermediate output layers added to deeper layers can be improved by increasing the weight of the error function for intermediate output layers added to deeper layers.

[0018] ≪Early Exit Inference Phase≫ In the Early Exit inference phase, if the confidence level of the inference result (e.g., the maximum value of the probability distribution) from a certain intermediate output layer of the Early Exit model is high, the inference process is terminated at that point. As a result, the inference result output from that intermediate output layer or the information generated according to that inference result is obtained as the final inference result.

[0019] <Examples of Creating Early Exit Models> Below, we will explain how to create Early Exit models from encoder-type Backbone models, decoder-type Backbone models, and encoder / decoder-type Backbone models, as examples. However, these are all just examples, and Backbone models are not limited to encoder-type, decoder-type, or encoder / decoder-type.

[0020] <<Encoder-type Backbone model>> Figure 2 shows an example of creating an Early Exit model 210 from an encoder-type Backbone model 200.

[0021] The encoder-type Backbone model 200 consists of an embedded layer 201, encoders 202-1 to 202-4, and an output layer 203. Encoders 202-1 to 202-3 are intermediate layers, and encoder 202-4 is the final layer. The embedded layer 201 is the layer that converts the input into a representation that can be handled by encoder 202-1, which is located in the subsequent stage (e.g., vector format).

[0022] The Early Exit model 210 consists of an embedded layer 201, encoders 202-1 to 202-4, an output layer 203, and intermediate output layers 204-1 to 204-3. Here, for n = 1, 2, and 3, the output layer added to encoder 202-n is the intermediate output layer 204-n. Note that the output layer 203 and intermediate output layers 204-1 to 204-3 are implemented, for example, by a linear transformation layer with the Softmax function as the activation function.

[0023] ≪Decoder-type Backbone Model≫ Figure 3 shows an example of creating an Early Exit model 310 from a decoder-type Backbone model 300.

[0024] The decoder-type Backbone model 300 consists of an embedded layer 301, decoders 302-1 to 302-4, and an output layer 303. Decoders 302-1 to 302-3 are intermediate layers, and decoder 302-4 is the final layer. The embedded layer 301 is the layer that converts the input into a representation that can be handled by decoder 302-1, which is located in the subsequent stage (e.g., vector format).

[0025] The Early Exit model 310 consists of an embedded layer 301, decoders 302-1 to 302-4, an output layer 303, and intermediate output layers 304-1 to 304-3. Here, for n=1, 2, and 3, the output layer added to decoder 302-n is the intermediate output layer 304-n. The output layer 303 and intermediate output layers 304-1 to 304-3 are implemented, for example, by a linear transformation layer with the Softmax function as the activation function.

[0026] <<Encoder / Decoder Type Backbone Model>> Figure 4 shows an example of creating an Early Exit model 410 from an encoder / decoder type Backbone model 400.

[0027] The encoder-decoder type Backbone model 400 consists of an embedded layer 401, encoders 402-1 to 402-4, decoders 403-1 to 403-4, and an output layer 404. Encoders 402-1 to 402-4 and decoders 403-1 to 403-3 are intermediate layers, and decoder 403-4 is the final layer. The embedded layer 401 is a layer that converts the input into a representation that can be handled by encoder 402-1, which is located in the subsequent stage (e.g., vector format).

[0028] The Early Exit model 410 consists of an embedded layer 401, encoders 402-1 to 402-4, decoders 403-1 to 403-4, an output layer 404, and intermediate output layers 405-1 to 405-3. Here, for n=1, 2, and 3, the output layer added to decoder 403-n is the intermediate output layer 405-n. Note that the output layer 404 and intermediate output layers 405-1 to 405-3 are implemented, for example, by a linear transformation layer with the Softmax function as the activation function.

[0029] <Problems with conventional methods> In Early Exit, in order to achieve faster inference while maintaining the desired accuracy, it is necessary to terminate the inference process early. In other words, it is necessary to terminate the inference process at a shallow layer of the Early Exit model.

[0030] However, the method described in Non-Patent Literature 1 prioritizes the accuracy of inference results output from the deep intermediate output layers in order to improve the overall accuracy of the Early Exit model. As a result, the accuracy of inference results output from the shallow intermediate output layers is not necessarily improved, and consequently, inference processing may not terminate at the shallow layers, preventing a reduction in inference time.

[0031] Furthermore, if we were to perform training that prioritizes the accuracy of inference results output from shallow intermediate output layers in order to solve the above problems, a new problem would arise: the accuracy of inference results output from deep intermediate output layers would decrease due to the influence of the inference results output from shallow intermediate output layers.

[0032] <Proposed Method> Below, we propose a method (hereinafter also referred to as the "proposed method") that improves the accuracy of inference results output from the shallow intermediate output layer while maintaining the accuracy of the inference results output from the final output layer and the deep intermediate output layer of the Early Exit model.

[0033] The proposed method retrains a pre-trained Early Exit model to improve the accuracy of inference results output from shallow intermediate output layers while maintaining the accuracy of inference results output from deep intermediate output layers. This improves the accuracy of inference results output from shallow intermediate output layers, causing the inference process to terminate at shallower layers, thus reducing inference time.

[0034] The following explanation will primarily focus on speech recognition, describing a case where speech is input and text (strings) representing the speech recognition result is output. However, speech recognition is just one example; the proposed method can be similarly applied to any text generation task. Examples of text generation tasks other than speech recognition include machine translation and question answering, which may involve images or text.

[0035] The proposed method retrains an Early Exit model that has been trained using an existing arbitrary method (e.g., the method described in Non-Patent Document 1). The data used to retrain the Early Exit model consists of the audio X that serves as the input to the Early Exit model and the correct transcription data of that audio X, which is the correct string Y = {y １ , , y Ｔ Training data consisting of} is given. Here, T is the length of the correct string (i.e., the number of tokens that make up the correct string). Note that a token is a string that serves as the processing unit when generating text by the Early Exit model, and for example, words or subwords are used. Below, each y ｔ Let's call this a token.

[0036] Furthermore, in the following, the total number of intermediate output layers in the Early Exit model is assumed to be L-1, and the intermediate output layers are numbered 1, ..., L-1 in order from the shallowest to the largest intermediate layer. The L-th layer is assumed to be the final output layer.

[0037] At this time, in the proposed method, the parameters of the intermediate output layers from the j = 1st to the j = Jth and the parameters of the respective intermediate layers corresponding to those intermediate output layers are updated so that the error between the character string estimated by the Early Exit model (hereinafter also referred to as the "estimated character string") when the voice X is input and the correct character string Y becomes small. Here, J is a predetermined integer satisfying 1 ≤ J ≤ L - 1. Hereinafter, the parameters to be updated will be referred to as "learning target parameters".

[0038] For example, when using cross-entropy as the error between the estimated character string and the correct character string, the learning target parameters are updated so as to minimize the error function E shown in the following formula (1).

[0039]

[0040] However, w in the above formula (1) is as follows. ｌ is as follows.

[0041]

[0042] Also, D is a set of learning data (hereinafter also referred to as the "learning data set"), and Θ is a set of learnable parameters (hereinafter also referred to as the "learnable parameters") of the Early Exit model. Further, P ｌ (y ｔ |y １ , ···, y ｔ－１ , X; Θ) is the probability distribution that the l-th output layer (that is, the final output layer when l = L, and the l-th intermediate output layer when l < L) outputs the token y １ , ···, y ｔ－１ when the voice X and the token sequence y ｔ are given. Note that P ｌ is an example of a value or score representing that the token y １ , ···, y ｔ－１ is output by the l-th output layer when the voice X and the token sequence y ｔ are given, and may be referred to as, for example, "score", "reliability", "likelihood", etc.

[0043] In this case, the parameters to be learned are the parameters of the intermediate output layers from j=1 to j=J, which are included in the set of learnable parameters Θ, and the parameters of the intermediate layers corresponding to each of those intermediate output layers. Hereafter, parameters other than the parameters to be learned will also be called "fixed parameters".

[0044] When updating the parameters to be learned, existing optimization methods such as gradient descent can be used. Furthermore, when calculating the gradient of the error function with respect to the learnable parameters, existing methods such as backpropagation can be used.

[0045] As an example, when the Early Exit model 410 shown in Figure 4 is retrained using the proposed method, Θ = {θ} ｅｍｂ , θ ｅｎｃ １ , ..., θ ｅｎｃ ４ , θ ｄｅｃ １ ,・・・θ ｄｅｃ ４ , θ ｌｉｎｅａｒ １ ,・・・θ ｌｉｎｅａｒ ４}. Here, θ ｅｍｂ θ is a learnable parameter of the embedding layer 401. ｅｎｃ １ , ..., θ ｅｎｃ ４ These are the learnable parameters of encoders 402-1 to 402-4, respectively, and θ ｄｅｃ １ ,・・・θ ｄｅｃ ４ These are the learnable parameters of decoders 403-1 to 403-4, respectively, and θ ｌｉｎｅａｒ １ ,・・・θ ｌｉｎｅａｒ ３ These are the learnable parameters of intermediate output layers 405-1 to 405-3, respectively, and θ ｌｉｎｅａｒ ４ These are the learnable parameters of the output layer 404.

[0046] In this case, the proposed method, for a predetermined J (1 ≤ J ≤ 3), learns the target parameter {θ} in such a way that it minimizes the error function E shown in equation (1) above. ｄｅｃ １ ,・・・θ ｄｅｃ Ｊ , θ ｌｉｎｅａｒ １ ,・・・θ ｌｉｎｅａｒ Ｊ Update {θ}. This is shown in Figure 5. Figure 5 shows the case where J = 1, and {θ} ｄｅｃ １ , θ ｌｉｎｅａｒ １ The parameters indicated by} are the parameters to be learned, while the other learnable parameters are fixed parameters. In Figure 5, the backpropagation of the error function when calculating the gradient with respect to the learnable parameters is represented by a dashed line.

[0047] The proposed method makes it possible to retrain the intermediate layers, including shallow layers, and their corresponding intermediate output layers. Therefore, according to the proposed method, it is possible to improve the accuracy of the inference results output from the shallow intermediate output layers while maintaining the accuracy of the inference results output from the final output layer and the deep intermediate output layers. As a result, the inference process will terminate at the shallow layers, thus reducing the inference time.

[0048] ≪Innovative Aspects of the Proposed Method≫ The innovative aspect of the proposed method is that, in order to maintain the accuracy of the inference results output from the final output layer and the intermediate output layers of the deep layers, it also uses the gradients related to the learnable parameters of the final layer, the final output layer, the intermediate layers in the deep layers, and the intermediate output layers corresponding to those intermediate layers.

[0049] For example, a simple method to improve the accuracy of inference results output from shallow intermediate output layers is to update the trained parameters using only the gradients with respect to the trainable parameters of those intermediate layers and intermediate output layers. However, using this method may reduce the accuracy of inference results output from the final output layer and deep intermediate output layers.

[0050] Therefore, in the proposed method described above, the parameters to be learned are updated not only with respect to the layer containing the parameters to be learned, but also with respect to the learnable parameters of other layers, in order to minimize the error of all output layers (final output layer and intermediate output layer). This makes it possible to learn in a way that simultaneously maximizes the accuracy of the inference results output from the intermediate output layer of the shallow layers and the accuracy of the inference results output from the final output layer and the intermediate output layer of the deep layers. Thus, it is possible to simultaneously improve the accuracy of the inference results output from the intermediate output layer of the shallow layers and maintain the accuracy of the inference results output from the final output layer and the intermediate output layer of the deep layers. Furthermore, since the accuracy of the inference results output from the final output layer and the intermediate output layer of the deep layers is also maximized, there is a possibility that their accuracy will improve even further.

[0051] The following describes a speech recognition device 10 that performs retraining of an Early Exit model and inference using the retrained Early Exit model, using speech recognition as an example of a text generation task. In the following, the phase in which the Early Exit model is retrained using the proposed method will be called the "learning phase," and the phase in which inference is performed using the retrained Early Exit model will be called the "inference phase." The speech recognition device 10 in the learning phase may also be called, for example, the "learning device" or "training device." On the other hand, the speech recognition device 10 in the inference phase may also be called, for example, the "inference device" or "estimation device."

[0052] <Example of Hardware Configuration of Speech Recognition Device 10> Figure 6 is a diagram showing an example of the hardware configuration of the speech recognition device 10 according to this embodiment. As shown in Figure 6, the speech recognition device 10 according to this embodiment includes an input device 501, a display device 502, an external I / F 503, a communication I / F 504, a RAM (Random Access Memory) 505, a ROM (Read Only Memory) 506, an auxiliary storage device 507, and a processor 508. Each of these hardware components is connected to communicate via a bus 509.

[0053] The input device 501 is, for example, a keyboard, mouse, touch panel, or physical button. The display device 502 is, for example, a display or display panel. The voice recognition device 10 does not necessarily have to have at least one of the input device 501 and the display device 502.

[0054] The external I / F 503 is an interface with external devices such as the recording medium 503a. Examples of the recording medium 503a include CD (Compact Disc), DVD (Digital Versatile Disk), SD memory card (Secure Digital memory card), and USB (Universal Serial Bus) memory card.

[0055] The communication interface 504 is an interface for connecting to a communication network. The RAM 505 is a volatile semiconductor memory (storage device) that temporarily holds programs and data. The ROM 506 is a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The auxiliary storage device 507 is a non-volatile storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. The processor 508 is a computing device such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit).

[0056] Note that the hardware configuration shown in Figure 6 is just one example, and the hardware configuration of the speech recognition device 10 is not limited to this. For example, the speech recognition device 10 may have multiple auxiliary storage devices 507 and multiple processors 508, it may not have some of the hardware shown, or it may have various other hardware besides the hardware shown.

[0057] <Example of Functional Configuration of Speech Recognition Device 10 in the Learning Phase> Figure 7 is a diagram showing an example of the functional configuration of speech recognition device 10 in the learning phase. As shown in Figure 7, the speech recognition device 10 in the learning phase has an input unit 601, a speech recognition unit 602, a parameter update unit 603, and a termination determination unit 604. Each of these units is realized, for example, by a process in which one or more programs installed in the speech recognition device 10 are executed by a processor 508 or the like. The speech recognition device 10 in the learning phase also has a learning data storage unit 701 and a model storage unit 702. Each of these storage units is realized, for example, by a storage area such as an auxiliary storage device 507. At least one of the learning data storage unit 701 and the model storage unit 702 may be realized by a storage area of ​​a storage device (e.g., a storage device provided by a database server) that is connected to the speech recognition device 10 in a way that allows communication.

[0058] The input unit 601 receives a learning dataset D, which consists of one or more learning data stored in the learning data storage unit 701.

[0059] The speech recognition unit 602 uses the speech contained in each training data that makes up the training dataset D input by the input unit 601 and the Early Exit model stored in the model storage unit 702 to calculate the score of each token that makes up the estimated string.

[0060] The parameter update unit 603 updates the learnable parameters of the Early Exit model using the score calculated by the speech recognition unit 602, a predetermined error function (e.g., the error function E shown in equation (1) above), and the gradient of the error function with respect to the learnable parameters of the Early Exit model.

[0061] The termination determination unit 604 determines whether or not to terminate the update of the learning target parameters.

[0062] The learning data storage unit 701 stores learning data consisting of audio input to the Early Exit model and the correct string corresponding to that audio.

[0063] The model storage unit 702 stores Early Exit models learned using any existing method (e.g., the method described in Non-Patent Document 1).

[0064] <Example of learning process> Figure 8 is a flowchart showing an example of the learning process according to this embodiment.

[0065] The input unit 601 receives a learning dataset D, which consists of one or more learning data stored in the learning data storage unit 701 (step S101).

[0066] The speech recognition unit 602 uses the speech contained in each training data that makes up the training dataset D input in step S101 above, and the Early Exit model stored in the model storage unit 702 to calculate the score of each token that makes up the estimated string (step S102). For example, for l = 1, ..., L, the speech recognition unit 602 calculates the score of each token that makes up the estimated string using the probability distribution P included in the error function E shown in equation (1) above. ｌ (y ｔ | １ , , y ｔ－１ Calculate X;Θ) (t∈[1,T]).

[0067] The parameter update unit 603 updates the parameters to be learned of the Early Exit model using the score calculated in step S103, a predetermined error function, and the gradient of the error function with respect to the learnable parameters of the Early Exit model (step S103). For example, the parameter update unit 603 calculates the gradient of the error function E shown in equation (1) above with respect to the learnable parameters using backpropagation, and then updates the parameters to be learned so as to minimize the value of the error function E using an optimization method such as gradient descent. The parameters to be learned are the parameters of the intermediate output layers from j=1 to j=J and the parameters of the intermediate layers corresponding to each of those intermediate output layers. However, J is a predetermined integer satisfying 1 ≤ J ≤ L-1.

[0068] The termination determination unit 604 determines whether or not to terminate the update of the learning target parameters (step S104). At this time, the termination determination unit 604 determines to terminate the update of the learning target parameters if a predetermined first termination condition is met, and to not terminate the update of the learning target parameters otherwise. Here, the predetermined first termination condition may be, for example, that the number of repetitions of steps S101 to S103 has reached a predetermined number, that the number of epochs has reached a predetermined number, or that the value of the error function for the development data has converged. The number of epochs is the number that represents how many times the entire learning data stored in the learning data storage unit 701 has been used for parameter updates. Development data is the data consisting of audio and correct answer strings, excluding the learning data stored in the learning data storage unit 701.

[0069] If it is determined in step S104 that the update of the parameters to be learned is complete, the process returns to step S101. On the other hand, if it is not determined in step S104 that the update of the parameters to be learned is complete, the learning process is terminated.

[0070] The learning dataset D input in step S101 above may be a set consisting of all the learning data stored in the learning data storage unit 701, or it may be a set called a minibatch. A minibatch is a subset obtained by dividing the entire learning data stored in the learning data storage unit 701.

[0071] <Example of Functional Configuration of Speech Recognition Device 10 in the Inference Phase> Figure 9 shows an example of a speech recognition device 10 in the inference phase. As shown in Figure 9, the speech recognition device 10 in the inference phase has an input unit 601, a speech recognition unit 602, a termination determination unit 604, and an output unit 605. Each of these units is realized, for example, by a process in which one or more programs installed in the speech recognition device 10 are executed by a processor 508 or the like. The speech recognition device 10 in the inference phase also has a model storage unit 702. The model storage unit 702 is realized, for example, by a storage area such as an auxiliary storage device 507. The model storage unit 702 may also be realized, for example, by a storage area of ​​a storage device (e.g., a storage device provided by a database server) that is communicably connected to the speech recognition device 10.

[0072] The input unit 601 receives the speech to be used for speech recognition (hereinafter also referred to as "speech to be recognized").

[0073] The speech recognition unit 602 uses the speech input by the input unit 601 and the Early Exit model stored in the model storage unit 702 to calculate the scores of the tokens that make up the estimated string, and then generates tokens according to those scores. At this time, similar to the existing Early Exit, if the score output from the intermediate output layer of the Early Exit model is high, the speech recognition unit 602 generates a token from that score at that point and calculates the score for the next token.

[0074] The termination determination unit 604 determines whether or not to terminate the generation of tokens that constitute the estimated string.

[0075] The output unit 605 outputs the estimated string, which consists of tokens generated by the speech recognition unit 602, to a predetermined output destination. This output destination is not limited to a specific destination, but could include, for example, a display device 502 such as a display, a storage area such as an auxiliary storage device 507, other programs, or other devices connected via communication.

[0076] The model memory unit 702 stores the Early Exit model that was retrained during the learning phase.

[0077] <Example of speech recognition processing> Figure 10 is a flowchart showing an example of speech recognition processing according to this embodiment. In the following, it is assumed that the speech recognition device 10 is provided with the speech to be recognized.

[0078] The input unit 601 receives the given speech to be recognized (step S201).

[0079] The speech recognition unit 602 uses the speech to be recognized input in step S201 and the Early Exit model stored in the model storage unit 702 to calculate the scores of the tokens that constitute the estimated string, and then generates tokens from those scores (step S202). At this time, the speech recognition unit 602 sequentially calculates the scores from the l-th output layer for l=1, ..., L (e.g., probability distribution P ｌ (y ｔ | １ , , y ｔ－１ After outputting the X;Θ)) the confidence level of that score (e.g., probability distribution P) ｌ (y ｔ | １ , , y ｔ－１ If the maximum value of X;Θ) exceeds a predetermined threshold, token y is drawn from that score. ｔ This generates the t-th token y. ｔ This is generated. Note that the score is a probability distribution P ｌ (y ｔ | １ , , y ｔ－１ If the case is X; Θ), the speech recognition unit 602 will determine the probability distribution P ｌ (y ｔ | １ , , y ｔ－１ The t-th token y according to X;Θ) ｔ You just need to generate it.

[0080] However, if the score output from the l-th output layer exceeds a predetermined threshold, token y is generated from that score. ｔ generate (i.e., token y) ｔ One example is terminating the inference process for token y.ｔ It may be possible to end the inference process. For example, when scores are output from the l-th output layer in order for l = 1,..., L, and the same token y ｔ is generated continuously a predetermined number of times, the inference process for token y ｔ may be ended at that point. For a specific example, if the score is the probability distribution P ｌ (y ｔ | y １ ,..., y ｔ－１ , X; Θ), when the token that takes the maximum value of the probability distribution P ｌ (y ｔ | y １ ,..., y ｔ－１ , X; Θ) is the same continuously for a predetermined number of times, the inference process for token y ｔ may be ended at that point.

[0081] Incidentally, by changing the threshold value for ending the inference process of token y ｔ (that is, the threshold value for the score output from the l-th output layer, the threshold value representing the number of times the same token y ｔ is generated continuously), it is possible to indirectly change the inference time.

[0082] The end determination unit 604 determines whether to end the generation of the tokens constituting the estimated string (step S203). At this time, when the end determination unit 604 satisfies a predetermined second end condition, it may determine to end the generation of the tokens constituting the estimated string, and when it does not, it may determine not to end the generation of the tokens constituting the estimated string. Here, examples of the predetermined second end condition include, for example, that a token representing the end of a sentence is generated as token y ｔ .

[0083] If it is not determined in step S203 above to end the generation of the tokens constituting the estimated string, the process returns to step S202. As a result, the (t + 1)-th token y ｔ is generated. Note that it should be noted that the initial value of t is 1.

[0084] The output unit 605 outputs the token y generated in step S202 aboveｔ Output the estimated string composed of (t = 1, 2,...) to a predetermined output destination (step S204).

[0085] <Experiment> An experiment was conducted to confirm the effect of the proposed method. In this experiment, learning data composed of Japanese speech and its correct string, and evaluation data composed of Japanese speech and its correct string were used. Note that the evaluation data is different from any learning data. Also, the total time of the speech included in each learning data is about 512.6 hours, and the total time of the speech included in each evaluation data is about 1.3 hours.

[0086] The evaluation metrics used were the character error rate (%) of the estimated string with respect to the correct string and the speed increase rate (%) representing the increase rate of the inference time. Also, the comparison targets with the proposed method were the method using the Backbone model that always executes the processing up to the final layer (hereinafter referred to as "baseline (always processes up to the final layer)") and the method of creating an Early Exit model by the method described in Non-Patent Document 1 (hereinafter referred to as "baseline").). Also, in the proposed method, the Early Exit model was re-learned using the above learning data with the baseline as the initial value.

[0087] The evaluation results of the baseline (always processes up to the final layer), baseline, and proposed method are shown in Table 1 below.

[0088]

[0089] Here, in the Early Exit model, since the inference time and accuracy can be controlled by the threshold for ending the inference process of token y ｔ it is difficult to strictly specify the accuracy and speed. Therefore, in order to explain the effect of the proposed method, when comparing the accuracy of the inference results, the character error rate will be compared between equivalent speed increase rates.

[0090] Table 1 compares the character error rate for the first line of the baseline with that of the proposed method, the character error rate for the second line of the baseline with that of the proposed method, and the character error rate for the third line of the baseline with that of the proposed method. It can be seen that the proposed method has a lower character error rate and higher accuracy than the baseline.

[0091] Furthermore, as shown in Table 1, when comparing the rate of speed improvement at equivalent character error rates (first line in the baseline vs. third line in the proposed method) between the baseline and the proposed method, it can be seen that the proposed method is approximately 2.48 times faster than the baseline.

[0092] Based on the above, it can be seen that the proposed method achieves higher accuracy in inference results and a reduction in inference time compared to the baseline.

[0093] <Summary> As described above, the speech recognition device 10 according to this embodiment can reduce the inference time while maintaining the accuracy of the inference results when applied to speech recognition. However, applying it to speech recognition is just one example, and it is possible to similarly reduce the inference time while maintaining the accuracy of the inference results when applied to text generation tasks other than speech recognition (e.g., machine translation, question answering, etc.).

[0094] The following additional information is disclosed regarding the embodiments described above.

[0095] (Note 1) A learning device comprising: a memory; and at least one processor connected to the memory, wherein the processor comprises: a trained model including one or more intermediate layers; a final layer located after the one or more intermediate layers; a first output layer located after the final layer; and a second output layer added to at least one of the one or more intermediate layers; outputting inference results from the first output layer and the second output layer based on data that is input to the trained model; and updating the parameters of at least one predetermined second output layer among the second output layers and the parameters of the intermediate layer to which the predetermined second output layer is added, based on the inference results output from the first output layer and the second output layer and information corresponding to the data. (Note 2) The learning device according to Note 1, wherein the predetermined second output layer is a second output layer added to a predetermined number of intermediate layers among the second output layers, determined in order of the processing order when the data is input to the trained model. (Note 3) The learning device according to Note 1 or 2, wherein the information corresponding to the data is the correct inference result when the data is input to the trained model, and the processor updates the parameters of the predetermined second output layer and the parameters of the intermediate layer to which the predetermined second output layer is added in order to minimize the error between the inference result output from the first output layer and the second output layer and the correct answer. (Note 4) The learning device according to Note 3, wherein the processor updates the parameters of the predetermined second output layer and the parameters of the intermediate layer to which the predetermined second output layer is added in order to minimize the error using the gradient of the error with respect to the trainable parameters of the trained model. (Note 5) The learning device according to Note 4, wherein the processor updates the parameters of the predetermined second output layer and the parameters of the intermediate layer to which the predetermined second output layer is added in order to fix the parameters of the trainable parameters of the trained model other than the parameters of the predetermined second output layer and the parameters of the intermediate layer to which the predetermined second output layer is added.(Appendix 6) An inference device comprising: a memory; and at least one processor connected to the memory, wherein the processor retrains a trained second model obtained by retraining a trained first model which includes one or more intermediate layers, a final layer located after the one or more intermediate layers, a first output layer located after the final layer, and a second output layer added to at least one of the one or more intermediate layers; and outputs inference results from the first output layer or the second output layer based on data that becomes input to the trained second model, wherein the trained second model is a model obtained by retraining the predetermined second output layer and an intermediate layer to which the predetermined second output layer is added so as to improve the accuracy of the inference results output from at least one predetermined second output layer of the trained first model. (Appendix 7) A non-temporary storage medium storing a program executable by a computer to perform a learning process, wherein the learning process includes a trained model comprising one or more intermediate layers, a final layer located after the one or more intermediate layers, a first output layer located after the final layer, and a second output layer added to at least one of the one or more intermediate layers; outputting inference results from the first output layer and the second output layer based on data that serves as input to the trained model; and updating the parameters of at least one predetermined second output layer among the second output layers and the parameters of the intermediate layer to which the predetermined second output layer is added, based on the inference results output from the first output layer and the second output layer and information corresponding to the data.

[0096] The present invention is not limited to the embodiments specifically disclosed above, and various modifications, changes, and combinations with known technologies are possible without departing from the spirit of the claims.

[0097] [References] Reference 1: M. Elbayad, J. Gu, E. Grave, and M. Auli, "Depth-adaptive transformer", in Proc. International Conference on Learning Representations (ICLR), 2020.

[0098] 10 Speech recognition device 501 Input device 502 Display device 503 External I / F 503a Recording medium 504 Communication I / F 505 RAM 506 ROM 507 Auxiliary storage device 508 Processor 509 Bus 601 Input unit 602 Speech recognition unit 603 Parameter update unit 604 Termination determination unit 605 Output unit 701 Learning data storage unit 702 Model storage unit

Claims

1. A learning device comprising: a trained model including one or more intermediate layers, a final layer located after the one or more intermediate layers, a first output layer located after the final layer, and a second output layer added to at least one of the one or more intermediate layers; an inference unit that outputs inference results from the first output layer and the second output layer based on data that serves as input to the trained model; and an update unit that updates the parameters of at least one predetermined second output layer among the second output layers and the parameters of the intermediate layer to which the predetermined second output layer is added, based on the inference results output from the first output layer and the second output layer and information corresponding to the data.

2. The learning device according to claim 1, wherein the predetermined second output layer is a second output layer added to a predetermined number of intermediate layers, which are determined in order of the processing order when the data is input to the trained model.

3. The learning device according to claim 1 or 2, wherein the information corresponding to the data is the correct inference result when the data is input to the trained model, and the update unit updates the predetermined parameters of the second output layer and the parameters of the intermediate layer to which the predetermined second output layer is added, so as to minimize the error between the inference results output from the first output layer and the second output layer and the correct answer.

4. The learning device according to claim 3, wherein the update unit updates the parameters of the predetermined second output layer and the parameters of the intermediate layer to which the predetermined second output layer is added, using the gradient of the error with respect to the learnable parameters of the trained model, in order to minimize the error.

5. The learning device according to claim 4, wherein the update unit fixes the parameters of the trained model other than the parameters of the predetermined second output layer and the parameters of the intermediate layer to which the predetermined second output layer is added, and then updates the parameters of the predetermined second output layer and the parameters of the intermediate layer to which the predetermined second output layer is added.

6. An inference device comprising: a trained second model obtained by retraining a trained first model which includes one or more intermediate layers, a final layer located after the one or more intermediate layers, a first output layer located after the final layer, and a second output layer added to at least one of the intermediate layers; and an inference unit which outputs an inference result from the first output layer or the second output layer based on data that is input to the trained second model, wherein the trained second model is a model obtained by retraining the predetermined second output layer and an intermediate layer to which the predetermined second output layer has been added so as to improve the accuracy of the inference result output from at least one predetermined second output layer of the trained first model.

7. A learning method in which a computer performs the following steps: a trained model including one or more intermediate layers, a final layer located after the one or more intermediate layers, a first output layer located after the final layer, and a second output layer added to at least one of the one or more intermediate layers; an inference procedure that outputs inference results from the first output layer and the second output layer based on data that serves as input to the trained model; and an update procedure that updates the parameters of at least one predetermined second output layer among the second output layers and the parameters of the intermediate layer to which the predetermined second output layer is added, based on the inference results output from the first output layer and the second output layer and information corresponding to the data.

8. A program that causes a computer to execute: a trained model including one or more intermediate layers, a final layer located after the one or more intermediate layers, a first output layer located after the final layer, and a second output layer added to at least one of the intermediate layers; an inference procedure that outputs inference results from the first output layer and the second output layer based on data that serves as input to the trained model; and an update procedure that updates the parameters of at least one predetermined second output layer among the second output layers and the parameters of the intermediate layer to which the predetermined second output layer is added, based on the inference results output from the first output layer and the second output layer and information corresponding to the data.

Images