Alternative Soft Label Generation

Abstract

A technique can be provided for identifying alternative soft labels for training a student model. A teacher model can generate soft labels for labeled training data. The training data can be audio files of utterances or spoken natural language. The pool of soft labels previously generated by the teacher model can be searched at the label level to identify soft labels similar to the generated soft labels. Similar soft labels can have similar lengths or sequences at the word phoneme and / or state level. The identified similar soft labels can be used, in conjunction with the generated soft labels, to train a student model.

Description

[Technical Field]

[0001] The present invention relates generally to the field of machine learning, and more particularly to training student models via knowledge distillation. [Background technology]

[0002] Training machine learning models requires large datasets and significant computational resources. Knowledge distillation techniques, which attempt to transfer knowledge from a complex teacher network to a simple student network, are used to train machine learning models. The student network is trained on the teacher's soft output, which minimizes the difference in distribution between the student and teacher. Student model training is achieved by using degraded features as inputs and soft targets for better features from the teacher network as targets. Such methods require additional parallel data to improve the accuracy of the student network. Summary of the Invention

[0003] Embodiments of the present disclosure include computer-implemented methods, computer program products, and systems for training a student acoustic model. Embodiments may include receiving labeled acoustic features. Additionally, embodiments may include generating soft labels for the acoustic features labeled by the teacher model. Embodiments may further include extracting, by one or more processors, a second soft label from the soft label pool that is similar to the first soft label. Finally, embodiments may include training a student model using the first soft label and the second soft label.

[0004] The above summary is not intended to describe every example embodiment of every implementation of the present disclosure. [Brief explanation of the drawings]

[0005] [Figure 1] FIG. 1 is a functional block diagram generally illustrating a soft label generation environment in accordance with an embodiment of the present invention. [Figure 2] FIG. 10 is a functional block diagram illustrating an alternative soft label identification engine, according to an embodiment of the present invention. [Figure 3] 10 is a flowchart illustrating an alternative method of soft label generation according to an embodiment of the present invention. [Figure 4] FIG. 1 is a functional block diagram of an exemplary computing system within a code refactoring naming environment, according to an embodiment of the present invention. [Figure 5] 1 is a diagram illustrating a cloud computing environment according to an embodiment of the present invention. [Figure 6] FIG. 2 is a functional block diagram illustrating an abstract model layer according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0006] While the embodiments described herein are amenable to various modifications and alternative forms, specific examples thereof have been shown by way of example in the drawings and will now be described in detail. It is to be understood, however, that the particular embodiments described are not to be taken in a limiting sense. On the contrary, it is intended to cover all modifications, equivalents, and alternatives falling within the scope of the present disclosure.

[0007] The depicted embodiment enables the generation of soft-label training data for a student model based on the generation of training data by a teacher model. Complex acoustic models cannot be deployed for real-time decoding of streaming speech data. This is primarily due to constraints related to the lack of computational resources on devices such as smartphones, tablets, and laptops. Another factor contributing to the difficulty of deploying complex acoustic models is the latency issue when devices attempt to transmit data over a network to a model located on a server. In an attempt to circumvent the computational and latency issues of complex acoustic models, compact acoustic models have been trained via knowledge distillation techniques. Compact acoustic models lack the layers or analytical capabilities of complex or wideband acoustic models. However, compact acoustic models offer the benefits of fast feedback and low computational resource requirements. In an attempt to increase the performance of compact acoustic model output, distilling training data from complex models allows more information to be included in the layers of the compact acoustic model.

[0008] In speech recognition, neural networks (NNs) are typically used for acoustic models to generate probability distributions over hidden Markov model (HMM) states from acoustic features derived from the speech signal. HMM states may correspond to clusters of context-dependent phoneme states, or simply context-independent phoneme states. One context-dependent model is the quinphone model, in which separate phone models are used for two different left and two different right phone contexts.

[0009] For example, English has 44 phonemes. By considering phonemes surrounding a central phoneme, along with parameter sharing, a typical speech recognition system will have thousands to tens of thousands of context-dependent phoneme states. A neural network model for an acoustic model typically includes an input layer that receives acoustic features derived from the speech signal, one or more hidden layers that process the acoustic features, and an output layer that outputs a probability distribution over the HMM states based on the activations of the hidden layers.

[0010] 1 is a functional block diagram illustrating a soft-label generation environment 100. The soft-label generation environment 100 includes a teacher model 104 and an alternative soft-label identification engine 106 operable on a server 102, a soft-label pool 108 stored on the server 102, a student model 114 operable on a server 112, and a network 110.

[0011] Servers 102 and 112 can be standalone computing devices, management servers, web servers, mobile computing devices, or any other electronic device or computing system capable of receiving, transmitting, and processing data. In other embodiments, servers 102 and 112 can represent a server computing system that utilizes multiple computers as a server system. In another embodiment, servers 102 and 112 can be laptop computers, tablet computers, netbook computers, personal computers, desktop computers, or any programmable electronic device capable of communicating with other computing devices (not shown) in soft label generation environment 100 over network 110.

[0012] In another embodiment, servers 102 and 112 represent a computing system utilizing clustered computers and components (e.g., database server computers, application server computers, etc.) that can function as a single pool of seamless resources when accessed within soft-label generation environment 100. Servers 102 and 112 can include internal and external hardware components, as shown and described in further detail with respect to FIG. 4. While only servers 102 and 112 are shown in FIG. 1, it should be noted that multiple computing devices can be present within soft-label generation environment 100. For example, a client computer (not shown) can communicate with servers 102 and 112 via network 110. In another example, servers 102 and 112 can be part of a cloud server network (e.g., the Internet) that allows computing devices (not shown) connected to network 110 to access servers 102 and 112.

[0013] The teacher model 104 is typically a high-resource classification model, including a single model or an ensemble of models. The teacher model 104 may even be too resource-heavy to actually deploy as a practical service. A single classification model or each classification model in the ensemble may be any one of standard neural networks, including deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and neural network models that combine features of several neural network types. In particular embodiments, the teacher model 104 includes, but is not limited to, a VGG model, a long short-term memory (LSTM), or a ResNet, or a combination thereof. In the described embodiment, the teacher model 104 is already trained using a sufficiently large collection of training data prior to knowledge distillation. Note that the teacher model 104 can be configured to classify labeled acoustic features into one or more classes. A soft label can be a prediction of the probability that an acoustic feature exists in a particular class. Additionally, in some embodiments, the teacher model 104 can generate hard features for the labeled acoustic features. A hard label is a yes or no classification of whether the input acoustic feature exists in a class. A soft label is a prediction (i.e., probability) that the input acoustic feature exists in a certain class.

[0014] The alternative soft-label identification engine 106 is a computer module that can be configured to identify one or more soft labels in the soft-label pool 108 that are similar to the reference labels output by the teacher model 104. The alternative soft-label identification engine 106 can search the reference labels generated by a trained teacher model. Furthermore, in some embodiments, the alternative soft-label identification engine 106 can identify candidate similar soft labels based on phonemes associated with the reference labels in the soft-label pool 108. Note that the teacher model that generates the soft labels in the soft-label pool 108 can be a different teacher model than the teacher model 104 in the soft-label generation environment 100. The alternative soft-label identification engine 106 can also extract similarity scores from the candidate soft labels to generate alternative soft labels for training the student model 114. The alternative soft-label identification engine 106 can transmit the generated soft labels to the student model 114 via the network 110.

[0015] The soft label pool 108 is configured to store collection soft labels and reference labels that label individual frames in the speech data. In an exemplary embodiment, the data units are frames in the speech data. The speech data collected to generate soft labels can include any speech data that records actual utterances spoken in conversation, narration, etc. Each soft label in the soft label pool can include a phoneme class that is aligned with a corresponding frame from the teacher model 104 through forced alignment. In other words, each frame is pre-labeled in parallel with the correct phoneme class selected from the teacher class set. The soft label pool 108 can be provided by a data storage or memory area of ​​any storage medium or device operably coupled to the alternative soft label identification engine 106, which implements alternative soft label generation for knowledge distillation on the student model.

[0016] The network 110 can be a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of the two, and can include wired, wireless, or fiber optic connections. In general, the network 110 can be any combination of connections and protocols that support communication between the servers 102, 112 and other computing devices (not shown).

[0017] The student model 114 is typically a lightweight, compact classification model that can be easily used as a production acoustic model. The student model 114 can be any one of a number of standard neural networks, including DNN-, CNN-, RNN-based models, and neural networks that combine features of several neural network types. In one or more embodiments, the student model 114 has a different structure than the teacher model 104. In an exemplary embodiment, the student model 114 is smaller and simpler (i.e., has fewer parameters) than the teacher model 104. However, models that are larger and / or more complex than the teacher model 104 may not be excluded from the model for the student model 114. Note that the student model 114 may be trained on the same classes as the teacher model 104. The student model 114 may also be trained on multiple classes and may receive training data from multiple teacher models (not shown).

[0018] FIG. 2 is a functional block diagram 200 illustrating an alternative soft label identification engine, according to an embodiment of the present invention.

[0019] 2 illustrates an alternative soft-label identification engine 106. Operable on the alternative soft-label identification engine 106 are a reference label search module 202 and a similarity extraction module 204.

[0020] The reference label search module 202 is a computer module that can be configured to identify candidate soft labels from the soft label pool 108 based on reference labels associated with soft labels in the soft label pool 108. In some embodiments, the reference label search module 202 can receive data associated with soft labels output by the teacher module 104. The data can include references to identified utterances or utterances. For example, a reference label for a soft label from the teacher model 104 can be "I have a question," targeting "question." The reference label search module 202 can search all of the reference labels associated with soft labels in the soft label pool 108 to find candidate soft labels. In the example immediately above, the reference label search module 202 identifies the following from the soft label pool 108: "...good question...," "...your question...," and "...two questions...." Candidate soft labels with "question" in the reference label can be sent to an alternative soft label identification engine 106. Note that the reference label can be a generated transcription of an audio file containing the utterance, where the audio file has been processed by an automatic speech recognition system. In some embodiments, the reference label may be a human-annotated transcription of the content of the audio file.

[0021] The similarity extraction module 204 is a computer module that can be configured to identify similar soft labels from the candidate soft labels identified by the reference label lookup module 202. The similarity extraction module 204 can identify similar soft labels based on an analysis of data within the soft-label utterance data. In some embodiments, the similarity extraction module 204 extracts word sequences of similar length at the word, phoneme, or state level. In other embodiments, the similarity extraction module 204 can extract identical or similar sequences of words at the word-phoneme or state level. In other embodiments, the similarity extraction module 204 can identify phoneme sequences of identical or similar length at the phoneme or state level. In some embodiments, the similarity extraction module 204 can be configured to generate a similarity score for the candidate soft labels based on various factors (e.g., word / phoneme / state match, gender, age, speaker attributes, etc.). Note that the similarity extraction module 204 can be configured to force-align transcriptions from the soft label pool 108. The forced-aligned transcripts are then aligned to the audio file, and the similarity extraction module 204 can further process the spectrogram of the audio file. In an embodiment, the similarity extraction module 204 analyzes linguistic similarities. If the target word is "question," the similarity extraction module 204 selects vocalization regions "question" with the same or similar duration from the soft label pool 208.For example, for the word "question" in the target domain, at the word level, the similarity extraction module 204 would search for "question(24)" in the soft label pool 208; at the Phong level, the similarity extraction module 204 would search for "K(3)-W(3)-EH(4)-S(3)-CH(5)-IH(3)-N(3)" in the soft label pool 208; and at the state level, the similarity extraction module 204 would search for "K(1-1-1)-W(1-1-1)-EH(1-2-1)-S(1-1-1)-CH(1-3-1)-IH(1-1-1)-N(1-1-1)." Note that acoustic features for “question” from the target domain, the original soft label for “question”, and the alternative soft label for “question” from the selected vocalization domain are used to train the student model.

[0022] In another example, in an embodiment in which the similarity extraction module 204 analyzes word sequences for similar length, the input for the teacher-generated soft label can be "question" in "I have a question." From the soft label pool 108, data for candidate soft labels and the following two word segments have been identified from the reference label: "...that's two questions, sir" and "...very good question." The similarity extraction module 204 recognizes that one of the candidate segments ends with the word "question" and one of the candidate segments has four words in it, like the input segment. In this example, the similarity extraction module 204 is configured to generate similarity scores for the candidate soft labels. The similarity score for "...very good question" is 96%, while the similarity score for "...that's two questions, sir" is 70%. In this example setting, only candidate soft labels with scores of approximately 83% are sent to the student model 114 for training.

[0023] In a further embodiment, if the similarity extraction model 204 determines that one or more candidate soft labels are similar to the soft labels generated by the teacher model 104, the similarity extraction model can further extract similar soft labels from the determined candidate soft labels. In an embodiment, the similarity extraction model 204 can send all matched soft labels to the student model 114 for training. In another embodiment, the similarity extraction model can send only the best matched (e.g., 1 best) or n-best soft labels (where n is a positive integer) to the student model 114 for training. In yet another embodiment, the similarity extraction model 204 can send soft labels with a high kurtosis distribution to train the student model 114.

[0024] FIG. 3 is a flowchart illustrating a method 300 for alternative soft label generation, according to an embodiment of the present invention. In step 302, training data is received by the teacher model 104. In some embodiments, the training data is audio files from a user or a database. In some embodiments, the training data is labeled using a transcription. Additionally, the training data may have a spectrogram representing sound waves associated with the audio file. In some embodiments, an automatic speech recognition system has generated feature vectors for the training data.

[0025] In step 304, soft labels are generated by the teacher model 104 for the training data. In some embodiments, the teacher model 104 can generate soft labels for the training data, which are probability predictions for the training data that place words or word segments into particular classes that the teacher model is designed to classify. In some embodiments, the teacher model 104 can also be configured to generate hard labels for the training data.

[0026] In step 306, an alternative soft label identification engine 106 is used to extract similar soft labels from the soft label pool 108. For example, the alternative soft label identification engine 106 can receive the soft labels and associated training data generated by the teacher model 104. The reference label search module 202 can search for candidate soft labels in the soft label pool 108 based on the reference labels associated with the training data. In some embodiments, the target word can be isolated within the training data reference label for further refinement. In another example, the similarity extraction module 204 can analyze the identified candidate soft labels based on speech data associated with the soft labels. The speech data can be force-aligned to the speech data of the soft target generated based on the target word. In some embodiments, the phoneme sequence of the speech data for the candidate soft labels can be analyzed against the phoneme sequence of the speech data for the generated soft label. In some embodiments, a similarity score can be generated by the similarity extraction module 204 for each identified candidate soft label. Furthermore, if a candidate soft label is identified as above a threshold similarity score, the candidate soft label may be sent for training to the student model 114. In some embodiments, one or more similar soft labels may be sent to the student model 114 for training based on alternative soft label identification engine 106 settings.

[0027] In step 308, the student model 114 is trained using the soft labels generated by the teacher model 104 and similar soft labels identified by the alternative soft label identification engine 106. In some embodiments, the student model 114 can be trained using two or more similar soft labels.

[0028] 4 illustrates an exemplary computer system, computer system 400, which may represent servers 102 and 112, or any other computing device in embodiments of the present invention. Computer system 400 includes a communications fabric 412 that provides communications between a computer processor 414, memory 416, persistent storage 418, a network adapter 428, and an input / output (I / O) interface 426. Communications fabric 412 may be implemented with any architecture designed to pass data and / or control information between processors (e.g., microprocessors, communications and network processors), system memory, peripheral devices, and any other hardware components in the system. For example, communications fabric 412 may be implemented using one or more buses.

[0029] Computer system 400 includes a processor 414, a cache 422, a memory 416, a network adapter 428, an input / output (I / O) interface 426, and a communications fabric 412. Communications fabric 412 provides communication between cache 422, memory 416, persistent storage 418, network adapter 428, and input / output (I / O) interface 426. Communications fabric 412 may be implemented with any architecture designed to pass data and / or control information between processors (e.g., microprocessors, communications and network processors), system memory, peripheral devices, and any other hardware components in the system. For example, communications fabric 412 may be implemented using one or more buses or crossbar switches.

[0030] Memory 416 and persistent storage 418 are computer-readable storage media. In this embodiment, memory 416 includes persistent storage 418, random access memory (RAM) 420, cache 422, and program modules 424. In general, memory 416 may include any suitable volatile or non-volatile computer-readable storage medium. Cache 422 is a high-speed memory that improves performance of processor 414 by retaining recently and most recently accessed data from memory 416. As further depicted and described below, memory 416 may include at least one of program modules 424 configured to perform functions of embodiments of the present invention.

[0031] A program / utility having at least one program module 424 can be stored in memory 416, as can, for example and without limitation, an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or some combination thereof, can include an implementation of a networking environment. The program module 424 generally performs the functions and / or methods of embodiments of the present invention as described herein.

[0032] Program instructions and data used to practice embodiments of the present invention may be stored in persistent storage 418 and memory 416 for execution by one or more of the individual processors 414 via cache 422. In an embodiment, persistent storage 418 includes a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, persistent storage 418 can include a solid-state hard drive, a semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0033] The media used by persistent storage 418 may also be removable. For example, a removable hard drive may be used for persistent storage 418. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer to another computer-readable storage medium that is also part of persistent storage 418.

[0034] Network adapter 428, in these examples, provides for communication with other data processing systems or devices. In these examples, network adapter 428 includes one or more network interface cards. Network adapter 428 may provide communications through the use of either or both physical and wireless communications links. Program instructions and data used to practice embodiments of the present invention may be downloaded to persistent storage 418 through network adapter 428.

[0035] The I / O interface 426 allows for the input and output of data with other devices that may be connected to each computer system. For example, the I / O interface 426 may provide a connection to external devices 430, such as a keyboard, keypad, touch screen, or any other suitable input device or combination thereof. The external devices 430 may also include portable computer-readable storage media, such as thumb drives, portable optical or magnetic disks, and memory cards. Software and data embodying embodiments of the present invention may be stored on such portable computer-readable storage media and loaded into persistent storage 418 via the I / O interface 426. The I / O interface 426 also connects to a display 432.

[0036] Display 432 provides a mechanism for displaying data to a user and may be, for example, a computer monitor or a virtual graphical user interface.

[0037] The components described herein are identified based on the application in which they are implemented in specific embodiments of the invention. However, it should be appreciated that any specific component nomenclature herein is used for convenience only, and thus the invention should not be limited to use in any specific application identified and / or implied by such nomenclature.

[0038] The present invention may be a system, a method, and / or a computer program product, and may include a computer-readable storage medium having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0039] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as instruction-recorded punch cards or grooved structures, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as ephemeral signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted over electrical wires.

[0040] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to an individual computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the individual computing / processing device.

[0041] The computer-readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming languages ​​such as Smalltalk®, C++, and traditional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer as a standalone software package, partly on the user's computer and partly on a remote computer, or all on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer readable program instructions to individualize the electronic circuitry by utilizing state information in the computer readable program instructions to implement aspects of the present invention.

[0042] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0043] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to make a machine, such that the instructions, which execute on the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium capable of directing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored comprises an article of manufacture containing instructions for implementing aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0044] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to create a computer-implemented process and cause the computer, other programmable apparatus, or other device to perform a series of operational steps, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0045] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may in fact be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by special-purpose hardware-based systems that perform the specified functions or actions or execute a combination of special-purpose hardware and computer instructions.

[0046] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings presented herein is not limited to cloud computing environments. Rather, embodiments of the present invention may be implemented in conjunction with any other type of computing environment now known or later developed.

[0047] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0048] Its features are as follows:

[0049] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capabilities such as server time and network storage automatically as needed without requiring human interaction with the service provider.

[0050] Broad network access: Functionality is available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms (e.g., mobile phones, laptops, and PDAs).

[0051] Pooling of resources: Provider computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated on demand. There is a sense of location independence in that consumers generally have no control or knowledge of the exact location of the resources provided, although they may be able to identify the location at a higher level of abstraction (e.g., country, state, or data center).

[0052] Rapid Scalability: Capabilities can be provisioned quickly and scalably, in some cases automatically, quickly scaled out, and quickly released and quickly scaled in. To the consumer, the capabilities available for provisioning often appear unlimited, and any amount can be purchased at any time.

[0053] Service Metering: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services used.

[0054] The service model is as follows:

[0055] Software as a Service (SaaS): The functionality offered to the consumer is the use of a provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, except for limited user-specific application configuration settings.

[0056] Platform as a Service (PaaS): The capability offered to consumers is the deployment of consumer-created or off-the-shelf applications, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the application hosting environment configuration.

[0057] Infrastructure as a Service (IaaS): The capability offered to consumers is the provisioning of processing, storage, network, and other basic computing resources on which the consumer can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does have control over the operating system, storage, deployed applications, and possibly limited control over select networking components (e.g., host firewalls).

[0058] The deployment model is as follows:

[0059] Private Cloud: Cloud infrastructure is operated exclusively for an organization, can be managed by that organization or a third party, and can exist on-premise or off-premise.

[0060] Community Cloud: Cloud infrastructure is shared by several organizations and supports a specific community with a shared mission (e.g., mission, security requirements, policies, and compliance concerns). It may be managed by the organization or a third party and can exist on-premises or off-premises.

[0061] Public Cloud: Cloud infrastructure is made available to the general public or large industry organizations and is owned by an organization that sells cloud services.

[0062] Hybrid Cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a unique entity, but are joined by standardized or proprietary technologies that allow data and application portability (e.g., cloud bursting for load balancing between clouds).

[0063] Cloud computing environments are service-oriented, focusing on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0064] FIG. 5 is a block diagram illustrating a cloud computing environment 50 in accordance with at least one embodiment of the present invention. The cloud computing environment 50 includes one or more cloud computing nodes 10 that can communicate with local computing devices used by cloud consumers, such as, for example, a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automobile computer system 54N. The nodes 10 can communicate with each other. They can be grouped physically or virtually in one or more networks (not shown), such as a private, community, public, or hybrid cloud, or a combination thereof, as described herein. This enables the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service, eliminating the need for cloud consumers to maintain resources on their local computing devices. It will be understood that the types of computing devices 54A-N shown in FIG. 6 are intended to be exemplary only, and that the computing node 10 and cloud computing environment 50 can communicate with any type of computerized device over any type of network and / or network-addressable connection (e.g., using a web browser).

[0065] Figure 6 is a block diagram illustrating a set of functional abstract model layers provided by the cloud computing environment 50 depicted in Figure 5 in accordance with at least one embodiment of the present invention. It should be understood in advance that the components, layers, and functions depicted in Figure 6 are intended to be merely exemplary, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0066] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframes 61, RISC (reduced instruction set computer) architecture-based servers 62, servers 63, blade servers 64, storage devices 65, and networks and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0067] The virtualization layer 70 provides an abstraction layer over which the following examples of virtual entities can be provided: virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.

[0068] In one example, the management layer 80 can provide the following functions: Resource provisioning 81 provides dynamic procurement of computing resources and other resources utilized to perform tasks within the cloud computing environment. Metering and billing 82 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 provides allocation and management of cloud computing resources to ensure required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides advance agreement on and procurement of cloud computing resources in anticipation of future demand according to SLAs.

[0069] The workload layer 90 provides examples of functionality for which a cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this layer include: mapping and navigation 91, software development and lifecycle management 92, virtual classroom instruction delivery 93, data analytics processing 94, transaction processing 95, and alternative soft label identification 96.

[0070] While descriptions of various embodiments of the present invention have been presented for illustrative purposes, they are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the present invention. The terminology used herein has been chosen to best explain the principles of the embodiments, practical applications, or technical improvements over techniques found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for training a student model via knowledge distillation with supplemented labeled training data, the method comprising: receiving, by one or more processors, labeled acoustic features; generating, by the one or more processors, first soft labels for the acoustic features labeled by the teacher model; extracting, by the one or more processors, a second soft label from a soft label pool that is similar to the first soft label; training, by the one or more processors, a student model using the first soft labels and the second soft labels; 11. A computer-implemented method comprising:

2. Extracting the second soft label comprises: identifying, by the one or more processors, one or more candidate soft labels based on each reference label associated with the first soft label; determining, by the one or more processors, whether each of the one or more candidate soft labels is similar to the labeled acoustic feature, wherein the one or more candidate soft labels are similar if they have the same phoneme sequence as the labeled acoustic feature; in response to the one or more candidate soft labels being determined to be similar to the labeled acoustic feature, outputting the one or more candidate soft labels if the one or more candidate soft labels are determined to be similar to the acoustic feature by the one or more processors; The computer-implemented method of claim 1 , further comprising:

3. The computer-implemented method of claim 1 or 2, wherein the soft label pool is generated by the teacher model.

4. The computer-implemented method of any one of claims 1 to 3, wherein the teacher model and the student model are automatic speech recognition systems.

5. 5. The computer-implemented method of claim 1, wherein the labeled acoustic features are generated by an acoustic feature generator, the acoustic feature generator being a convolutional neural network.

6. The computer-implemented method of any one of claims 1 to 5, wherein the teacher model is a visual geometry group model.

7. generating, by the one or more processors, hard labels for the labeled acoustic features; training, by the one or more processors, the student model using the hard labels; The computer-implemented method of any one of claims 1 to 6, further comprising:

8. 1. A system for generating labeled training data for a student model, the system comprising: one or more computer processors; one or more computer-readable storage media; receiving first training data including a plurality of pairs of a first hard label and a first soft label, each of the first soft labels being generated by a teacher model; detecting, by the one or more computer processors, a second soft label from a soft label pool that is similar to the first soft label; outputting the second soft labels as additional training data for each first soft label; training a student model using the first soft labels and the second soft labels; and computer program instructions for performing A system comprising:

9. 9. The system of claim 8, wherein detecting the second soft label further comprises extracting, by the one or more computer processors, word sequences from the soft label pool that have the same length at a state level.

10. 10. The system of claim 8 or 9, wherein detecting the second soft label further comprises extracting phoneme sequences of the same length from the soft label pool.

11. The system of any one of claims 8 to 10, wherein the teacher model and the student model are automatic speech recognition systems.

12. The system according to any one of claims 8 to 11, wherein the teacher model is a visual geometry group model.

13. The system of any one of claims 8 to 12, wherein the student model is a convolutional neural network.

14. A computer program comprising: receiving first training data including a plurality of pairs of a first hard label and a first soft label, each of the first soft labels being generated by a teacher model; finding a second soft label from a soft label pool that is similar to the first soft label; outputting the second soft labels as additional training data for each first soft label; training a student model using the first soft labels and the second soft labels; A computer program for executing the above.

15. 15. The computer program product of claim 14, wherein detecting the second soft label further comprises extracting word sequences from the soft label pool that have the same length at a state level.

16. 16. The computer program product of claim 14 or 15, wherein detecting the second soft label further comprises extracting phoneme sequences of the same length from the soft label pool.

17. 17. A computer program product according to any one of claims 14 to 16, wherein the teacher model and the student model are automatic speech recognition systems.

18. The computer program according to any one of claims 14 to 17, wherein the teacher model is a visual geometry group model.

19. 19. The computer program of claim 14, wherein the student model is a convolutional neural network.

Images