Speech recognition and model training method, device, equipment and computer program product

Abstract

The invention discloses a voice recognition and model training method, device and equipment and a computer program product, and relates to the technical field of voice recognition, a voice recognition model and a text reconstruction model are jointly trained, the voice recognition model and the text reconstruction model share the same decoder, the text reconstruction model further comprises a text encoder constructed on the basis of a large language model, and the voice recognition and model training efficiency is improved. According to the method, the text semantic representation can be extracted from the recognition text label, the feature alignment loss value between the audio semantic representation extracted from the audio sample by the audio encoder and the text semantic representation is calculated, the text recognition loss value between the text output by the decoder and the recognition text label is calculated, the large language model can extract rich semantic information, and the recognition accuracy is improved. Rich semantic information of the large language model can be migrated to the audio encoder through the feature alignment loss value, and the recognition effect of the speech recognition model in the specific field is improved. In the reasoning stage, the acoustic features of the voice signals to be recognized are input into the voice recognition model, and a voice recognition result output by the model is obtained.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and more specifically, to a speech recognition and model training method, device, equipment and computer program product. Background Art

[0002] In recent years, with the development of intelligent voice technology, end-to-end speech recognition systems have become a research hotspot and have been widely used in voice assistants, conference transcription, call transcription and other usage scenarios.

[0003] Compared with traditional speech recognition systems, end-to-end speech recognition systems implicitly integrate acoustic models, speech models, pronunciation dictionaries and other modules, directly learn the mapping relationship from speech signals to text, simplify the training and reasoning process, and avoid the error accumulation of cascade systems. With the rise of deep learning technology, especially the application of deep neural networks (DNN), the accuracy of end-to-end speech recognition systems has been significantly improved compared with traditional speech recognition systems. However, end-to-end speech recognition systems still face many challenges when facing specific (professional) fields, such as medical, legal, financial and other professional fields.

[0004] Since end-to-end speech recognition models rely on a large amount of training data to achieve good results, for some specific fields, on the one hand, due to the long-tail effect of data, the proportion of specific field data in the training data is usually very low. On the other hand, it is particularly difficult to obtain sufficient amount of speech text data in specific fields. Therefore, when facing specific fields, the recognition effect of the end-to-end speech recognition model will be greatly reduced. Summary of the invention

[0005] In view of the above problems, this application is proposed to provide a speech recognition and model training method, device, equipment and computer program product to improve the recognition effect of the speech recognition model in a specific field. The specific solution is as follows: In a first aspect of the present application, a speech recognition method is provided, comprising: Acquiring acoustic features of a speech signal to be recognized; The acoustic features are input into the trained speech recognition model to obtain the speech recognition results output by the model; wherein: The speech recognition model is jointly trained with the text reconstruction model during the training phase. The speech recognition model and the text reconstruction model share the same decoder. The text reconstruction model also includes a text encoder built based on a large language model. The joint training process uses audio samples as input to the audio encoder in the speech recognition model, and uses the recognized text labels corresponding to the audio samples as input to the text encoder. The total loss value of the joint training includes a text recognition loss value calculated based on the text output by the decoder and the recognized text label, and a feature alignment loss value calculated based on the audio semantic representation extracted by the audio encoder and the text semantic representation extracted by the text encoder.

[0006] A second aspect of the present application provides a speech recognition model training method, comprising: Acquire audio-text pair training data, wherein the audio-text pair training data includes audio samples and corresponding recognition text labels; The audio sample is used as the input of an audio encoder in a speech recognition model, and the recognized text label is used as the input of a text encoder constructed based on a large language model in a text reconstruction model, wherein the text reconstruction model and the speech recognition model share the same decoder, wherein the audio encoder is used to extract an audio semantic representation and feed it into the decoder with a first probability, wherein the text encoder is used to extract a text semantic representation and feed it into the decoder with a second probability, wherein the decoder is used to output a decoded text, and the sum of the first probability and the second probability is equal to 1; Calculating a text recognition loss value based on the text output by the decoder and the recognized text label, calculating a feature alignment loss value based on the audio semantic representation and the text semantic representation, and calculating a total loss value of the text recognition loss value and the feature alignment loss value; The model parameters are updated according to the total loss value until the training end condition is reached, thereby obtaining a trained speech recognition model.

[0007] In one possible design, in another implementation of the second aspect of the embodiment of the present application, the speech recognition model also includes a shared encoder located between the audio encoder and the decoder, and the shared encoder encodes the input features and sends the encoded features to the decoder; the audio semantic representation extracted by the audio encoder is sent to the shared encoder with a first probability, and the text semantic representation extracted by the text encoder is sent to the shared encoder with a second probability.

[0008] In one possible design, in another implementation of the second aspect of the embodiments of the present application, the speech recognition model further includes an alignment module; When the input of the shared encoder is the audio semantic representation, the alignment module is used to predict audio-text alignment information based on the audio semantic representation; The total loss value also includes: a temporal classification CTC loss calculated based on the audio-text alignment information and the recognition text label.

[0009] In one possible design, in another implementation of the second aspect of the embodiment of the present application, the text encoder includes a large language model, a resampling module, and a time length prediction module; The large language model is used to extract initial text semantic representation from the input text; When the input of the shared encoder is the audio semantic representation, the resampling module is used to resample the initial text semantic representation based on the audio-text alignment information output by the alignment module to obtain a resampled text semantic representation with the same frame number as the audio semantic representation, and calculate the feature alignment loss value using the resampled text semantic representation and the audio semantic representation; The duration prediction module is used to predict the duration of each frame in the initial text semantic representation to obtain duration prediction information; The total loss value also includes: a duration prediction loss value calculated based on the duration prediction information and the audio-text alignment information.

[0010] In a possible design, in another implementation of the second aspect of the embodiment of the present application, after updating the model parameters according to the total loss value until the training end condition is reached, it also includes: Obtain text corpus in the target domain; Sending the text corpus into the large language model to obtain a target text semantic representation of the text corpus; Predicting the duration of each frame in the semantic representation of the target text by the duration prediction module to obtain target duration prediction information; Resampling the target text semantic representation according to the target duration prediction information through the resampling module to obtain a resampled text semantic representation; Encoding the resampled text semantic representation through the shared encoder, sending the encoded features to the decoder, and predicting the decoded text through the decoder; A text reconstruction loss value is calculated based on the decoded text predicted by the decoder and the text corpus, and the model parameters are updated according to the text reconstruction loss value until the training end condition is reached. The audio encoder, the shared encoder and the decoder form a final speech recognition model.

[0011] In a possible design, in another implementation manner of the second aspect of the embodiment of the present application, after obtaining the target duration prediction information, the method further includes: The target duration prediction information is adjusted according to the configured duration disturbance parameter to obtain the disturbed target duration prediction information to guide the resampling module to perform resampling.

[0012] In a possible design, in another implementation of the second aspect of the embodiment of the present application, the process of acquiring text corpus in the target domain includes: The first prompt instruction prompt is sent to the large language model to obtain the text corpus of the target domain generated by the large language model, wherein the first prompt instruction is used to instruct the model to generate the text corpus of the target domain.

[0013] In a third aspect of the present application, a speech recognition device is provided, comprising: An acoustic feature acquisition unit, used to acquire acoustic features of a speech signal to be recognized; A model recognition unit is used to input the acoustic features into the trained speech recognition model to obtain the speech recognition results output by the model; wherein: The speech recognition model is jointly trained with the text reconstruction model during the training phase. The speech recognition model and the text reconstruction model share the same decoder. The text reconstruction model also includes a text encoder built based on a large language model. The joint training process uses audio samples as input to the audio encoder in the speech recognition model, and uses the recognized text labels corresponding to the audio samples as input to the text encoder. The total loss value of the joint training includes a text recognition loss value calculated based on the text output by the decoder and the recognized text label, and a feature alignment loss value calculated based on the audio semantic representation extracted by the audio encoder and the text semantic representation extracted by the text encoder.

[0014] In a fourth aspect of the present application, a speech recognition model training device is provided, comprising: A training data acquisition unit, used to acquire audio-text pair training data, wherein the audio-text pair training data includes an audio sample and a corresponding recognition text label; A model calculation unit, used to use the audio sample as an input of an audio encoder in a speech recognition model, and use the recognized text label as an input of a text encoder constructed based on a large language model in a text reconstruction model, wherein the text reconstruction model and the speech recognition model share the same decoder, wherein the audio encoder is used to extract an audio semantic representation and feed it into the decoder with a first probability, wherein the text encoder is used to extract a text semantic representation and feed it into the decoder with a second probability, wherein the decoder is used to output a decoded text, and wherein the sum of the first probability and the second probability is equal to 1; a loss value calculation unit, configured to calculate a text recognition loss value based on the text output by the decoder and the recognized text label, calculate a feature alignment loss value based on the audio semantic representation and the text semantic representation, and calculate a total loss value of the text recognition loss value and the feature alignment loss value; A parameter updating unit is used to update the model parameters according to the total loss value until the training end condition is reached to obtain a trained speech recognition model.

[0015] In a fifth aspect of the present application, an electronic device is provided, comprising: a memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement the speech recognition method described in the first aspect of the present application, or to implement the various steps of the speech recognition model training method described in any one of the second aspects of the present application.

[0016] The sixth aspect of the present application provides a readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, it implements the speech recognition method described in the first aspect of the present application, or implements the various steps of the speech recognition model training method described in any one of the second aspects of the present application.

[0017] The seventh aspect of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the speech recognition method described in the first aspect of the present application, or implements the various steps of the speech recognition model training method described in any one of the second aspects of the present application.

[0018] With the above technical solution, the present application adopts a joint training strategy to jointly train the speech recognition model and the text reconstruction model, and the two share the same decoder. The text reconstruction model also includes a text encoder built based on a large language model, which can extract text semantic representations for the recognized text labels corresponding to the audio samples. During the training process, the feature alignment loss value between the audio semantic representation extracted by the audio encoder for the audio sample and the text semantic representation is calculated, and the text recognition loss value between the text output by the decoder and the recognized text label is calculated. The semantic representations of audio and text are unified into the same semantic representation space through the feature alignment loss value. At the same time, since the large language model can extract rich semantic information, the rich semantic information of the large language model can be migrated to the audio encoder through the feature alignment loss value, so that the audio encoder can extract richer audio semantic representations, thereby improving the domain customization capability of the speech recognition model, that is, improving the recognition effect of the trained speech recognition model in a specific domain. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 A flow chart of a speech recognition model training method provided in an embodiment of the present application; Figure 2 An example of a speech recognition model training architecture diagram is shown; Figure 3 Another example of a speech recognition model training architecture diagram is shown; Figure 4 Another example of a speech recognition model training architecture diagram is shown; Figure 5 Another example of a speech recognition model training architecture diagram is shown; Figure 6 A schematic diagram of the structure of a speech recognition device provided in an embodiment of the present application; Figure 7 A schematic diagram of the structure of a speech recognition model training device provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] Before introducing this application solution, the relevant concepts involved in this article are first explained: prompt: Instructions. When interacting with AI (such as an artificial intelligence model), you need to send instructions to AI. It can be a text description, such as "Please recommend me a pop song" when you interact with AI, or it can be a parameter description in a certain format. For example, if you want AI to draw in a certain format, you need to describe the relevant drawing parameters.

[0021] Large language model: (LLM) generally refers to a language model with a large number of parameters and capabilities, which learns the statistical laws and semantic relations of language by pre-training on large-scale text data. These models usually use unsupervised learning methods to predict the next word or fill in missing words to capture the context and semantic information of the language. Large language models are able to generate coherent sentences, answer questions, complete translation tasks, etc. LLMs are characterized by their large scale, containing billions or even more parameters, which help them learn complex patterns in language data. The emerging capabilities of large language models include contextual learning, instruction following, and step-by-step reasoning capabilities.

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] Traditional end-to-end speech recognition systems face many challenges when faced with specific fields, such as medical, legal, financial and other professional fields.

[0024] These challenges mainly stem from the following aspects: First, since the terms and concepts in professional fields are often quite different from everyday language, their frequency of appearance in training data is extremely low, resulting in the end-to-end speech recognition system usually being unable to accurately identify these professional terms and concepts (such as diseases and drugs in the medical field, professional terms in the financial field, etc.); Second, although it is possible to construct domain-specific labeled data to perform domain adaptation training on the end-to-end speech recognition system, the labeling of domain-specific data requires huge costs, and some fields are highly open, and the labeled data is difficult to cover all domain scenarios, resulting in insufficient generalization ability of the model when facing domain scenarios.

[0025] In order to improve the recognition effect of the end-to-end speech recognition system in a specific field, one solution is to use the text corpus of the target field to train an external language model (Language Model), and integrate it into the end-to-end speech recognition model during the inference (decoding) process. In the decoding stage, the probability of the external language model is combined with the probability of the end-to-end speech recognition model to increase the probability of the decoding results related to the target field, thereby improving the recognition effect of the end-to-end speech recognition system in the target field.

[0026] Although this method can improve the recognition effect in the target domain, it involves the use of an external language model during inference, introduces a more complex model architecture, greatly increases the computational cost of the decoding process, slows down the inference speed, and cannot be applied to offline devices with limited computing resources. In addition, this method also requires the adjustment of additional hyperparameters, which increases the complexity of training and deployment of end-to-end speech recognition systems.

[0027] In order to avoid introducing an external language model (LM) in the decoding process and thus increasing the computational cost, one solution is to use text to speech (TTS) technology to generate paired speech and text data based on the target domain text library, and then use this data to train or fine-tune the end-to-end speech recognition system to improve the recognition effect of the end-to-end speech recognition system in the target domain.

[0028] However, the above solution requires the introduction of an additional TTS system. The generated voice data is usually based on rules or deep learning algorithms, which cannot fully simulate the voice changes and complexity in the real world. The data authenticity and diversity are far behind the real audio data, resulting in a decline in the effectiveness of the voice recognition system in real-world applications.

[0029] To this end, the present application further provides a speech recognition method and a speech recognition model training method to improve at least some of the problems existing in the aforementioned examples and enhance the recognition effect of the speech recognition model in specific fields.

[0030] The following embodiments of the present application provide a speech recognition method and a speech recognition model training method, respectively. The speech recognition model training method provided can be applied to an electronic device. For example, the electronic device can be, for example, a server, a robot, a smart phone, a personal computer (PC), a laptop computer, a wireless electronic device in an industrial control, etc.

[0031] The method provided in the present application can be divided into a training phase and an inference phase. The training phase is the phase of training the speech recognition model, and the inference phase is the phase of using the trained speech recognition model to perform speech recognition on the speech signal to be recognized. Among them, the training phase and the inference phase can be deployed in the same device or in different devices. For example, the training phase can be deployed in the cloud or in a server, and the inference phase can be deployed in an intelligent terminal, such as a mobile phone, a tablet, a voice recorder, a translator, a robot, a vehicle-mounted terminal, or a wearable device.

[0032] For ease of understanding, this application introduces the processes of the training phase and the reasoning phase respectively.

[0033] 1. Training Phase It should be noted that, generally, the training process of the model can be divided into multiple iterations or multiple executions. This application exemplarily takes one of the iterations or execution processes as an example for illustrative introduction. The steps mentioned below can be repeated and will not be repeated below.

[0034] See also Figure 1 , a flow chart of a method for training a speech recognition model provided in the present application is as follows, the method comprising: Step S100: Acquire audio-text pair training data, where the audio-text pair training data includes audio samples and corresponding recognition text labels.

[0035] Specifically, audio samples and paired recognition text labels may be collected to form audio-text pair training data.

[0036] Step S110: jointly train the speech recognition model and the text reconstruction model using the audio text training data. The two models share the same decoder. The text reconstruction model also includes a text encoder built based on a large language model. The joint training process uses audio samples as input to the audio encoder in the speech recognition model and recognizes text tags as input to the text encoder.

[0037] Specifically, combined Figure 2 As shown, the speech recognition model may include an audio encoder and a decoder, wherein the audio sample is used as the input of the audio encoder, and the audio semantic representation of the audio sample is extracted through the audio encoder.

[0038] The text reconstruction model includes a text encoder and a decoder, and the speech recognition model and the text reconstruction model share the same decoder.

[0039] The text encoder is built based on a large language model, specifically the ability of the large language model to extract rich text semantic representations. During the training phase, the recognized text label corresponding to the audio sample can be input into the text encoder to obtain the text semantic representation extracted by the text encoder. In this embodiment, in order to facilitate the subsequent steps to calculate the feature alignment loss between the text semantic representation and the audio semantic representation, the text semantic representation extracted by the text encoder here can be a feature of the same dimension as the audio semantic representation.

[0040] For a piece of audio-text pair training data, only one feature of the audio semantic representation extracted by the audio encoder and the text semantic representation extracted by the text encoder can be sent to the decoder. In this embodiment, the first probability P1 and the second probability (1-P1) can be pre-set, and the audio semantic representation is sent to the decoder with the first probability P1, and the text semantic representation is sent to the decoder with the second probability (1-P1). The size of P1 can be set by the user, and the value range is (0,1).

[0041] The decoder decodes the input feature representation to obtain the output text.

[0042] It can be understood that when the input of the decoder is an audio semantic representation, the text output by the decoder can be regarded as a speech recognition result; when the input of the decoder is a text semantic representation, the text output by the decoder can be regarded as a reconstructed text.

[0043] Step S120, calculating a text recognition loss value based on the text output by the decoder and the recognized text label, calculating a feature alignment loss value based on the audio semantic representation and the text semantic representation, and calculating a total loss value of the text recognition loss value and the feature alignment loss value.

[0044] Combination Figure 2As shown, when the input of the decoder is an audio semantic representation, the text recognition loss value calculated between the decoder output text and the recognized text label can be a speech recognition loss value, and when the input of the decoder is a text semantic representation, the text recognition loss value calculated between the decoder output text and the recognized text label can be a text reconstruction loss value. In order to simplify the expression in this embodiment, the text recognition loss function is defined as ASR_Loss.

[0045] The text recognition loss value can be used to update the parameters of the encoder and decoder as a whole.

[0046] For the audio semantic representation output by the audio encoder and the text semantic representation output by the text encoder, calculate the feature alignment loss value between the two, and combine Figure 2 As shown, the feature alignment loss function is defined as MSE_Loss.

[0047] Under the constraint of the feature alignment loss function, the rich semantic information of the large language model (the text encoder is built based on the large language model and has the feature processing capabilities of the large language model) can be migrated to the audio encoder, allowing the audio encoder to extract richer audio semantic representations. Under the premise of limited audio-text pair training data, the text data can be fully utilized to improve the feature extraction capability of the audio encoder and improve the recognition effect of the speech recognition model in specific fields.

[0048] Step S130, updating the model parameters according to the total loss value until the training end condition is reached, thereby obtaining a trained speech recognition model.

[0049] Specifically, when updating the parameters of the speech recognition model and the text reconstruction model according to the above total loss value, for the large language model in the text encoder, a pre-trained large language model can be used, which does not participate in the parameter update process. Of course, in some other optional implementations, the large language model can also be combined with other network modules to participate in parameter update.

[0050] After the set training end condition is reached, the trained speech recognition model can be obtained.

[0051] Obviously, the speech recognition model training method provided in this embodiment adopts a text reconstruction model to assist in the joint training of the speech recognition model, and unifies the semantic representations of audio and text into the same semantic representation space through the feature alignment loss value. At the same time, since the large language model can extract rich semantic information, the rich semantic information of the large language model can be migrated to the audio encoder through the feature alignment loss value, so that the audio encoder can extract richer audio semantic representations, thereby improving the domain customization capability of the speech recognition model, that is, improving the recognition effect of the trained speech recognition model in a specific domain.

[0052] The speech recognition model training method provided in this embodiment does not require the language model to be loaded inside or outside the trained speech recognition model, which simplifies the structure of the speech recognition model and reduces the time consumption of reasoning. In addition, the present application does not need to synthesize audio text training data through audio synthesis technology to train the speech recognition model. By making full use of the text data in the existing audio text training data, the audio encoder is guided to extract richer audio semantic representations, thereby improving the field customization capability of the speech recognition model and improving the recognition effect of the speech recognition model in a specific field.

[0053] Reference Figure 3 , another speech recognition model training architecture is provided. Compared with the previous embodiment, the speech recognition model in this embodiment can further include a shared encoder. During the training phase, the shared encoder is shared by the speech recognition model and the text reconstruction model.

[0054] The shared encoder is located between the audio encoder and decoder, and also between the text encoder and decoder. The audio semantic representation extracted by the audio encoder is fed into the shared encoder with a first probability P1, and the text semantic representation extracted by the text encoder is fed into the shared encoder with a second probability (1-P1).

[0055] The shared encoder re-encodes the input feature representation and sends the encoded features to the decoder.

[0056] In this embodiment, a shared encoder is added between the audio encoder and decoder of the speech recognition model. In the joint training stage, when the text semantic representation is sent to the shared encoder, the loss function is the text reconstruction loss function. When the parameters are updated according to the text reconstruction loss function, the parameters of the decoder and the shared encoder can be updated at the same time, that is, the network module involved in the parameter update further adds a shared encoder. In the end-to-end speech recognition model, the role of the encoder is usually greater than that of the decoder. Therefore, by adding a shared encoder to participate in the parameter update, the shared encoder can also perform domain fine-tuning, further improving the recognition effect of the trained speech recognition model on specific fields.

[0057] Reference Figure 4 , provides another speech recognition model training architecture. Compared with the aforementioned embodiment, the speech recognition model in this embodiment can further include an alignment module.

[0058] When the input of the shared encoder is an audio semantic representation, the alignment module is used to predict the audio-text alignment information based on the audio semantic representation, that is, the alignment relationship between each frame in the audio sample and each word segmentation unit in the text.

[0059] On this basis, the temporal classification CTC loss can be calculated based on the audio-text alignment information and the recognition text label, and the total loss value of the joint training process can further include the CTC loss value.

[0060] In this embodiment, by adding the CTC loss function, the speech recognition model can be guided to learn the alignment relationship between audio and text, thereby improving the recognition effect of the speech recognition model.

[0061] It should be noted that according to Figure 4 After the architecture training is completed, only the audio encoder, shared encoder and decoder can be taken to form the final speech recognition model for subsequent reasoning.

[0062] Reference Figure 4 As shown, for the text encoder introduced in the above-mentioned embodiment, this embodiment provides an optional composition structure of the text encoder, which may include: a large language model, a resampling module and a time length prediction module.

[0063] The large language model is used to extract the initial text semantic representation from the input text.

[0064] When the input of the shared encoder is an audio semantic representation, the resampling module is used to resample the initial text semantic representation based on the audio-text alignment information output by the alignment module to obtain a resampled text semantic representation with the same frame number as the audio semantic representation.

[0065] Specifically, due to the difference in the granularity of modeling units, text sequences are usually converted into unit sequences such as character sequences, pinyin sequences, and phoneme sequences and sent to the large language model. The semantic representation of the text sequence extracted by the large language model usually has a smaller frame number than the audio semantic representation extracted by the acoustic encoder. In this embodiment, a resampling module is used to expand the frames of the text semantic representation so that its frame number is consistent with the audio semantic representation, and then the resampled text semantic representation and audio semantic representation are used to calculate the feature alignment loss MSE_Loss.

[0066] The duration prediction module is used to predict the duration of each frame in the initial text semantic representation and obtain the duration prediction information. On this basis, the duration prediction loss value can be calculated based on the duration prediction information and the audio text alignment information output by the alignment module, and the duration prediction loss function is defined as Duration_Loss.

[0067] The total loss value of the joint training process may also include the above-mentioned duration prediction loss value.

[0068] In summary, the speech recognition model training process provided in this embodiment is jointly trained with the text reconstruction model. The total loss function of the joint training process can be expressed as: Losstotal =ASR_Loss+αMSE_Loss+βCTC_Loss+γDuration_Loss Among them, α, β, and γ are the weights of MSE_Loss, CTC_Loss, and Duration_Loss respectively.

[0069] The speech recognition model training method provided in this embodiment designs the structure of the text encoder, ensures that it has the rich semantic feature extraction capability of the large language model, and guides the resampling module to perform feature resampling through the alignment information output by the alignment module, so that the dimension of the resampled text semantic representation is the same as that of the audio semantic representation, thereby facilitating the calculation of MSE_Loss. In addition, the text encoder also includes a duration prediction module, which can perform duration prediction on the initial text semantic representation extracted by the large language model, and calculate the duration prediction loss value by combining the duration prediction information with the audio text alignment information output by the alignment module, so as to guide the training process of the duration prediction module.

[0070] When there is a lack of training data for audio text, only text data in the target domain can be obtained. After the initial text semantic representation is extracted through the large language model, the duration prediction module predicts the duration information of each frame feature in the initial text semantic representation, and then guides the resampling module to perform resampling. Since the aforementioned training stage maps the text semantic representation and the audio semantic representation to the same semantic representation space, the resampled text semantic representation can simulate the audio semantic representation extracted by the corresponding audio encoder, and be sent to the subsequent shared encoder and decoder for processing, output the decoded text, and calculate the text recognition loss value between the decoded text and the input text. The model parameter update process is guided by the text recognition loss value, which can greatly improve the domain adaptation effect of the shared encoder in the absence of corresponding audio.

[0071] After the training is completed according to the training strategy of the above-mentioned embodiment, the present embodiment may further include a process of performing domain adaptive training on the speech recognition model using target domain text.

[0072] Combination Figure 5 As shown in Figure 2, at this stage, there is only domain text but no corresponding audio, so Figure 4 In the corresponding training architecture, the network modules related to the audio branch are removed, and the training architecture of this stage is as follows Figure 5 As shown, it includes a large language model, a duration prediction module, a resampling module, a shared encoder and a decoder.

[0073] Through the training strategy introduced in the above embodiment, the duration prediction module has the ability to predict duration information.

[0074] The domain adaptation training phase can include the following training steps: S1. Obtain text corpus in the target domain.

[0075] Specifically, the target domain may be the domain in which the speech recognition model is to be applied. For example, if the speech recognition model is to be applied in the medical field, the medical field may be used as the target domain in this step.

[0076] For scenarios where audio-text pairing data in the target domain is scarce, in this embodiment, only text corpus in the target domain may be obtained. Compared with audio data, text corpus is easier to obtain.

[0077] In this step, the text corpus of the target domain can be obtained from a public data set or automatically generated. This embodiment provides a method for automatically generating text corpus of the target domain, which can make use of the text generation capability of a large language model, specifically: The first prompt instruction prompt is sent to the large language model to obtain the text corpus of the target domain generated by the large language model, wherein the first prompt instruction is used to instruct the model to generate the text corpus of the target domain.

[0078] The first prompt instruction prompt may include the target domain theme, reference keywords, reference examples, word count requirements for the generated text, text magnitude requirements, etc. An example of the first prompt instruction prompt is provided below: "The target field is medical conferences. Related topics include Topic 1, Topic 2, Topic 3, etc. Possible keywords include Keyword 1, Keyword 2, Keyword 3, etc. Typical sample texts in this field are Sample Text 1, Sample Text 2, Sample Text 3. Please generate 1,000 lines of text related to this field based on the above information, with each line of text containing 10 to 50 words."

[0079] S2. Send the text corpus into the large language model to obtain the target text semantic representation of the text corpus.

[0080] Combination Figure 5 As shown, the text corpus generated in the previous step is used as an input text sequence and input into the large language model, and the large language model is used to extract the target text semantic representation of the input text corpus.

[0081] S3. Predict the duration of each frame in the semantic representation of the target text through the duration prediction module to obtain target duration prediction information.

[0082] After the training process of the aforementioned embodiment, the duration prediction module has acquired the duration prediction capability. In this step, the duration prediction module can be used to predict the duration of each frame in the semantic representation of the target text to obtain target duration prediction information.

[0083] S4. Through the resampling module, the target text semantic representation is resampled according to the target duration prediction information to obtain the resampled text semantic representation.

[0084] Specifically, the resampling module resamples the target text semantic representation under the guidance of the target duration prediction information to obtain the resampled text semantic representation. Since the aforementioned training phase maps the text semantic representation and the audio semantic representation to the same semantic representation space, the resampled text semantic representation can simulate the audio semantic representation extracted by the corresponding audio encoder.

[0085] S5. Encode the resampled text semantic representation through the shared encoder, send the encoded features to the decoder, and predict the decoded text through the decoder.

[0086] S6. Calculate the text reconstruction loss value based on the decoded text and text corpus predicted by the decoder, and update the model parameters according to the text reconstruction loss value until the training end condition is reached. The final speech recognition model is composed of the audio encoder, shared encoder and decoder.

[0087] As in the previous article, the text reconstruction loss function is defined as ASR_Loss. The text corpus obtained in the aforementioned step S1 can be used as the text label to calculate the text reconstruction loss value between the decoded text predicted by the decoder and the text label. The text reconstruction loss function can adopt a variety of loss function types, such as the cross entropy CE loss function.

[0088] After the above-mentioned domain adaptation training stage, the shared encoder and decoder can be domain-adaptively fine-tuned using the text corpus of the target domain, further improving the recognition effect of the speech recognition model in the target domain.

[0089] In the above domain adaptation training stage, the parameters of the large language model can be updated synchronously. The loss function of the domain adaptation training stage can be expressed as: Loss total =ASR_Loss Furthermore, before the above steps S3 and S4, the following steps may be further added: The target duration prediction information is adjusted according to the configured duration disturbance parameters to obtain the disturbed target duration prediction information to guide the resampling module in step S4 to perform resampling.

[0090] That is, in the domain adaptation training stage, by adding appropriate duration perturbation parameters to the duration prediction module (such as multiplying the predicted duration of each frame by a parameter greater than 1 or less than 1 with the set probability P2), the generated resampled text semantic representation can be made richer, thereby better training the shared encoder and decoder.

[0091] To sum up, the speech recognition model training method provided by the present application, on the one hand, does not require an external language model for decoding fusion, does not increase the amount of inference calculations, and does not increase the complexity of training and deployment; on the other hand, with the help of the rich semantic representation capabilities of the large language model, the audio semantic representation and the text semantic representation are mapped to the same semantic representation space, and the rich semantic feature extraction capabilities of the large language model are migrated to the audio encoder, thereby improving the semantic perception ability of the audio encoder.

[0092] In addition, the present application further adds a domain adaptation training stage, and uses text corpus in the target domain to adaptively train the model. This process can simultaneously perform domain fine-tuning on the shared encoder and decoder, thereby improving the customization capability of the speech recognition model for the target domain. It can simulate the diversity of audio semantic representation in the absence of audio text training data, and achieve better domain adaptation effects.

[0093] On the basis of the speech recognition model training method introduced in any of the above embodiments, the speech to be recognized can be recognized based on the trained speech recognition model.

[0094] In some embodiments of the present application, a speech recognition method is further provided, which may include the following steps: S1. Acquire acoustic features of a speech signal to be recognized.

[0095] S2. Input the acoustic features into the trained speech recognition model to obtain the speech recognition results output by the model; wherein: The speech recognition model is jointly trained with the text reconstruction model during the training phase. The speech recognition model and the text reconstruction model share the same decoder. The text reconstruction model also includes a text encoder built based on a large language model. The joint training process uses audio samples as input to the audio encoder in the speech recognition model, and uses the recognized text labels corresponding to the audio samples as input to the text encoder. The total loss value of the joint training includes a text recognition loss value calculated based on the text output by the decoder and the recognized text label, and a feature alignment loss value calculated based on the audio semantic representation extracted by the audio encoder and the text semantic representation extracted by the text encoder.

[0096] For the training process of the speech recognition model, details may be referred to the introduction of the speech recognition model training process in the aforementioned embodiments, which will not be repeated in this embodiment.

[0097] The speech recognition model trained by the above training strategy has the ability to be customized to the target domain. Therefore, for the speech signal to be recognized in the target domain, the speech recognition model can be used to obtain a better recognition effect.

[0098] The following is a description of a speech recognition device provided in an embodiment of the present application. The speech recognition device described below and the speech recognition method described above can be referenced to each other.

[0099] See also Figure 6 , Figure 6 A schematic diagram of the structure of a speech recognition device disclosed in an embodiment of the present application.

[0100] like Figure 6 As shown, the device may include: An acoustic feature acquisition unit 11, used to acquire acoustic features of a speech signal to be recognized; The model recognition unit 12 is used to input the acoustic features into the trained speech recognition model to obtain the speech recognition results output by the model; wherein: The speech recognition model is jointly trained with the text reconstruction model during the training phase. The speech recognition model and the text reconstruction model share the same decoder. The text reconstruction model also includes a text encoder built based on a large language model. The joint training process uses audio samples as input to the audio encoder in the speech recognition model, and uses the recognized text labels corresponding to the audio samples as input to the text encoder. The total loss value of the joint training includes a text recognition loss value calculated based on the text output by the decoder and the recognized text label, and a feature alignment loss value calculated based on the audio semantic representation extracted by the audio encoder and the text semantic representation extracted by the text encoder.

[0101] Furthermore, the speech recognition model training device provided in the embodiment of the present application is described. The speech recognition model training device described below and the speech recognition model training method described above can refer to each other.

[0102] See also Figure 7 , Figure 7 A schematic diagram of the structure of a speech recognition model training device disclosed in an embodiment of the present application.

[0103] like Figure 7 As shown, the device may include: A training data acquisition unit 21 is used to acquire audio-text pair training data, wherein the audio-text pair training data includes an audio sample and a corresponding recognition text label; A model calculation unit 22 is used to use the audio sample as an input of an audio encoder in a speech recognition model, and use the recognized text label as an input of a text encoder constructed based on a large language model in a text reconstruction model, wherein the text reconstruction model and the speech recognition model share the same decoder, wherein the audio encoder is used to extract an audio semantic representation and feed it into the decoder with a first probability, wherein the text encoder is used to extract a text semantic representation and feed it into the decoder with a second probability, wherein the decoder is used to output a decoded text, and wherein the sum of the first probability and the second probability is equal to 1; A loss value calculation unit 23, used to calculate a text recognition loss value based on the text output by the decoder and the recognized text label, calculate a feature alignment loss value based on the audio semantic representation and the text semantic representation, and calculate a total loss value of the text recognition loss value and the feature alignment loss value; The parameter updating unit 24 is used to update the model parameters according to the total loss value until the training end condition is reached to obtain a trained speech recognition model.

[0104] In one possible implementation, the speech recognition model also includes a shared encoder located between the audio encoder and the decoder, which encodes the input features and sends the encoded features to the decoder; the audio semantic representation extracted by the audio encoder is sent to the shared encoder with a first probability, and the text semantic representation extracted by the text encoder is sent to the shared encoder with a second probability.

[0105] In a possible implementation, the speech recognition model further includes an alignment module; When the input of the shared encoder is the audio semantic representation, the alignment module is used to predict audio-text alignment information based on the audio semantic representation; The total loss value also includes: a temporal classification CTC loss calculated based on the audio-text alignment information and the recognition text label.

[0106] In one possible implementation, the text encoder includes a large language model, a resampling module, and a time length prediction module; The large language model is used to extract initial text semantic representation from the input text; When the input of the shared encoder is the audio semantic representation, the resampling module is used to resample the initial text semantic representation based on the audio-text alignment information output by the alignment module to obtain a resampled text semantic representation with the same frame number as the audio semantic representation, and calculate the feature alignment loss value using the resampled text semantic representation and the audio semantic representation; The duration prediction module is used to predict the duration of each frame in the initial text semantic representation to obtain duration prediction information; The total loss value also includes: a duration prediction loss value calculated based on the duration prediction information and the audio-text alignment information.

[0107] In a possible implementation, the apparatus of the present application may further include: a domain adaptive fine-tuning training unit, configured to further adopt a domain adaptive fine-tuning training strategy to perform model training after the parameter updating unit updates the model parameters according to the total loss value until the training end condition is reached. The process may include: Obtain text corpus in the target domain; Sending the text corpus into the large language model to obtain a target text semantic representation of the text corpus; Predicting the duration of each frame in the semantic representation of the target text by the duration prediction module to obtain target duration prediction information; Resampling the target text semantic representation according to the target duration prediction information through the resampling module to obtain a resampled text semantic representation; Encoding the resampled text semantic representation through the shared encoder, sending the encoded features to the decoder, and predicting the decoded text through the decoder; A text reconstruction loss value is calculated based on the decoded text predicted by the decoder and the text corpus, and the model parameters are updated according to the text reconstruction loss value until the training end condition is reached. The audio encoder, the shared encoder and the decoder form a final speech recognition model.

[0108] In one possible implementation, after obtaining the target duration prediction information, the domain adaptive fine-tuning training unit can also be used to adjust the target duration prediction information according to the configured duration perturbation parameters to obtain the perturbed target duration prediction information to guide the resampling module to perform resampling.

[0109] In a possible implementation, the process of the domain adaptive fine-tuning training unit acquiring the text corpus of the target domain includes: The first prompt instruction prompt is sent to the large language model to obtain the text corpus of the target domain generated by the large language model, wherein the first prompt instruction is used to instruct the model to generate the text corpus of the target domain.

[0110] The present application also provides an electronic device in an embodiment. Figure 8As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as servers, personal computers, mobile phones, etc. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0111] like Figure 8 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603 to implement the speech recognition method or speech recognition model training method of the aforementioned embodiment of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0112] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0113] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any speech recognition method or speech recognition model training method provided in the embodiment of the present application.

[0114] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement any speech recognition method or speech recognition model training method provided in an embodiment of the present application.

[0115] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0116] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0117] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0118] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0119] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

Claims

1. A speech recognition method, characterized in that: include: Acquiring acoustic features of a speech signal to be recognized; The acoustic features are input into the trained speech recognition model to obtain the speech recognition results output by the model; wherein: The speech recognition model is jointly trained with the text reconstruction model during the training phase. The speech recognition model and the text reconstruction model share the same decoder. The text reconstruction model also includes a text encoder built based on a large language model. The joint training process uses audio samples as input to the audio encoder in the speech recognition model, and uses the recognized text labels corresponding to the audio samples as input to the text encoder. The total loss value of the joint training includes a text recognition loss value calculated based on the text output by the decoder and the recognized text label, and a feature alignment loss value calculated based on the audio semantic representation extracted by the audio encoder and the text semantic representation extracted by the text encoder.

2. A speech recognition model training method, characterized in that: include: Acquire audio-text pair training data, wherein the audio-text pair training data includes audio samples and corresponding recognition text labels; The audio sample is used as the input of an audio encoder in a speech recognition model, and the recognized text label is used as the input of a text encoder constructed based on a large language model in a text reconstruction model, wherein the text reconstruction model and the speech recognition model share the same decoder, wherein the audio encoder is used to extract an audio semantic representation and feed it into the decoder with a first probability, wherein the text encoder is used to extract a text semantic representation and feed it into the decoder with a second probability, wherein the decoder is used to output a decoded text, and the sum of the first probability and the second probability is equal to 1; Calculating a text recognition loss value based on the text output by the decoder and the recognized text label, calculating a feature alignment loss value based on the audio semantic representation and the text semantic representation, and calculating a total loss value of the text recognition loss value and the feature alignment loss value; The model parameters are updated according to the total loss value until the training end condition is reached, thereby obtaining a trained speech recognition model.

3. The method according to claim 2, characterized in that The speech recognition model also includes a shared encoder located between the audio encoder and the decoder, which encodes input features and sends the encoded features to the decoder; the audio semantic representation extracted by the audio encoder is sent to the shared encoder with a first probability, and the text semantic representation extracted by the text encoder is sent to the shared encoder with a second probability.

4. The method according to claim 3, characterized in that The speech recognition model also includes an alignment module; When the input of the shared encoder is the audio semantic representation, the alignment module is used to predict audio-text alignment information based on the audio semantic representation; The total loss value also includes: a temporal classification CTC loss calculated based on the audio-text alignment information and the recognition text label.

5. The method according to claim 4, characterized in that The text encoder includes a large language model, a resampling module and a time length prediction module; The large language model is used to extract initial text semantic representation from the input text; When the input of the shared encoder is the audio semantic representation, the resampling module is used to resample the initial text semantic representation based on the audio-text alignment information output by the alignment module to obtain a resampled text semantic representation with the same frame number as the audio semantic representation, and calculate the feature alignment loss value using the resampled text semantic representation and the audio semantic representation; The duration prediction module is used to predict the duration of each frame in the initial text semantic representation to obtain duration prediction information; The total loss value also includes: a duration prediction loss value calculated based on the duration prediction information and the audio-text alignment information.

6. The method according to claim 5, characterized in that After updating the model parameters according to the total loss value until the training end condition is reached, the method further includes: Obtain text corpus in the target domain; Sending the text corpus into the large language model to obtain a target text semantic representation of the text corpus; Predicting the duration of each frame in the semantic representation of the target text by the duration prediction module to obtain target duration prediction information; Resampling the target text semantic representation according to the target duration prediction information through the resampling module to obtain a resampled text semantic representation; Encoding the resampled text semantic representation through the shared encoder, sending the encoded features to the decoder, and predicting the decoded text through the decoder; A text reconstruction loss value is calculated based on the decoded text predicted by the decoder and the text corpus, and the model parameters are updated according to the text reconstruction loss value until the training end condition is reached. The audio encoder, the shared encoder and the decoder form a final speech recognition model.

7. The method according to claim 6, characterized in that After obtaining the target duration prediction information, it also includes: The target duration prediction information is adjusted according to the configured duration disturbance parameter to obtain the disturbed target duration prediction information to guide the resampling module to perform resampling.

8. The method according to claim 6 or 7, characterized in that: The process of obtaining text corpus in the target domain includes: The first prompt instruction prompt is sent to the large language model to obtain the text corpus of the target domain generated by the large language model, wherein the first prompt instruction is used to instruct the model to generate the text corpus of the target domain.

9. A speech recognition device, characterized in that: include: An acoustic feature acquisition unit, used to acquire acoustic features of a speech signal to be recognized; A model recognition unit is used to input the acoustic features into the trained speech recognition model to obtain the speech recognition results output by the model; wherein: The speech recognition model is jointly trained with the text reconstruction model during the training phase. The speech recognition model and the text reconstruction model share the same decoder. The text reconstruction model also includes a text encoder built based on a large language model. The joint training process uses audio samples as input to the audio encoder in the speech recognition model, and uses the recognized text labels corresponding to the audio samples as input to the text encoder. The total loss value of the joint training includes a text recognition loss value calculated based on the text output by the decoder and the recognized text label, and a feature alignment loss value calculated based on the audio semantic representation extracted by the audio encoder and the text semantic representation extracted by the text encoder.

10. A speech recognition model training device, characterized in that: include: A training data acquisition unit, used to acquire audio-text pair training data, wherein the audio-text pair training data includes an audio sample and a corresponding recognition text label; A model calculation unit, used to use the audio sample as an input of an audio encoder in a speech recognition model, and use the recognized text label as an input of a text encoder constructed based on a large language model in a text reconstruction model, wherein the text reconstruction model and the speech recognition model share the same decoder, wherein the audio encoder is used to extract an audio semantic representation and feed it into the decoder with a first probability, wherein the text encoder is used to extract a text semantic representation and feed it into the decoder with a second probability, wherein the decoder is used to output a decoded text, and wherein the sum of the first probability and the second probability is equal to 1; a loss value calculation unit, configured to calculate a text recognition loss value based on the text output by the decoder and the recognized text label, calculate a feature alignment loss value based on the audio semantic representation and the text semantic representation, and calculate a total loss value of the text recognition loss value and the feature alignment loss value; A parameter updating unit is used to update the model parameters according to the total loss value until the training end condition is reached to obtain a trained speech recognition model.

11. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the speech recognition method as described in claim 1, or to implement each step of the speech recognition model training method as described in any one of claims 2 to 8.

12. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the speech recognition method as claimed in claim 1, or implements the various steps of the speech recognition model training method as claimed in any one of claims 2 to 8.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the speech recognition method as claimed in claim 1, or implements the various steps of the speech recognition model training method as claimed in any one of claims 2 to 8.

Images