Speech recognition methods, devices, electronic equipment and storage media

By combining acoustic, text, and language embedding vectors into a target speech recognition model, and training the encoder and predictor, the problem of multilingual recognition in monolingual models is solved, achieving higher recognition accuracy and wider applicability.

CN117765927BActive Publication Date: 2026-11-10GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202311672423.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-06
Publication Date
2026-11-10
Estimated Expiration
2043-12-06

AI Technical Summary

Technical Problem

Existing automatic speech recognition models only support single-language recognition, resulting in poor accuracy and reliability in multilingual recognition, and thus failing to meet the needs of multilingual recognition.

Method used

The acoustic embedding vector, text embedding vector, and language embedding vector of speech are determined by the target speech recognition model. The speech recognition is performed by combining the concatenation unit, and the initial encoder, predictor, and concatenation unit are trained to improve the applicability of the model.

Benefits of technology

It improves the accuracy of multilingual speech recognition, enabling the recognition of different languages ​​in the same model and expanding the applicable scenarios of speech recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117765927B_ABST
    Figure CN117765927B_ABST
Patent Text Reader

Abstract

This application provides a speech recognition method, apparatus, electronic device, and storage medium. The method includes determining an acoustic embedding vector of a first frame of speech using a target speech recognition model; determining a text embedding vector corresponding to the first frame of speech using the target speech recognition model; obtaining a language embedding vector of the first frame of speech; and determining first text information corresponding to the first frame of speech based on the acoustic embedding vector, text embedding vector, and language embedding vector. The speech recognition method provided in this application, by combining the language embedding vector corresponding to the speech during speech recognition, takes into account language-related information during the recognition process, thereby more accurately recognizing the corresponding text information and improving the accuracy of speech recognition. Furthermore, it can utilize the same speech recognition model to recognize different languages, increasing the applicability of the speech recognition model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of speech processing technology, specifically to a speech recognition method, device, electronic device, and storage medium. Background Technology

[0002] Automatic Speech Recognition (ASR) is a core technology for transcribing input speech data into corresponding text content; it is currently widely used in e-commerce, finance, logistics, and other fields. In existing ASR scenarios, ASR models only support single-language speech recognition tasks; that is, a single ASR model can only recognize one specific language. For example, a Chinese ASR model can only be used for Chinese speech recognition, and an English ASR model can only be used for English speech recognition. With the increasing prevalence of ASR technology, the need for a single ASR model to recognize multiple languages ​​is becoming increasingly urgent. By unifying ASR models for multiple languages ​​into a single model, the costs of model iteration and maintenance can be reduced, while also avoiding recognition errors caused by language misidentification. Currently, existing technologies for multilingual speech recognition have significant limitations, resulting in poor accuracy and reliability for multilingual recognition. Summary of the Invention

[0003] This application provides a speech recognition method, apparatus, electronic device, and storage medium, which can improve the accuracy of multilingual speech recognition.

[0004] In a first aspect, this application provides a speech recognition method, wherein the speech sequence to be recognized includes a first frame of speech, the method comprising:

[0005] The acoustic embedding vector of the first frame of speech is determined using a target speech recognition model;

[0006] The target speech recognition model is used to determine the text embedding vector corresponding to the first frame of speech;

[0007] Obtain the language embedding vector of the first frame of speech;

[0008] The first text information corresponding to the first frame of speech is determined based on the acoustic embedding vector, the text embedding vector, and the language embedding vector.

[0009] In some embodiments, determining the acoustic embedding vector of the first frame of speech using a target speech recognition model includes:

[0010] The audio features of the first frame of speech are determined using the target speech recognition model;

[0011] The audio features are encoded to obtain the acoustic embedding vector of the first frame of speech.

[0012] In some embodiments, the speech sequence to be identified includes a second frame of speech, wherein the first frame of speech and the second frame of speech are two adjacent frames of speech, and the second frame of speech precedes the first frame of speech;

[0013] The step of determining the text embedding vector corresponding to the first frame of speech using the target speech recognition model includes:

[0014] Obtain the second text information corresponding to the second frame of speech;

[0015] The target speech recognition model is used to determine the text features corresponding to the second text information;

[0016] Based on the text features and the target speech recognition model, the text embedding vector corresponding to the first frame of speech is determined.

[0017] In some embodiments, before determining the acoustic embedding vector of the first frame of speech using a target speech recognition model, the method further includes:

[0018] Obtain an initial speech recognition model, which includes an initial encoder;

[0019] The initial encoder is trained to obtain a first speech recognition model, wherein the first speech recognition model includes the trained first encoder;

[0020] The first speech recognition model is trained using the first learning rate to obtain the second speech recognition model;

[0021] The second speech recognition model is trained using a second learning rate to obtain the target speech recognition model.

[0022] In some embodiments, training the initial encoder to obtain a first speech recognition model includes:

[0023] Obtain speech recognition training samples, which include speech training sequences;

[0024] The speech training sequence is labeled to determine the first phoneme label corresponding to the speech training sequence;

[0025] The initial encoder is used to determine the acoustic embedding training vector corresponding to the speech training sequence;

[0026] Determine the second phoneme annotation corresponding to the acoustic embedding training vector;

[0027] Based on the first loss function, the first phoneme annotation, and the second phoneme annotation, the parameters of the initial encoder are adjusted to obtain a trained first speech recognition model, which includes the trained first encoder.

[0028] In some embodiments, the first speech recognition model includes an initial predictor and an initial combiner;

[0029] The step of training the first speech recognition model using a first learning rate to obtain a second speech recognition model includes:

[0030] The encoder parameters of the first encoder are fixed;

[0031] Obtain the training text corresponding to the speech recognition training sequence;

[0032] The initial predictor and the initial concatenator are trained using the second loss function, the training text, the speech recognition training sequence, and the first learning rate to obtain the second speech recognition model, which includes the trained first encoder, the first predictor, and the first concatenator.

[0033] In some embodiments, training the second speech recognition model using a second learning rate to obtain the target speech recognition model includes:

[0034] The second speech recognition model is trained using the second loss function, the training text, the speech recognition training sequence, and the second learning rate to adjust the parameters of the first encoder, the first predictor, and the first jointer, thereby obtaining the trained target speech recognition model.

[0035] The target speech recognition model includes a trained target encoder, a target predictor, and a target combiner.

[0036] Secondly, embodiments of this application provide a voice recognition device, the voice recognition device comprising:

[0037] An acoustic embedding vector determination module is used to determine the acoustic embedding vector of the first frame of speech using a target speech recognition model.

[0038] The text embedding vector determination module is used to determine the text embedding vector corresponding to the first frame of speech using the target speech recognition model.

[0039] The language embedding vector determination module is used to obtain the language embedding vector of the first frame of speech;

[0040] The text prediction module is used to determine the first text information corresponding to the first frame of speech based on the acoustic embedding vector, the text embedding vector, and the language embedding vector.

[0041] Thirdly, the electronic device provided in this application includes a memory and a processor. The memory stores a computer program, and the processor runs the computer program in the memory to implement the steps in the speech recognition method provided in this application.

[0042] Fourthly, the computer-readable storage medium provided in this application stores multiple instructions that are adapted for loading by a processor to implement the steps in the speech recognition method provided in this application.

[0043] This application provides a speech recognition method, apparatus, electronic device, and storage medium. The method includes determining an acoustic embedding vector of a first frame of speech using a target speech recognition model; determining a text embedding vector corresponding to the first frame of speech using the target speech recognition model; obtaining a language embedding vector of the first frame of speech; and determining first text information corresponding to the first frame of speech based on the acoustic embedding vector, text embedding vector, and language embedding vector. The speech recognition method provided in this application, by combining the language embedding vector corresponding to the speech during speech recognition, takes into account language-related information during the recognition process, thereby more accurately recognizing the corresponding text information and improving the accuracy of speech recognition. Furthermore, it can utilize the same speech recognition model to recognize different languages, increasing the applicability of the speech recognition model. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of a scenario for the speech recognition system provided in an embodiment of this application;

[0046] Figure 2 This is a flowchart illustrating one embodiment of the speech recognition method provided in this application.

[0047] Figure 3 This is a schematic diagram of the architecture of the target speech recognition model provided in an embodiment of this application;

[0048] Figure 4 This is a flowchart illustrating one embodiment of the training of the initial speech recognition model provided in this application.

[0049] Figure 5This is a schematic diagram of an embodiment of the training process for obtaining the first speech recognition model provided in this application.

[0050] Figure 6 This is a schematic diagram of the encoder training architecture provided in an embodiment of this application;

[0051] Figure 7 This is a schematic diagram of the structure of the target speech recognition model device provided in the embodiments of this application;

[0052] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0053] Figure 9 This illustration shows a block diagram of a portion of the structure of a mobile phone related to a terminal device provided in an embodiment of this application;

[0054] Figure 10 This is a schematic diagram of a server structure provided in an embodiment of this application. Detailed Implementation

[0055] It should be noted that the principles of this application are illustrated by example in a suitable computing environment. The following description is based on the specific embodiments of this application that are illustrated, and should not be regarded as limiting other specific embodiments not detailed herein.

[0056] In the following description of this application, "some embodiments" are referred to, which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments, and may be combined with each other without conflict.

[0057] In the following description of this application, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0059] To improve the performance of speech recognition, embodiments of this application provide a speech recognition method, apparatus, electronic device, and storage medium. The speech recognition method can be executed by a target speech recognition model apparatus or by an electronic device integrating the target speech recognition model apparatus.

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] Please refer to Figure 1 This application also provides a speech recognition system, such as Figure 1 As shown, the speech recognition system includes an electronic device 100, which integrates the target speech recognition model device provided in this application.

[0062] Among them, electronic device 100 can be any device equipped with a processor and having processing capabilities, such as mobile electronic devices with processors such as smartphones, tablets, PDAs, laptops, and smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, and industrial equipment.

[0063] In addition, such as Figure 1 As shown, the speech recognition system may also include a memory 200 for storing speech sequences.

[0064] In this embodiment of the application, the memory 200 can be a cloud memory. Cloud storage is a new concept that is extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file system functions to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) through application software or application interfaces to work together to provide data storage and business access functions to the outside world.

[0065] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0066] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0067] It should be noted that, Figure 1 The schematic diagram of the speech recognition system shown is merely an example. The speech recognition system and scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of speech recognition systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0068] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0069] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating one embodiment of the speech recognition method provided in this application, as shown below. Figure 2 As shown, the flow of the speech recognition method provided in this application is as follows:

[0070] 201. Use the target speech recognition model to determine the acoustic embedding vector of the first frame of speech.

[0071] This application provides a speech recognition method that, while recognizing a speech sequence, also needs to translate the speech into text, i.e., convert speech into text; therefore, this application provides a target speech recognition model for speech recognition. Specifically, this application needs to perform speech recognition on a speech sequence to be recognized, which includes multiple consecutive frames of speech; taking any one frame as the first frame as an example: the target speech recognition model needs to be used to determine the acoustic embedding vector of the first frame of speech.

[0072] like Figure 3 The image shown is a schematic diagram of the architecture of a target speech recognition model provided in an embodiment of this application. Figure 3 In the illustrated embodiment, the target speech recognition model may include an encoder, a predictor, a language embedding vector layer, and a combiner. Determining the acoustic embedding vector of the first frame of speech using the target speech recognition model essentially involves using the encoder within the target speech recognition model to determine the acoustic embedding vector of the first frame of speech.

[0073] Specifically, in the embodiments of this application, determining the acoustic embedding vector of the first frame of speech using a target speech recognition model may include:

[0074] The audio features of the first frame of speech are determined using a target speech recognition model; the audio features are then encoded to obtain the acoustic embedding vector of the first frame of speech.

[0075] In this embodiment, the encoder can encode audio features to obtain high-dimensional acoustic continuity information, i.e., a high-dimensional acoustic embedding vector. However, the encoder cannot directly determine the audio features of the speech; instead, it uses other functional modules in the target speech recognition model to determine the audio features, and then uses these audio features as input to the encoder. The encoder encodes the input audio features and finally outputs the high-dimensional acoustic embedding vector corresponding to the speech of the current frame, i.e., the high-dimensional acoustic embedding vector of the first frame.

[0076] 202. Use the target speech recognition model to determine the text embedding vector corresponding to the first frame of speech.

[0077] In practical speech recognition, it is usually necessary to determine the specific text of the current frame of speech by considering the context; therefore, a text embedding vector also exists. The text embedding vector is typically determined based on the already determined text. Specifically, the speech sequence to be recognized in this application includes a second frame of speech, where the first and second frames are adjacent, with the second frame preceding the first. Determining the text embedding vector corresponding to the first frame of speech using a target speech recognition model can include:

[0078] Obtain the second text information corresponding to the second frame of speech; determine the text features corresponding to the second text information using the target speech recognition model; determine the text embedding vector corresponding to the first frame of speech based on the text features and the target speech recognition model.

[0079] Specifically, the second text information corresponding to the second frame of speech can be obtained first, then the text features corresponding to the second text information can be determined, and then the text features and the target speech recognition model can be used to determine the text embedding vector corresponding to the first frame of speech. In the embodiments of this application, the predictor in the target speech recognition model can be used to determine the text embedding vector; for the predictor, the predictor can encode the text information, thereby providing linguistic contextual information for speech recognition, thereby improving the accuracy of speech recognition. In the embodiments of this application, the input of the predictor can be the text features of the text corresponding to the second frame of speech; in fact, the text information corresponding to the second frame of speech is converted into data that the predictor can process, and then input into the predictor.

[0080] 203. Obtain the language embedding vector of the first frame of speech.

[0081] For the embodiments of this application, it is also necessary to obtain the language embedding vector of the first frame of speech. This allows for the integration of language-related information during speech recognition, thereby improving the accuracy of speech recognition. The language can be multiple different languages ​​such as Chinese, English, and French, each with its own specific features. The language embedding vector obtained in this application can be understood as the unique features of each language. Combining the language embedding vector with speech recognition can effectively improve the accuracy of speech recognition.

[0082] like Figure 3 As shown in the embodiments of this application, the target speech recognition model also includes a language embedding vector layer. This layer determines the language embedding vector corresponding to the current speech to be recognized. In the embodiments of this application, the input to the language embedding vector layer can be a one-hot vector for the language; a one-hot vector is a feature vector unique to each language. In the embodiments of this application, the one-hot vector corresponding to each language can be customized; for example, a one-hot vector can be a 1*3 vector, where the one-hot vector for Chinese can be 001, for English it can be 010, and for French it can be 100. Since the language one-hot vector is usually not in the same dimension as the acoustic embedding vector and the text embedding vector, it is necessary to use the language embedding vector layer to transform the one-hot vector to the dimension of the acoustic embedding vector to obtain a language embedding vector of the same dimension for subsequent calculations.

[0083] It should be noted that in the embodiments of this application, the acoustic embedding vector and the text embedding vector are usually in the same dimension, so no dimension conversion is required; however, the dimension of the one-hot vector is much smaller than that of the acoustic embedding vector and the text embedding vector, so a dimension conversion is required to obtain the language embedding vector.

[0084] 204. Determine the first text information corresponding to the first frame of speech based on the acoustic embedding vector, text embedding vector, and language embedding vector.

[0085] After obtaining the acoustic embedding vector, text embedding vector, and language embedding vector corresponding to the first frame of speech, these three can be used to determine the first text information corresponding to the first frame of speech, that is, to translate the first frame of speech into text. In this application, these three can be input into a concatenator, which combines the high-dimensional theological continuity information (i.e., the acoustic embedding vector), the linguistic context information (i.e., the text embedding vector), and the total embedding vector to predict the first text information corresponding to the first frame of speech.

[0086] Compared to existing technologies, the speech recognition method provided in this application additionally incorporates a language embedding vector. This allows language-related information to be considered during speech recognition, thereby improving accuracy. Furthermore, in this embodiment, once the language embedding vector is determined, subsequent speech recognition processes do not require converting the language's one-hot vector into a corresponding language embedding vector; instead, the already determined language embedding vector is directly saved and utilized, thus saving computational and storage resources.

[0087] It should be noted that, in the embodiments of this application, a special starting identifier representing the language can be used as the input character of the first frame of the predictor; and the first character in the first frame can also be used to identify the corresponding text information. That is, the first character in the speech sequence to be identified can be used to determine both the language corresponding to the speech sequence and the one-hot vector corresponding to the language, and the text information corresponding to the first character.

[0088] The foregoing embodiments described how to perform speech recognition using a target speech recognition model. However, in the embodiments of this application, before performing speech recognition using the target speech recognition model, it is necessary to obtain an initial speech recognition model and train it to obtain a trained target speech recognition model. The trained target speech recognition model is then used for speech recognition. In the embodiments of this application, the initial speech recognition model needs to be trained multiple times; specifically as follows... Figure 4 As shown, it may include:

[0089] 401. Obtain the initial speech recognition model.

[0090] 402. Train the initial encoder to obtain the first speech recognition model.

[0091] In this embodiment, an initial speech recognition model is first obtained, which includes an initial encoder, an initial predictor, an initial combiner, and a language-embedded vector layer. The training process in this embodiment requires training the initial encoder first, followed by training the initial predictor and the initial combiner.

[0092] like Figure 5 The diagram shown is a flowchart illustrating an embodiment of the training process for obtaining a first speech recognition model provided in this application, which may include:

[0093] 501. Obtain speech recognition training samples, which include speech training sequences.

[0094] 502. Label the speech training sequence and determine the first phoneme label corresponding to the speech training sequence.

[0095] 503. Use the initial encoder to determine the acoustic embedding training vector corresponding to the speech training sequence.

[0096] 504. Determine the second phoneme annotation corresponding to the acoustic embedding training vector.

[0097] 505. Based on the first loss function, the first phoneme annotation, and the second phoneme annotation, adjust the parameters of the initial encoder to obtain the trained first speech recognition model.

[0098] like Figure 6 The diagram shown is a schematic of the training architecture of the encoder provided in an embodiment of this application. Figure 6 The training architecture shown is only a portion of the initial speech recognition training model; the initial speech recognition model also includes other functional modules, which are not shown. The following section combines... Figure 5 and Figure 6 Detailed instructions on how to train the encoder:

[0099] for Figure 6 The training architecture shown first requires acquiring speech recognition training samples, primarily speech training sequences, i.e., acquiring training samples of speech sequences. Next, the speech in the training sequences needs to be labeled to determine the first phoneme label corresponding to each frame of speech in the training sequence; that is, obtaining the first phoneme label corresponding to the speech training sequence. Here, phoneme labeling is the smallest unit of speech defined based on the natural attributes of speech; in this application, phonemes universally used in various languages ​​are used for labeling, such as using the International Phonetic Alphabet (IPA) or Romanization.

[0100] Next, the audio features corresponding to the speech training samples are input into the initial encoder. The initial encoder encodes the audio features to obtain the acoustic embedding training vector corresponding to the speech training sequence. The acoustic embedding vector is then input into the phoneme classifier to determine the predicted phoneme label, i.e., the second phoneme label, corresponding to the speech training sequence. The first loss function can then be used to calculate the loss between the true first phoneme label and the predicted second phoneme label. The parameters of the initial encoder are then adjusted based on the loss until the loss function converges or meets a preset requirement. In this training process, only the parameters of the initial encoder are adjusted, not the parameters of the initial predictor and the initial concatenator; that is, the parameters of the initial predictor and the initial concatenator are frozen. The resulting first speech recognition model includes the trained first encoder and the unadjusted initial predictor and initial concatenator. In some embodiments, the first loss function can be the Connectionist Temporal Classification (CTC) loss function.

[0101] In the above embodiments, the speech training samples can be speech samples from different languages, but training is performed using common phoneme annotations across different languages. This guides the encoder to learn language-independent human pronunciation information, avoiding the problem of significant performance differences in speech recognition models between different languages ​​that may result from an imbalance in the number of speech sequences across languages. This allows for good speech recognition results even with a small number of speech samples from a particular language.

[0102] 403. Train the first speech recognition model using the first learning rate to obtain the second speech recognition model.

[0103] After obtaining the first speech recognition model, further training is required. However, at this stage, the encoder parameters of the first encoder need to be fixed, with training primarily focused on the parameters of the initial predictor and the initial combiner. This is because the encoder has the largest number of parameters in the entire speech recognition model, making it most susceptible to overfitting. If multilingual data exists, the encoder may overfit to the language with the largest amount of data, resulting in poor recognition performance for languages ​​with fewer speech sequences. Therefore, the encoder is trained separately first, and its parameters are frozen. Subsequently, only the predictor and combiner are trained. In this application, when training the predictor and combiner, a pre-trained, language-independent first encoder trained using phonemes is used. This avoids overfitting. Furthermore, the encoder pre-trained using common phoneme annotations shows greater differences in acoustic features across different languages, primarily in the continuity of acoustic features. This information also helps the combiner learn the linguistic characteristics of different languages.

[0104] In the specific training process, after fixing the encoder parameters of the first encoder, it is also necessary to obtain the training text corresponding to the speech recognition training sequence, i.e., the text information. Then, the initial predictor and the initial joint unit can be trained using the second loss function, the training text, and the speech recognition training sequence; training also requires incorporating the first learning rate. The learning rate determines whether and when the loss function converges to a local minimum; a suitable learning rate enables the loss function to converge to a local minimum within a suitable time. Here, a relatively large learning rate can be used for training, and the specific training process can be referenced from existing technologies, which are not limited here.

[0105] 404. Train the second speech recognition model using the second learning rate to obtain the target speech recognition model.

[0106] In the aforementioned embodiments, the encoder, predictor, and concatenator were trained separately, ultimately resulting in a second speech recognition model comprising a first encoder, a first predictor, and a first concatenator. A third training step is then required. This third training differs from the previously described parameter-freezing training process; it involves simultaneously adjusting the parameters of the first encoder, first predictor, and first concatenator. Specifically, the second speech recognition model can be trained using a second loss function, training text, and speech training recognition sequences to adjust the parameters of the first encoder, first predictor, and first concatenator. Simultaneously, a second learning rate is also incorporated into the training process to obtain the final target speech recognition model comprising a target encoder, target predictor, and target concatenator.

[0107] In this embodiment, the second learning rate can be a smaller learning rate, and the second learning rate is smaller than the first learning rate. The specific values ​​of the first and second learning rates can be selected according to actual needs and are not limited here. When training the first speech recognition model and the second speech recognition model, the same loss function, such as a recurrent neural network (RNNT), can be used.

[0108] It should be noted that the parameters of the language embedding vector layer are not adjusted during the training process described above. The language embedding vector layer only needs to convert the one-hot vector corresponding to the language into the dimension of the acoustic embedding vector to obtain the language embedding vector. Adjusting the parameters does not affect the obtained one-hot vector and the corresponding language embedding vector.

[0109] To facilitate better implementation of the speech recognition method provided in the embodiments of this application, the embodiments of this application also provide a speech recognition device based on the above speech recognition method. The meanings of the terms used are the same as in the above speech recognition method; for specific implementation details, please refer to the descriptions in the above method embodiments.

[0110] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of the target speech recognition model device provided in the embodiments of this application. The target speech recognition model device may include an acoustic embedding vector determination module 701, a text embedding vector determination module 702, a language embedding vector determination module 703, and a text prediction module 704, wherein...

[0111] The acoustic embedding vector determination module 701 is used to determine the acoustic embedding vector of the first frame of speech using the target speech recognition model.

[0112] The text embedding vector determination module 702 is used to determine the text embedding vector corresponding to the first frame of speech using the target speech recognition model.

[0113] The language embedding vector determination module 703 is used to obtain the language embedding vector of the first frame of speech.

[0114] The text prediction module 704 is used to determine the first text information corresponding to the first frame of speech based on the acoustic embedding vector, the text embedding vector and the language embedding vector.

[0115] This application provides a speech recognition device. First, it uses a target speech recognition model to determine the acoustic embedding vector of a first frame of speech. Then, it uses the target speech recognition model to determine the text embedding vector corresponding to the first frame of speech. Next, it obtains the language embedding vector of the first frame of speech. Finally, based on the acoustic embedding vector, text embedding vector, and language embedding vector, it determines the first text information corresponding to the first frame of speech. The speech recognition method provided in this application, by combining the language embedding vector corresponding to the speech, takes into account language-related information during the recognition process, thereby more accurately recognizing the corresponding text information and improving the accuracy of speech recognition. Furthermore, it can utilize the same speech recognition model to recognize different languages, increasing the applicability of the speech recognition model.

[0116] In some embodiments, the acoustic embedding vector determination module 701 may specifically be used for:

[0117] The audio features of the first frame of speech are determined using a target speech recognition model;

[0118] The audio features are encoded to obtain the acoustic embedding vector of the first frame of speech.

[0119] In some embodiments, the speech sequence to be identified includes a second frame of speech, where the first and second frames are two adjacent frames, with the second frame preceding the first frame. The text embedding vector determination module 702 can specifically be used for:

[0120] Obtain the second text information corresponding to the second frame of speech;

[0121] The text features corresponding to the second text information are determined using a target speech recognition model.

[0122] Based on text features and the target speech recognition model, the text embedding vector corresponding to the first frame of speech is determined.

[0123] In some embodiments, the speech recognition device further includes a speech recognition model training module 705, which can be specifically used for:

[0124] Obtain the initial speech recognition model, which includes the initial encoder;

[0125] Train the initial encoder to obtain the first speech recognition model, which includes the trained first encoder.

[0126] The first speech recognition model is trained using the first learning rate to obtain the second speech recognition model;

[0127] The second speech recognition model is trained using a second learning rate to obtain the target speech recognition model.

[0128] In some embodiments, the speech recognition model training module 705 can specifically be used for:

[0129] Obtain speech recognition training samples, which include speech training sequences;

[0130] The speech training sequence is labeled to determine the first phoneme label corresponding to the speech training sequence;

[0131] The acoustic embedding training vectors corresponding to the speech training sequence are determined using the initial encoder.

[0132] Determine the second phoneme annotation corresponding to the acoustic embedding training vector;

[0133] Based on the first loss function, the first phoneme annotation, and the second phoneme annotation, the parameters of the initial encoder are adjusted to obtain the trained first speech recognition model, which includes the trained first encoder.

[0134] In some embodiments, the first speech recognition model includes an initial predictor and an initial jointer. The speech recognition model training module 705 can specifically be used for:

[0135] Fix the encoder parameters of the first encoder;

[0136] Obtain the training text corresponding to the speech recognition training sequence;

[0137] The initial predictor and initial concatenator are trained using training text, speech recognition training sequence and a first learning rate to obtain a second speech recognition model. The second speech recognition model includes a trained first encoder, a first predictor and a first concatenator.

[0138] In some embodiments, the speech recognition model training module 705 can specifically be used for:

[0139] The second speech recognition model is trained using training text, speech recognition training sequence, and second learning rate to adjust the parameters of the first encoder, first predictor, and first combiner, thereby obtaining the trained target speech recognition model.

[0140] The target speech recognition model includes a trained target encoder, a target predictor, and a target combiner.

[0141] For details on the implementation of each of the above modules, please refer to the previous examples, which will not be repeated here.

[0142] This application also provides an electronic device, including a memory and a processor, wherein the processor executes the steps in the speech recognition method provided in this embodiment by calling a computer program stored in the memory.

[0143] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0144] The electronic device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will understand that... Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0145] The processor 801 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802. Optionally, the processor 801 may include one or more processing cores; optionally, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 801.

[0146] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0147] The electronic device also includes a power supply 803 that supplies power to the various components. Optionally, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0148] The electronic device may also include an input unit 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0149] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the electronic device will load one or more executable codes corresponding to computer programs into the memory 802 according to the following instructions, and the processor 801 will execute the steps in the speech recognition method provided in this application, such as:

[0150] The acoustic embedding vector of the first frame of speech is determined using a target speech recognition model;

[0151] The text embedding vector corresponding to the first frame of speech is determined using a target speech recognition model.

[0152] Obtain the language embedding vector of the first frame of speech;

[0153] Based on the acoustic embedding vector, text embedding vector, and language embedding vector, the first text information corresponding to the first frame of speech is determined.

[0154] It should be noted that the electronic device provided in this application embodiment and the speech recognition method in the above embodiment belong to the same concept. The specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0155] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program stored thereon is executed on the processor of the electronic device provided in the embodiments of this application, the processor of the electronic device performs the steps in the speech recognition method provided in this application. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0156] Since the computer program stored in the computer-readable storage medium can execute the steps in any of the short-term data storage methods provided in the embodiments of this application, the beneficial effects that any of the speech recognition methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0157] When the computing device in this application embodiment is a terminal device, this application embodiment also provides a terminal device, such as... Figure 9 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal device can be any terminal device including mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a mobile phone as an example:

[0158] Figure 9 This diagram illustrates a partial structural representation of a mobile phone related to the terminal device provided in this embodiment. (Reference) Figure 9 The mobile phone includes components such as a radio frequency (RF) circuit 910, a memory 920, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a wireless fidelity (WiFi) module 970, a processor 980, and a power supply 990. Those skilled in the art will understand that... Figure 9 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0159] The following is combined Figure 9 A detailed introduction to each component of a mobile phone:

[0160] The RF circuit 910 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with the processor 980; additionally, it transmits uplink data to the base station. Typically, the RF circuit 910 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF circuit 910 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, and Short Messaging Service (SMS).

[0161] The memory 920 can be used to store software programs and modules. The processor 980 executes various functions and data processing of the mobile phone by running the software programs and modules stored in the memory 920. The memory 920 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 920 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0162] The input unit 930 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone. Specifically, the input unit 930 may include a touch panel 931 and other input devices 932. The touch panel 931, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 931), and drive the corresponding connected devices according to a pre-set program. Optionally, the touch panel 931 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 980, and can also receive and execute commands sent by the processor 980. In addition, the touch panel 931 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 931, the input unit 930 may also include other input devices 932. Specifically, other input devices 932 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0163] The display unit 940 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. The display unit 940 may include a display panel 941, which may optionally be configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar display. Further, a touch panel 931 may cover the display panel 941. When the touch panel 931 detects a touch operation on or near it, it transmits the information to the processor 980 to determine the type of touch event. Subsequently, the processor 980 provides corresponding visual output on the display panel 941 based on the type of touch event. Although in Figure 9 In this embodiment, the touch panel 931 and the display panel 941 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch panel 931 and the display panel 941 can be integrated to realize the input and output functions of the mobile phone.

[0164] The mobile phone may also include at least one sensor 950, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 941 according to the ambient light level, and the proximity sensor can turn off the display panel 941 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, which can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, taps), etc. Other sensors that may be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0165] Audio circuit 960, speaker 961, and microphone 962 provide an audio interface between the user and the mobile phone. Audio circuit 960 converts received audio data into electrical signals and transmits them to speaker 961, where speaker 961 converts them into sound signals for output. On the other hand, microphone 962 converts collected sound signals into electrical signals, which are received by audio circuit 960, converted into audio data, and then processed by processor 980 before being transmitted via RF circuit 910 to, for example, another mobile phone, or the audio data can be output to memory 920 for further processing.

[0166] Wi-Fi is a short-range wireless transmission technology. Through the Wi-Fi module 970, mobile phones can help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access. Although Figure 9 The Wi-Fi module 970 is shown, but it is understood that it is not an essential component of a mobile phone and can be omitted as needed without changing the essence of the invention.

[0167] The processor 980 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It executes software programs and / or modules stored in the memory 920, and calls data stored in the memory 920 to perform various functions and process data, thereby providing overall monitoring of the phone. Optionally, the processor 980 may include one or more processing units; optionally, the processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 980.

[0168] The phone also includes a power supply 990 (such as a battery) that supplies power to various components. Optionally, the power supply can be logically connected to the processor 980 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0169] Although not shown, mobile phones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0170] In this embodiment of the application, the processor 980 included in the mobile phone also has a process for controlling the execution of the short-term data storage method performed by the video authentication device.

[0171] This application also provides a server; please refer to [link / reference]. Figure 10 , Figure 10 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 1000 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1022 (e.g., one or more processors) and memory 1032, and one or more storage media 1030 (e.g., one or more mass storage devices) for storing application programs 1042 or data 1044. The memory 1032 and storage media 1030 can be temporary or persistent storage. The program stored in the storage media 1030 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the CPU 1022 may be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1030 on the server 1000.

[0172] Server 1000 may also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, and / or one or more operating systems 1041, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0173] The steps in the federated learning-based model training method described in the above embodiments can be based on this... Figure 10 The structure of server 1000 is shown. For example, central processing unit 1022 performs the following operations by calling instructions in memory 1032:

[0174] The acoustic embedding vector of the first frame of speech is determined using a target speech recognition model;

[0175] The text embedding vector corresponding to the first frame of speech is determined using a target speech recognition model.

[0176] Obtain the language embedding vector of the first frame of speech;

[0177] Based on the acoustic embedding vector, text embedding vector, and language embedding vector, the first text information corresponding to the first frame of speech is determined.

[0178] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0179] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0180] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0181] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0183] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0184] The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0185] The technical solutions provided in the embodiments of this application have been described in detail above. Specific examples have been used in the embodiments of this application to illustrate the principles and implementation methods of the embodiments of this application. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A speech recognition method, characterized in that, The speech sequence to be recognized includes a first frame of speech, and the method includes: The acoustic embedding vector of the first frame of speech is determined using a target speech recognition model. Before determining the acoustic embedding vector of the first frame of speech using the target speech recognition model, the method further includes: obtaining an initial speech recognition model, which includes an initial encoder, an initial predictor, and an initial jointer; training the initial encoder to obtain a first speech recognition model, which includes a trained first encoder; fixing the encoder parameters of the first encoder; training the initial predictor and the initial jointer of the first speech recognition model using a first learning rate to obtain a second speech recognition model; and training the second speech recognition model using a second learning rate to obtain the target speech recognition model. The target speech recognition model is used to determine the text embedding vector corresponding to the first frame of speech; Obtain the language embedding vector of the first frame of speech; The first text information corresponding to the first frame of speech is determined based on the acoustic embedding vector, the text embedding vector, and the language embedding vector.

2. The speech recognition method according to claim 1, characterized in that, The step of determining the acoustic embedding vector of the first frame of speech using a target speech recognition model includes: The audio features of the first frame of speech are determined using the target speech recognition model; The audio features are encoded to obtain the acoustic embedding vector of the first frame of speech.

3. The speech recognition method according to claim 1, characterized in that, The speech sequence to be identified includes a second frame of speech, the first frame of speech and the second frame of speech are two adjacent frames of speech, and the second frame of speech precedes the first frame of speech; The step of determining the text embedding vector corresponding to the first frame of speech using the target speech recognition model includes: Obtain the second text information corresponding to the second frame of speech; The target speech recognition model is used to determine the text features corresponding to the second text information; Based on the text features and the target speech recognition model, the text embedding vector corresponding to the first frame of speech is determined.

4. The speech recognition method according to claim 3, characterized in that, The training of the initial encoder to obtain the first speech recognition model includes: Obtain speech recognition training samples, which include speech training sequences; The speech training sequence is labeled to determine the first phoneme label corresponding to the speech training sequence; The initial encoder is used to determine the acoustic embedding training vector corresponding to the speech training sequence; Determine the second phoneme annotation corresponding to the acoustic embedding training vector; Based on the first loss function, the first phoneme annotation, and the second phoneme annotation, the parameters of the initial encoder are adjusted to obtain a trained first speech recognition model, which includes the trained first encoder.

5. The speech recognition method according to claim 4, characterized in that, The first speech recognition model includes an initial predictor and an initial combiner; The step of training the first speech recognition model using a first learning rate to obtain a second speech recognition model includes: The encoder parameters of the first encoder are fixed; Obtain the training text corresponding to the speech recognition training sequence; The initial predictor and the initial concatenator are trained using the second loss function, the training text, the speech recognition training sequence, and the first learning rate to obtain the second speech recognition model, which includes the trained first encoder, the first predictor, and the first concatenator.

6. The speech recognition method according to claim 5, characterized in that, The step of training the second speech recognition model using a second learning rate to obtain the target speech recognition model includes: The second speech recognition model is trained using the second loss function, the training text, the speech recognition training sequence, and the second learning rate to adjust the parameters of the first encoder, the first predictor, and the first jointer, thereby obtaining the trained target speech recognition model. The target speech recognition model includes a trained target encoder, a target predictor, and a target combiner.

7. A voice recognition device, characterized in that, The speech recognition device includes: An acoustic embedding vector determination module is used to determine the acoustic embedding vector of a first frame of speech using a target speech recognition model. Before determining the acoustic embedding vector of the first frame of speech using the target speech recognition model, the module further includes: obtaining an initial speech recognition model, which includes an initial encoder, an initial predictor, and an initial jointer; training the initial encoder to obtain a first speech recognition model, which includes a trained first encoder; fixing the encoder parameters of the first encoder; training the initial predictor and the initial jointer of the first speech recognition model using a first learning rate to obtain a second speech recognition model; and training the second speech recognition model using a second learning rate to obtain the target speech recognition model. The text embedding vector determination module is used to determine the text embedding vector corresponding to the first frame of speech using the target speech recognition model. The language embedding vector determination module is used to obtain the language embedding vector of the first frame of speech; The text prediction module is used to determine the first text information corresponding to the first frame of speech based on the acoustic embedding vector, the text embedding vector, and the language embedding vector.

8. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing a computer program and the processor running the computer program in the memory to perform the steps of the speech recognition method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the speech recognition method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Speech recognition method and device, computer readable storage medium and computer equipment

    CN115312028A