Speech recognition method and device and electronic equipment

Abstract

The invention discloses a voice recognition method and device and electronic equipment. The method comprises the steps that to-be-recognized audio data and preset first text description data of all classification categories are acquired; inputting the audio data and the first text description data of each classification category into a pre-trained voice classification model, and outputting a similarity score between the audio data and each classification category; based on the similarity score between the audio data and each classification category, determining a target classification category corresponding to the audio data in each classification category; and performing voice recognition processing on the audio data according to the target classification category corresponding to the audio data. According to the invention, the problem of large deviation between the classification category predicted by the voice classification model and the classification category of the classification task can be avoided, the accuracy of voice classification is improved, and the accuracy of voice recognition is further improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and electronic device for speech recognition. Background Art

[0002] Intelligent voice interaction is an automatic communication technology based on artificial intelligence technology, which involves many fields such as live interaction, intelligent customer service, and intelligent question and answer. Intelligent voice interaction can automatically interact with users through preset scripts, voice recognition, and natural language processing technologies. Intelligent voice interaction can even analyze the user's tone, speech speed and other characteristics to judge the user's emotional response and interact with the user in a more appropriate way.

[0003] However, the actual voice interaction process is often accompanied by noise pollution and complex call environment, which leads to the unsatisfactory sound quality of the generated voice signal. For example, the voice signal contains valid audio and invalid audio. Among them, valid voice refers to relatively clear and coherent human conversations; invalid audio refers to channel noise, environmental noise, action sounds, mobile phone ringtones, non-human voices, and audio such as coughing and sneezing. If all voice signals are directly input into the intelligent voice interaction system, the efficiency of voice interaction will be reduced. Therefore, a voice classification model is usually introduced in the intelligent voice interaction system. The voice classification model accurately divides the voice signals sent to the intelligent voice interaction system into two categories: valid audio and invalid audio. After the voice signal is classified by the voice classification model, the valid audio is accurately screened and advanced to the next stage of the main process (i.e., automatic speech recognition and natural language processing), while the invalid audio is appropriately excluded from the main process for other processing.

[0004] At present, the speech classification model trained with zero-shot training mainly relies on the pre-trained model capabilities to predict some classification categories for the input audio data. However, due to the lack of the classification category constraints of the classification task, the predicted classification categories deviate greatly from the classification categories of the classification task, resulting in low accuracy of speech classification and, in turn, low accuracy of speech recognition. Summary of the invention

[0005] In order to solve the above problems, the embodiments of the present application provide a method, device and electronic device for speech recognition, which can avoid the problem that the classification category predicted by the speech classification model is greatly different from the classification category of the classification task, improve the accuracy of speech classification, and further improve the accuracy of speech recognition. Specifically, the embodiments of the present application disclose the following technical solutions:

[0006] A first aspect of an embodiment of the present application provides a method for speech recognition, the method comprising:

[0007] Acquire audio data to be identified and first text description data of each preset classification category;

[0008] Input the audio data and the first text description data of each classification category into a pre-trained speech classification model, and output the similarity score between the audio data and each classification category;

[0009] Based on the similarity scores between the audio data and each classification category, determining the target classification category corresponding to the audio data in each classification category;

[0010] The audio data is processed for speech recognition according to the target classification category corresponding to the audio data.

[0011] In some embodiments, the speech classification model includes an audio encoder, a text encoder, a linear projection module, and a similarity discrimination module; the audio data and the first text description data of each classification category are input into the pre-trained speech classification model, and the similarity score between the audio data and each classification category is output, including:

[0012] Inputting the audio data into an audio encoder and outputting a first audio feature of the audio data;

[0013] Inputting the first text description data of each classification category into a text encoder, and outputting the first text feature of each classification category;

[0014] Inputting the first audio feature of the audio data and the first text feature of each classification category into a linear projection module, outputting the second audio feature of the audio data and the second text feature of each classification category, wherein the second audio feature and each second text feature have the same dimension;

[0015] The second audio feature of the audio data and the second text feature of each classification category are input into the similarity determination module, and the similarity score between the audio data and each classification category is output.

[0016] In some embodiments, the audio encoder includes a first feature encoding unit and a second feature encoding unit, the first feature encoding unit is multiple, the multiple first feature encoding units are stacked, the first feature encoding unit includes a feature grouping subunit, a feature processing subunit and a feature splicing subunit, and the feature processing subunit includes multiple feature processing layers that are distributed in parallel; inputting audio data into the audio encoder, outputting the first audio feature of the audio data, including:

[0017] The audio input feature is divided into a plurality of first audio sub-features by a feature grouping sub-unit; the audio input feature includes a Mel-spectrogram feature of the audio data and / or an audio output feature of a first feature encoding unit at a previous level;

[0018] Performing feature encoding processing on the plurality of first audio sub-features respectively through a plurality of feature processing layers to obtain a plurality of second audio sub-features;

[0019] The plurality of second audio sub-features are spliced ​​together by a feature splicing sub-unit to obtain an audio output feature;

[0020] The second feature encoding unit performs feature fusion processing on the audio output features output by the plurality of first feature encoding units to obtain the first audio feature of the audio data.

[0021] In some embodiments, the feature processing subunit further includes a plurality of attention fusion layers, and the plurality of attention fusion layers correspond one-to-one to the plurality of feature processing layers; the method further includes:

[0022] An attention fusion layer is used to calculate an attention weight according to the first audio sub-feature and the second audio sub-feature output by another feature processing layer, and a feature fusion process is performed on the first audio sub-feature and the second audio sub-feature output by another feature processing layer according to the attention weight to obtain a fusion result;

[0023] The fusion result is subjected to feature encoding processing through the feature processing layer to obtain the second audio sub-feature.

[0024] In some embodiments, the method further comprises:

[0025] Acquire a training sample set, where the training sample set includes sample audio data and second text description data of the sample audio data;

[0026] Based on the training sample set, the initial speech classification model is trained to obtain a trained speech classification model.

[0027] In some embodiments, the initial speech classification model includes an audio encoder, a text encoder, a linear projection module, and a similarity discrimination module; based on the training sample set, the initial speech classification model is trained to obtain a trained speech classification model, including:

[0028] Inputting the sample audio data into an audio encoder and outputting a third audio feature of the sample audio data;

[0029] Inputting the second text description data of the sample audio data into a text encoder, and outputting the third text features of the second text description data;

[0030] Inputting the third audio feature of the sample audio data and the third text feature of the second text description data into a linear projection module, outputting the fourth audio feature of the sample audio data and the fourth text feature of the second text description data, wherein the fourth audio feature and the fourth text feature have the same dimension;

[0031] Based on the fourth audio feature, the fourth text feature and the loss function corresponding to the similarity discrimination module, the audio encoder, the text encoder and the linear projection module are jointly trained to obtain a trained speech classification model.

[0032] In some embodiments, the loss function is:

[0033]

[0034] Among them, L represents the function value of the loss function, N represents the number of training sample pairs formed by sample audio data and text description data in the training sample set, represents the second audio feature corresponding to the i-th sample audio data, represents the second audio feature corresponding to the j-th sample audio data, represents the second text feature corresponding to the text description data of the i-th sample audio data, represents the second text feature corresponding to the text description data of the j-th sample audio data, and τ represents the temperature parameter.

[0035] In some embodiments, obtaining a training sample set includes:

[0036] Obtain sample audio data and keywords of the sample audio data;

[0037] Based on the keywords, second text description data of the sample audio data is generated.

[0038] A second aspect of an embodiment of the present application provides a speech recognition device, the device comprising:

[0039] A first acquisition module is configured to: acquire audio data to be recognized and first text description data of each preset classification category;

[0040] An output module is configured to: input the audio data and the first text description data of each classification category into a pre-trained speech classification model, and output a similarity score between the audio data and each classification category;

[0041] A determination module is configured to: determine a target classification category corresponding to the audio data in each classification category based on a similarity score between the audio data and each classification category;

[0042] The recognition module is configured to perform speech recognition processing on the audio data according to the target classification category corresponding to the audio data.

[0043] A third aspect of an embodiment of the present application provides an electronic device, comprising: a processor; and a memory for storing a program; wherein the program comprises instructions, which, when executed by the processor, cause the processor to execute the method for speech recognition of the first aspect described above.

[0044] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the method for speech recognition of the first aspect when the computer program is executed by a processor.

[0045] A fifth aspect of an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the method of speech recognition of the first aspect mentioned above.

[0046] A sixth aspect of an embodiment of the present application provides a computer program, which, when executed by a processor, can implement the method for speech recognition in the first aspect described above.

[0047] In the method of speech recognition provided in the embodiment of the present application, the electronic device obtains the audio data to be recognized and the first text description data of each preset classification category, inputs the audio data and the first text description data of each classification category into a pre-trained speech classification model, and outputs the similarity score between the audio data and each classification category. Afterwards, the electronic device determines the target classification category corresponding to the audio data in each classification category based on the similarity score between the audio data and each classification category, and performs speech recognition processing on the audio data according to the target classification category corresponding to the audio data. In this way, when the audio data to be recognized is classified by the pre-trained speech classification model, the first text description data of each classification category of the classification task is input, so as to ensure that the predicted classification category of the audio data output by the speech classification model is consistent with the classification category of the classification task, thereby improving the accuracy of speech classification, and then improving the accuracy of speech recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A flowchart of a method for speech recognition provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of the structure of a speech classification model provided in an embodiment of the present application;

[0051] Figure 3 A flowchart of a method for speech recognition provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the structure of an audio encoder provided in an embodiment of the present application;

[0053] Figure 5 A flowchart of a method for speech recognition provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the structure of an AFF provided in an embodiment of the present application;

[0055] Figure 7 A flowchart of a method for speech recognition provided in an embodiment of the present application;

[0056] Figure 8 A flowchart of a method for training a speech classification model provided in an embodiment of the present application;

[0057] Fig. 9 A flowchart of a method for training a speech classification model provided in an embodiment of the present application;

[0058] Fig.10 A schematic diagram of an example of a method for speech recognition provided in an embodiment of the present application;

[0059] Fig.11 A schematic diagram of the structure of a speech recognition device provided in an embodiment of the present application;

[0060] Fig.12 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0062] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, the words "first" and "second" are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not limit them to be different.

[0063] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0064] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0065] With the popularity of smart phones and the development of mobile Internet, mobile phones have become an indispensable part of people's daily lives. As an important part of mobile phone customization, mobile phone interactive themes can not only provide users with a unique visual experience, but also enhance users' sense of participation and experience through interaction. However, the mobile phone interactive themes on the market currently generally have problems such as single interactive form and poor flexibility. Therefore, a processing method that can improve the flexibility of interactive themes is urgently needed.

[0066] Intelligent voice interaction is an automatic communication technology based on artificial intelligence technology, which involves many fields such as live interaction, intelligent customer service, and intelligent question and answer. Intelligent voice interaction can automatically interact with users through preset scripts, voice recognition, and natural language processing technologies. Intelligent voice interaction can even analyze the user's tone, speech speed and other characteristics to judge the user's emotional response and interact with the user in a more appropriate way.

[0067] However, the actual voice interaction process is often accompanied by noise pollution and complex call environment, which leads to the unsatisfactory sound quality of the generated voice signal. For example, the voice signal contains valid audio and invalid audio. Among them, valid voice refers to relatively clear and coherent human conversations; invalid audio refers to channel noise, environmental noise, action sounds, mobile phone ringtones, non-human voices, and audio such as coughing and sneezing. If all voice signals are directly input into the intelligent voice interaction system, the efficiency of voice interaction will be reduced. Therefore, a voice classification model is usually introduced in the intelligent voice interaction system. The voice classification model accurately divides the voice signals sent to the intelligent voice interaction system into two categories: valid audio and invalid audio. After the voice signal is classified by the voice classification model, the valid audio is accurately screened and advanced to the next stage of the main process (i.e., automatic speech recognition and natural language processing), while the invalid audio is appropriately excluded from the main process for other processing.

[0068] At present, the speech classification model trained with zero-shot training mainly relies on the pre-trained model capabilities to predict some classification categories for the input audio data. However, due to the lack of classification task classification constraints, the predicted classification categories deviate greatly from the classification categories of the classification task, resulting in low accuracy of speech classification, and thus low accuracy of speech recognition.

[0069] In order to solve the above problems, the embodiment of the present application provides a method for speech recognition. The method can be applied to any speech recognition scenario. The method of the embodiment of the present application is described below. Figure 1 As shown, the specific processing process is as follows.

[0070] Step 101: Acquire audio data to be identified and first text description data of each preset classification category.

[0071] In one embodiment, in order to avoid a large deviation between the predicted classification category output by the speech classification model and the classification category of the classification task, when the electronic device classifies the audio data to be recognized through the speech classification model, the electronic device can also obtain the first text description data of each preset classification category. Among them, each preset classification category is a classification category pre-set based on different classification tasks. The classification task can be a classification task for distinguishing invalid audio, or a classification task for distinguishing emotions in valid audio, or other classification tasks, which are not limited in the embodiments of the present application.

[0072] In one example, for the classification task of distinguishing invalid audio, the preset classification categories may include: {cough, breathing, knocking, wind, telephone ring, human voice...} and other classification categories. The first text description data corresponding to each classification category is "This is the sound of {cough / breathing / knocking / wind / telephone ring / human voice...}".

[0073] In another example, for the task of distinguishing emotions in valid audio, the preset classification categories may include: {angry, happy, sad, surprised...} and other classification categories. The first text description data corresponding to each classification category is: "This is a human voice with {angry / happy / sad / surprised...}".

[0074] Step 102: input the audio data and the first text description data of each classification category into a pre-trained speech classification model, and output the similarity score between the audio data and each classification category.

[0075] In one embodiment, after the electronic device obtains the audio data to be identified and the first text description data of each classification category, it can further input the audio data and the first text description data of each classification category into a pre-trained speech classification model. Correspondingly, the pre-trained speech classification model will output the similarity score between the audio data and each classification category. Among them, the similarity score can characterize the degree of similarity between the audio data and each classification category. The higher the similarity score between the audio data and a certain classification category, the higher the similarity between the audio data and the classification category. Furthermore, the electronic device can also use the similarity score as logits and input it into the softmax function to obtain the predicted probability of each classification category. Among them, the sum of the predicted probabilities of each classification category is 1.

[0076] It should be noted that the structure of the speech classification model, the process of the speech classification model outputting the similarity scores between the audio data and each classification category, and the training process of the speech classification model will be introduced in detail in the subsequent embodiments of the present application, which will not be repeated here.

[0077] Step 103: Based on the similarity scores between the audio data and each classification category, determine the target classification category corresponding to the audio data in each classification category.

[0078] In one embodiment, the similarity between the audio data and each classification category can be characterized based on the similarity score. After the electronic device obtains the similarity score between the audio data and each classification category, it can further determine the target classification category corresponding to the audio data in each classification category based on the similarity score between the audio data and each classification category.

[0079] In one example, the electronic device may determine, among various classification categories, a classification category corresponding to a maximum similarity score as a target classification category corresponding to the audio data.

[0080] Step 104: Perform speech recognition processing on the audio data according to the target classification category corresponding to the audio data.

[0081] In one implementation, after the electronic device obtains the target classification category corresponding to the audio data, it may further perform speech recognition processing on the audio data based on the target classification category corresponding to the audio data.

[0082] In one example, for the classification task of distinguishing invalid audio, if the target classification category corresponding to the audio data belongs to the invalid audio category, the audio data is directly discarded. If the target classification category corresponding to the audio data belongs to the valid audio category, further speech recognition processing can be performed.

[0083] In an embodiment of the present application, the electronic device obtains the audio data to be recognized and the first text description data of each classification category of the classification task, inputs the audio data and the first text description data of each classification category into a pre-trained speech classification model, and outputs the similarity score between the audio data and each classification category. Afterwards, the electronic device determines the target classification category corresponding to the audio data in each classification category based on the similarity score between the audio data and each classification category, and performs speech recognition processing on the audio data according to the target classification category corresponding to the audio data. In this way, when the audio data to be recognized is classified by the pre-trained speech classification model, the first text description data of each classification category of the classification task is input, thereby ensuring that the predicted classification category of the audio data output by the speech classification model is consistent with the classification category of the classification task, thereby improving the accuracy of speech classification.

[0084] In some embodiments, Figure 2 As shown, the speech classification model includes an audio encoder 210 , a text encoder 220 , a linear projection module 230 and a similarity determination module 240 .

[0085] The audio encoder (Audio Encoder) 210 is used to extract audio features of input audio data.

[0086] The text encoder (Text Encoder) 220 is mainly composed of a token_embedding encoding layer, a texttransformer layer and a normalization LayerNorm layer, and is used to extract text features of input text description data.

[0087] The linear projection module 230 is used to perform linear transformation on the audio features and text features extracted by the audio encoder 210 and the text encoder 220 to make the dimensions of the two consistent, and then normalize them and input them into the similarity determination module 240 for similarity determination.

[0088] The similarity determination module 240 mainly includes a cosine similarity matrix, and uses similarity and dissimilarity to map similar instances together closely in the latent space and separate dissimilar instances to achieve similarity determination.

[0089] In some embodiments, Figure 3 As shown, the electronic device inputs the audio data and the first text description data of each classification category into the pre-trained speech classification model, and the processing process of outputting the similarity score between the audio data and each classification category is as follows:

[0090] Step 301: input audio data into an audio encoder, and output a first audio feature of the audio data.

[0091] In one implementation, after the electronic device acquires the audio data to be recognized, the audio data may be input into an audio encoder, and accordingly, the audio encoder may output the first audio feature of the audio data.

[0092] Step 302: input the first text description data of each classification category into a text encoder, and output the first text feature of each classification category.

[0093] In one embodiment, after the electronic device obtains the first text description data of each classification category, the first text description data of each classification category can be input into a text encoder respectively. Correspondingly, the text encoder outputs the first text feature of each classification category. The first text description data of the classification category corresponds to the first text feature of the classification category one by one.

[0094] Step 303: Input the first audio feature of the audio data and the first text feature of each classification category into a linear projection module, and output the second audio feature of the audio data and the second text feature of each classification category, wherein the second audio feature and each second text feature have the same dimension.

[0095] In one embodiment, in order to facilitate similarity determination, after the electronic device obtains the first audio feature of the audio data and the first text feature of each classification category, the first audio feature of the audio data and the first text feature of each classification category can be further input into the linear projection module at the same time. Correspondingly, the linear projection module outputs the second audio feature of the audio data and the second text feature of each classification category. The second audio feature and each second text feature have the same dimension.

[0096] Step 304: input the second audio feature of the audio data and the second text feature of each classification category into a similarity determination module, and output the similarity score between the audio data and each classification category.

[0097] In one embodiment, after the electronic device obtains the second audio features of the audio data with the same dimension and the second text features of each classification category, it can further input the second audio features of the audio data and the second text features of each classification category into a similarity determination module at the same time. Accordingly, the similarity determination module outputs the similarity score between the audio data and each classification category.

[0098] In some embodiments, Figure 4 As shown, the audio encoder mainly includes a Mel spectrum extractor (SpectrogramExtractor) and an improved ResNet structure. The present invention further adjusts the ResNet structure, which mainly includes two convolution modules with residual connections (hereinafter referred to as convolution modules). The ResNet structure extracts multi-scale features in the channel dimension by constructing a hierarchical residual-like connection in a single residual block, and then captures the local features and global features in the input audio data through a feature fusion mechanism (local feature fusion and global feature fusion) to improve the performance of speech classification. Among them, the ResNet structure mainly includes a first feature coding unit (i.e., Stage1 to Stage4) and a second feature coding unit (Fusion). There are multiple first feature coding units, and multiple first feature coding units are stacked. Stage1, Stage2, Stage3 and Stage4 include 3, 4, 6 and 3 convolution modules respectively, a total of 16 convolution modules. Each convolution module includes a convolution module structure and attention feature fusion (AFF), which connects the grouped features to enhance the local information through a step-by-step fusion mechanism, so as to obtain finer-grained features. The first feature encoding unit mainly includes a feature grouping subunit (i.e., 1×1conv at the top), a feature processing subunit (i.e., 3×3conv) and a feature splicing subunit (i.e., 1×1conv at the bottom). The feature processing subunit includes multiple feature processing layers with parallel distribution.

[0099] like Figure 5 As shown, the electronic device inputs the audio data into the audio encoder, and the processing process of the first audio feature of the output audio data is as follows:

[0100] Step 501: Divide the audio input feature into a plurality of first audio sub-features by a feature grouping sub-unit, wherein the audio input feature includes a Mel-spectrogram feature of the audio data and / or an audio output feature of a first feature encoding unit at a previous level.

[0101] In one embodiment, in combination Figure 3As shown, taking Stage 1 as an example, the electronic device first extracts spectrogram features from the audio data through the Mel spectrum extractor, and then the spectrogram features are extracted through Stage 1, Stage 2, Stage 3 and Stage 4 for local feature extraction. Specifically, the spectrogram feature X∈R DxTxC . Where D, T and C represent the spectral dimension, time and number of channels respectively. First, the spectrogram feature is input into the feature grouping subunit (1x1 convolution layer) of the first convolution module of Stage1 to extract the first layer of features. Then, the first layer of features is divided into 3 subsets (x1, x2 and x3) to extract sub-features (i.e., the first audio sub-features). Each sub-feature has the same scale size D, but the channel is 1 / s of the input feature, where s is {1, 2, 3}. For Stage2 to Stage4, the input is the audio output feature of the first feature encoding unit of the previous level. The processing process of Stage2 to Stage4 is similar to that of Stage1, and will not be repeated here.

[0102] Step 502: Perform feature encoding processing on the plurality of first audio sub-features through a plurality of feature processing layers to obtain a plurality of second audio sub-features.

[0103] In one embodiment, the electronic device performs feature encoding processing on the x1 sub-feature, the x2 sub-feature and the x3 sub-feature through multiple feature processing layers in the feature processing subunit (3x3 convolution layer), and outputs the sub-local features y1, y2 and y3 (i.e., the second audio sub-feature) of the corresponding subset.

[0104] Step 503: Splice the plurality of second audio sub-features through a feature splicing sub-unit to obtain an audio output feature.

[0105] In one implementation, the electronic device outputs the sub-local features y1, y2, and y3 (ie, the second audio sub-features) of the corresponding subset through a feature concatenation subunit (1×1conv) to obtain audio output features.

[0106] Step 504: Perform feature fusion processing on the audio output features output by the plurality of first feature encoding units through the second feature encoding unit to obtain the first audio feature of the audio data.

[0107] In one embodiment, global feature fusion uses the fusion of acoustic features of different times and scales output by different Stage levels to enhance the information of speech classification. The fusion mechanism is based on the AFF method. For example, the audio encoder performs global feature fusion on the output of Stage2 and the output of Stage1. In the process of global feature fusion, the output of Stage1 first extracts features through a layer of 3x3 convolution, and then downsamples and expands the channel dimension to 2 times. Among them, the number of channels from top to bottom is 64, 128, 256 and 512, respectively, to increase the receptive field of local features. Then, AFF fusion is performed with the output of Stage2, and the local features of each stage are fused in a bottom-up manner to enhance the global feature interaction capability of features of different times and scales, and the features are fused in turn through Stage3 and Stage4, and finally the vectors (embeddings) are output through the polling pooling layer and the fc fully connected layer. The electronic device performs feature fusion processing on the audio output features output by multiple first feature coding units through the second feature coding unit to obtain the first audio features of the audio data.

[0108] In some embodiments, the feature processing subunit further includes a plurality of attention fusion layers, and the plurality of attention fusion layers correspond one to one to the plurality of feature processing layers. Figure 6 As shown in Figure 2, the AFF step-by-step fusion mechanism mainly includes a 1x1 convolution layer, a BatchNorm normalization layer, a SiLU activation layer, and a tanh activation function. The AFF step-by-step fusion mechanism combines adjacent feature maps x i and i-1 (i is {1, 2, 3}) are merged as input and the local attention weight U is calculated:

[0109] U=tanh(BN(W2·SiLU(BN(W1·[x i ],y i-1 ))))

[0110] Among them, W1 and W2 represent point-by-point convolutions with output channel sizes of C / r and C, respectively. C is the number of channels, and r is the channel reduction rate, which is 4 here.

[0111] like Figure 7 As shown, the electronic device further performs the following steps:

[0112] Step 701: The attention fusion layer calculates the attention weight based on the first audio sub-feature and the second audio sub-feature output by another feature processing layer, and performs feature fusion processing on the first audio sub-feature and the second audio sub-feature output by another feature processing layer according to the attention weight to obtain a fusion result.

[0113] In one embodiment, the electronic device calculates the attention weight according to the first audio sub-feature and the second audio sub-feature output by another feature processing layer through the attention fusion layer, and performs feature fusion processing on the first audio sub-feature and the second audio sub-feature output by another feature processing layer according to the attention weight to obtain a fusion result. That is, in addition to the x1 sub-feature, the other sub-features (i.e., the x2 sub-feature and the x3 sub-feature) are connected to extract the sub-features through the AFF step-by-step fusion mechanism to obtain finer-grained features and strengthen the interaction of local information.

[0114] Step 702: Perform feature encoding processing on the fusion result through a feature processing layer to obtain a second audio sub-feature.

[0115] In one implementation, the electronic device performs feature encoding processing on the fusion result through a feature processing layer to obtain a second audio sub-feature, that is, the x1 sub-feature, the x2 sub-feature and the x3 sub-feature are processed through a 3x3 convolution layer to output sub-local features y1, y2 and y3 of the corresponding subset.

[0116] In an embodiment of the present application, local features in the audio data are optimized by a first feature encoding unit, and global features of the audio data are optimized by a second feature encoding unit. By combining the two, the effects of audio recognition, classification, and subsequent interaction processes are made more ideal.

[0117] In some embodiments, the present application also provides a method for training a speech classification model, such as Figure 8 As shown in the figure, the training process of the speech classification model is as follows:

[0118] Step 801: Obtain a training sample set, wherein the training sample set includes sample audio data and second text description data of the sample audio data.

[0119] In one embodiment, when an electronic device is training a speech classification model, it can obtain a training sample set. The training sample set includes sample audio data and second text description data of the sample audio data. The sample audio data is not the sample audio data corresponding to a certain feature classification task, but various natural audio data. The second text description data of the sample audio data refers to a rough text description of the content expressed by the sample audio data. For example, the sample audio data is: a lady wearing nice clothes. Correspondingly, the second text description data of the sample audio data can be: (1) the lady's dress is decent and appropriate; (2) the lady's dress taste is high.

[0120] It should be noted that the sample audio data and the text description data may be in a one-to-many relationship, that is, in a training sample set, one sample audio data and N second text description data of the sample audio data constitute N training sample pairs (Batch).

[0121] Step 802: Based on the training sample set, the initial speech classification model is trained to obtain a trained speech classification model.

[0122] In one implementation, after the electronic device acquires the training sample set, it may train the initial speech classification model based on the training sample set to obtain a trained speech classification model.

[0123] In some embodiments, based on Figure 2 The structure of the speech classification model shown in Fig. 9 As shown, the electronic device trains the initial speech classification model based on the training sample set, and the processing process of obtaining the trained speech classification model is as follows:

[0124] Step 901: input sample audio data into an audio encoder, and output a third audio feature of the sample audio data.

[0125] In one embodiment, after the electronic device obtains the training sample set, for each sample audio data Xa in the training sample set, the electronic device can input the sample audio data Xa into the audio encoder of the initial speech classification model. Correspondingly, the audio encoder can output the third audio feature X^a of the sample audio data Xa. Since all training sample pairs (Batch) in the training sample set are trained together. Therefore, the dimension of the third audio feature X^a output by the audio encoder is N*V. Among them, the training sample pair includes a sample audio data and a second text description data; N represents the number of training sample pairs (Batch); V represents the dimension of a third audio feature X^a.

[0126] It should be noted that, for each sample audio data Xa, the electronic device may also pre-process the sample audio data Xa (such as audio sampling, length processing, etc.) before inputting it into the audio encoder.

[0127] Step 902: input the second text description data of the sample audio data into a text encoder, and output the third text feature of the second text description data.

[0128] In one embodiment, after the electronic device obtains the training sample set, for each sample audio data Xa in the training sample set, the electronic device can input the second text description data Xt of the sample audio data Xa into the text encoder. Correspondingly, the text encoder outputs the third text feature X^t of the second text description data Xt. Since all training sample pairs (Batch) in the training sample set are trained together. Therefore, the dimension of the third text feature X^t output by the audio encoder is N*U. Among them, the training sample pair includes a sample audio data and a second text description data; N represents the number of training sample pairs (Batch); U represents the dimension of a third text feature X^t.

[0129] Step 903: Input the third audio feature of the sample audio data and the third text feature of the second text description data into a linear projection module, and output the fourth audio feature of the sample audio data and the fourth text feature of the second text description data, wherein the fourth audio feature and the fourth text feature have the same dimension.

[0130] In one embodiment, after the electronic device obtains the third audio feature X^a of the sample audio data Xa and the third text feature X^t of the second text description data Xt, it can further input the third audio feature X^a of the sample audio data Xa and the third text feature X^t of the second text description data Xt into the linear projection module at the same time. Correspondingly, the linear projection module can output the fourth audio feature Ea of the sample audio data Xa and the fourth text feature Et of the second text description data Xt. Wherein, Ea=La(X^a), Et=Lt(X^t), La and Lt represent linear projection, the fourth audio feature Ea and the fourth text feature Et have the same dimension, and the dimensions of the two can be the dimension d of the joint multimodal space corresponding to the linear projection module.

[0131] Step 904: Based on the fourth audio feature, the fourth text feature and the loss function corresponding to the similarity discrimination module, jointly train the audio encoder, the text encoder and the linear projection module to obtain a trained speech classification model.

[0132] In one embodiment, after the electronic device obtains the fourth audio feature Ea of the sample audio data Xa and the fourth text feature Et of the second text description data Xt, it can further jointly train the audio encoder, the text encoder and the linear projection module based on the fourth audio feature Ea, the fourth text feature Et and the loss function corresponding to the similarity discrimination module to obtain a trained speech classification model. In the process of joint training, the loss function is minimized.

[0133] In some embodiments, the above loss function is:

[0134]

[0135] Among them, L represents the function value of the loss function, N represents the number of training sample pairs formed by sample audio data and text description data in the training sample set, represents the second audio feature corresponding to the i-th sample audio data, represents the second audio feature corresponding to the j-th sample audio data, represents the second text feature corresponding to the text description data of the i-th sample audio data, represents the second text feature corresponding to the text description data of the j-th sample audio data, and τ represents the temperature parameter.

[0136] In some embodiments, the process of the electronic device acquiring the training sample set is as follows:

[0137] Step 1: Obtain sample audio data and keywords of the sample audio data.

[0138] In one implementation, in order to enrich the training samples, the electronic device obtains sample audio data and keywords of the sample audio data.

[0139] Step 2: Generate second text description data of the sample audio data based on the keywords.

[0140] In one embodiment, after the electronic device obtains the keyword of the sample audio data, the keyword can be input into a text expansion model (such as a T5 model). Accordingly, the text expansion model can output the second text description data of the multiple sample audio data. Subsequently, the electronic device can form multiple training sample pairs with one sample audio data and multiple second text description data, thereby enriching the training samples.

[0141] Fig.10 This is a schematic diagram of an example of a method for speech recognition provided in an embodiment of the present application. Figure 8As shown, first, the electronic device obtains the audio data to be identified and the first text description data of each classification category preset. Then, the electronic device inputs the audio data into the audio encoder, outputs the first audio feature of the audio data, and inputs the first text description data of each classification category into the text encoder, and outputs the first text feature of each classification category. After that, the electronic device inputs the first audio feature of the audio data and the first text feature of each classification category into the linear projection module, and outputs the second audio feature of the audio data and the second text feature of each classification category. Among them, the second audio feature and the second text feature have the same dimension. After that, the electronic device inputs the second audio feature of the audio data and the second text feature of each classification category into the similarity judgment module, and outputs the similarity score between the audio data and each classification category. Finally, the electronic device determines the target classification category corresponding to the audio data in each classification category based on the similarity score between the audio data and each classification category.

[0142] Fig.11 The structure diagram of a speech recognition device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the speech recognition apparatus 1100 includes: a first acquisition module 1110 , an output module 1120 , a determination module 1130 and a recognition module 1140 .

[0143] in:

[0144] The first acquisition module 1110 is configured to: acquire audio data to be recognized and first text description data of each preset classification category;

[0145] The output module 1120 is configured to: input the audio data and the first text description data of each classification category into a pre-trained speech classification model, and output a similarity score between the audio data and each classification category;

[0146] The determination module 1130 is configured to: determine, in each of the classification categories, a target classification category corresponding to the audio data based on a similarity score between the audio data and each of the classification categories;

[0147] The recognition module 1140 is configured to perform speech recognition processing on the audio data according to the target classification category corresponding to the audio data.

[0148] In some embodiments, the speech classification model includes an audio encoder, a text encoder, a linear projection module and a similarity determination module; the output module 1120 is specifically used to:

[0149] Inputting the audio data into the audio encoder and outputting a first audio feature of the audio data;

[0150] Inputting the first text description data of each of the classification categories into the text encoder, and outputting the first text features of each of the classification categories;

[0151] Inputting the first audio feature of the audio data and the first text feature of each of the classification categories into a linear projection module, outputting the second audio feature of the audio data and the second text feature of each of the classification categories, wherein the second audio feature and each of the second text features have the same dimension;

[0152] The second audio feature of the audio data and the second text feature of each of the classification categories are input into a similarity determination module, and a similarity score between the audio data and each of the classification categories is output.

[0153] In some embodiments, the audio encoder includes a first feature encoding unit and a second feature encoding unit, the first feature encoding unit is multiple, the multiple first feature encoding units are stacked, the first feature encoding unit includes a feature grouping subunit, a feature processing subunit and a feature splicing subunit, the feature processing subunit includes multiple feature processing layers distributed in parallel; the output module 1120 is specifically used to:

[0154] The audio input feature is divided into a plurality of first audio sub-features by the feature grouping sub-unit; the audio input feature includes the Mel spectrum feature of the audio data and / or the audio output feature of the first feature encoding unit at the previous level;

[0155] Performing feature encoding processing on the plurality of first audio sub-features respectively through the plurality of feature processing layers to obtain a plurality of second audio sub-features;

[0156] The feature splicing subunit splices a plurality of the second audio sub-features to obtain an audio output feature;

[0157] The second feature encoding unit performs feature fusion processing on the audio output features output by the plurality of the first feature encoding units to obtain a first audio feature of the audio data.

[0158] In some embodiments, the feature processing subunit further includes a plurality of attention fusion layers, and the plurality of attention fusion layers correspond one-to-one to the plurality of feature processing layers; the method further includes:

[0159] a fusion module, configured to calculate an attention weight according to the first audio sub-feature and the second audio sub-feature output by the other feature processing layer through the attention fusion layer, and perform feature fusion processing on the first audio sub-feature and the second audio sub-feature output by the other feature processing layer according to the attention weight to obtain a fusion result;

[0160] The processing module is configured to perform feature encoding processing on the fusion result through the feature processing layer to obtain a second audio sub-feature.

[0161] In some embodiments, the apparatus further comprises:

[0162] A second acquisition module is configured to: acquire a training sample set, wherein the training sample set includes sample audio data and second text description data of the sample audio data;

[0163] The training module is configured to: train the initial speech classification model based on the training sample set to obtain a trained speech classification model.

[0164] In some embodiments, the training module is specifically used to:

[0165] Inputting the sample audio data into the audio encoder, and outputting the third audio feature of the sample audio data;

[0166] Inputting the second text description data of the sample audio data into the text encoder, and outputting the third text feature of the second text description data;

[0167] Inputting the third audio feature of the sample audio data and the third text feature of the second text description data into a linear projection module, outputting the fourth audio feature of the sample audio data and the fourth text feature of the second text description data, wherein the fourth audio feature and the fourth text feature have the same dimension;

[0168] Based on the fourth audio feature, the fourth text feature and the loss function corresponding to the similarity discrimination module, the audio encoder, the text encoder and the linear projection module are jointly trained to obtain a trained speech classification model.

[0169] In some embodiments, the loss function is:

[0170]

[0171] Among them, L represents the function value of the loss function, N represents the number of training sample pairs formed by sample audio data and text description data in the training sample set, represents the second audio feature corresponding to the i-th sample audio data, represents the second audio feature corresponding to the j-th sample audio data, represents the second text feature corresponding to the text description data of the i-th sample audio data, represents the second text feature corresponding to the text description data of the j-th sample audio data, and τ represents the temperature parameter.

[0172] In some embodiments, the second acquisition module is specifically used to:

[0173] Acquire sample audio data and keywords of the sample audio data;

[0174] Based on the keywords, second text description data of the sample audio data is generated.

[0175] Fig.12 A schematic diagram of an electronic device provided for an embodiment of the present application. In some embodiments, the electronic device includes one or more processors and a memory. The memory is configured to store one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of speech recognition in the above-mentioned embodiment.

[0176] like Fig.12 As shown, the electronic device 1200 includes: a processor (processor) 1201 and a memory 1202. Exemplarily, the electronic device 1200 may also include: a communication interface (Communications Interface) 1203 and a communication bus 1204.

[0177] The processor 1201, the memory 1202 and the communication interface 1203 communicate with each other via the communication bus 1204. The communication interface 1203 is used to communicate with other devices such as client or other server network elements.

[0178] In some embodiments, the processor 1201 is used to execute the program 1205, and specifically can execute the relevant steps in the above-mentioned speech recognition method embodiment. Specifically, the program 1205 can include program code, and the program code includes computer executable instructions.

[0179] Exemplarily, the processor 1201 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The electronic device 1200 may include one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0180] In some embodiments, the memory 1202 is used to store the program 1205. The memory 1202 may include a high-speed RAM memory, and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0181] Program 1205 can be specifically called by processor 1201 to enable electronic device 1200 to perform operations of the method for voice recognition.

[0182] An embodiment of the present application provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on the electronic device 1200, the electronic device 1200 executes the voice recognition method in the above embodiment.

[0183] The executable instructions can be specifically used to enable the electronic device 1200 to perform operations of the voice recognition method.

[0184] For example, the computer readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0185] In some embodiments, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the method of speech recognition described in any of the above embodiments.

[0186] In some embodiments, the embodiments of the present application further provide a computer program, which, when executed by a processor, can implement the method of speech recognition described in any of the above embodiments.

[0187] The beneficial effects that can be achieved by the speech classification generation device, electronic device, computer-readable storage medium, computer program product and computer program provided in the embodiments of the present application can refer to the beneficial effects of the speech recognition method provided above, and will not be repeated here.

[0188] It should be noted that, in the application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0189] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0190] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0191] For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by an instruction execution system, apparatus, or device or in conjunction with such instruction execution system, apparatus, or device.

[0192] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM).

[0193] In addition, the computer readable medium can even be a paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, and then editing, interpreting or processing in other suitable ways as necessary, and then storing it in the computer memory. It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof.

[0194] In the above embodiments, multiple steps or methods may be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it may be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0195] The above-described embodiments of the present application do not constitute a limitation on the protection scope of the present application.

Claims

1. A method for speech recognition, characterized in that: The method comprises: Acquire audio data to be identified and first text description data of each preset classification category; Inputting the audio data and the first text description data of each classification category into a pre-trained speech classification model, and outputting a similarity score between the audio data and each classification category; Based on the similarity scores between the audio data and each of the classification categories, determining a target classification category corresponding to the audio data in each of the classification categories; The audio data is subjected to speech recognition processing according to the target classification category corresponding to the audio data.

2. The method according to claim 1, characterized in that The speech classification model includes an audio encoder, a text encoder, a linear projection module and a similarity discrimination module; the audio data and the first text description data of each classification category are input into the pre-trained speech classification model, and the similarity score between the audio data and each classification category is output, including: Inputting the audio data into the audio encoder and outputting a first audio feature of the audio data; Inputting the first text description data of each of the classification categories into the text encoder, and outputting the first text features of each of the classification categories; Inputting the first audio feature of the audio data and the first text feature of each of the classification categories into a linear projection module, outputting the second audio feature of the audio data and the second text feature of each of the classification categories, wherein the second audio feature and each of the second text features have the same dimension; The second audio feature of the audio data and the second text feature of each of the classification categories are input into a similarity determination module, and a similarity score between the audio data and each of the classification categories is output.

3. The method according to claim 2, characterized in that The audio encoder includes a first feature encoding unit and a second feature encoding unit, the first feature encoding unit is multiple, the multiple first feature encoding units are stacked, the first feature encoding unit includes a feature grouping subunit, a feature processing subunit and a feature splicing subunit, the feature processing subunit includes multiple feature processing layers that are distributed in parallel; the audio data is input into the audio encoder, and the first audio feature of the audio data is output, including: The audio input feature is divided into a plurality of first audio sub-features by the feature grouping sub-unit; the audio input feature includes the Mel spectrum feature of the audio data and / or the audio output feature of the first feature encoding unit at the previous level; Performing feature encoding processing on the plurality of first audio sub-features respectively through the plurality of feature processing layers to obtain a plurality of second audio sub-features; The feature splicing subunit splices a plurality of the second audio sub-features to obtain an audio output feature; The second feature encoding unit performs feature fusion processing on the audio output features output by the plurality of the first feature encoding units to obtain a first audio feature of the audio data.

4. The method according to claim 3, characterized in that The feature processing subunit further includes a plurality of attention fusion layers, and the plurality of attention fusion layers correspond one-to-one to the plurality of feature processing layers; the method further includes: The attention fusion layer calculates an attention weight according to the first audio sub-feature and the second audio sub-feature output by the other feature processing layer, and performs feature fusion processing on the first audio sub-feature and the second audio sub-feature output by the other feature processing layer according to the attention weight to obtain a fusion result; The fusion result is subjected to feature encoding processing by the feature processing layer to obtain a second audio sub-feature.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a training sample set, wherein the training sample set includes sample audio data and second text description data of the sample audio data; Based on the training sample set, the initial speech classification model is trained to obtain a trained speech classification model.

6. The method according to claim 5, characterized in that The initial speech classification model includes an audio encoder, a text encoder, a linear projection module and a similarity discrimination module; The initial speech classification model is trained based on the training sample set to obtain a trained speech classification model, including: Inputting the sample audio data into the audio encoder, and outputting the third audio feature of the sample audio data; Inputting the second text description data of the sample audio data into the text encoder, and outputting the third text feature of the second text description data; Inputting the third audio feature of the sample audio data and the third text feature of the second text description data into a linear projection module, outputting the fourth audio feature of the sample audio data and the fourth text feature of the second text description data, wherein the fourth audio feature and the fourth text feature have the same dimension; Based on the fourth audio feature, the fourth text feature and the loss function corresponding to the similarity discrimination module, the audio encoder, the text encoder and the linear projection module are jointly trained to obtain a trained speech classification model.

7. The method according to claim 6, characterized in that The loss function is: Among them, L represents the function value of the loss function, N represents the number of training sample pairs formed by sample audio data and text description data in the training sample set, represents the second audio feature corresponding to the i-th sample audio data, represents the second audio feature corresponding to the j-th sample audio data, represents the second text feature corresponding to the text description data of the i-th sample audio data, represents the second text feature corresponding to the text description data of the j-th sample audio data, and τ represents the temperature parameter.

8. The method according to claim 5, characterized in that The step of obtaining a training sample set includes: Acquire sample audio data and keywords of the sample audio data; Based on the keywords, second text description data of the sample audio data is generated.

9. A speech recognition device, characterized in that: The device comprises: A first acquisition module is configured to: acquire audio data to be recognized and first text description data of each preset classification category; An output module is configured to: input the audio data and the first text description data of each classification category into a pre-trained speech classification model, and output a similarity score between the audio data and each classification category; A determination module is configured to: determine, in each of the classification categories, a target classification category corresponding to the audio data based on a similarity score between the audio data and each of the classification categories; The recognition module is configured to perform speech recognition processing on the audio data according to the target classification category corresponding to the audio data.

10. An electronic device comprising: processor; and a memory storing a program; wherein the program comprises instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method steps of any one of claims 1 to 8 are implemented.

Images