A speech recognition method, device, electronic device, and storage medium

Abstract

This application provides a speech recognition method applied in the field of speech recognition. By acquiring current speech data, already recognized speech data, and a hot word library matching the current context, the method determines the model score of the current speech data. Based on the model score, it identifies a first speech data point from the current speech data. The first speech data is combined with the already recognized speech data to obtain a second speech data point. The method determines the matching degree with preset hot words in the hot word library and groups the second speech data according to the matching degree. It then determines the first path score corresponding to each second speech data point in the group and identifies a third speech data group from the group based on the first path score. The third speech data group is then filtered to obtain a target speech result, which is then determined as the recognition result of the current speech data. Through filtering at different stages, the method effectively reduces the speech recognition error rate when the speech data contains hot words.

Description

Technical Field

[0001] This application relates to the field of speech recognition, and in particular to speech recognition methods, devices, electronic devices and storage media. Background Technology

[0002] In the field of speech recognition, hotword identification is of great significance, especially in domain-specific applications. Hotwords typically refer to domain-specific vocabulary that represents the semantic core of a speech segment. Therefore, the accuracy of hotword identification is often more critical than the accuracy of the entire speech segment. Incorrect hotword identification directly impacts the understanding of the entire speech segment; thus, hotwords require close attention during the speech recognition process.

[0003] In speech recognition implementations, path selection is typically used to determine the optimal recognition result. However, due to the diversity of paths and limitations in computational resources, it's impossible to traverse all possible paths. Therefore, during frame-by-frame recognition, a top-k sampling method is usually employed. This method retains only the k highest-scoring paths each time a new path is obtained, discarding the others. While top-k sampling effectively reduces computation, it is a locally optimal strategy, not a globally optimal one. This means that in some cases, correct path prefixes may be prematurely discarded due to low scores, resulting in the failure to obtain results containing hot words. This phenomenon significantly reduces the recall rate of hot words, impacting the overall performance of speech recognition. Summary of the Invention

[0004] In view of the above problems, a speech recognition method, apparatus, electronic device, and storage medium are proposed to overcome or at least partially solve the above problems, including:

[0005] A speech recognition method, the method comprising:

[0006] Acquire current speech data, recognized vocalization data, and a hot word library matching the current context;

[0007] Based on the identified vocal data and the current speech data, determine the current vocal data corresponding to the current speech data and the model score corresponding to the current vocal data;

[0008] Based on the model score, a first vocal data is determined from the current vocal data, and the first vocal data is combined with the identified vocal data to obtain a second vocal data.

[0009] Determine the matching degree between the second vocal data and the preset hot words in the hot word library, and group the second vocal data according to the matching degree to obtain the second vocal data group;

[0010] For each second vocal data group, determine the first path score corresponding to each second vocal data in the second vocal data group, and determine the third vocal data group from the second vocal data group based on the first path score;

[0011] The third vocal data group is filtered to obtain the target vocal result;

[0012] The target vocalization result is determined as the recognition result of the current speech data.

[0013] In an optional embodiment of this application, determining the current voice data corresponding to the current voice data and the model score corresponding to the current voice data based on the identified voice data and the current voice data includes:

[0014] Feature extraction is performed on the current speech data to obtain audio features;

[0015] The audio features are input into a preset encoder to obtain a first feature matrix;

[0016] The identified vocal data is input into a preset decoder to obtain a second feature matrix;

[0017] The first feature matrix and the second feature matrix are input to a preset connector to obtain the current vocal data corresponding to the current speech data and the model score corresponding to the current vocal data.

[0018] In an optional embodiment of this application, determining the first vocal data from the current vocal data based on the model score includes:

[0019] The current vocal data is sorted according to the model score to obtain the sorted current vocal data;

[0020] The first target number of first vocal data is determined from the sorted current vocal data according to the preset first target number.

[0021] In an optional embodiment of this application, after determining the first target number of first vocal data from the sorted current vocal data according to a preset first target number, the process includes:

[0022] The bias score between the first vocal data and the identified vocal data is calculated using a pre-set hot word detection algorithm.

[0023] In an optional embodiment of this application, determining the first path score corresponding to each second vocal data piece in each second vocal data group includes:

[0024] Obtain the second path score of the identified vocal data in the second vocal data;

[0025] The second path score, the model score, and the bias score are summed to obtain the first path score corresponding to the second vocal data.

[0026] In an optional embodiment of this application, determining the third vocal data group from the second vocal data group based on the first path score includes:

[0027] Based on the first path score, the second vocal data contained in the second vocal data group is sorted to obtain the sorted second vocal data group;

[0028] According to the preset second target number, the second voice data of the second target number is selected from the sorted second voice data group to obtain the third voice data group.

[0029] In an optional embodiment of this application, the third vocal data group includes a fourth vocal data group and a fifth vocal data group, and the step of filtering the third vocal data group to obtain the target vocal result includes:

[0030] Calculate the difference between the first path score of the fourth vocal data and the first path score of the fifth vocal data to obtain the path score difference;

[0031] Calculate the difference between the matching degree of the fourth vocal data and the matching degree of the fifth vocal data to obtain the matching degree difference;

[0032] The relationship between the path score difference and the matching degree difference is determined, and the third vocal data group is filtered based on the relationship between the path score difference and the matching degree difference to obtain the target vocal result.

[0033] This application also discloses a speech recognition device, the device comprising:

[0034] The data acquisition module is configured to acquire current speech data, recognized speech data, and a hot word library that matches the current context;

[0035] The model score determination module is configured to determine the current voice data corresponding to the current voice data and the model score corresponding to the current voice data based on the identified voice data and the current voice data.

[0036] The first sampling module is configured to determine the first vocal data from the current vocal data based on the model score, and combine the first vocal data with the identified vocal data to obtain the second vocal data.

[0037] The data binning module is configured to determine the matching degree between the second vocal data and preset hot words in the hot word library, and to group the second vocal data according to the matching degree to obtain second vocal data groups;

[0038] The second sampling module is configured to, for each second vocal data group, determine the first path score corresponding to each second vocal data in the second vocal data group, and determine the third vocal data group from the second vocal data group based on the first path score;

[0039] The third sampling module is configured to filter the third vocal data group to obtain the target vocal result;

[0040] The result determination module is configured to determine the target vocalization result as the recognition result of the current speech data.

[0041] This application also discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the speech recognition method as described above.

[0042] This application also discloses a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, implements the speech recognition method as described above.

[0043] The embodiments of this application have the following advantages:

[0044] By acquiring current speech data, identified speech data, and a hot word library matching the current context, the system determines the current speech data and its corresponding model score based on the identified speech data and the current speech data. Based on the model score, a first speech data point is identified from the current speech data. This first speech data point is then combined with the identified speech data to obtain a second speech data point. The matching degree between the second speech data point and preset hot words in the hot word library is determined, and the second speech data point is grouped according to the matching degree. For each second speech data group, a first path score is determined for each second speech data point within that group. Based on the first path score, a third speech data group is identified from the second speech data group. This third speech data group is then filtered to obtain the target speech result, which is then used as the recognition result for the current speech data. This effectively reduces the speech recognition error rate when the speech data contains hot words. Attached Figure Description

[0045] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating the steps of a speech recognition method according to an embodiment of this application;

[0047] Figure 2 This is a structural block diagram of a speech recognition device provided in one embodiment of this application. Detailed Implementation

[0048] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0049] Reference Figure 1 The diagram illustrates a flowchart of a speech recognition method according to an embodiment of this application, which may specifically include the following steps:

[0050] Step 101: Obtain the current speech data, the recognized speech data, and the hot word library that matches the current context.

[0051] In this context, "current speech data" refers to all data contained in the current audio frame (or audio segment within the current time window) during speech recognition. Current speech data can include acoustic features such as Mel-frequency cepstral coefficients, spectrograms, and linear prediction coefficients. In speech recognition, "recognized speech data" refers to the multiple possible paths identified at the current moment. These paths are typically represented as a set of tokens, where each token can be a single character. This set of tokens constitutes the recognized speech data. The "hot word library" matching the current context refers to a medical terminology library, a vehicle inspection terminology library, or other specialized terminology libraries.

[0052] Before extracting the first frame, the identified speech data can be used as an initial node representing silence. Then, the speech data of the first frame is used as the current speech data and is recognized. Assuming the current context is a medical environment, it is also necessary to obtain a medical hot word database and start the specific steps of subsequent speech recognition.

[0053] Step 102: Based on the identified vocal data and the current speech data, determine the current vocal data corresponding to the current speech data and the model score corresponding to the current vocal data.

[0054] In speech recognition tasks, speech data is time-series data, and the pronunciation data corresponding to the current speech is usually related to the previously recognized pronunciation data. The model score refers to the probability distribution from the previous frame token to the current frame token.

[0055] In the embodiments of this application, the current voice data corresponding to the current voice data refers to the voice data that the current voice data may correspond to under the influence of the already recognized voice data.

[0056] In some embodiments of this application, step 102 may include the following sub-steps:

[0057] Sub-step 11: Extract features from the current speech data to obtain audio features.

[0058] Sub-step 12: Input the audio features into a preset encoder to obtain the first feature information;

[0059] Sub-step 13: Input the identified vocal data into the preset decoder to obtain the second feature matrix;

[0060] Sub-step 14: Input the first feature information and the second feature information into the preset connector to obtain the current speech data and the model score corresponding to the current speech data.

[0061] Among them, feature extraction of the current speech data to obtain audio features specifically may include steps such as preprocessing, time-frequency conversion, feature calculation, and feature selection. Preprocessing includes noise reduction, normalization, frame segmentation, etc. of the audio signal of the current speech data; time-frequency conversion refers to converting the audio signal from the time domain to the frequency domain (such as STFT, wavelet transform, etc.); feature calculation refers to calculating the required features (such as MFCC, energy, zero-crossing rate, etc.); feature selection refers to selecting a suitable feature subset according to the task requirements; feature normalization is to perform normalization processing on the features for subsequent classification or recognition tasks.

[0062] In some embodiments of the present application, the Zipformer model can be used to extract features from the collected speech data. Zipformer is an efficient speech recognition model architecture that combines three core components: an encoder, a decoder, and a joiner. Among them, the encoder of Zipformer is responsible for extracting the features of the speech data. The input of the encoder is the original data, such as the audio waveform or the feature sequence after feature extraction; the output of the encoder is the feature matrix after processing the feature sequence.

[0063] In some implementation manners of this embodiment, first, feature extraction is performed on the current speech data, and the extracted audio features are input into the encoder of the Zipformer model. The encoder processes the features to obtain a first feature matrix. This first feature matrix is the encoder feature, and the first feature matrix can be a high-dimensional representation of the audio features. The recognized vocal data is input into the preset decoder, and a corresponding second feature matrix is obtained. This second feature matrix is a high-dimensional representation of the recognized vocal data. The first feature matrix (i.e., the feature representation of the current vocal data) and the second feature matrix (i.e., the feature representation of the recognized vocal data) are input into the joiner of the Zipformer model. The joiner calculates the current vocal data that the current speech data may correspond to, and the model score corresponding to the current vocal data. This model score reflects the matching degree between the current vocal data and the recognized vocal data. In a specific implementation, the model score refers to the probability distribution from the recognized vocal data to the current speech data. Assuming there are only 3 Chinese characters, namely ammonia, ethyl, and group, the obtained model score (i.e., the probability distribution) can be a 3×3 matrix M

[0064]

[0065] Among them, Mij represents the probability from the i-th row to the j-th row. For example, M 12 represents that the probability from "ammonia" to "ethyl" is 0.3, M 23The probability of representing "B" to "base" is 0.3.

[0066] Step 103: Based on the model score, determine the first vocal data from the current vocal data, and combine the first vocal data with the identified vocal data to obtain the second vocal data;

[0067] In some implementations of this embodiment, a new corresponding token is added to the identified vocal data to obtain a new path. This new path is the second vocal data in this application embodiment. Since the identified vocal data (current path set) is not a single path, but a current path set composed of multiple paths, and multiple different tokens may be added to each current path, thus generating more branches, multiple new paths are obtained, that is, multiple second vocal data can be obtained.

[0068] In some embodiments of this application, step 103, "determining the first vocal data from the current vocal data based on the model score," may further include the following sub-steps:

[0069] Sub-step 21: Sort the current vocal data according to the model score to obtain the sorted current vocal data;

[0070] Sub-step 22: Determine the first vocal data of the first target quantity from the sorted current vocal data according to the preset first target quantity.

[0071] In some implementations of this embodiment, the model scores of each token calculated in sub-step 13 are sorted in descending order. That is, the current vocal data is sorted in descending order of model scores, and k1 of the highest scores are selected to determine the corresponding tokens. In other words, a first target number k1 first vocal data points are determined from the sorted current vocal data according to a preset first target number k1. Here, k1 can be adjusted according to actual needs. For example, as computer hardware performance continues to improve, k1 can be set to be larger, leaving more data and making the speech recognition results more accurate. Alternatively, if hardware performance cannot be guaranteed, k1 can be set to be smaller, leaving less data and reducing the amount of calculation in subsequent steps.

[0072] In some embodiments of this application, sub-step 22 may be followed by the following sub-steps:

[0073] Sub-step 31: Calculate the bias score between the first vocal data and the identified vocal data using a pre-set hot word detection algorithm.

[0074] The pre-defined hot word detection algorithm refers to the Aho-Corasick Automaton (AC Automaton), which is an efficient multi-pattern matching algorithm. This application uses the AC Automaton for hot word detection. For example, multiple pattern strings (such as hot words or keywords) are constructed into an automaton, so that multiple patterns can be matched simultaneously in one scan.

[0075] In some implementations of this embodiment, the identified speech data and the corresponding token (i.e., the first speech data) are processed by AC-automaton to perform hot word detection calculation, which can obtain the bias score corresponding to the first speech data and the identified speech data.

[0076] After the above sub-steps 21-22 are executed, k1 first vocal data can be determined. Through the execution of sub-step 31, the bias scores of k1 first vocal data and the identified vocal data can be obtained. Then, the identified vocal data and k1 first vocal data can be combined to obtain multiple new paths, i.e. multiple second vocal data.

[0077] Step 104: Determine the matching degree between the second vocal data and the preset hot words in the hot word library, and group the second vocal data according to the matching degree to obtain the second vocal data group.

[0078] A hot word database is a collection of pre-defined hot words. For example, a medical hot word database might include commonly used drug names, disease names, symptom descriptions, and medical terminology. These hot words play a crucial role in scenarios such as medical speech recognition, electronic medical records, and intelligent consultation systems.

[0079] In some implementations of this embodiment, the obtained new path (i.e., the second vocal data) is grouped according to the prefix length of the hot words matched with the hot word library. Assuming that the hot word library in the medical context contains the pre-set hot word "aminoethyl indole," and the second vocal data composed of the identified vocal data and the second vocal data corresponding to the current voice data (determined via the first vocal data corresponding to the current voice data) includes phrases like "want some catering," "some catering," "catering," "want some ammonia," "some ammonia," "ammonia," "want some aminoethyl," "some aminoethyl," and "aminoethyl," then the matching degree between the second vocal data and the pre-set hot word can be determined. Multiple second vocal data are then grouped according to the matching degree, also known as binning, resulting in different groups of second vocal data. For example, the matching degree between "want some aminoethyl," "some aminoethyl," and "aminoethyl" and the pre-set hot word "aminoethyl indole" is 2; the matching degree between "want some ammonia," "some ammonia," and "ammonia" and the pre-set hot word "aminoethyl indole" is 1; while the matching degree between "want some catering," "some catering," and "catering" and the pre-set hot word is 0. These nine second voice data can then be divided into three groups: the second voice data group with a matching degree of 2 contains "want some ammonia B", "some ammonia B", "ammonia B", the second voice data group with a matching degree of 1 contains "want some ammonia", "some ammonia", "ammonia", and the second voice data group with a matching degree of 0 contains "want some catering", "some catering", "catering".

[0080] Step 105: For each second vocal data group, determine the first path score corresponding to each second vocal data in the second vocal data group, and determine the third vocal data group from the second vocal data group based on the first path score.

[0081] After determining multiple second vocal data groups, in order to reduce the subsequent computational load, this embodiment of the application performs a second sampling on each group, and can determine a third vocal data group that is more in line with the actual situation from the second vocal data groups based on the first path score corresponding to the second vocal data.

[0082] The first path score can refer to the sum of the path score of the identified vocal data, the model score calculated by the Joiner model in the aforementioned steps, and the bias score calculated by the AC-automaton in the aforementioned steps. The second vocal data group is then filtered based on the first path score. In some embodiments of this example, the use of the bias score can be illustrated by the following example: Suppose the original path score for "want some ammonia" is 7. Since the AC-automaton calculates its matching degree as 1 and the bias score as 3, the biased score for this path can be 7 + 3 × 1. Similarly, the original path score for "want some ammonia B" is 9. Since the matching degree is 2, the biased score is 9 + 2 × 6 = 15.

[0083] In some embodiments of this application, step 105, "for each second vocal data group, determine the first path score corresponding to each second vocal data in the second vocal data group," may include the following sub-steps:

[0084] Sub-step 41: Obtain the second path score of the identified vocal data in the second vocal data.

[0085] Sub-step 42: Sum the second path score, model score, and bias score to obtain the first path score corresponding to the second vocal data.

[0086] The second path score of the identified speech data refers to the path score calculated during the recognition stage of the identified speech data. In order to distinguish it from the path score corresponding to the second speech data, this application refers to the path score of the identified speech data as the second path score. The model score and bias score obtained in the aforementioned steps are summed with the second path score of the identified speech data to obtain the first path score of the second speech data, which is the path score of the newly generated path.

[0087] In some embodiments of this application, step 105, "determining the third vocal data group from the second vocal data group based on the first path score," may include the following sub-steps:

[0088] Sub-step 51: Based on the first path score, sort the second vocal data contained in the second vocal data group to obtain the sorted second vocal data group.

[0089] Sub-step 52: Select the second target number of second vocal data from the sorted second vocal data group according to the preset second target number to obtain the third vocal data group.

[0090] In some implementations of this embodiment, the second vocal data in each group is sorted by score, and the k2 paths with the highest scores are retained. The number of second targets is k2. Specifically, the number of second targets k2 can be 4. Assuming k2 is 4, all data in the second vocal data groups with matching degrees of 2, 1, and 0 can be retained. Of course, the value of k2 can be adjusted according to actual needs. Assuming k2 is adjusted to 2, in this example, "Ammonium B" in the second vocal data with matching degree of 2 can be retained. The second vocal data group with matching degree of 2 may retain "Want some Ammonium B" and "Some Ammonium B". The second vocal data group with matching degree of 1 may retain "Want some Ammonium" and "Some Ammonium". The second vocal data group with matching degree of 0 may retain "Want some Catering" and "Some Catering". That is, the resulting third vocal data groups are the matching degree group 2, the matching degree group 1, and the matching degree group 0, respectively.

[0091] Step 106: Filter the third vocal data group to obtain the target vocal result.

[0092] After obtaining the third group of vocal data, this application further filters the vocal data in each third group to further reduce the number of vocal data and further reduce the amount of subsequent calculations.

[0093] In some embodiments of this application, the third acoustic data group includes the fourth acoustic data and the fifth acoustic data, and step 106 may include the following sub-steps:

[0094] Sub-step 41: Calculate the difference between the first path score of the fourth vocal data and the first path score of the fifth vocal data to obtain the path score difference;

[0095] Sub-step 42: Calculate the difference between the matching degree of the fourth vocal data and the matching degree of the fifth vocal data to obtain the matching degree difference;

[0096] Sub-step 43: Determine the relationship between the path score difference and the matching degree difference, and filter the third vocal data group based on the relationship between the path score difference and the matching degree difference to obtain the target vocal result.

[0097] The fourth voice data refers to the high-matching path between two adjacent groups in the third voice data group, and the fifth voice data refers to the low-matching path between two adjacent groups in the third voice data group.

[0098] In some implementations of this embodiment, the third vocal data grouping can be filtered using a combination of reverse sampling and forward sampling. Reverse sampling refers to sampling from high to low matching lengths, while forward sampling refers to sampling from low to high matching lengths. Assuming that after grouping according to matching degree, we get group 0, group 1, group 2, group 3...group 8, where group 0 represents a matching degree of 1, group 8 represents a matching degree of 8; for downward sampling, the path score difference between group 0 and group 1 is compared with the matching degree difference for cropping; then group 1 is compared with group 2 for cropping, and so on. If group 2 is empty when comparing group 1 and group 2, then group 1 is compared with group 3. Cropping from high to low is the reverse. In some implementations, the third vocal data grouping can be filtered using a combination of reverse sampling and forward sampling; in other implementations, it can be filtered using a combination of forward sampling and reverse sampling.

[0099] If the following conditions are met, retain the high-matching path and discard the low-matching path;

[0100] High matching path score - low matching path score > poor matching degree * bias score.

[0101] For the group with a matching degree of 1 in the above example, "want some ammonia", assuming the calculated path score is 10, and for the group with a matching degree of 2 in the above example, "want some ammonia B", assuming the calculated path score is 14, if the bias score is 3, then in the implementation of this embodiment, sampling from high to low is: 14-10>(2-1)*3, so the path "want some ammonia" with a matching degree of 1 can be discarded.

[0102] Low-matching paths are retained and high-matching paths are discarded when the following conditions are met;

[0103] High matching path score - Low matching path score < Matching difference * Bias score - beam

[0104] Considering that there may be a length difference of 1 to 2 tokens between different target speech data, some implementations can set the threshold beam to 3 times the bias score.

[0105] For the group with a matching degree of 1 in the above example, "want some ammonia", assuming the calculated path score is 14, and for the group with a matching degree of 2, "want some ammonia B", assuming the calculated path score is 7, if the bias score is 3, the beam is 3×3=9, then in the implementation of this example, when sampling from low to high: 7-14<(2-1)*3-9, the path "want some ammonia B" with a matching degree of 2 can be discarded.

[0106] Step 107: Determine the target speech result as the recognition result of the current speech data.

[0107] As can be seen from the above steps, after screening the grouped third vocalization data, the target vocalization result is obtained, and the finally retained target vocalization result is used as the recognition result of the current voice data, and the processing of the next frame is continued until all frames are traversed, and the path with the highest score is selected as the final result.

[0108] In the prior art, top-k sampling is often used for speech recognition. When top-k obtains a new path each time, only the top k paths with the highest scores are retained, and other paths are trimmed. However, since the top-k sampling method is locally optimal rather than globally optimal, there may be a situation where the correct path prefix is discarded in advance due to a too low score in the middle. The embodiment of the present application effectively avoids the following two most common problems in top-k sampling in the prior art through the above speech recognition steps.

[0109] Problem 1: Since the bias score accumulates gradually with the matching degree. When inferring the first few words at the beginning of the inference, even after adding the current bias score, the path score of the candidate vocalization data containing the hot word prefix is still not high enough. As a result, in subsequent top-k sampling, the candidate vocalization data containing the hot word prefix is trimmed. For example:

[0110] Hot word: aminoethyl indole

[0111] Correct result: Aminoethyl indole is a commonly used anti-inflammatory drug

[0112] Incorrect result: Dining machine drink flower is a commonly used anti-inflammatory drug

[0113] In top-k sampling in the prior art, when inferring the first word 'amino', since the bias score has not been accumulated yet, the path score is not high, and it may be trimmed in advance. Assume that k = 4 in top-k, and the possible current paths and their corresponding path scores are respectively 'dining': 10 points, 'catering': 12 points, 'ginseng': 7 points, 'peace': 6 points, 'amino': 2 points. Among them, 'amino' matches the hot word, and the matching degree is 1. The path score after bias is 2 + 3 = 5. Since k = 4, 4 paths are retained, resulting in the correct path 'amino' being trimmed. In the embodiment of the present application, since multiple vocalization data are grouped according to the matching degree between the candidate vocalization data and the preset hot words in the hot word library, the matching degree of the first word 'amino' is 1, and it is not in the same group as other high-score paths with a matching degree of 0, so it is retained.

[0114] Problem 2: In fact, the voice data does not contain hot words, but due to the prefix part being similar to the hot word, affected by the bias score, the path score of the candidate vocalization data that matches the hot word is relatively high. When top-k trims, the correct candidate vocalization data is trimmed in advance. For example:

[0115] Hot topic: Pediatric four-dimensional calcium dry suspension

[0116] Correct identification result: Children's thinking training is beneficial to brain development.

[0117] Incorrect identification result: 4D training for children is beneficial to brain development

[0118] The reason for the error: Assuming k=4 in topk, the current path and its score may be as follows: "Xiaoersi": 8 points, "Liangxiaoersi": 2 points, after bias the score to 2+3*3=11, "Xiaoersi": 4 points, after bias the score to 4+3*3=13, "Liangxiaoersiwei" scores 1, after bias the score to 1+4*3=13, "Xiaoersiwei": 2 points, after bias the score to 2+4*3=14. Since 4 paths are retained, the correct path "Xiaoersi" is cut off, resulting in the erroneous conclusion that "Xiaoersiwei training is beneficial to brain development".

[0119] In this embodiment, candidate vocal data is grouped according to their matching degree with pre-set hot words in the hot word library. Paths containing "children's thinking" (matching degree 0) and paths containing "children's four dimensions" (matching degree 4) are not in the same group and are therefore retained. Furthermore, since "children's thinking" receives a higher model score when the token "training" is added next, it is ultimately retained, resulting in the correct final result "children's thinking training is beneficial to brain development." Moreover, by grouping candidate vocal data and filtering within each group based on its first path score, and then performing forward and reverse sampling between different groups based on matching degree, the number of retained filtering results is effectively reduced, thus lowering the computational load, with minimal impact on the final result.

[0120] This application's embodiments perform performance tests on a medical terminology test set, specifically streaming and non-streaming methods. Streaming refers to real-time processing of speech data frame by frame or segment by segment as it arrives, without waiting for the entire speech data to be input before processing. This is suitable for scenarios requiring real-time response, such as real-time speech recognition, voice assistants, and speech translation. Non-streaming refers to processing all speech data at once after it has been input, outputting the final recognition result. This is suitable for scenarios that do not require real-time response, such as offline speech recognition and speech-to-text.

[0121] Test data: Medical terminology, 1869 audio clips, total duration 11330s, hot word database contains 1495 hot words; Test environment: CPU (Central Processing Unit) single thread, including vad (Voice Activity Detection) module, using streaming and non-streaming models; Test parameters: single token bias score 3.5, k is 4 in topk sampling. In this application, when determining the current vocal data of the current speech data, k1 is set to 20; when determining the third vocal data from the second vocal data contained in the second vocal data group, k2 is set to 4, that is, there are 4 vocal data in each third vocal data group; the specific test results are shown in Tables 1 and 2 below. As can be calculated from the data in the tables, under the condition that the increase in time consumption is not significant, the speech recognition method of this application embodiment effectively reduces the word error rate compared with the related technology topk sampling method for speech recognition, with a relative reduction of 29.5% for non-streaming and 38.2% for streaming and 29.5% for non-streaming. At the same time, the speech recognition method of this application embodiment is equivalent to the related technology topk method for speech recognition, which improves the recall rate of hot words, with a relative improvement of 24.3% for non-streaming and 40% for streaming.

[0122] Table 1: Non-flow cytometry

[0123] Word error rate recall time consuming Do not use hot words 5.24％ 44.1％ 626s Related technologies topk 4.34％ 63.8％ 1601s Examples of this application 3.06％ 79.3％ 1914s

[0124] Table 2: Flow Cytometry

[0125] Word error rate recall time consuming Do not use hot words 7.77％ 33.2％ 1215s Related technologies topk 7.90％ 55.7％ 2092s Examples of this application 4.88％ 78.0％ 2385s

[0126] This application embodiment acquires current speech data, identified speech data, and a hot word library matching the current context. Based on the identified speech data and the current speech data, it determines the current speech data and the corresponding model score. Based on the model score, it identifies the first speech data from the current speech data. The first speech data is combined with the identified speech data to obtain the second speech data. It determines the matching degree between the second speech data and preset hot words in the hot word library, and groups the second speech data according to the matching degree. For each second speech data group, it determines the first path score corresponding to each second speech data in the second speech data group, and determines the third speech data group from the second speech data group based on the first path score. It then filters the third speech data group to obtain the target speech result, and determines the target speech result as the recognition result of the current speech data. This effectively reduces the speech recognition error rate when the speech data contains hot words.

[0127] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0128] Reference Figure 2 The diagram shows a structural schematic of a speech recognition device according to an embodiment of this application, which may specifically include the following modules:

[0129] The data acquisition module 201 is configured to acquire current speech data, recognized speech data, and a hot word library that matches the current context;

[0130] The model score determination module 202 is configured to determine the current voice data corresponding to the current voice data and the model score corresponding to the current voice data based on the identified voice data and the current voice data.

[0131] The first sampling module 203 is configured to determine the first vocal data from the current vocal data based on the model score, and combine the first vocal data with the identified vocal data to obtain the second vocal data.

[0132] The data binning module 204 is configured to determine the matching degree between the second vocal data and the preset hot words in the hot word library, and to group the second vocal data according to the matching degree to obtain the second vocal data group;

[0133] The second sampling module 205 is configured to, for each second sound data group, determine the first path score corresponding to each second sound data in the second sound data group, and determine the third sound data group from the second sound data group based on the first path score;

[0134] The third sampling module 206 is configured to filter the third vocal data group to obtain the target vocal result;

[0135] The result determination module 207 is configured to determine the target vocalization result as the recognition result of the current speech data.

[0136] In an optional embodiment of this application, the sound data determination module 202 includes:

[0137] The first feature extraction submodule is configured to extract features from the current speech data to obtain audio features;

[0138] The encoding submodule is configured to input the audio features into a preset encoder to obtain a first feature matrix;

[0139] The decoding submodule is configured to input the identified vocal data into a preset decoder to obtain a second feature matrix;

[0140] The score determination submodule is configured to input the first feature matrix and the second feature matrix into a preset connector to obtain the current vocal data corresponding to the current vocal data and the model score corresponding to the current vocal data.

[0141] In an optional embodiment of this application, the first sampling module 203 includes:

[0142] The first determining subunit is configured to sort the current vocal data according to the model score to obtain sorted current vocal data;

[0143] The second determining subunit is configured to determine the first target number of first vocal data from the sorted current vocal data according to a preset first target number.

[0144] In an optional embodiment of this application, the apparatus further includes:

[0145] The hot word detection module is configured to calculate the bias score between the first vocal data and the identified vocal data using a preset hot word detection algorithm.

[0146] In an optional embodiment of this application, the second sampling module 205 includes:

[0147] The path score acquisition submodule is configured to acquire the second path score of the identified vocal data in the second vocal data;

[0148] The path score calculation submodule is configured to sum the second path score, the model score, and the bias score to obtain the first path score corresponding to the second vocal data.

[0149] In an optional embodiment of this application, the second sampling module 205 includes:

[0150] The first grouping submodule is configured to sort the second vocal data contained in the second vocal data group according to the first path score, so as to obtain the sorted second vocal data group.

[0151] The second grouping submodule is configured to select a second target number of second vocal data from the sorted second vocal data group according to a preset second target number, to obtain a third vocal data group.

[0152] In an optional embodiment of this application, the third acoustic data group includes fourth acoustic data and fifth acoustic data, and the third sampling module 206 includes:

[0153] The first difference calculation module is configured to calculate the difference between the first path score of the fourth vocal data and the first path score of the fifth vocal data to obtain the path score difference.

[0154] The second difference calculation module is configured to calculate the difference between the matching degree of the fourth vocal data and the matching degree of the fifth vocal data to obtain the matching degree difference.

[0155] The first filtering submodule is configured to determine the relationship between the path score difference and the matching degree difference, and to filter the third vocal data group based on the relationship between the path score difference and the matching degree difference to obtain the target vocal result.

[0156] An embodiment of this application also provides an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the above-described speech recognition method.

[0157] An embodiment of this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for generating game voice.

[0158] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0160] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0161] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0163] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0165] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0166] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.

[0167] The above provides a detailed description of the provided speech recognition method, apparatus, electronic device, and storage medium. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A speech recognition method, characterized in that, The method includes: Acquire current speech data, recognized vocalization data, and a hot word library matching the current context; Based on the identified vocal data and the current speech data, determine the current vocal data corresponding to the current speech data and the model score corresponding to the current vocal data; Based on the model score, a first vocal data is determined from the current vocal data, and the first vocal data is combined with the identified vocal data to obtain a second vocal data. Determine the matching degree between the second vocal data and the preset hot words in the hot word library, and group the second vocal data according to the matching degree to obtain the second vocal data group; For each second vocal data group, determine the first path score corresponding to each second vocal data in the second vocal data group, and determine the third vocal data group from the second vocal data group based on the first path score; The third vocal data group is filtered to obtain the target vocal result; The target vocalization result is determined as the recognition result of the current speech data.

2. The method according to claim 1, characterized in that, The step of determining the current voice data corresponding to the current voice data and the model score corresponding to the current voice data based on the identified voice data and the current voice data includes: Feature extraction is performed on the current speech data to obtain audio features; The audio features are input into a preset encoder to obtain a first feature matrix; The identified vocal data is input into a preset decoder to obtain a second feature matrix; The first feature matrix and the second feature matrix are input to a preset connector to obtain the current vocal data corresponding to the current speech data and the model score corresponding to the current vocal data.

3. The method according to claim 2, characterized in that, The step of determining the first vocal data from the current vocal data based on the model score includes: The current vocal data is sorted according to the model score to obtain the sorted current vocal data; The first target number of first vocal data is determined from the sorted current vocal data according to the preset first target number.

4. The method according to claim 3, characterized in that, After determining the first target number of first vocal data from the sorted current vocal data according to the preset first target number, the process includes: The bias score between the first vocal data and the identified vocal data is calculated using a pre-set hot word detection algorithm.

5. The method according to claim 4, characterized in that, For each second vocal data group, determining the first path score corresponding to each second vocal data item in the second vocal data group includes: Obtain the second path score of the identified vocal data in the second vocal data; The second path score, the model score, and the bias score are summed to obtain the first path score corresponding to the second vocal data.

6. The method according to any one of claims 1-5, characterized in that, The step of determining the third vocal data group from the second vocal data group based on the first path score includes: Based on the first path score, the second vocal data contained in the second vocal data group is sorted to obtain the sorted second vocal data group; According to the preset second target number, the second voice data of the second target number is selected from the sorted second voice data group to obtain the third voice data group.

7. The method according to any one of claims 1-5, characterized in that, The third vocal data group includes the fourth and fifth vocal data. The step of filtering the third vocal data group to obtain the target vocal result includes: Calculate the difference between the first path score of the fourth vocal data and the first path score of the fifth vocal data to obtain the path score difference; Calculate the difference between the matching degree of the fourth vocal data and the matching degree of the fifth vocal data to obtain the matching degree difference; The relationship between the path score difference and the matching degree difference is determined, and the third vocal data group is filtered based on the relationship between the path score difference and the matching degree difference to obtain the target vocal result.

8. A voice recognition device, characterized in that, The device includes: The data acquisition module is configured to acquire current speech data, recognized speech data, and a hot word library that matches the current context; The model score determination module is configured to determine the current voice data corresponding to the current voice data and the model score corresponding to the current voice data based on the identified voice data and the current voice data. The first sampling module is configured to determine the first vocal data from the current vocal data based on the model score, and combine the first vocal data with the identified vocal data to obtain the second vocal data. The data binning module is configured to determine the matching degree between the second vocal data and preset hot words in the hot word library, and to group the second vocal data according to the matching degree to obtain second vocal data groups; The second sampling module is configured to, for each second vocal data group, determine the first path score corresponding to each second vocal data in the second vocal data group, and determine the third vocal data group from the second vocal data group based on the first path score; The third sampling module is configured to filter the third vocal data group to obtain the target vocal result; The result determination module is configured to determine the target vocalization result as the recognition result of the current speech data.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the speech recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the speech recognition method as described in any one of claims 1 to 7.

Images