Speech recognition method, related device, electronic equipment and storage medium

Abstract

The invention discloses a voice recognition method, a related device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the coding based on a to-be-recognized audio, and obtaining a target coding feature; decoding based on the target coding feature to obtain a target hidden layer feature in the decoding process, and selecting at least one semantic entity related to the to-be-recognized audio in a semantic entity library based on the target coding feature; based on the target hidden layer feature and at least one semantic entity, generating a recognition confidence coefficient of the first recognition text; wherein the target coding feature is decoded to finally obtain a first recognition text; and selecting to maintain the first recognition text or revise the first recognition text by referring to the at least one semantic entity based on the recognition confidence, and obtaining a second recognition text of the to-be-recognized audio. According to the scheme, the speech recognition efficiency can be improved on the premise of ensuring that the accuracy of the final recognized text meets the requirement as much as possible.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a speech recognition method and related apparatus, electronic device and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, voice recognition technology, as an important means of human-computer interaction, has been widely used in many fields such as smart homes, intelligent customer service, and in-vehicle systems.

[0003] In existing technologies, the first text predicted from the audio to be recognized by an end-to-end speech recognition model is input into an artificial intelligence model for secondary processing to generate the final recognized text. However, in practical applications, this secondary processing method leads to a decrease in processing speed, thereby reducing the efficiency of speech recognition. Therefore, how to improve the efficiency of speech recognition while ensuring the accuracy of the final recognized text meets the requirements has become an urgent problem to be solved. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a speech recognition method and related devices, electronic devices, and storage media that can improve the efficiency of speech recognition while ensuring the accuracy of the final recognized text meets the requirements as much as possible.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a speech recognition method, comprising: encoding based on audio to be recognized to obtain target encoded features; decoding based on the target encoded features to obtain target hidden layer features during the decoding process, and selecting at least one semantic entity related to the audio to be recognized from a semantic entity library based on the target encoded features; generating a recognition confidence score of a first recognized text based on the target hidden layer features and at least one semantic entity; wherein, decoding the target encoded features ultimately yields the first recognized text; and selecting, based on the recognition confidence score, to maintain the first recognized text or to modify the first recognized text by referring to at least one semantic entity, to obtain a second recognized text of the audio to be recognized.

[0006] To address the aforementioned technical problems, a second aspect of this application provides a speech recognition device, comprising: a target encoding module, a target decoding module, a confidence generation module, and a text determination module. The target encoding module is used to encode based on the audio to be recognized to obtain target encoded features; the target decoding module is used to decode based on the target encoded features to obtain target hidden layer features during the decoding process, and to select at least one semantic entity related to the audio to be recognized from a semantic entity library based on the target encoded features; the confidence generation module is used to generate a recognition confidence of a first recognized text based on the target hidden layer features and at least one semantic entity; wherein, the target encoded features are decoded to finally obtain the first recognized text; the text determination module is used to select whether to maintain the first recognized text or modify the first recognized text by referring to at least one semantic entity based on the recognition confidence to obtain a second recognized text of the audio to be recognized.

[0007] To address the aforementioned technical problems, a third aspect of this application provides an electronic device, including a memory and a processor coupled to each other, wherein the memory stores at least program instructions, and the processor executes the program instructions to implement the speech recognition method of the first aspect.

[0008] To address the aforementioned technical problems, a fourth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the speech recognition method of the first aspect described above.

[0009] The above scheme encodes the audio to be recognized to obtain target encoded features, decodes these features to obtain target hidden features, and finally obtains the first recognized text. Based on the target encoded features, at least one semantic entity related to the audio to be recognized is selected from a semantic entity library. Based on the target hidden features and at least one semantic entity, a recognition confidence score for the first recognized text is generated. Based on the recognition confidence score, the first recognized text is either maintained or modified with reference to at least one semantic entity to obtain the second recognized text of the audio to be recognized. Since the selection of semantic entities is determined based on the matching results between the target encoded features and the semantic entity library, the matching results at the entity level can eliminate interference factors that do not affect the actual semantics of the first recognized text as much as possible. By generating recognition confidence scores from the target hidden features and semantic entities, the degree of fit between the first recognized text and the true semantics of the audio to be recognized can be more accurately reflected, thereby improving the accuracy of the recognition confidence score generation. Furthermore, the decision to modify or maintain the first recognized text based on the recognition confidence score can more accurately determine whether a large language model needs to be introduced to modify the first recognized text, thereby avoiding a decrease in processing speed due to over-reliance on a large language model. Therefore, it can improve the efficiency of speech recognition while ensuring the accuracy of the final recognized text meets the requirements as much as possible. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating an embodiment of the speech recognition method of this application; Figure 2 This is a schematic diagram of the structure of a semantic extraction unit in the speech recognition method of this application; Figure 3 This is a schematic diagram of the structure of an embodiment of the confidence generation unit in the speech recognition method of this application; Figure 4 This is a schematic diagram of the framework of an embodiment of the speech recognition system in the speech recognition method of this application; Figure 5 This is a schematic diagram of the framework of an embodiment of the speech recognition device of this application; Figure 6 This is a schematic diagram of the framework of an embodiment of the electronic device of this application; Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0011] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0012] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0013] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the slash " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper means two or more.

[0014] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0015] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the speech recognition method of this application. Specifically, it may include the following steps: Step S11: Encode the audio to be identified to obtain the target encoded features.

[0016] In one implementation scenario, the audio to be identified can be a voice signal input by a user through an audio acquisition device such as a microphone, or an audio file pre-stored in an electronic device. Specifically, the type of audio to be identified can be streaming audio or non-streaming audio, which is not limited in this application.

[0017] In one implementation scenario, the audio signal corresponding to the audio to be identified is a discrete sampling sequence in the time or frequency dimension. The audio to be identified is encoded to extract the target encoded features of the audio to be identified. Specifically, the encoding process can use a deep neural network model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN) and its variant Long Short-Term Memory (LSTM), to capture the temporal information and local features in the audio signal, thereby obtaining the target encoded features that characterize the acoustic features of the audio to be identified.

[0018] In a specific implementation scenario, the target encoding features are obtained by encoding the audio to be recognized based on the target encoder in the audio recognition model. The target encoder can adopt various structures, such as a convolutional coding structure composed of multiple convolutional layers, which extracts features at different levels by sliding the convolutional kernel on the audio signal, or a cyclic coding structure containing cyclic units, which uses cyclic units to model the temporal dependencies of the audio sequence, etc.

[0019] In a specific implementation scenario, the target encoder in the audio recognition model is an Attention encoder. By introducing an attention mechanism, the Attention encoder automatically learns and focuses on the parts of the audio signal most relevant to the speech recognition task, thereby extracting more discriminative target coding features. Specifically, when processing the audio signal, the Attention encoder assigns an attention weight to each time step or feature location. These weights reflect the importance of the current location to the recognition task. Through weighted summation, the Attention encoder dynamically aggregates key information from the audio signal to generate accurate target coding features.

[0020] In a specific implementation scenario, before generating the target coding features through the Attention encoder, a set of bandpass filters can be used to divide the spectrum of the audio to be recognized into different frequency ranges, extract information such as energy in each range, and then further process the preliminary features output by the filter group, such as performing interpolation and normalization, and input the processed features into the Attention encoder to obtain the target coding features of the audio to be recognized.

[0021] Step S12: Decode based on target coding features to obtain target hidden layer features during the decoding process, and select at least one semantic entity related to the audio to be identified from the semantic entity library based on the target coding features.

[0022] In one implementation scenario, the decoding process can also employ deep neural network models, such as attention-based decoders (e.g., Transformer decoders), to generate corresponding text sequences based on the target encoded features. During decoding, hidden features reflecting the intermediate states of the audio signal are obtained, such as prior and posterior probability matrices related to the target encoded features, to capture the dynamic changes of the audio signal during decoding. These hidden features, which at least partially characterize the intermediate states of the audio signal during decoding, are denoted as the target hidden features.

[0023] In one implementation scenario, based on the matching results between the target encoded features and the semantic entity library, at least one semantic entity related to the audio to be identified is selected. Specifically, the semantic entity library is pre-built and contains various possible semantic units, such as words and phrases. In particular, these semantic entities are associated with specific semantics or concepts.

[0024] In a specific implementation scenario, the selection of semantic entities can be achieved by calculating the similarity between the target encoded features and each entity in the semantic entity library. For example, measures such as cosine similarity and Euclidean distance can be used to calculate the similarity between the target encoded features and the vector representation of each semantic entity, and then select the semantic entities with the highest similarity as entities related to the audio to be identified, or select the semantic entities whose similarity meets a preset minimum threshold as entities related to the audio to be identified. This application does not impose any limitations on this.

[0025] In a specific implementation scenario, feature extraction is performed on each semantic entity in the semantic entity library to obtain the entity semantic features of each semantic entity. Based on the matching results between the target encoded features and the entity semantic features of each semantic entity, at least one semantic entity related to the audio to be identified is selected.

[0026] In a specific implementation scenario, it is understandable that the audio to be identified represents a semantically continuous sentence, while semantic entities are discrete semantic units.

[0027] In a specific implementation scenario, contextual information can be introduced to improve the accuracy of semantic entity selection. Contextual information can include the preceding and following text or scene information of the audio to be identified. By combining contextual information, it is possible to more accurately determine which semantic entities are related to the audio to be identified, thereby improving the accuracy of semantic entity selection.

[0028] In a specific implementation scenario, based on the current audio acquisition scenario, a sub-entity library related to the current audio acquisition scenario can be selected from the semantic entity library. For example, in an in-vehicle voice recognition scenario, priority is given to filtering from sub-libraries containing entity categories such as "navigation," "traffic conditions," and "air conditioning control"; in a smart home scenario, sub-libraries containing entity categories such as "device on / off," "temperature adjustment," and "lighting mode" are used. This scenario-adaptive entity library selection mechanism can significantly narrow the semantic matching range, improving entity selection efficiency while reducing computational resource consumption. Specifically, the corresponding semantic sub-library can be automatically switched based on the pre-defined mapping relationship between scenario tags and entity sub-libraries, combined with the current operating environment of the device.

[0029] In a specific implementation scenario, a target decoder based on an audio recognition model decodes target encoded features. Specifically, the target decoder is an Attention decoder. The decoding of the target encoded features ultimately yields the first recognized text. The Attention decoder dynamically adjusts the focus at each time step during the decoding process through an attention mechanism, automatically focusing on the most relevant parts of the target encoded features when generating each character or word. Specifically, the Attention decoder calculates the association weights between the current decoding state and each position of the target encoded features, and obtains a context vector through weighted summation. This vector, together with the current hidden state of the decoder, generates the character probability distribution of the first recognized text. During the generation of the first recognized text, the Attention decoder simultaneously outputs hidden features reflecting intermediate decoding states. These features, along with entity information obtained through a semantic entity selection mechanism, constitute the basis for calculating the recognition confidence. This scheme, by combining the dynamic features of the decoding process with the semantic entity matching results, can more comprehensively evaluate the semantic consistency between the first recognized text and the original audio, providing a reliable basis for subsequent correction decisions.

[0030] In one implementation scenario, after selecting at least one semantic entity related to the audio to be identified from the semantic entity library, feature extraction is performed based on at least one semantic entity to obtain target semantic features. These target semantic features and target encoded features are then fused to obtain a first fused feature. Decoding is performed based on the first fused feature to obtain the first identified text of the audio. This utilizes the semantic information carried by the semantic entities to complement the acoustic features in the target encoded features. For example, the target semantic features may contain key semantic information such as professional terms related to the audio to be identified and common expressions in specific scenarios, while the target encoded features reflect the acoustic characteristics of the audio, such as pitch and rhythm. The first fused feature obtained by fusing the two more comprehensively covers the information of the audio in both semantic and acoustic dimensions, making the decoded first identified text more accurate and consistent with the actual semantics.

[0031] In a specific implementation scenario, the fusion process can be carried out in various ways. For example, by concatenating the target semantic features and target coding features in a specific dimension, a longer feature vector can be formed. Alternatively, a weighted fusion method can be used, which assigns different weights to the target semantic features and target coding features according to their importance, and then adds the weighted features together to obtain the first fused feature.

[0032] In a specific implementation scenario, to further improve the fusion decoding effect, the target semantic features and target encoded features can be preprocessed. For example, the target semantic features can be normalized to match their numerical range with the target encoded features, avoiding the impact of excessively large differences in feature values ​​on the fusion effect. The target encoded features can be dimensionality reduced to remove redundant information and improve feature compactness, thereby enhancing the representational power of the fused features. These methods make the fused features more reasonable and effective, further improving the accuracy of the first identified text.

[0033] In one implementation scenario, a semantic entity library is invoked based on a confidence assessment model to select at least one semantic entity related to the audio to be identified from the semantic entity library based on the confidence assessment model.

[0034] In a specific implementation scenario, the semantic extraction unit in the confidence assessment model selects at least one semantic entity related to the audio to be identified from the semantic entity library and performs feature extraction based on at least one semantic entity to obtain the target semantic features.

[0035] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a semantic extraction unit in the speech recognition method of this application. Figure 2 As shown, the semantic extraction unit consists of a sequentially connected embedding module and a Transformer module. The embedding module converts each semantic entity in the semantic entity library into a low-dimensional vector representation. These vector representations capture the essential features of the semantic entities, providing a foundation for subsequent entity selection. The Transformer module receives the vector representations output by the embedding module and performs in-depth analysis and comparison of each semantic entity through a self-attention mechanism. Inside the Transformer module, multiple self-attention layers and feedforward neural network layers work together to automatically learn the complex relationships between semantic entities. Based on the feature vectors output by the Transformer module and the target encoded features, it selects and generates at least one semantic entity related to the audio to be recognized, and extracts its features to obtain the target semantic features.

[0036] Step S13: Based on the target hidden layer features and at least one semantic entity, generate the recognition confidence of the first recognition text.

[0037] In this embodiment of the disclosure, the target encoded features are decoded to obtain the first recognized text. For details, please refer to the foregoing embodiments for a detailed description. For the sake of brevity, it will not be repeated here.

[0038] In one implementation scenario, the target hidden layer features and the target semantic features of at least one semantic entity are fused to obtain a second fused feature. The second fused feature is then linearly transformed to obtain a first encoded feature. Position information is injected into the first encoded feature to obtain a second encoded feature. The second encoded feature is then processed using a multi-head attention mechanism to obtain a target modeling feature. Based on the target modeling feature, the recognition confidence of the first recognized text is predicted.

[0039] In a specific implementation scenario, when fusing target hidden features and target semantic features, a weighted summation method can be used. Based on the importance of the target hidden features and target semantic features in the recognition task, they are assigned corresponding weights respectively, and then the weighted features are added together to obtain the second fused feature.

[0040] In a specific implementation scenario, during the process of obtaining the first encoded feature by performing a linear transformation based on the second fusion feature, the parameters of the linear transformation can be learned and optimized through a large amount of training data, so that the first encoded feature can better represent the fusion information of the target hidden layer features and the target semantic features, and provide stronger feature support for subsequent steps.

[0041] In a specific implementation scenario, when injecting position information encoding based on the first encoding feature, the position information encoding can be generated by combining sine and cosine functions. This encoding method can give elements at different positions unique position identifiers, enabling the model to better capture the positional relationships of elements in the sequence, thereby improving the accuracy of confidence prediction.

[0042] In a specific implementation scenario, when using a multi-head attention mechanism to process the second encoded feature, the multi-head attention mechanism can segment the input feature into multiple different attention heads. Each attention head independently calculates its attention weight, and then the results of each attention head are concatenated. This mechanism can analyze and process the feature from multiple different perspectives, extract richer information, and thus obtain more accurate target modeling features.

[0043] In a specific implementation scenario, when predicting the recognition confidence of the first recognized text based on the target modeling features, a fully connected neural network can be used. The fully connected neural network can learn the complex mapping relationship between the target modeling features and the recognition confidence. By adjusting the network parameters through a large amount of training data, the predicted recognition confidence can more accurately reflect the degree of matching between the first recognized text and the original audio.

[0044] In a specific implementation scenario, the recognition confidence of the first identified text is generated based on the confidence generation unit in the confidence evaluation model. Specifically, the confidence generation unit adopts a deep learning network structure, including a sequentially connected linear module, a Transform module, and a classification layer.

[0045] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an embodiment of the confidence generation unit in the speech recognition method of this application. For example... Figure 3 As shown, the confidence generation unit includes sequentially connected linear modules, six Transform modules, and a classification layer. Each linear module consists of two linear layers, and each Transform module includes positional encoding, a multi-head attention mechanism, normalization, a residual structure, and an FFN structure. The classification layer consists of one linear layer and softmax computation. Specifically, the input layer of the linear module takes the target hidden layer features and the target semantic features of at least one semantic entity as input. The positional encoding in the Transform modules includes absolute positional encoding, RoPE rotational positional encoding, Alibi positional encoding, etc., which are not limited in this application. After obtaining the encoded sequence Z from the linear module, the second encoded feature obtained after positional encoding embedding is:

[0046]

[0047]

[0048] In formulas (1) to (3), pos is the current position index, i is the dimension index, and d is the dimension of the vector.

[0049] In a specific implementation scenario, multi-head attention allows the model to learn information in different subspaces. Assuming there are h heads, each with dimension dk, the formula for multi-head attention is:

[0050]

[0051]

[0052] In formulas (4) to (6), Q, K, and V are all generated by applying different weights W to the second encoded features after adding positional encoding. , , , All of these are weights used in training and learning.

[0053] In a specific implementation scenario, the residual structure is used to preserve the original information, as detailed in the following formula (7):

[0054] The normalization structure can be either batch-norm or layer-norm; there is no specific limitation. Taking layer-norm as an example, please refer to the following formula:

[0055]

[0056] Among them, in formulas (8) to (9), This represents the output after multi-head attention residuals. and These are the mean and variance of the input vector, respectively. and These are learnable scaling and offset parameters. It is a very small constant to prevent division by zero errors.

[0057] In a specific implementation scenario, a feedforward neural network (FFN) consists of two linear layers and an activation function. Specifically, the activation function can be ReLU, Sigmoid, Tanh, etc. Taking ReLU as an example, the activation function formulas can be found in formulas (10) and (11) below:

[0058]

[0059] In a specific implementation scenario, the classification layer includes a linear layer and a softmax calculation, as detailed in the following formula:

[0060] In formula (12), and These are the weights and biases of the classification layer, respectively. The output after passing through the above 6 Transform modules is then subjected to a softmax classification, as detailed in the following formula:

[0061] In formula (13), P is the output confidence score.

[0062] In a specific implementation scenario, the confidence calculation unit can utilize the graded label sequence for cross-entropy loss calculation during training, and update the weights through backpropagation. Specifically, the cross-entropy loss function is as follows:

[0063] In formula (14), C represents the number of categories. Specifically, taking a category with 1 category as an example, the category is whether it is a correct label, and the true label is... , This is the probability predicted for the current class c after softmax.

[0064] In a specific implementation scenario, the first recognized text is obtained by predicting the audio to be recognized through an audio recognition model that includes a target encoder and a target decoder with sequential connections. Specifically, the audio recognition model can update the weights based on cross-entropy loss constraints to achieve the training of the audio recognition model.

[0065] Step S14: Based on the recognition confidence, choose to maintain the first recognized text or modify the first recognized text by referring to at least one semantic entity to obtain the second recognized text of the audio to be recognized.

[0066] In one implementation scenario, the first contribution of the target hidden layer features in determining the recognition confidence is obtained, and the second contribution of at least one semantic entity in determining the recognition confidence is obtained. Based on the difference between the first and second contributions, an adjustment factor for the recognition confidence is determined. The recognition confidence is adjusted based on the adjustment factor to obtain the final recognition confidence of the first recognized text. Based on the final recognition confidence, the first recognized text is either corrected or maintained to obtain the second recognized text of the audio to be recognized. Specifically, the greater the difference between the first and second contributions, the lower the credibility of the recognition confidence. Correspondingly, the adjustment factor can adjust the original recognition confidence to a smaller recognition confidence. The above scheme, by comprehensively considering the difference in contributions of the target hidden layer features and semantic entities in the recognition confidence, can dynamically adjust the confidence evaluation results and avoid evaluation bias caused by a single feature dominating the evaluation.

[0067] In a specific implementation scenario, determining the difference between the first and second contributions can be achieved through various quantification methods, such as calculating their ratio or the absolute value of the difference. Different quantification methods can reflect the differences in the degree of influence of the target hidden features and semantic entities on the recognition confidence from different perspectives. For example, by calculating the absolute value of the difference, a large absolute value indicates a significant difference in the influence of the target hidden features and semantic entities on the recognition confidence; a small absolute value indicates that the difference in their influence is not significant.

[0068] In a specific implementation scenario, when determining the adjustment factor for the identification confidence level, different adjustment strategies can be set according to the magnitude of the difference between the first contribution and the second contribution. When the difference is large, the adjustment factor can be set relatively large to make a larger adjustment to the identification confidence level; when the difference is small, the adjustment factor can be set relatively small to make a smaller adjustment.

[0069] In a specific implementation scenario, when adjusting the recognition confidence based on the adjustment factor, a multiplicative adjustment method can be used to multiply the original recognition confidence by the adjustment factor to obtain the final recognition confidence of the first recognition text.

[0070] In one implementation scenario, the system detects whether the confidence level of the identification meets the screening conditions regarding the confidence threshold. In response to the detection that the screening conditions are not met, the first identified text is corrected based on at least one semantic entity to obtain the second identified text. In response to the detection that the screening conditions are met, the first identified text is maintained as the second identified text.

[0071] In a specific implementation scenario, the selection criterion is that the recognition confidence level is not less than the confidence threshold. Specifically, the confidence threshold can be flexibly set according to the needs of the actual application scenario. For example, in some scenarios with extremely high recognition accuracy requirements, such as medical diagnostic audio recognition and legal document audio transcription, the confidence threshold can be set higher, such as 0.95. Only when the recognition confidence level of the first recognized text reaches or exceeds this threshold is the recognition result considered reliable, thus maintaining the first recognized text as the second recognized text. In some scenarios with high recognition efficiency requirements and relatively low accuracy requirements, such as ordinary meeting minutes and daily conversation transcription, the confidence threshold can be set lower, such as 0.7 or 0.8. When the recognition confidence level of the first recognized text reaches this threshold, it is maintained as the second recognized text. If it does not reach this threshold, the first recognized text is corrected based on at least one semantic entity. For example, by searching for entities in the semantic entity library that are semantically similar to the current recognition result but more in line with the context to replace the incorrectly recognized words, or by combining the grammatical features of semantic entities to perform grammatical correction on the recognition result to obtain a more accurate second recognized text.

[0072] In a specific implementation scenario, when the recognition confidence does not meet the screening criteria, the first recognition text is corrected based on the text correction model. Specifically, the text correction model can combine at least one semantic entity and the context to accurately replace and adjust the erroneous words or expressions in the first recognition text, so as to integrate these replacement contents into the first recognition text and make grammatical and semantic adjustments to ensure that the corrected second recognition text is more accurate.

[0073] In a specific implementation scenario, the text correction model is a large language model that outputs a corrected second recognized text based on the first recognized text to be corrected input to the text correction model.

[0074] In a specific implementation scenario, based on at least one semantic entity and the first recognized text, a large model instruction is constructed. This instruction directs a large language model to correct the first recognized text by referring to at least one semantic entity. The output text of the large language model in response to the instruction serves as the second recognized text. The large model is a deep learning model with powerful language understanding and generation capabilities. It can receive information from the large model instruction and deeply analyze the relationship between at least one semantic entity and the first recognized text. During processing, the large language model uses contextual understanding to locate potential errors or inaccuracies in the first recognized text. For example, when there is a significant semantic discrepancy between the semantic entity and a word in the first recognized text, or when the word is used inappropriately according to the context, the large language model combines the accurate meaning of the semantic entity with the overall context to generate a more suitable alternative word or expression. In this approach, the text output by the large language model in response to the large model instruction maximizes the grammatical and semantic accuracy of the second recognized text, thereby effectively improving the final performance of speech recognition.

[0075] In a specific implementation scenario, before constructing the large model instructions based on at least one semantic entity and the first recognized text, the probability value of each character in the first recognized text that needs correction is analyzed based on at least one semantic entity. Based on the probability value of each character that needs correction, the target sentence in the first recognized text that needs correction is obtained. Specifically, the probability value of each character that needs correction can be calculated by statistically analyzing the frequency of errors of each character under the influence of different semantic entities and combining the relevance of the context. For example, for a certain character, if in the context of multiple semantic entities, the word or phrase formed by this character and the surrounding characters is semantically unreasonable, and this unreasonableness occurs frequently, then a higher probability value that needs correction can be assigned to this character. Based on these probability values, consecutive characters with probability values ​​exceeding a certain threshold are combined to form the target sentence that needs correction. The above scheme, when constructing the large model instructions, can more specifically instruct the large language model to correct unreasonable target sentences, improving the efficiency and accuracy of correction.

[0076] In a specific implementation scenario, a sliding window method can be used to analyze the probability value of each character in the first recognized text that needs correction. For example, a fixed-length sliding window is set, and the window is slid sequentially across the first recognized text. Each time the window slides, the frequency of errors occurring in characters within the window under the influence of different semantic entities is counted. Specifically, for each character within the window, the semantic rationality of the words or phrases formed with its surrounding characters in the context of each semantic entity is analyzed. If an irrationality exists, an error is recorded. By statistically analyzing and combining the relevance of the context, the probability value of each character needing correction is calculated. For example, if a character frequently exhibits semantically irrational behavior when forming words or phrases with its surrounding characters in multiple semantic entity contexts, then the probability of that character needing correction is relatively high. Specifically, based on these calculated probability values, consecutive characters with probability values ​​exceeding a pre-set threshold are combined to form the target sentences that need correction. This approach can accurately instruct the large language model to correct these target sentences during subsequent large-scale model construction, thereby effectively improving correction efficiency and accuracy.

[0077] In a specific implementation scenario, a large model instruction is constructed based on at least one semantic entity, the first recognized text, and its target sentence. This large model instruction specifically indicates that the target sentence in the first recognized text needs to be corrected by referring to at least one semantic entity. This approach, by accurately constructing the large model instruction, provides a clear and definite correction direction for the large language model. The corrected text output by the large language model in response to the large model instruction is then used as the second recognized text, effectively improving the accuracy and reliability of speech recognition results.

[0078] In a specific implementation scenario, the first phoneme sequence of at least one semantic entity and the second phoneme sequence of the first recognized text are obtained. The first recognized text is then segmented into several sub-texts. Sliding matching is performed based on the first and second phoneme sequences to obtain the pronunciation matching degree of each character in the first recognized text. Furthermore, feature matching is performed between the entity semantic features of at least one semantic entity and the text semantic features of each sub-text to obtain the semantic matching degree of each character in the first recognized text. Finally, the pronunciation matching degree and semantic matching degree of the same character are fused to obtain the probability value of the corresponding character that needs correction. This scheme, by comprehensively considering information from both pronunciation and semantic dimensions, can more comprehensively and accurately evaluate the correctness of each character in the first recognized text.

[0079] In a specific implementation scenario, after obtaining the first phoneme sequence of at least one semantic entity and the second phoneme sequence of the first recognized text, the first phoneme sequence is used as a template for sliding matching on the second phoneme sequence. During the sliding process, for each position in the second phoneme sequence, the similarity with the first phoneme sequence is calculated to obtain the pronunciation-level matching degree of each character in the first recognized text. For example, the similarity between two phoneme sequences can be measured using a dynamic time warping algorithm. By finding the optimal matching path, the similarity between the two sequences is obtained, and thus the pronunciation matching degree of each character is obtained. Simultaneously, the first recognized text is segmented into several sub-texts. Subsequently, the semantic features of at least one semantic entity are extracted, and the textual semantic features of each sub-text are extracted. Specifically, these features can be obtained through a pre-trained language model or a specific semantic feature extraction method, which will not be elaborated here. Next, the semantic features of the semantic entities are matched with the textual semantic features of each sub-text. Methods such as cosine similarity can be used to measure the similarity between the two, thereby determining the semantic-level matching degree of each character in the first recognized text. Finally, for each identical character in the first identified text, its pronunciation matching degree and semantic matching degree are fused according to a certain weight.

[0080] In a specific implementation scenario, the weights of pronunciation matching and semantic matching can be adjusted according to the importance of pronunciation and semantics in the actual application scenario. For example, in scenarios where pronunciation accuracy is extremely important, pronunciation matching can be given a higher weight; while in scenarios where semantic understanding is more important, semantic matching can be given a higher weight.

[0081] Please see Figure 4 , Figure 4 This is a schematic diagram of the framework of an embodiment of the speech recognition system in the speech recognition method of this application. Figure 4 As shown, the second recognized text is obtained by processing the audio to be recognized by a speech recognition system. The speech recognition system includes an audio recognition model, a confidence evaluation model, and a text correction model. The speech recognition system includes at least a sequentially connected target encoder and a target decoder. The confidence evaluation model includes a sequentially connected semantic extraction unit and a confidence generation unit. The output of the target encoder and the semantic entity library are connected to the input of the semantic extraction unit, respectively. The output of the target network layer in the target decoder, used to output the target hidden features, is connected to the input of the confidence generation unit. The output of the target encoder is connected to the input of the text correction model. The speech recognition method implemented based on the speech recognition system can be found in the detailed description of the foregoing embodiments; for brevity, it will not be repeated here.

[0082] In a specific implementation scenario, the output of the semantic extraction unit is also connected to the input of the target decoder. After extracting semantic information, the semantic extraction unit can not only pass the information to the confidence generation unit for confidence evaluation, but also feed it back to the target decoder, providing it with richer semantic context and helping the target decoder to decode the information in the audio more accurately. This approach reduces decoding errors, thereby improving the accuracy and robustness of the entire speech recognition system.

[0083] The above scheme encodes the audio to be recognized to obtain target encoded features, decodes these features to obtain target hidden features, and finally obtains the first recognized text. Based on the target encoded features, at least one semantic entity related to the audio to be recognized is selected from a semantic entity library. Based on the target hidden features and at least one semantic entity, a recognition confidence score for the first recognized text is generated. Based on the recognition confidence score, the first recognized text is either modified or maintained to obtain the second recognized text of the audio to be recognized. Since the selection of semantic entities is determined based on the matching results between the target encoded features and the semantic entity library, the matching results at the entity level can eliminate interference factors that do not affect the actual semantics of the first recognized text as much as possible. By generating a recognition confidence score from the target hidden features and semantic entities, the degree of fit between the first recognized text and the true semantics of the audio to be recognized can be more accurately reflected, thereby improving the accuracy of the generated recognition confidence score. Furthermore, the decision to modify or maintain the first recognized text based on this recognition confidence score can more accurately determine whether a large language model needs to be introduced to modify the first recognized text, thereby avoiding a decrease in processing speed due to over-reliance on a large language model. Therefore, it can improve the efficiency of speech recognition while ensuring the accuracy of the final recognized text meets the requirements as much as possible.

[0084] Please see Figure 5 , Figure 5 This is a schematic diagram of the framework of an embodiment of the speech recognition device of this application. Figure 5As shown, the speech recognition device 50 includes a target encoding module 51, a target decoding module 52, a confidence generation module 53, and a text determination module 54. The target encoding module 51 is used to encode based on the audio to be recognized to obtain target encoding features; the target decoding module 52 is used to decode based on the target encoding features to obtain target hidden layer features during the decoding process, and select at least one semantic entity related to the audio to be recognized from the semantic entity library based on the target encoding features; the confidence generation module 53 is used to generate the recognition confidence of the first recognized text based on the target hidden layer features and at least one semantic entity; wherein, the target encoding features are decoded to finally obtain the first recognized text; the text determination module 54 is used to select to maintain the first recognized text or modify the first recognized text by referring to at least one semantic entity based on the recognition confidence to obtain the second recognized text of the audio to be recognized.

[0085] Therefore, the speech recognition device 50 encodes the audio to be recognized to obtain target encoded features, decodes the target encoded features to obtain target hidden features during the decoding process, and finally obtains the first recognized text by decoding the target encoded features. Based on the target encoded features, at least one semantic entity related to the audio to be recognized is selected from the semantic entity library. Based on the target hidden features and at least one semantic entity, a recognition confidence score of the first recognized text is generated. Based on the recognition confidence score, the device chooses to maintain the first recognized text or modify the first recognized text by referring to at least one semantic entity to obtain the second recognized text of the audio to be recognized. Since the selection of semantic entities is determined based on the matching results between the target encoded features and the semantic entity library, the matching results at the entity level can eliminate interference factors that do not affect the actual semantics of the first recognized text as much as possible. By generating a recognition confidence score by combining the target hidden features and semantic entities, the device can more accurately reflect the degree of fit between the first recognized text and the true semantics of the audio to be recognized, thereby improving the accuracy of the generated recognition confidence score. Furthermore, the decision to correct or maintain the first recognized text based on this recognition confidence level can more accurately determine whether a large language model needs to be introduced to correct the first recognized text, thereby avoiding a decrease in processing speed due to over-reliance on a large language model. Therefore, it can improve the efficiency of speech recognition while ensuring that the accuracy of the final recognized text meets the requirements as much as possible.

[0086] In some disclosed embodiments, the speech recognition device 50 further includes a semantic feature extraction module (not shown) for extracting features based on at least one semantic entity to obtain target semantic features; the speech recognition device 50 further includes a first feature fusion module (not shown) for fusing the target semantic features and target encoded features to obtain a first fused feature; the speech recognition device 50 further includes a first feature decoding module (not shown) for decoding based on the first fused feature to obtain a first recognized text of the audio to be recognized.

[0087] In some disclosed embodiments, the confidence generation module 53 further includes a second feature fusion module (not shown), used to fuse target hidden layer features and target semantic features of at least one semantic entity to obtain a second fused feature; the confidence generation module 53 further includes a linear transformation module (not shown), used to perform a linear transformation based on the second fused feature to obtain a first encoded feature; the confidence generation module 53 further includes a position encoding module (not shown), used to inject position information encoding based on the first encoded feature to obtain a second encoded feature; the confidence generation module 53 further includes a multi-head attention module (not shown), used to process the second encoded feature based on a multi-head attention mechanism to obtain target modeling features; the confidence generation module 53 further includes a confidence prediction module (not shown), used to predict the recognition confidence of the first recognized text based on the target modeling features.

[0088] In some disclosed embodiments, the target decoding module 52 further includes an entity feature extraction module (not shown), which is used to extract features based on each semantic entity in the semantic entity library to obtain the entity semantic features of each semantic entity; the target decoding module 52 further includes an entity selection module (not shown), which is used to select at least one semantic entity related to the audio to be identified based on the matching results between the target encoded features and the entity semantic features of each semantic entity.

[0089] In some disclosed embodiments, the text determination module 54 further includes a condition filtering module (not shown) for detecting whether the recognition confidence level meets the filtering conditions regarding the confidence threshold; the text determination module 54 further includes a first response module (not shown) for correcting the first recognized text based on at least one semantic entity to obtain the second recognized text in response to the detection that the filtering conditions are not met; the text determination module 54 further includes a second response module (not shown) for maintaining the first recognized text as the second recognized text in response to the detection that the filtering conditions are met.

[0090] In some disclosed embodiments, the first response module (not shown) further includes a first instruction construction module (not shown), used to construct a large model instruction based on at least one semantic entity and the first identified text; wherein, the large model instruction is used to instruct the large language model to correct the first identified text by referring to at least one semantic entity; the first response module (not shown) further includes a first text output module (not shown), used to obtain the output text of the large language model in response to the large model instruction, as the second identified text.

[0091] In some disclosed embodiments, before constructing a large model instruction based on at least one semantic entity and the first recognized text, the speech recognition device 50 further includes a probability analysis module (not shown), used to analyze and obtain the probability value of each character in the first recognized text that needs to be corrected based on at least one semantic entity; the speech recognition device 50 further includes a target sentence determination module (not shown), used to obtain the target sentence in the first recognized text that needs to be corrected based on the probability value of each character in the first recognized text that needs to be corrected; the first instruction construction module (not shown) further includes a construction submodule (not shown), used to construct a large model instruction based on at least one semantic entity, the first recognized text and its target sentence; wherein, the large model instruction specifically indicates that the target sentence in the first recognized text needs to be corrected with reference to at least one semantic entity.

[0092] In some disclosed embodiments, the target sentence determination module (not shown) further includes a determination submodule (not shown), used to obtain the first phoneme sequence of at least one semantic entity and the second phoneme sequence of the first identified text, and to perform word segmentation based on the first identified text to obtain several sub-texts; the target sentence determination module (not shown) further includes a sliding matching module (not shown), used to perform sliding matching based on the first phoneme sequence and the second phoneme sequence to obtain the pronunciation matching degree of each character in the first identified text, and to perform feature matching based on the entity semantic features of at least one semantic entity and the text semantic features of each sub-text to obtain the semantic matching degree of each character in the first identified text; the target sentence determination module (not shown) further includes a matching fusion module (not shown), used to fuse the pronunciation matching degree and semantic matching degree of the same character to obtain the probability value of the corresponding character that needs to be corrected.

[0093] In some disclosed embodiments, the second recognized text in the speech recognition device 50 is obtained by processing the audio to be recognized by the speech recognition system. The speech recognition system includes an audio recognition model, a confidence evaluation model, and a text correction model. The speech recognition system includes at least a target encoder and a target decoder connected in sequence. The confidence evaluation model includes a semantic extraction unit and a confidence generation unit connected in sequence. The output of the target encoder and the semantic entity library are respectively connected to the input of the semantic extraction unit. The output of the target network layer in the target decoder, which is used to output the target hidden layer features, is connected to the input of the confidence generation unit. The output of the target encoder is connected to the input of the text correction model.

[0094] Please see Figure 6 , Figure 6This is a schematic diagram of a framework of an embodiment of the electronic device of this application. The electronic device 60 includes at least a memory 61 and a processor 62 coupled to each other. The memory 61 stores at least program instructions, and the processor 62 is used to execute the program instructions to implement the steps in any of the above-described embodiments of the speech recognition method. For details, please refer to the foregoing disclosed embodiments, which will not be repeated here. It should be noted that the specific type of the electronic device 60 is not limited here.

[0095] Specifically, processor 62 controls itself and memory 61 to implement the steps in any of the above-described speech recognition method embodiments. Processor 62 can also be referred to as a CPU (Central Processing Unit). Processor 62 may be an integrated circuit chip with signal processing capabilities. Processor 62 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 62 can be implemented using integrated circuit chips.

[0096] In the above scheme, the electronic device 60 encodes the audio to be recognized to obtain target encoded features, decodes the target encoded features to obtain target hidden features during the decoding process, and finally obtains the first recognized text by decoding the target encoded features. Based on the target encoded features, at least one semantic entity related to the audio to be recognized is selected from the semantic entity library. Based on the target hidden features and at least one semantic entity, a recognition confidence score of the first recognized text is generated. Based on the recognition confidence score, the first recognized text is either maintained or modified by referring to at least one semantic entity to obtain the second recognized text of the audio to be recognized. Since the selection of semantic entities is determined based on the matching results between the target encoded features and the semantic entity library, the matching results at the entity level can eliminate interference factors that do not affect the actual semantics of the first recognized text as much as possible. By generating a recognition confidence score by combining the target hidden features and semantic entities, the degree of fit between the first recognized text and the true semantics of the audio to be recognized can be more accurately reflected, thereby improving the accuracy of the generated recognition confidence score. Furthermore, the decision to correct or maintain the first recognized text based on this recognition confidence level can more accurately determine whether a large language model needs to be introduced to correct the first recognized text, thereby avoiding a decrease in processing speed due to over-reliance on a large language model. Therefore, it can improve the efficiency of speech recognition while ensuring that the accuracy of the final recognized text meets the requirements as much as possible.

[0097] Please see Figure 7 , Figure 7 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 70 stores program instructions 71 that can be executed by a processor. The program instructions 71 are used to implement the steps in any of the above-described embodiments of the speech recognition method.

[0098] In the above scheme, the computer-readable storage medium 70 encodes the audio to be recognized to obtain target encoded features, decodes the target encoded features to obtain target hidden features during the decoding process, and finally obtains the first recognized text by decoding the target encoded features. Based on the target encoded features, at least one semantic entity related to the audio to be recognized is selected from the semantic entity library. Based on the target hidden features and at least one semantic entity, a recognition confidence score of the first recognized text is generated. Based on the recognition confidence score, the first recognized text is either maintained or modified with reference to at least one semantic entity to obtain the second recognized text of the audio to be recognized. Since the selection of semantic entities is determined based on the matching results between the target encoded features and the semantic entity library, the matching results at the entity level can eliminate interference factors that do not affect the actual semantics of the first recognized text as much as possible. By generating a recognition confidence score by combining the target hidden features and semantic entities, the degree of fit between the first recognized text and the true semantics of the audio to be recognized can be more accurately reflected, thereby improving the accuracy of the generated recognition confidence score. Furthermore, the decision to correct or maintain the first recognized text based on this recognition confidence level can more accurately determine whether a large language model needs to be introduced to correct the first recognized text, thereby avoiding a decrease in processing speed due to over-reliance on a large language model. Therefore, it can improve the efficiency of speech recognition while ensuring that the accuracy of the final recognized text meets the requirements as much as possible.

[0099] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0100] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A speech recognition method, characterized in that, include: Encode the target encoded features based on the audio to be identified; Decoding is performed based on the target encoding features to obtain the target hidden layer features during the decoding process, and at least one semantic entity related to the audio to be identified is selected from the semantic entity library based on the target encoding features. Based on the target hidden layer features and the at least one semantic entity, a recognition confidence score for the first recognition text is generated; wherein, the target encoded features are decoded to finally obtain the first recognition text; Based on the recognition confidence level, the first recognized text is either maintained or the first recognized text is modified with reference to at least one semantic entity to obtain the second recognized text of the audio to be recognized.

2. The method according to claim 1, characterized in that, The method further includes: Based on the at least one semantic entity, feature extraction is performed to obtain the target semantic features; The first fused feature is obtained by fusing the target semantic features and the target coding features. Decoding is performed based on the first fusion feature to obtain the first recognized text of the audio to be recognized.

3. The method according to claim 1, characterized in that, The step of generating the recognition confidence score of the first recognized text based on the target hidden layer features and the at least one semantic entity includes: The second fused feature is formed by fusing the target hidden layer features and the target semantic features of at least one semantic entity. A linear transformation is performed based on the second fusion feature to obtain the first encoding feature; Based on the first encoded feature, position information is injected and encoded to obtain the second encoded feature; The second encoded feature is processed using a multi-head attention mechanism to obtain the target modeling feature; Based on the target modeling features, the recognition confidence of the first recognized text is predicted.

4. The method according to claim 1, characterized in that, The step of selecting at least one semantic entity related to the audio to be identified from the semantic entity library based on the target encoding features includes: Based on the semantic entity library, feature extraction is performed on each of the semantic entities to obtain the entity semantic features of each semantic entity; Based on the matching results between the target encoded features and the entity semantic features of each semantic entity, at least one semantic entity related to the audio to be identified is selected.

5. The method according to claim 1, characterized in that, The step of selecting to maintain the first identified text or modify the first identified text by referring to at least one semantic entity based on the identification confidence level to obtain the second identified text of the audio to be identified includes: Detect whether the identification confidence level meets the screening conditions regarding the confidence threshold; In response to the detection that the filtering conditions are not met, the first identified text is corrected based on the at least one semantic entity to obtain the second identified text; In response to the detection that the filtering conditions are met, the first identified text is maintained as the second identified text.

6. The method according to claim 5, characterized in that, The step of correcting the first identified text based on the at least one semantic entity to obtain the second identified text includes: Based on the at least one semantic entity and the first identified text, a large model instruction is constructed; wherein, the large model instruction is used to instruct the large language model to correct the first identified text by referring to the at least one semantic entity; The output text of the large language model in response to the large model instruction is obtained and used as the second recognized text.

7. The method according to claim 6, characterized in that, Before constructing the large model instruction based on the at least one semantic entity and the first recognized text, the method further includes: Based on the at least one semantic entity, the probability value of each character in the first recognized text that needs to be corrected is obtained through analysis; Based on the probability value of each character in the first identified text that needs to be corrected, the target sentence in the first identified text that needs to be corrected is obtained. The large model instruction constructed based on the at least one semantic entity and the first recognized text includes: Based on the at least one semantic entity, the first identified text, and the target statement, the large model instruction is constructed; wherein, the large model instruction specifically indicates that the target statement in the first identified text needs to be corrected with reference to the at least one semantic entity.

8. The method according to claim 7, characterized in that, The step of analyzing and obtaining the probability value of each character in the first recognized text that needs to be corrected based on the at least one semantic entity includes: Obtain the first phoneme sequence of each of the at least one semantic entity and the second phoneme sequence of the first identified text, and perform word segmentation based on the first identified text to obtain several sub-texts; Based on the first phoneme sequence, sliding matching is performed on the second phoneme sequence to obtain the pronunciation matching degree of each character in the first recognized text. Based on the entity semantic features of each of the at least one semantic entity, feature matching is performed with the text semantic features of each of the sub-texts to obtain the semantic matching degree of each character in the first recognized text. The probability value of the corresponding character needing to be corrected is obtained by fusing the pronunciation matching degree and semantic matching degree of the same character.

9. The method according to any one of claims 1 to 8, characterized in that, The second recognized text is obtained by processing the audio to be recognized by a speech recognition system. The speech recognition system includes an audio recognition model, a confidence evaluation model, and a text correction model. The speech recognition system includes at least a target encoder and a target decoder connected in sequence. The confidence evaluation model includes a semantic extraction unit and a confidence generation unit connected in sequence. The output of the target encoder and the semantic entity library are respectively connected to the input of the semantic extraction unit. The output of the target network layer in the target decoder, which is used to output the target hidden layer features, is connected to the input of the confidence generation unit. The output of the target encoder is connected to the input of the text correction model.

10. A voice recognition device, characterized in that, include: The target encoding module is used to encode the audio to be recognized to obtain target encoded features; The target decoding module is used to decode based on the target encoding features to obtain the target hidden layer features during the decoding process, and to select at least one semantic entity related to the audio to be identified from the semantic entity library based on the target encoding features. The confidence generation module is used to generate the recognition confidence of the first recognition text based on the target hidden layer features and the at least one semantic entity; wherein, the target encoded features are decoded to finally obtain the first recognition text; The text determination module is used to select, based on the recognition confidence level, to maintain the first recognized text or to modify the first recognized text by referring to at least one semantic entity, so as to obtain the second recognized text of the audio to be recognized.

11. An electronic device, characterized in that, It includes at least a memory and a processor, wherein the memory stores at least program instructions, and the processor is used to execute the program instructions to implement the speech recognition method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The device stores program instructions that can be executed by a processor, the program instructions being used to implement the speech recognition method according to any one of claims 1 to 9.

Images