Speech recognition hallucination suppression method and system based on dual-decoding decision network, and medium

Abstract

The application provides a hallucination suppression method and system for speech recognition based on a dual-decoding decision network, and a medium. The method comprises: obtaining speech data to be recognized; decoding the speech data using a biased decoding model to obtain a customized transcription result A; decoding the speech data using an unbiased decoding model to obtain a preliminary transcription result B; inputting representation tensors of the customized transcription result A and the preliminary transcription result B into a pre-trained semantic encoder to obtain high-level representations; inputting the high-level representations into a trained prediction projection network to obtain a hallucination probability; comparing the hallucination probability with a set discrimination threshold; if the hallucination probability is less than or equal to the discrimination threshold, outputting the preliminary transcription result B; if the hallucination probability is higher than the discrimination threshold, outputting the customized transcription result A; by running two decoding paths in parallel for the same speech, hallucination insertion errors caused by prompt words can be effectively eliminated, and the overall recognition accuracy is significantly improved in a scenario containing user-customized hot words.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and more specifically, to a speech recognition hallucination suppression method, system, and medium based on a dual-decoding decision network. Background Technology

[0002] Currently, large-scale pre-trained models are widely used in automatic speech recognition (ASR). Among them, prompt engineering and hotword injection are commonly used personalized customization methods, which improve the recognition accuracy of specific terms by providing specific context or vocabulary during decoding. For example, NVIDIA's NeMo model allows input text prompts as context during inference to improve the recognition rate of proper nouns or hot words. However, such biased methods based on prompts or context may cause "illusion" outputs - that is, the transcription results given by the model contain content that is seriously inconsistent with the actual audio. Studies have shown that when prompts are used improperly (e.g., simply listing hot words as prompts), the model is prone to producing false inserted content when processing spontaneous speech with code mixing. Koenecke et al. [1] also found in their evaluation of Whisper that about 1% of the audio segment recognition results contained entirely fictitious content that did not exist in the original audio. Such illusion outputs are often fluent but have no relation to the audio, are not easily detected, and seriously affect the credibility of the recognition results.

[0003] Faced with the above problems, the academic and industry communities have proposed a series of improvement schemes, most of which are based on the ideas of bias weight adjustment, model structure fusion or limiting the scale of hot words. For example, shallow language model fusion is used to increase the probability of specific words during the decoding process. This method improves the probability of hot words by fusing external language models during end-to-end ASR decoding, which has achieved certain results, but it requires training on external models and is very sensitive to the selection of fusion coefficients (weights). In addition, shallow fusion only biases the probability distribution at the decoding level and lacks the use of acoustic information. To this end, in recent years, deep bias methods have emerged that directly fuse context hot words into the internal state of the model, that is, adding a context bias module (such as a word encoder and bias attention mechanism) to the model architecture [2]. These methods are jointly optimized with the main ASR task during model training and usually achieve higher hot word recognition performance than shallow fusion. However, even if the above bias mechanism is introduced, improper control may still cause performance degradation in the recognition of regular content. To mitigate this generalization performance degradation, some studies have used loss functions and structural design to suppress excessive bias. On the other hand, in practical applications, the impact of the size of the hot word list on system performance also needs to be considered: when the number of biased words is too large, decoding latency and false recognition rate often increase. To address this, existing technologies include limiting the length of the hot word list or dynamically filtering relevant hot words (such as using a Top-K truncation strategy) to reduce the negative impact. In summary, existing solutions mainly focus on the model decoding process or structure, and lack a strategy for correcting the results after decoding output to further enhance recognition accuracy and eliminate hallucination errors. Summary of the Invention

[0004] The purpose of this application is to provide a speech recognition hallucination suppression method, system and medium based on a dual-decoding decision network. By running two decoding operations in parallel on the same speech segment, it can effectively eliminate hallucination insertion errors caused by prompt words, significantly reduce the probability of irrelevant content appearing in the transcription results, and significantly improve the overall recognition accuracy in scenarios containing user-customized hot words.

[0005] This application also provides a speech recognition hallucination suppression method based on a dual-decoding decision network, including: Acquire the speech data to be recognized; The speech data is decoded using a biased decoding model, which embeds target hot word prompts to obtain a customized transcription result A. The speech data is decoded using an unbiased decoding model, which does not embed customized hot word prompts, to obtain a preliminary transcription result B; The representation tensors of the customized transcription result A and the preliminary transcription result B are input into the pre-trained semantic encoder to obtain the high-level representation. The high-level representation is then input into the trained prediction projection network to obtain the hallucination probability. The hallucination probability is compared with the set discrimination threshold. If the hallucination probability is less than or equal to the discrimination threshold, the customized transcription result A is output; if the hallucination probability is greater than the discrimination threshold, the preliminary transcription result B is output.

[0006] Optionally, in the speech recognition hallucination suppression method based on a dual-decoding decision network described in this application embodiment, a biased decoding model is used to decode the speech data. The biased decoding model embeds target hot word prompts to obtain a customized transcription result A, specifically including: The target hot words are obtained, and the target hot word prompts are preprocessed to generate a hot word prompt vector set. The preprocessing includes word segmentation, normalization and vector embedding. The speech data to be recognized is converted into an acoustic feature sequence, input into a biased decoding model, and outputs speech feature encoding. The hot word prompt vector set is fused with the speech feature encoding to obtain the initial decoding state; Autoregressive decoding is performed based on the initial decoding state, and the association weight between the hot word prompt vector set and the current decoding is strengthened based on the attention mechanism to obtain the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form a customized transcription result A.

[0007] Optionally, in the speech recognition hallucination suppression method based on a dual-decoding decision network described in this application embodiment, the method for fusing the hot word prompt vector set and the speech feature encoding is as follows: The hot word prompt vector set is mapped to the same dimensional space as the speech feature encoding through a multilayer perceptron; Element-level addition and fusion are performed based on residual connection to obtain the initial decoding state.

[0008] Optionally, in the speech recognition hallucination suppression method based on a dual-decoding decision network described in this application embodiment, the biased decoding model training method is as follows: Construct a training dataset, which includes speech data, corresponding text transcriptions, and hot word annotations; Initialize the biased decoding model parameters, batch sample the training data, convert speech data into acoustic features, convert text transcription into unit sequences, and convert hot word annotations into hot word vector sets; The acoustic features are encoded into speech to obtain speech feature codes, and the hot word vector set is encoded to obtain hot word feature codes; Based on the fusion mechanism, speech feature encoding and hot word feature encoding are fused, and the fused feature encoding is then subjected to autoregressive decoding to generate a prediction unit sequence. Calculate the cross-entropy loss between the predicted unit sequence and the real text transcription, adjust the model parameters based on the cross-entropy loss value, and obtain the trained biased decoding model.

[0009] Optionally, in the speech recognition hallucination suppression method based on a dual-decoding decision network described in this application embodiment, an unbiased decoding model is used to decode the speech data. The unbiased decoding model does not embed customized hot word prompts, resulting in a preliminary transcription result B, specifically including: Acquire the speech data to be recognized, extract features from the speech data, and generate an acoustic feature sequence; The acoustic feature sequence is input into the unbiased decoding model to generate speech feature codes without hot word bias. Initialize the decoder state to obtain the initial state, perform autoregressive decoding based on the initial state, and calculate the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form the preliminary transcription result B.

[0010] Optionally, in the speech recognition hallucination suppression method based on a dual-decoding decision network described in the embodiments of this application, the hallucination probability analysis method is as follows: The customized transcription result A and the preliminary transcription result B are preprocessed to obtain standardized text sequences A' and B'. Input text sequences A' and B' into the text embedding layer to generate corresponding word embedding sequences EA and EB; The word embedding sequence EA and EB are fused for features, and a joint embedding sequence EAB is generated by concatenation. The joint embedding sequence EAB is input into the pre-trained semantic encoder, and the output is a high-level representation containing semantic association information. By inputting the high-level representation into the predictive projection network, the probability of hallucination is obtained.

[0011] Secondly, embodiments of this application provide a speech recognition hallucination suppression system based on a dual-decoding decision network. The system includes a memory and a processor. The memory includes a program for a speech recognition hallucination suppression method based on a dual-decoding decision network. When the program for the speech recognition hallucination suppression method based on a dual-decoding decision network is executed by the processor, it implements the following steps: Acquire the speech data to be recognized; The speech data is decoded using a biased decoding model, which embeds target hot word prompts to obtain a customized transcription result A. The speech data is decoded using an unbiased decoding model, which does not embed customized hot word prompts, to obtain a preliminary transcription result B; The representation tensors of the customized transcription result A and the preliminary transcription result B are input into the pre-trained semantic encoder to obtain the high-level representation. The high-level representation is then input into the trained prediction projection network to obtain the hallucination probability. The hallucination probability is compared with the set discrimination threshold. If the hallucination probability is less than or equal to the discrimination threshold, the customized transcription result A is output; if the hallucination probability is greater than the discrimination threshold, the preliminary transcription result B is output.

[0012] Optionally, in the speech recognition hallucination suppression system based on a dual-decoding decision network described in this application embodiment, a biased decoding model is used to decode the speech data. The biased decoding model embeds target hot word prompts to obtain a customized transcription result A, specifically including: The target hot words are obtained, and the target hot word prompts are preprocessed to generate a hot word prompt vector set. The preprocessing includes word segmentation, normalization and vector embedding. The speech data to be recognized is converted into an acoustic feature sequence, input into a biased decoding model, and outputs speech feature encoding. The hot word prompt vector set is fused with the speech feature encoding to obtain the initial decoding state; Autoregressive decoding is performed based on the initial decoding state, and the association weight between the hot word prompt vector set and the current decoding is strengthened based on the attention mechanism to obtain the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form a customized transcription result A.

[0013] Optionally, in the speech recognition hallucination suppression system based on a dual-decoding decision network described in this application embodiment, the method for fusing the hot word prompt vector set and the speech feature encoding is as follows: The hot word prompt vector set is mapped to the same dimensional space as the speech feature encoding through a multilayer perceptron; Element-level addition and fusion are performed based on residual connection to obtain the initial decoding state.

[0014] Thirdly, embodiments of this application also provide a computer-readable storage medium, which includes a speech recognition hallucination suppression method program based on a dual-decoding decision network. When the speech recognition hallucination suppression method program based on a dual-decoding decision network is executed by a processor, it implements the steps of the speech recognition hallucination suppression method based on a dual-decoding decision network as described in any of the preceding claims.

[0015] As can be seen from the above, the speech recognition hallucination suppression method, system, and medium based on a dual-decoding decision network provided in this application involves: acquiring speech data to be recognized; decoding the speech data using a biased decoding model, which embeds target hot word prompts to obtain a customized transcription result A; decoding the speech data using an unbiased decoding model, which does not embed customized hot word prompts to obtain a preliminary transcription result B; and inputting the representation tensors of the customized transcription result A and the preliminary transcription result B into a pre-trained semantic encoder to obtain a high-level... The high-level representation is input into a trained prediction projection network to obtain the hallucination probability. The hallucination probability is compared with a set discrimination threshold. If the hallucination probability is less than or equal to the discrimination threshold, the customized transcription result A is output; if the hallucination probability is greater than the discrimination threshold, the preliminary transcription result B is output. By running two decoding paths in parallel on the same speech segment, the hallucination insertion error caused by prompt words can be effectively eliminated, and the probability of irrelevant content appearing in the transcription result can be greatly reduced. In scenarios containing user-customized hot words, the overall recognition accuracy is significantly improved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of a speech recognition hallucination suppression method based on a dual-decoding decision network provided in an embodiment of this application; Figure 2 A block diagram of a speech recognition hallucination suppression system based on a dual-decoding decision network provided in this application embodiment; Figure 3 This is a schematic diagram of the hallucination decision network architecture of a speech recognition hallucination suppression system based on a dual-decoding decision network, provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0019] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0020] Please refer to Figure 1 , Figure 1 This is a flowchart of a speech recognition hallucination suppression method based on a dual-decoding decision network, as described in some embodiments of this application. This speech recognition hallucination suppression method based on a dual-decoding decision network is used in a terminal device and includes the following steps: S101, acquire the speech data to be recognized; S102, the biased decoding model is used to decode the speech data, and the biased decoding model embeds target hot word prompts to obtain customized transcription result A; S103, the speech data is decoded using an unbiased decoding model. The unbiased decoding model does not embed customized hot word prompts, and the preliminary transcription result B is obtained. S104, Input the representation tensors of the customized transcription result A and the preliminary transcription result B into the pre-trained semantic encoder to obtain the high-level representation, and input the high-level representation into the trained prediction projection network to obtain the hallucination probability; S105, compare the hallucination probability with the set discrimination threshold. If the hallucination probability is less than or equal to the discrimination threshold, output the customized transcription result A; if the hallucination probability is greater than the discrimination threshold, output the preliminary transcription result B.

[0021] According to an embodiment of the present invention, a biased decoding model is used to decode speech data, and the biased decoding model embeds target hot word prompts to obtain a customized transcription result A, specifically including: Obtain target hot words, preprocess the target hot word prompts, and generate a hot word prompt vector set. The preprocessing includes word segmentation, normalization, and vector embedding. The speech data to be recognized is converted into an acoustic feature sequence, input into a biased decoding model, and outputs speech feature encoding. The initial decoding state is obtained by fusing the hot word prompt vector set with the speech feature encoding; Autoregressive decoding is performed based on the initial decoding state, and the association weight between the hot word prompt vector set and the current decoding is strengthened based on the attention mechanism to obtain the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form a customized transcription result A.

[0022] According to an embodiment of the present invention, the method for fusing hot word suggestion vector sets and speech feature encoding is as follows: The hot word prompt vector set is mapped to the same dimensional space as the speech feature encoding through a multilayer perceptron; Element-level addition and fusion are performed based on residual connection to obtain the initial decoding state.

[0023] According to an embodiment of the present invention, the biased decoding model training method is as follows: Construct a training dataset, which includes speech data, corresponding text transcriptions, and hot word annotations; Initialize the biased decoding model parameters, batch sample the training data, convert speech data into acoustic features, convert text transcription into unit sequences, and convert hot word annotations into hot word vector sets; The acoustic features are encoded into speech to obtain speech feature codes, and the hot word vector set is encoded to obtain hot word feature codes; Based on the fusion mechanism, speech feature encoding and hot word feature encoding are fused, and the fused feature encoding is then subjected to autoregressive decoding to generate a prediction unit sequence. Calculate the cross-entropy loss between the predicted unit sequence and the real text transcription, adjust the model parameters based on the cross-entropy loss value, and obtain the trained biased decoding model.

[0024] According to an embodiment of the present invention, a biasless decoding model is used to decode the speech data. The biasless decoding model does not embed customized hot word prompts, resulting in a preliminary transcription result B, which specifically includes: Acquire the speech data to be recognized, extract features from the speech data, and generate an acoustic feature sequence; The acoustic feature sequence is input into the unbiased decoding model to generate speech feature codes without hot word bias. Initialize the decoder state to obtain the initial state, perform autoregressive decoding based on the initial state, and calculate the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form the preliminary transcription result B.

[0025] According to an embodiment of the present invention, the hallucination probability analysis method is as follows: The customized transcription result A and the preliminary transcription result B are preprocessed to obtain standardized text sequences A' and B'. Input text sequences A' and B' into the text embedding layer to generate corresponding word embedding sequences EA and EB; The word embedding sequence EA and EB are fused for features, and a joint embedding sequence EAB is generated by concatenation. The joint embedding sequence EAB is input into the pre-trained semantic encoder, and the output is a high-level representation containing semantic association information. By inputting the high-level representation into the predictive projection network, the probability of hallucination is obtained.

[0026] like Figures 2-3 As shown, in a second aspect, embodiments of this application provide a speech recognition hallucination suppression system based on a dual-decoding decision network. The system includes a memory and a processor. The memory includes a program for a speech recognition hallucination suppression method based on a dual-decoding decision network. When the program for the speech recognition hallucination suppression method based on a dual-decoding decision network is executed by the processor, it implements the following steps: Acquire the speech data to be recognized; The speech data is decoded using a biased decoding model, and the target hot word prompts are embedded in the biased decoding model to obtain a customized transcription result A. The speech data is decoded using an unbiased decoding model. The unbiased decoding model does not embed custom hot word prompts, and the preliminary transcription result B is obtained. The representation tensors of the customized transcription result A and the preliminary transcription result B are input into the pre-trained semantic encoder to obtain the high-level representation. The high-level representation is then input into the trained prediction projection network to obtain the hallucination probability. The hallucination probability is compared with the set discrimination threshold. If the hallucination probability is less than or equal to the discrimination threshold, the customized transcription result A is output; if the hallucination probability is greater than the discrimination threshold, the preliminary transcription result B is output.

[0027] According to an embodiment of the present invention, a biased decoding model is used to decode speech data, and the biased decoding model embeds target hot word prompts to obtain a customized transcription result A, specifically including: Obtain target hot words, preprocess the target hot word prompts, and generate a hot word prompt vector set. The preprocessing includes word segmentation, normalization, and vector embedding. The speech data to be recognized is converted into an acoustic feature sequence, input into a biased decoding model, and outputs speech feature encoding. The initial decoding state is obtained by fusing the hot word prompt vector set with the speech feature encoding; Autoregressive decoding is performed based on the initial decoding state, and the association weight between the hot word prompt vector set and the current decoding is strengthened based on the attention mechanism to obtain the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form a customized transcription result A.

[0028] According to an embodiment of the present invention, the method for fusing hot word suggestion vector sets and speech feature encoding is as follows: The hot word prompt vector set is mapped to the same dimensional space as the speech feature encoding through a multilayer perceptron; Element-level addition and fusion are performed based on residual connection to obtain the initial decoding state.

[0029] A third aspect of the present invention provides a computer-readable storage medium including a speech recognition hallucination suppression method program based on a dual-decoding decision network. When the speech recognition hallucination suppression method program based on a dual-decoding decision network is executed by a processor, it implements the steps of the speech recognition hallucination suppression method based on a dual-decoding decision network as described in any of the above claims.

[0030] This invention discloses a speech recognition hallucination suppression method, system, and medium based on a dual-decoding decision network. The method involves acquiring speech data to be recognized; decoding the speech data using a biased decoding model, which embeds target hot word prompts to obtain a customized transcription result A; decoding the speech data using an unbiased decoding model, which does not embed customized hot word prompts to obtain a preliminary transcription result B; inputting the representation tensors of the customized transcription result A and the preliminary transcription result B into a pre-trained semantic encoder to obtain a high-level representation; inputting the high-level representation into a trained prediction projection network to obtain a hallucination probability; comparing the hallucination probability with a set discrimination threshold; if the hallucination probability is less than or equal to the discrimination threshold, outputting the customized transcription result A; if the hallucination probability is greater than the discrimination threshold, outputting the preliminary transcription result B. By running two decoding paths in parallel on the same speech segment, the method effectively eliminates hallucination insertion errors caused by prompts, significantly reduces the probability of irrelevant content appearing in the transcription result, and significantly improves the overall recognition accuracy in scenarios containing user-customized hot words.

[0031] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0032] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0033] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0034] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0035] Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for suppressing hallucinations in speech recognition based on a dual-decoding decision network, characterized in that, include: Acquire the speech data to be recognized; The speech data is decoded using a biased decoding model, which embeds target hot word prompts to obtain a customized transcription result A. The speech data is decoded using an unbiased decoding model, which does not embed customized hot word prompts, to obtain a preliminary transcription result B; The representation tensors of the customized transcription result A and the preliminary transcription result B are input into the pre-trained semantic encoder to obtain the high-level representation. The high-level representation is then input into the trained prediction projection network to obtain the hallucination probability. The hallucination probability is compared with the set discrimination threshold. If the hallucination probability is less than or equal to the discrimination threshold, the customized transcription result A is output; if the hallucination probability is greater than the discrimination threshold, the preliminary transcription result B is output. The method for analyzing the probability of hallucinations is as follows: The customized transcription result A and the preliminary transcription result B are preprocessed to obtain standardized text sequences A' and B'. Input text sequences A' and B' into the text embedding layer to generate corresponding word embedding sequences EA and EB; The word embedding sequence EA and EB are fused for features, and a joint embedding sequence EAB is generated by concatenation. The joint embedding sequence EAB is input into the pre-trained semantic encoder, and the output is a high-level representation containing semantic association information. By inputting the high-level representation into the predictive projection network, the probability of hallucination is obtained.

2. The speech recognition hallucination suppression method based on a dual-decoding decision network according to claim 1, characterized in that, The speech data is decoded using a biased decoding model, which embeds target hot word prompts to obtain a customized transcription result A, specifically including: The target hot words are obtained, and the target hot word prompts are preprocessed to generate a hot word prompt vector set. The preprocessing includes word segmentation, normalization and vector embedding. The speech data to be recognized is converted into an acoustic feature sequence, input into a biased decoding model, and outputs speech feature encoding. The hot word prompt vector set is fused with the speech feature encoding to obtain the initial decoding state; Autoregressive decoding is performed based on the initial decoding state, and the association weight between the hot word prompt vector set and the current decoding is strengthened based on the attention mechanism to obtain the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form a customized transcription result A.

3. The speech recognition hallucination suppression method based on a dual-decoding decision network according to claim 2, characterized in that, The method for fusing hot word suggestion vector sets with speech feature encoding is as follows: The hot word prompt vector set is mapped to the same dimensional space as the speech feature encoding through a multilayer perceptron; Element-level addition and fusion are performed based on residual connection to obtain the initial decoding state.

4. The speech recognition hallucination suppression method based on a dual-decoding decision network according to claim 3, characterized in that, The biased decoding model training method is as follows: Construct a training dataset, which includes speech data, corresponding text transcriptions, and hot word annotations; Initialize the biased decoding model parameters, batch sample the training data, convert speech data into acoustic features, convert text transcription into unit sequences, and convert hot word annotations into hot word vector sets; The acoustic features are encoded into speech to obtain speech feature codes, and the hot word vector set is encoded to obtain hot word feature codes; Based on the fusion mechanism, speech feature encoding and hot word feature encoding are fused, and the fused feature encoding is then subjected to autoregressive decoding to generate a prediction unit sequence. Calculate the cross-entropy loss between the predicted unit sequence and the real text transcription, adjust the model parameters based on the cross-entropy loss value, and obtain the trained biased decoding model.

5. The speech recognition hallucination suppression method based on a dual-decoding decision network according to claim 4, characterized in that, The speech data is decoded using an unbiased decoding model that does not embed custom hot word prompts, resulting in a preliminary transcription result B, which specifically includes: Acquire the speech data to be recognized, extract features from the speech data, and generate an acoustic feature sequence; The acoustic feature sequence is input into the unbiased decoding model to generate speech feature codes without hot word bias. Initialize the decoder state to obtain the initial state, perform autoregressive decoding based on the initial state, and calculate the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form the preliminary transcription result B.

6. A speech recognition hallucination suppression system based on a dual-decoding decision network, characterized in that, The system includes a memory and a processor. The memory contains a program for a speech recognition hallucination suppression method based on a dual-decoding decision network. When the program for the speech recognition hallucination suppression method based on the dual-decoding decision network is executed by the processor, it performs the following steps: Acquire the speech data to be recognized; The speech data is decoded using a biased decoding model, which embeds target hot word prompts to obtain a customized transcription result A. The speech data is decoded using an unbiased decoding model, which does not embed customized hot word prompts, to obtain a preliminary transcription result B; The representation tensors of the customized transcription result A and the preliminary transcription result B are input into the pre-trained semantic encoder to obtain the high-level representation. The high-level representation is then input into the trained prediction projection network to obtain the hallucination probability. The hallucination probability is compared with the set discrimination threshold. If the hallucination probability is less than or equal to the discrimination threshold, the customized transcription result A is output; if the hallucination probability is greater than the discrimination threshold, the preliminary transcription result B is output. The method for analyzing the probability of hallucinations is as follows: The customized transcription result A and the preliminary transcription result B are preprocessed to obtain standardized text sequences A' and B'. Input text sequences A' and B' into the text embedding layer to generate corresponding word embedding sequences EA and EB; The word embedding sequence EA and EB are fused for features, and a joint embedding sequence EAB is generated by concatenation. The joint embedding sequence EAB is input into the pre-trained semantic encoder, and the output is a high-level representation containing semantic association information. By inputting the high-level representation into the predictive projection network, the probability of hallucination is obtained.

7. The speech recognition hallucination suppression system based on a dual-decoding decision network according to claim 6, characterized in that, The speech data is decoded using a biased decoding model, which embeds target hot word prompts to obtain a customized transcription result A, specifically including: The target hot words are obtained, and the target hot word prompts are preprocessed to generate a hot word prompt vector set. The preprocessing includes word segmentation, normalization and vector embedding. The speech data to be recognized is converted into an acoustic feature sequence, input into a biased decoding model, and outputs speech feature encoding. The hot word prompt vector set is fused with the speech feature encoding to obtain the initial decoding state; Autoregressive decoding is performed based on the initial decoding state, and the association weight between the hot word prompt vector set and the current decoding is strengthened based on the attention mechanism to obtain the decoding probability distribution; The optimal decoding sequence is selected based on the decoding probability distribution to form a customized transcription result A.

8. The speech recognition hallucination suppression system based on a dual-decoding decision network according to claim 7, characterized in that, The method for fusing hot word suggestion vector sets with speech feature encoding is as follows: The hot word prompt vector set is mapped to the same dimensional space as the speech feature encoding through a multilayer perceptron; Element-level addition and fusion are performed based on residual connection to obtain the initial decoding state.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a speech recognition hallucination suppression method program based on a dual-decoding decision network. When the speech recognition hallucination suppression method program based on a dual-decoding decision network is executed by a processor, it implements the steps of the speech recognition hallucination suppression method based on a dual-decoding decision network as described in any one of claims 1 to 5.

Images