Speech processing method, apparatus, device, and storage medium

Abstract

Embodiments of the present application disclose a speech processing method, device and equipment and a storage medium. First, it is judged whether the speech data belongs to noise. In the case that the speech data does not belong to noise, speech recognition is performed on the speech data. In addition to obtaining a speech recognition result, a confidence of the speech recognition result is also obtained. It is judged whether the speech recognition result belongs to an instruction set corresponding to a target scene. Only in the case that the speech recognition result belongs to the instruction set corresponding to the target scene and the confidence is greater than a first confidence threshold, the speech recognition result is output. Through multi-level judgment, maximum noise rejection is achieved.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and more specifically, to a speech processing method, apparatus, device, and storage medium. Background Technology

[0002] Speech recognition is a technology that uses machine learning to convert spoken language into corresponding written language, giving machines auditory abilities similar to humans. It is an important branch of artificial intelligence applications. With the development of artificial intelligence technology and the accumulation of a large amount of available training data, the accuracy of speech recognition has reached commercial standards and has been successfully implemented on a large scale on various smart terminal devices, bringing users a more convenient experience and lowering the barrier to human-machine interaction. However, in the interaction scenarios of smart terminal devices, noise is often misidentified by the speech recognition system, resulting in a very poor user experience. Summary of the Invention

[0003] In view of this, this application provides a speech processing method, apparatus, device and storage medium to improve noise rejection performance.

[0004] To achieve the above objectives, the following solution is proposed:

[0005] A speech processing method, the method comprising:

[0006] Noise discrimination of speech data;

[0007] If the noise discrimination result indicates that the speech data does not belong to noise, speech recognition is performed on the speech data to obtain the speech recognition result and the confidence level of the speech recognition result;

[0008] Determine whether the speech recognition result belongs to the instruction set corresponding to the target scene;

[0009] If the speech recognition result belongs to the instruction set corresponding to the target scene, and the confidence level is greater than the first confidence level threshold, the speech recognition result is output.

[0010] The above methods may optionally include:

[0011] If the speech recognition result does not belong to the instruction set corresponding to the target scene, the speech recognition result is output.

[0012] Optionally, the noise discrimination of the speech data in the above method includes:

[0013] Extract the acoustic features of the speech data;

[0014] The acoustic features are encoded to obtain the encoding result;

[0015] The encoding result is processed at least to obtain the noise discrimination result.

[0016] Optionally, in the above method, processing the encoding result to obtain the noise discrimination result includes:

[0017] Extract the attribute features of the voice data;

[0018] The attribute features are fused with the encoding results to obtain fused features;

[0019] The fusion features are processed to obtain the noise discrimination result.

[0020] Optionally, in the above method, performing speech recognition on the speech data includes:

[0021] The encoding result obtained by encoding the acoustic features of the speech data is used to obtain the noise discrimination result.

[0022] The encoded result is then subjected to speech recognition decoding to obtain the speech recognition result and its confidence level.

[0023] Optionally, in the above method, determining whether the speech recognition result belongs to the instruction set corresponding to the target scene includes:

[0024] If the confidence level is greater than the second confidence threshold, determine whether the speech recognition result belongs to the instruction set corresponding to the target scene;

[0025] The second confidence threshold is less than the first confidence threshold.

[0026] Optionally, the noise discrimination process for the speech data is implemented by a noise discrimination module, and the speech recognition process for the speech data is implemented by an acoustic decoding module.

[0027] The noise discrimination module and the acoustic decoding module are jointly trained.

[0028] A voice processing device, the device comprising:

[0029] The first discrimination module is used to perform noise discrimination on the speech data;

[0030] The recognition module is used to perform speech recognition on the speech data if the noise discrimination result indicates that the speech data does not belong to noise, and to obtain the speech recognition result and the confidence level of the speech recognition result;

[0031] The second discrimination module is used to determine whether the speech recognition result belongs to the instruction set corresponding to the target scene;

[0032] The output module is used to output the speech recognition result if the speech recognition result belongs to the instruction set corresponding to the target scene and the confidence level is greater than the first confidence level threshold.

[0033] A voice processing device, including a memory and a processor;

[0034] The memory is used to store programs;

[0035] The processor is configured to execute the program to implement the various steps of the speech processing method as described in any of the preceding claims.

[0036] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the speech processing method as described in any of the preceding claims.

[0037] As can be seen from the above technical solutions, the speech processing method, apparatus, device and storage medium provided in the embodiments of this application first determine whether the speech data belongs to noise. If the speech data does not belong to noise, then speech recognition is performed on the speech data. In addition to obtaining the speech recognition result, the confidence level of the speech recognition result is also obtained. It is then determined whether the speech recognition result belongs to the instruction set corresponding to the target scene. Only when the speech recognition result belongs to the instruction set corresponding to the target scene and the confidence level is greater than the first confidence level threshold, the speech recognition result is output. Through multi-level judgment, the maximum noise rejection is achieved, and the noise rejection effect is improved. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating one implementation of the speech processing method disclosed in this application.

[0040] Figure 2 This is a flowchart illustrating an implementation of noise discrimination of speech data disclosed in an embodiment of this application;

[0041] Figure 3 This is a flowchart of an implementation of at least processing the encoding result to obtain a noise discrimination result, as disclosed in an embodiment of this application.

[0042] Figure 4 This is a schematic diagram illustrating the structure and connection relationship of the noise discrimination module and acoustic decoding module disclosed in an embodiment of this application.

[0043] Figure 5 This is another structural diagram illustrating the structure and connection relationship of the noise discrimination module and acoustic decoding module disclosed in the embodiments of this application;

[0044] Figure 6 This is a schematic diagram of the structure of the voice processing device disclosed in an embodiment of this application;

[0045] Figure 7 This is a hardware structure block diagram of the voice processing device disclosed in the embodiments of this application. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] Currently, in the interaction scenarios of smart terminal devices, noise has a certain probability of being identified as control commands, causing smart devices to perform some unexpected actions, which is unacceptable for some business products and has become a technical problem that urgently needs to be solved.

[0048] To reduce noise interference, one noise rejection scheme is to combine speech with the text semantics of the previous round of dialogue in continuous dialogue scenarios for speech recognition. However, this scheme is based on the existence of multi-round dialogue scenarios, and there will be problems with noise rejection of the first round of dialogue where there is a lack of semantic prior knowledge.

[0049] like Figure 1 The diagram shown is a flowchart of one implementation of the speech processing method provided in this application, which may include:

[0050] Step S101: Perform noise discrimination on the speech data.

[0051] Voice data is the voice data to be processed. It can be received by the sound pickup device (microphone or microphone array, etc.) of an electronic device. As an example, the electronic device can be a smart terminal with a voice recognition system installed, which can include, but is not limited to, any of the following: mobile phone, smart speaker, home appliance, in-vehicle computer, etc.

[0052] The purpose of noise identification in speech data is to determine whether the speech data is noise.

[0053] Noise identification of speech data can be performed using existing noise discrimination methods or the noise discrimination method provided in this application (see below for the specific implementation process).

[0054] Step S102: If the noise discrimination result indicates that the speech data does not belong to noise, perform speech recognition on the speech data to obtain the speech recognition result and the confidence level of the speech recognition result.

[0055] This application performs speech recognition only when the speech data is not noise. Moreover, during speech recognition, it obtains not only the speech recognition result (i.e., the recognized text) but also the confidence level of the speech recognition result.

[0056] Step S103: Determine whether the speech recognition result belongs to the instruction set corresponding to the target scene.

[0057] The target scenario is the current scenario. In other words, the target scenario is determined based on the application scenario of the speech processing method in this application; that is, this application uses the application scenario of the speech processing method as the target scenario.

[0058] Optionally, a pre-trained natural language understanding (NLU) model can be used to determine whether the speech recognition result belongs to the instruction set corresponding to the target scene.

[0059] Step S104: If the speech recognition result belongs to the instruction set corresponding to the target scene and the confidence level is greater than the first confidence level threshold, output the speech recognition result.

[0060] In this application, if the speech recognition result belongs to the instruction set corresponding to the target scene, and the confidence level of the speech recognition result is greater than the first confidence level threshold, it indicates that the recognition result is reliable. At this time, the speech recognition result can be output so as to achieve effective control in the target scene.

[0061] In this embodiment of the application, outputting the speech recognition result may refer to outputting the speech recognition result to a downstream module for semantic understanding or intent recognition, etc. Alternatively,

[0062] Outputting speech recognition results can refer to displaying the results on a display device. Alternatively,

[0063] Outputting speech recognition results can refer to simultaneously sending the speech recognition results to downstream modules and displaying the speech recognition results through a display device.

[0064] The speech processing method provided in this application first determines whether the speech data is noise after obtaining the speech data. If the speech data is not noise, speech recognition is then performed on the speech data. In addition to obtaining the speech recognition result, the confidence level of the speech recognition result is also obtained. It is then determined whether the speech recognition result belongs to the instruction set corresponding to the target scene. Only when the speech recognition result belongs to the instruction set corresponding to the target scene and the confidence level is greater than the first confidence level threshold is the speech recognition result output. Through multi-level judgment, noise rejection is maximized and the noise rejection effect is improved.

[0065] In an optional embodiment, if the speech recognition result belongs to the instruction set corresponding to the target scene, and the confidence level is less than or equal to a first confidence threshold, then the speech recognition result is prohibited from being output. That is, even if the speech recognition result is determined to belong to the instruction set corresponding to the target scene, since the confidence level of the speech recognition result is less than or equal to the first confidence threshold, it indicates that the result of determining that the speech recognition result belongs to the instruction set corresponding to the target scene is unreliable. Therefore, the speech recognition result will not be output to avoid misrecognition in the target scene.

[0066] In an optional embodiment, if the speech recognition result does not belong to the instruction set corresponding to the target scene, the speech recognition result is output.

[0067] In other words, if the speech result does not belong to the instruction set corresponding to the target scene, the speech recognition result can be output directly without worrying about the confidence level of the speech recognition result.

[0068] In an optional embodiment, one way to implement the above-mentioned determination of whether the speech recognition result belongs to the instruction set corresponding to the target scene can be:

[0069] If the confidence level of the speech recognition result is greater than the second confidence level threshold, determine whether the speech recognition result corresponds to the instruction set of the speech target scene.

[0070] The second confidence threshold is less than the first confidence threshold.

[0071] In other words, this application will only determine whether the speech recognition result belongs to the instruction set corresponding to the speech target scene if the confidence level of the speech recognition result is greater than the second confidence level threshold.

[0072] Optionally, if the confidence level of the speech recognition result is less than or equal to the second confidence threshold, the speech recognition result is prohibited from being output. In other words, if the confidence level of the speech recognition result is less than or equal to the second confidence threshold, it indicates that the speech recognition result is unreliable, in order to avoid the adverse effects caused by misrecognized speech results.

[0073] By using multiple confidence thresholds, speech recognition results that are mistakenly triggered as instruction sets can be effectively rejected. Based on this, multi-level judgment can further improve the effectiveness of noise rejection.

[0074] Furthermore, based on the first confidence threshold and the second confidence threshold, speech recognition results with confidence levels between the first confidence threshold and the second confidence threshold can be continuously collected. Through simple manual verification, audio that is unclear or incomprehensible is labeled as noise, which greatly reduces the scope of manual detection of noise label data and reduces labor costs. Moreover, based on this application, noise data in the existing network can be efficiently retrieved and added to the training data to update the model, further shortening the response time of noise rejection and improving the user experience.

[0075] In an optional embodiment, the flowchart for one implementation of noise discrimination of the above-described speech data is as follows: Figure 2 As shown, it may include:

[0076] Step S201: Extract the acoustic features of the speech data.

[0077] Optionally, the acoustic features can be spectral features of the speech data, including but not limited to at least one of the following: Mel Frequency Cepstrum Coefficient (MFCC), Perceptual Linear Prediction (PLP), Filter-Bank (FBank), or Constant Q Cepstrum Coefficient (CQCC), etc.

[0078] Optionally, the speech data can be segmented into frames to obtain multiple speech frames. Each speech frame can be pre-emphasized, and then acoustic features can be extracted from each pre-emphasized speech frame to obtain the acoustic features of each speech frame. The acoustic feature of each speech frame is a multi-dimensional vector, where each dimension of the multi-dimensional vector is a feature of one dimension of the speech frame corresponding to the multi-dimensional vector.

[0079] Step S202: Encode the acoustic features to obtain the encoding result.

[0080] Optionally, the acoustic features of each speech frame can be encoded separately to obtain the encoding results of each speech frame.

[0081] or,

[0082] For each speech frame (denoted as the i-th speech frame for ease of description), the i-th speech frame can be encoded using the i-th speech frame and at least one speech frame before and after it to obtain the encoding result of the i-th speech frame. Compared with the previous encoding method, this method can improve the discriminative power of the encoded speech frame for acoustic features, thereby improving the accuracy of subsequent speech recognition.

[0083] Step S203: At least process the encoding result to obtain the noise discrimination result.

[0084] In other words, in this application, the noise discrimination result can be obtained solely based on the above coding result, or the noise coding result can be obtained by combining the above coding result with other features. As an example, other features may include, but are not limited to, attribute features of the speech data. Optionally, attribute features refer to higher-order audio features of the speech data, which may include, but are not limited to, at least one of the following: volume, speech rate, signal-to-noise ratio, etc.

[0085] Optionally, the implementation flowchart of at least processing the encoding result to obtain the noise discrimination result provided in the embodiments of this application is as follows: Figure 3 As shown, it may include:

[0086] Step S301: Extract the attribute features of the speech data.

[0087] Step S302: Fuse the attribute features with the encoding results to obtain fused features.

[0088] Optionally, attribute features and encoded features can be fused using a fusion network to obtain fused features. Alternatively,

[0089] The attribute features and encoding results can be concatenated to obtain fused features; or,

[0090] The attribute features can be added to or averaged with the encoding results to obtain the fused features.

[0091] Step S303: Process the fusion features to obtain the noise discrimination result.

[0092] This application improves the accuracy of noise discrimination results by fusing attribute features with coding features and performing noise discrimination based on the fused features.

[0093] Optionally, one implementation method for performing speech recognition on the above-mentioned speech data can be:

[0094] The above encoding results are then decoded using speech recognition to obtain the speech recognition result and its confidence level.

[0095] In other words, in this application, the noise discrimination and speech recognition processes share the encoding results, which can improve the speech processing speed. Moreover, when the speech processing method is implemented through a model, the model results and the number of model parameters can be reduced.

[0096] In an optional embodiment, the above-described process of noise discrimination of speech data is implemented by a noise discrimination module; the process of speech recognition of speech data is implemented by an acoustic decoding module.

[0097] The noise discrimination module and the acoustic decoding module were jointly trained.

[0098] like Figure 4 The diagram shown illustrates the structure and connection relationship between the noise discrimination module and the acoustic decoding module provided in an embodiment of this application.

[0099] The noise discrimination module includes at least: an acoustic feature extraction module 401, an encoding module 402, a noise discrimination decoding module 403, and a judgment module 404.

[0100] The acoustic feature extraction module 401 is used to extract acoustic features from speech data.

[0101] The encoding module 402 is used to encode the acoustic features to obtain the encoding result.

[0102] The noise discrimination decoding module 403 is used to process at least the encoding result to obtain the noise discrimination result.

[0103] The judgment module 404 is used to determine whether the speech data is noise based on the noise discrimination result. If it is not noise, the speech recognition decoding module is triggered to perform speech decoding; if it is noise, speech recognition is refused.

[0104] The acoustic decoding module includes at least one speech recognition module 405. The speech recognition module 405 is used to perform speech recognition decoding on the encoding result output by the encoding module 402 to obtain the speech recognition result and the confidence level of the speech recognition result.

[0105] Optionally, speech data labeled with speech recognition and noise discrimination results can be used as training samples to jointly train the noise discrimination module and the acoustic decoding module. Specifically, the training samples include positive and negative samples. Positive samples are labeled with text recognition results and non-noise labels, while negative samples are labeled with empty results and noise labels. Optionally, the labeling of the sample data is related to the modeling units of the noise discrimination module and the acoustic decoding module. For example, if the modeling unit of the noise discrimination module and the acoustic decoding module is a phoneme, then phone-level labeling should be performed when labeling the sample data. That is, each speech frame of the positive sample data is labeled with a phoneme and a non-noise label, and each speech frame of the negative sample data is labeled with a specific symbol (representing an empty result) and a noise label. If the modeling unit of the noise discrimination module and the acoustic decoding module is a word, then word-level labeling should be performed when labeling the sample data. That is, positive sample data is labeled with the corresponding text and non-noise labels, and negative sample data is labeled with an end symbol (representing an empty result) and a noise label.

[0106] Optionally, the noise discrimination module and the acoustic decoding module can be jointly trained in the following way:

[0107] The sample data is input into the noise discrimination module, and the acoustic feature extraction module 401 extracts acoustic features from the sample data to obtain the acoustic features of the sample data. The encoding module 402 encodes the acoustic features output by the acoustic feature extraction module 401 to obtain the encoding result. The noise discrimination decoding module 403 decodes the encoding result output by the encoding module 402 to obtain the noise discrimination result. The judgment module 404 judges the noise discrimination result output by the noise discrimination decoding module 403. If the noise discrimination result indicates that the sample data is not noise, a trigger signal is sent to the speech recognition module 405 of the acoustic decoding module to trigger the speech recognition module 405 to decode the encoding result output by the encoding module 402 to obtain the speech recognition result and the confidence level of the speech recognition result.

[0108] The goal is to make the noise discrimination result output by the noise discrimination decoding module 403 and the speech recognition result output by the speech recognition module 405 approximate the label of the sample data. The parameters of the noise discrimination module and the acoustic decoding module (mainly the parameters of the encoding module 402, noise discrimination decoding module 403, and acoustic decoding module) are updated until the training termination condition is met. Optionally, since the noise discrimination module and the acoustic decoding module belong to two different branches, the sum of the loss functions of the two branches can be used as the joint training loss function L. The definition is as follows:

[0109] L=αL n +(1-α)L asr

[0110] Among them, L n L represents the loss function of the noise discrimination module branch. asr Let α represent the loss function of the acoustic decoding module branch, where 0 ≤ α ≤ 1.

[0111] Alternatively, the stochastic steepest descent method (SGD) can be used to update the parameters of the noise discrimination module and the acoustic decoding module based on the above loss function L. This can both speed up the convergence and prevent the model from getting stuck in local optima.

[0112] Optionally, the activation functions of the output layers in both the noise discrimination decoding module 403 and the speech recognition module 405 can use softmax to optimize the noise discrimination and speech recognition results. The softmax function converts the outputs of both the noise discrimination decoding module 403 and the speech recognition module 405 into probability distributions. Based on this, the speech recognition result output by the speech recognition module 405 can be determined by the maximum probability corresponding to each speech frame output by the speech recognition module 405. The confidence level of the speech recognition result output by the speech recognition module 405 can be represented by the maximum probability among the probabilities corresponding to the speech recognition result (the larger the probability value, the higher the confidence level; the smaller the probability value, the lower the confidence level). Alternatively, the confidence level of the speech recognition result output by the speech recognition module 405 can be represented by the difference between the maximum probability and the second-highest probability among the probabilities corresponding to the speech recognition result (the larger the difference, the higher the confidence level; the smaller the difference, the lower the confidence level).

[0113] Optional, such as Figure 5 The diagram shown illustrates another structural configuration of the noise discrimination module and the acoustic decoding module provided in this embodiment, including their connections.

[0114] The noise discrimination module includes at least: an acoustic feature extraction module 401, an encoding module 402, an audio high-order feature extraction module 501, a noise discrimination decoding module 403, and a judgment module 404.

[0115] The acoustic feature extraction module 401 is used to extract acoustic features from speech data.

[0116] The encoding module 402 is used to encode the acoustic features to obtain the encoding result.

[0117] The audio high-order feature extraction module 501 is used to extract the attribute features of the speech data from the speech data.

[0118] The noise discrimination decoding module 403 is used to fuse attribute features and encoding results to obtain fused features, process the fused features, and obtain noise discrimination results.

[0119] The judgment module 404 is used to determine whether the speech data is noise based on the noise discrimination result. If it is not noise, the acoustic decoding module is triggered to perform speech decoding; if it is noise, speech recognition is refused.

[0120] The acoustic decoding module includes at least one speech recognition module 405. The speech recognition module 405 is used to perform speech recognition decoding on the encoding result output by the encoding module 402 to obtain the speech recognition result and the confidence level of the speech recognition result.

[0121] Optionally, speech data labeled with speech recognition and noise discrimination results can be used as training samples to jointly train the noise discrimination module and the acoustic decoding module. For details on the composition of the training samples, please refer to [link / reference needed]. Figure 4 Related embodiments will not be described in detail here.

[0122] Optionally, the noise discrimination module and the acoustic decoding module can be jointly trained in the following way:

[0123] The sample data is input into the noise discrimination module, where the acoustic feature extraction module 401 extracts acoustic features from the sample data to obtain the acoustic features of the sample data. The audio high-order feature extraction module 501 extracts the attribute features of the speech data from the speech data, and the encoding module 402 encodes the acoustic features output by the acoustic feature extraction module 401 to obtain the encoding result. The noise discrimination decoding module 403 fuses the encoding result output by the encoding module 402 and the attribute features extracted by the audio high-order feature extraction module 501 to obtain the fused feature, and decodes the fused feature to obtain the noise discrimination result. The judgment module 404 judges the noise discrimination result output by the noise discrimination decoding module 403. If the noise discrimination result indicates that the sample data is not noise, a trigger signal is sent to the speech recognition module 405 of the acoustic decoding module to trigger the speech recognition module 405 to decode the encoding result output by the encoding module 402 to obtain the speech recognition result and the confidence level of the speech recognition result.

[0124] With the goal of ensuring that the noise discrimination result output by the noise discrimination decoding module 403 and the speech recognition result output by the speech recognition module 405 approximate the labels of the sample data, the parameters of the noise discrimination module and the acoustic decoding module (mainly the parameters of the encoding module 402, the noise discrimination decoding module 403, and the acoustic decoding module) are updated until the training termination condition is met. For details of the update process, please refer to [link to documentation]. Figure 4 Related embodiments will not be described in detail here.

[0125] and Figure 4 The embodiments shown are different. Figure 5 The illustrated embodiment is in Figure 4Based on the embodiment shown, an audio high-order feature extraction module 501 for extracting attribute features of speech data from speech data is added, thereby improving the noise discrimination accuracy of the noise discrimination module.

[0126] Figure 4 and Figure 5 In the illustrated embodiment, the noise discrimination module and the acoustic decoding module share the encoding module. In other embodiments of this application, the noise discrimination module and the acoustic decoding module may not share the encoding module, but instead use independent encoding modules. However, the parameters of the noise discrimination module and the acoustic decoding module still need to be updated based on the loss function L obtained from joint training. The two independent encoding modules may or may not share parameters.

[0127] The following is an exemplary description of the solution proposed in this application.

[0128] Example 1

[0129] Assuming the speech data is a piece of pure noise audio, then based on this application:

[0130] Acoustic features can be extracted from the speech data to obtain its acoustic features. These features can then be encoded to obtain the encoded results. Decoding these results yields a noise discrimination result. If the noise discrimination result indicates that the speech data is noise, it is directly rejected, and the interaction ends. Alternatively,

[0131] Attribute features can be extracted from speech data, and acoustic features can be extracted from the speech data to obtain acoustic features of the speech data. The acoustic features are encoded to obtain the encoding result. The encoding result and attribute features are fused to obtain fused features. The fused features are decoded to obtain noise discrimination result. If the noise discrimination result indicates that the speech data is noise, it is directly rejected and the interaction ends.

[0132] Example 2

[0133] Assuming the speech data is a noisy speech containing human voices, then based on this application:

[0134] Acoustic features can be extracted from the speech data to obtain its acoustic features. These features are then encoded to obtain an encoding result. The encoding result is then decoded to obtain a noise discrimination result, which indicates that the speech data is not noise. Speech recognition decoding is then performed on the encoded result to obtain the speech recognition result and its confidence level. If the confidence level of the speech recognition result is less than a second confidence threshold, it is directly rejected. If the confidence level of the speech recognition result is greater than or equal to the second confidence threshold, the speech recognition result is processed by NLU to determine whether it corresponds to the instruction in the target scene. If it does not correspond to the instruction in the target scene, the speech recognition result can be output. If it corresponds to the instruction in the target scene, and the confidence level of the speech recognition result is less than a first confidence threshold (which is greater than the second confidence threshold), it is directly rejected, i.e., no speech recognition result is output. If the confidence level of the speech recognition result is greater than or equal to the first confidence threshold, the speech recognition result can be output. Alternatively,

[0135] Attribute features can be extracted from speech data, and acoustic features can be extracted to obtain acoustic features of the speech data. These acoustic features are then encoded to obtain an encoding result. The encoding result and attribute features are fused to obtain a fused feature, which is then decoded to obtain a noise discrimination result. This noise discrimination result indicates that the speech data is not noise. Speech recognition decoding is then performed on the encoded result to obtain the speech recognition result and its confidence level. If the confidence level of the speech recognition result is less than a second confidence threshold, it is directly rejected. If the confidence level of the speech recognition result is greater than or equal to the second confidence threshold, the speech recognition result is processed by NLU to determine whether it corresponds to the instruction of the target scene. If it does not correspond to the instruction of the target scene, the speech recognition result can be output. If it corresponds to the instruction of the target scene, and the confidence level of the speech recognition result is less than a first confidence threshold (the first confidence threshold is greater than the second confidence threshold), it is directly rejected, i.e., no speech recognition result is output. If the confidence level of the speech recognition result is greater than or equal to the first confidence threshold, the speech recognition result can be output.

[0136] Example 3

[0137] In the locomotive-vehicle interaction scenario, the first confidence threshold can be set to 0.7, and the second confidence threshold can be set to 0.5.

[0138] Assuming the speech recognition result is "turn off the engine," and the confidence level of the speech recognition result is 0.6, then, since 0.6 is greater than 0.5, the pre-trained NLU is used to determine whether "turn off the engine" belongs to the locomotive-vehicle interaction scenario. Assuming the determination result is that "turn off the engine" belongs to the locomotive-vehicle interaction scenario, the confidence level of the speech recognition result is further compared with the first confidence level threshold. Since 0.6 is less than 0.7, the output of the speech recognition result "turn off the engine" is prohibited.

[0139] Assuming the speech recognition result is "turn off the engine," and the confidence level of the speech recognition result is 0.85, then, since 0.85 is greater than 0.5, the pre-trained NLU is used to determine whether "turn off the engine" belongs to the locomotive-vehicle interaction scenario. Assuming the determination result is that "turn off the engine" belongs to the locomotive-vehicle interaction scenario, the confidence level of the speech recognition result is further compared with the first confidence level threshold. Since 0.85 is greater than 0.7, the speech recognition result "turn off the engine" is output.

[0140] Suppose the speech recognition result is "Check today's weather", and the confidence level of the speech recognition result is 0.6. Since 0.6 is greater than 0.5, the pre-trained NLU is used to determine whether "Check today's weather" belongs to the locomotive-vehicle interaction scenario. If the determination result is that "Check today's weather" does not belong to the locomotive-vehicle interaction scenario, then the speech recognition result "Check today's weather" is output.

[0141] Suppose the speech recognition result is "Check today's weather", and the confidence level of the speech recognition result is 0.4. Then, since 0.4 is less than 0.5, the speech recognition result "Check today's weather" is prohibited from being output, that is, it is directly rejected, and there is no need to execute the step of determining whether "Check today's weather" belongs to the locomotive-vehicle interaction scenario and subsequent steps.

[0142] Corresponding to the method embodiments, this application also provides a voice processing device. A schematic diagram of the structure of the voice processing device provided in this application embodiment is shown below. Figure 6 As shown, it may include:

[0143] The system comprises a first discrimination module 601, an identification module 602, a second discrimination module 603, and an output module 604; wherein...

[0144] The first discrimination module 601 is used to perform noise discrimination on the speech data;

[0145] The recognition module 602 is used to perform speech recognition on the speech data if the noise discrimination result indicates that the speech data does not belong to noise, and to obtain the speech recognition result and the confidence level of the speech recognition result;

[0146] The second discrimination module 603 is used to determine whether the speech recognition result belongs to the instruction set corresponding to the target scene;

[0147] The output module 604 is used to output the speech recognition result if the speech recognition result belongs to the instruction set corresponding to the target scene and the confidence level is greater than the first confidence level threshold.

[0148] The speech processing device provided in this application first determines whether the speech data is noise. If the speech data is not noise, it then performs speech recognition on the speech data. In addition to obtaining the speech recognition result, it also obtains the confidence level of the speech recognition result and determines whether the speech recognition result belongs to the instruction set corresponding to the target scene. Only when the speech recognition result belongs to the instruction set corresponding to the target scene and the confidence level is greater than the first confidence level threshold, the speech recognition result is output. Through multi-level judgment, the maximum noise rejection is achieved.

[0149] In an optional embodiment, the output module 604 may also be used for:

[0150] If the speech recognition result does not belong to the instruction set corresponding to the target scene, the speech recognition result is output.

[0151] In an optional embodiment, the first discrimination module 601 includes:

[0152] An acoustic feature extraction module is used to extract the acoustic features of the speech data;

[0153] The encoding module is used to encode the acoustic features to obtain the encoding result;

[0154] A decoding module is used to process at least the encoding result to obtain the noise discrimination result.

[0155] In an optional embodiment, the apparatus further includes an attribute feature extraction module for extracting attribute features from the speech data;

[0156] The decoding module is specifically used for:

[0157] The attribute features are fused with the encoding results to obtain fused features;

[0158] The fusion features are processed to obtain the noise discrimination result.

[0159] In an optional embodiment, when the recognition module 602 performs speech recognition on the speech data, it is used to:

[0160] The encoding result obtained by encoding the acoustic features of the speech data is used to obtain the noise discrimination result.

[0161] The encoded result is then subjected to speech recognition decoding to obtain the speech recognition result and its confidence level.

[0162] In an optional embodiment, the second discrimination module 603 is specifically used for:

[0163] If the confidence level is greater than the second confidence threshold, determine whether the speech recognition result belongs to the instruction set corresponding to the target scene;

[0164] The second confidence threshold is less than the first confidence threshold.

[0165] In an optional embodiment, the first discrimination module 601 performs noise discrimination on the speech data through a noise discrimination module; the recognition module 602 performs speech recognition on the speech data through an acoustic decoding module.

[0166] The noise discrimination module and the acoustic decoding module are jointly trained.

[0167] The voice processing device provided in this application embodiment can be applied to voice processing equipment, such as PC terminals, cloud platforms, servers, and server clusters. Optionally, Figure 7 A hardware block diagram of the voice processing device is shown, with reference to... Figure 7 The hardware structure of a voice processing device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;

[0168] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0169] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0170] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0171] The memory stores a program, which the processor can call. The program is used for:

[0172] Noise discrimination of speech data;

[0173] If the noise discrimination result indicates that the speech data does not belong to noise, speech recognition is performed on the speech data to obtain the speech recognition result and the confidence level of the speech recognition result;

[0174] Determine whether the speech recognition result belongs to the instruction set corresponding to the target scene;

[0175] If the speech recognition result belongs to the instruction set corresponding to the target scene, and the confidence level is greater than the first confidence level threshold, the speech recognition result is output.

[0176] Optionally, the refined and extended functions of the program can be found in the description above.

[0177] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for:

[0178] Noise discrimination of speech data;

[0179] If the noise discrimination result indicates that the speech data does not belong to noise, speech recognition is performed on the speech data to obtain the speech recognition result and the confidence level of the speech recognition result;

[0180] Determine whether the speech recognition result belongs to the instruction set corresponding to the target scene;

[0181] If the speech recognition result belongs to the instruction set corresponding to the target scene, and the confidence level is greater than the first confidence level threshold, the speech recognition result is output.

[0182] Optionally, the refined and extended functions of the program can be found in the description above.

[0183] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0184] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0185] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0186] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0187] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0188] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0189] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0190] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A speech processing method, characterized in that, The method includes: Noise discrimination of speech data; If the noise discrimination result indicates that the speech data does not belong to noise, speech recognition is performed on the speech data to obtain the speech recognition result and the confidence level of the speech recognition result. The confidence level is represented by the difference between the maximum probability corresponding to the speech recognition result and the second maximum probability among the recognition results corresponding to the speech data. The confidence level is positively correlated with the difference. If the confidence level is greater than the second confidence threshold, determine whether the speech recognition result belongs to the instruction set corresponding to the target scene; If the speech recognition result belongs to the instruction set corresponding to the target scene, and the confidence level is greater than the first confidence level threshold, the speech recognition result is output, and the second confidence level threshold is less than the first confidence level threshold. If the confidence level of the speech recognition result is less than or equal to the second confidence level threshold, then the speech recognition result is prohibited from being output.

2. The method according to claim 1, characterized in that, Also includes: If the speech recognition result does not belong to the instruction set corresponding to the target scene, the speech recognition result is output.

3. The method according to claim 1, characterized in that, The noise discrimination of the speech data includes: Extract the acoustic features of the speech data; The acoustic features are encoded to obtain the encoding result; The encoding result is processed at least to obtain the noise discrimination result.

4. The method according to claim 3, characterized in that, The step of processing the encoding result at least to obtain the noise discrimination result includes: Extract the attribute features of the voice data; The attribute features are fused with the encoding results to obtain fused features; The fusion features are processed to obtain the noise discrimination result.

5. The method according to claim 1, characterized in that, The process of performing speech recognition on the speech data includes: The encoding result obtained by encoding the acoustic features of the speech data is used to obtain the noise discrimination result. The encoded result is then subjected to speech recognition decoding to obtain the speech recognition result and its confidence level.

6. The method according to claim 1, characterized in that, The noise discrimination process for the speech data is implemented by a noise discrimination module; the speech recognition process for the speech data is implemented by an acoustic decoding module. The noise discrimination module and the acoustic decoding module are jointly trained.

7. A voice processing device, characterized in that, The device includes: The first discrimination module is used to perform noise discrimination on the speech data; The recognition module is used to perform speech recognition on the speech data if the noise discrimination result indicates that the speech data does not belong to noise, and obtain the speech recognition result and the confidence of the speech recognition result. The confidence is represented by the difference between the maximum probability corresponding to the speech recognition result and the second maximum probability among the recognition results corresponding to the speech data. The confidence is positively correlated with the difference. The second discrimination module is used to determine whether the speech recognition result belongs to the instruction set corresponding to the target scene; The output module is used to output the speech recognition result if the speech recognition result belongs to the instruction set corresponding to the target scene and the confidence level is greater than the first confidence level threshold. Specifically, the second discrimination module is used for: If the confidence level is greater than the second confidence threshold, determine whether the speech recognition result belongs to the instruction set corresponding to the target scene; the second confidence threshold is less than the first confidence threshold. If the confidence level of the speech recognition result is less than or equal to the second confidence level threshold, then the speech recognition result is prohibited from being output.

8. A voice processing device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the various steps of the speech processing method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the various steps of the speech processing method as described in any one of claims 1-6.

Images