Speech recognition method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a speech recognition method and device, electronic equipment and storage medium. The method comprises: obtaining a to-be-recognized speech signal; preprocessing the to-be-recognized speech signal to obtain a spectrogram feature of the to-be-recognized speech signal; processing the spectrogram feature based on a pre-set quality inspection rule to obtain a speech keyword and a confidence score of the speech keyword; and determining a risk recognition result of the to-be-recognized speech signal according to the confidence score and a confidence threshold. The embodiments of the present application can improve the recognition efficiency and accuracy of the risk speech.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a speech recognition method, apparatus, electronic device and storage medium. Background Technology

[0002] Enterprise voice quality inspection can be viewed as a type of voice recognition task. From a modeling perspective, it involves rapidly classifying voice content using a classification model. Voice quality inspection can effectively and quickly detect whether voice content contains illegal or sensitive words, thus determining the risk of leaking core corporate secrets and playing an indispensable role in protecting corporate interests.

[0003] Existing speech quality inspection tasks typically fall into two categories. One involves random sampling and manual identification. However, as companies grow, the amount of speech data increases significantly. Existing manual sampling methods suffer from inefficiency. The other approach uses speech recognition technology for speech quality inspection. This method relies on Hidden Markov Models (HMMs) to build acoustic and language models. For each model, engineers need to provide different types of labeled datasets and manually extract speech features as model input, resulting in lower accuracy. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of this application is to provide a speech recognition method, device, electronic device and storage medium to improve the efficiency and accuracy of risk speech recognition.

[0005] In a first aspect, embodiments of this application provide a speech recognition method, the method comprising:

[0006] Acquire the speech signal to be recognized;

[0007] The speech signal to be identified is preprocessed to obtain the spectrogram features of the speech signal to be identified;

[0008] The spectrogram features are processed based on pre-set quality inspection rules to obtain speech keywords and their confidence scores.

[0009] The risk identification result of the speech signal to be identified is determined based on the confidence score and confidence threshold.

[0010] Optionally, the preprocessing of the speech signal to be recognized to obtain the spectrogram features of the speech signal to be recognized includes:

[0011] The speech signal to be identified is denoised and enhanced based on a pre-trained signal detection model to obtain the processed speech signal.

[0012] The processed speech signal is preprocessed to obtain the spectrogram features.

[0013] Optionally, before performing denoising and enhancement processing on the speech signal to be recognized based on the pre-trained signal detection model to obtain the processed speech signal, the method further includes:

[0014] Obtain model training samples; the model training samples include positive samples and negative samples, the negative samples are speech signal samples with the risk of leakage, and the positive samples are speech signal samples without the risk of leakage;

[0015] The training samples of the model are input into the signal detection model to be trained; the signal detection model to be trained includes: an attention mechanism layer and a DRNN network layer;

[0016] The attention mechanism layer is invoked to extract the temporal information of each speech frame in the model training samples, and the temporal information is concatenated with the corresponding speech frame to generate concatenated speech information;

[0017] The DRNN network layer is invoked to denoise the spliced ​​speech information to obtain a predicted speech signal.

[0018] The loss value of the signal detection model to be trained is calculated based on the predicted speech signal and the processed speech signal corresponding to the model training sample.

[0019] If the loss value is within a preset range, the trained signal detection model is used as the final signal detection model.

[0020] Optionally, the attention mechanism layer includes: a speech frame splicing layer, a multilayer perceptron, a normalization layer, a temporal information acquisition layer, and a temporal information splicing layer.

[0021] The step of calling the attention mechanism layer to extract the temporal information of each speech frame in the model training samples, and concatenating the temporal information with the corresponding speech frame to generate concatenated speech information includes:

[0022] The speech frame splicing layer is invoked to splice each speech frame in the model training samples and the speech frames within the attention range to obtain spliced ​​speech frames;

[0023] The multilayer perceptron is invoked to process the spliced ​​speech frames to obtain an attention score for each speech frame;

[0024] The attention score is normalized by calling the normalization layer to obtain the attention weight for each speech frame;

[0025] The timing information acquisition layer is invoked to process the attention weights and each speech frame to obtain the timing information corresponding to each speech frame;

[0026] The timing information splicing layer is invoked to splice each speech frame and its corresponding timing information to obtain the spliced ​​speech information.

[0027] Optionally, the preprocessing of the processed speech signal to obtain the spectrogram features includes:

[0028] The processed speech signal is pre-emphasized to obtain a pre-emphasized speech signal;

[0029] The pre-emphasized speech signal is subjected to overlapping frame segmentation processing to obtain a framed speech signal;

[0030] The framed audio signal is windowed to obtain a windowed audio signal;

[0031] Perform a Fourier transform on the windowed speech signal to obtain the speech spectrum;

[0032] The logarithmic energy spectrum of each frame of speech signal is obtained based on the logarithm of the energy spectrum corresponding to the speech spectrum.

[0033] The logarithmic energy spectrum is rotated and concatenated into a feature vector matrix, and the spectrogram features corresponding to the processed speech signal are determined based on the feature vector matrix.

[0034] Optionally, the process of processing the spectrogram features based on pre-set quality control rules to obtain speech keywords and their confidence scores includes:

[0035] The keywords in the quality inspection rules are processed to obtain the keyword embedding vectors corresponding to the quality inspection rules.

[0036] Based on the feature vector matrix corresponding to the spectrogram features, the audio feature vector of the speech signal to be identified is determined;

[0037] Extract the high-level speech features from the audio feature vector;

[0038] The bidirectional long short-term memory network model is invoked to process the high-level speech features and the keyword embedding vector to obtain the attention score between the high-level speech features and the keyword embedding vector;

[0039] The high-level speech features and the keyword embedding vector are fused based on the attention score to obtain the fused feature vector of the speech keywords.

[0040] The confidence score of the speech keyword is determined based on the fused feature vector.

[0041] Optionally, the step of fusing the high-level speech features and the keyword embedding vector based on the attention score to obtain the fused feature vector of the speech keywords includes:

[0042] Based on the attention score, target speech high-level features are extracted from the speech high-level features;

[0043] The target speech high-level features are processed based on the target keyword embedding vector to obtain the initial weight vector corresponding to the target speech high-level features; the target keyword embedding vector is the keyword embedding vector that matches the target speech high-level features in the keyword embedding vector;

[0044] The initial weight vector is normalized to obtain the weight vector corresponding to the initial weight vector;

[0045] The target speech high-level features are weighted according to the weight vector to obtain the fusion feature vector of the speech keywords corresponding to the target speech high-level features.

[0046] Optionally, determining the risk identification result of the speech signal to be identified based on the confidence score and the confidence threshold includes:

[0047] If the confidence score is greater than or equal to the confidence threshold, a risk identification result indicating that the voice signal to be identified has a risk of leakage is obtained;

[0048] If the confidence score is less than the confidence threshold, a risk identification result indicating that the voice signal to be identified does not pose a risk of leakage is obtained.

[0049] Secondly, embodiments of this application provide a voice recognition device, the device comprising:

[0050] The speech signal acquisition module is used to acquire the speech signal to be recognized;

[0051] The spectrogram acquisition module is used to preprocess the speech signal to be recognized to obtain the spectrogram features of the speech signal to be recognized.

[0052] The confidence score acquisition module is used to process the spectrogram features based on pre-set quality inspection rules to obtain speech keywords and their confidence scores.

[0053] The recognition result determination module is used to determine the risk recognition result of the speech signal to be recognized based on the confidence score and the confidence threshold.

[0054] Optionally, the spectrogram acquisition module includes:

[0055] The speech signal processing unit is used to perform noise reduction and enhancement processing on the speech signal to be recognized based on a pre-trained signal detection model to obtain the processed speech signal.

[0056] The spectrogram acquisition unit is used to preprocess the processed speech signal to obtain the spectrogram features.

[0057] Optionally, the device further includes:

[0058] The training sample acquisition module is used to acquire model training samples; the model training samples include positive samples and negative samples, the negative samples are speech signal samples with the risk of leakage, and the positive samples are speech signal samples without the risk of leakage.

[0059] The training sample input module is used to input the model training samples into the signal detection model to be trained; the signal detection model to be trained includes: an attention mechanism layer and a DRNN network layer;

[0060] The concatenated speech generation module is used to call the attention mechanism layer to extract the temporal information of each speech frame in the model training samples, and concatenate the temporal information with the corresponding speech frame to generate concatenated speech information;

[0061] The predicted speech acquisition module is used to call the DRNN network layer to perform noise reduction processing on the spliced ​​speech information to obtain the predicted speech signal;

[0062] The loss value calculation module is used to calculate the loss value of the signal detection model to be trained based on the predicted speech signal and the processed speech signal corresponding to the model training sample.

[0063] The signal detection model acquisition module is used to use the trained signal detection model as the final signal detection model when the loss value is within a preset range.

[0064] Optionally, the attention mechanism layer includes: a speech frame splicing layer, a multilayer perceptron, a normalization layer, a temporal information acquisition layer, and a temporal information splicing layer.

[0065] The speech generation module includes:

[0066] The speech acquisition unit is used to call the speech frame splicing layer to splice each speech frame and the speech frame within the attention range in the model training sample to obtain a spliced ​​speech frame.

[0067] The attention score acquisition unit is used to call the multilayer perceptron to process the spliced ​​speech frames and obtain the attention score of each speech frame.

[0068] The attention weight acquisition unit is used to call the normalization layer to normalize the attention score and obtain the attention weight of each speech frame.

[0069] The timing information acquisition unit is used to call the timing information acquisition layer to process the attention weight and each speech frame to obtain the timing information corresponding to each speech frame;

[0070] The speech acquisition unit is used to call the timing information splicing layer to splice each speech frame and the corresponding timing information to obtain the spliced ​​speech information.

[0071] Optionally, the spectrogram acquisition module includes:

[0072] A pre-emphasis speech acquisition unit is used to pre-emphasize the processed speech signal to obtain a pre-emphasis speech signal.

[0073] The framed speech acquisition unit is used to perform overlapping framed processing on the pre-emphasized speech signal to obtain a framed speech signal.

[0074] A windowed speech acquisition unit is used to perform windowing processing on the framed speech signal to obtain a windowed speech signal;

[0075] The speech spectrum acquisition unit is used to perform Fourier transform on the windowed speech signal to obtain the speech spectrum;

[0076] The logarithmic energy spectrum acquisition unit is used to acquire the logarithmic energy spectrum of each frame of speech signal based on the logarithm of the energy spectrum corresponding to the speech spectrum;

[0077] The spectrogram determination unit is used to rotate and concatenate the logarithmic energy spectrum into a feature vector matrix, and determine the spectrogram features corresponding to the processed speech signal based on the feature vector matrix.

[0078] Optionally, the confidence score acquisition module includes:

[0079] An embedding vector acquisition unit is used to process the keywords in the quality inspection rules to obtain the keyword embedding vectors corresponding to the quality inspection rules.

[0080] An audio feature determination unit is used to determine the audio feature vector of the speech signal to be identified based on the feature vector matrix corresponding to the spectrogram features.

[0081] A high-level speech feature extraction unit is used to extract high-level speech features from the audio feature vector;

[0082] The attention score acquisition unit is used to call a bidirectional long short-term memory network model to process the high-level speech features and the keyword embedding vector to obtain the attention score between the high-level speech features and the keyword embedding vector.

[0083] The fusion feature vector acquisition unit is used to fuse the high-level speech features and the keyword embedding vector based on the attention score to obtain the fusion feature vector of the speech keywords.

[0084] The confidence score determination unit is used to determine the confidence score of the speech keyword based on the fused feature vector.

[0085] Optionally, the fusion feature vector acquisition unit includes:

[0086] The target high-level feature extraction subunit is used to extract target high-level speech features from the high-level speech features based on the attention score;

[0087] An initial weight vector acquisition subunit is used to process the target speech high-level features based on the target keyword embedding vector to obtain the initial weight vector corresponding to the target speech high-level features; the target keyword embedding vector is the keyword embedding vector that matches the target speech high-level features in the keyword embedding vector;

[0088] The weight vector acquisition sub-unit is used to normalize the initial weight vector to obtain the weight vector corresponding to the initial weight vector.

[0089] The feature vector fusion acquisition subunit is used to perform weighted processing on the high-level features of the target speech according to the weight vector to obtain the fusion feature vector of the speech keywords corresponding to the high-level features of the target speech.

[0090] Optionally, the identification result determination module includes:

[0091] The first recognition result acquisition unit is used to obtain a risk recognition result indicating that the voice signal to be recognized has a risk of leakage when the confidence score is greater than or equal to the confidence threshold.

[0092] The second recognition result acquisition unit is used to obtain a risk recognition result indicating that the voice signal to be recognized does not pose a risk of leakage when the confidence score is less than the confidence threshold.

[0093] Thirdly, embodiments of this application provide an electronic device, including:

[0094] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the speech recognition method described above.

[0095] Fourthly, embodiments of this application provide a computer-readable storage medium that, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform any of the above-described speech recognition methods.

[0096] Compared with the prior art, the embodiments of this application have the following advantages:

[0097] In this embodiment, the speech signal to be recognized is acquired, preprocessed to obtain its spectrogram features, and then subjected to speech keyword detection based on pre-set quality control rules to obtain confidence scores for the speech keywords. Based on the confidence scores and confidence thresholds, the risk identification result of the speech signal to be recognized is determined. This embodiment converts the speech signal to be recognized into spectrogram features and combines this with pre-set quality control rules to detect the features and identify the confidence scores of the speech keywords in the speech signal to determine the risk identification result, effectively improving the accuracy of risk speech recognition. Furthermore, no manual intervention is required during the speech recognition process, effectively improving the efficiency of risk speech recognition.

[0098] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0099] Figure 1 A flowchart illustrating the steps of a speech recognition method provided in this application embodiment;

[0100] Figure 2 A flowchart illustrating the steps of a spectrogram feature acquisition method provided in this application embodiment;

[0101] Figure 3 A flowchart illustrating the steps of a signal detection model acquisition method provided in this application embodiment;

[0102] Figure 4 A flowchart illustrating the steps of a method for acquiring spliced ​​voice information provided in this application embodiment;

[0103] Figure 5 A flowchart illustrating the steps of a spectrogram feature determination method provided in this application embodiment;

[0104] Figure 6 A flowchart illustrating the steps of a confidence score determination method provided in this application embodiment;

[0105] Figure 7 A flowchart illustrating the steps of a method for obtaining fused feature vectors provided in this application embodiment;

[0106] Figure 8 A flowchart illustrating the steps of a risk identification result acquisition method provided in this application embodiment;

[0107] Figure 9 This is a schematic diagram of the structure of an ADR model provided in an embodiment of this application;

[0108] Figure 10 This is a schematic diagram of the structure of a speech preprocessing module provided in an embodiment of this application;

[0109] Figure 11 This is a schematic diagram of the structure of a speech keyword detection model provided in an embodiment of this application;

[0110] Figure 12 This is a schematic diagram of the structure of a speech recognition device provided in an embodiment of this application;

[0111] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0112] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0113] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0114] Reference Figure 1 The diagram illustrates a flowchart of a speech recognition method provided in an embodiment of this application. Figure 1 As shown, the speech recognition method may include steps 101, 102, 103 and 104.

[0115] Step 101: Acquire the speech signal to be recognized.

[0116] The embodiments of this application can be applied to scenarios where leakage risk identification is performed on the voice signal to be identified in conjunction with set quality inspection rules.

[0117] The voice signal to be identified refers to the voice signal used for identifying the risk of information leakage. In specific implementations, the voice signal to be identified can be internal call information within an enterprise, such as call information when employees answer calls via company phones. For example, calls on telephones installed within the enterprise can be monitored and recorded to obtain the voice messages of employees in real time, and then the obtained voice messages can be regarded as the voice signal to be identified.

[0118] In practical implementation, other methods can also be used to acquire the voice signal to be recognized, such as monitoring equipment (e.g., surveillance cameras). Specifically, the method for acquiring the voice signal to be recognized can be determined according to business needs, and this embodiment does not impose any restrictions on it.

[0119] It is understood that the above examples are merely examples listed to better understand the technical solutions of the embodiments of this application, and are not intended to be the only limitation on the embodiments.

[0120] After acquiring the speech signal to be recognized, proceed to step 102.

[0121] Step 102: Preprocess the speech signal to be recognized to obtain the spectrogram features of the speech signal to be recognized.

[0122] A spectrogram, also known as a speech spectrum diagram, is a spectrum obtained by processing the received time-domain signal. In practice, the horizontal axis of the spectrogram represents time, the vertical axis represents frequency, and the values ​​at each coordinate point represent the energy of the speech data.

[0123] After acquiring the speech signal to be recognized, denoising and enhancement processes can be performed to obtain a clean speech signal. Then, the clean speech signal is preprocessed to obtain spectrogram features. The process of obtaining the clean speech signal can be combined with... Figure 2 The following is a detailed description.

[0124] Reference Figure 2 The diagram illustrates a flowchart of the steps in a spectrogram feature acquisition method provided in an embodiment of this application. Figure 2 As shown, the spectrogram feature acquisition method may include steps 201 and 202.

[0125] Step 201: Based on the pre-trained signal detection model, the speech signal to be identified is denoised and enhanced to obtain the processed speech signal.

[0126] In this embodiment, the signal detection model is the ADR model, and the structure of the ADR model can be as follows: Figure 9 As shown. This ADR model can be an innovative network model based on DRNN. The training process of the ADR model can be combined with... Figure 3 The following is a detailed description.

[0127] Reference Figure 3 The flowchart illustrates the steps of a signal detection model acquisition method provided in an embodiment of this application, as follows: Figure 3 As shown, the signal detection model acquisition method may include steps 301, 302, 303, 304, 305 and 306.

[0128] Step 301: Obtain model training samples; the model training samples include positive samples and negative samples, the negative samples are speech signal samples with the risk of leakage, and the positive samples are speech signal samples without the risk of leakage.

[0129] In this embodiment of the application, model training samples refer to speech signal samples used to train the signal detection model. In this example, model training samples may include positive samples and negative samples, wherein positive samples are speech signal samples that do not pose a risk of leakage, and negative samples are speech signal samples that pose a risk of leakage.

[0130] In some examples, the model training samples can be voice signals based on the company's internal historical records. For example, two databases can be set up in advance within the company. One database can record voice signals that do not pose a risk of leakage, and the other database can record voice signals that do pose a risk of leakage. When obtaining model training samples, a first preset number of voice signals that pose a risk of leakage can be obtained from one database as negative samples, and a second preset number of voice signals that do not pose a risk of leakage can be obtained from the other database as positive samples, and so on.

[0131] In some examples, the training samples for the model can be speech signals obtained from the Internet. For example, one can search for speech signals that pose a risk of leakage and speech signals that do not pose a risk of leakage on the Internet, and download the corresponding speech texts as negative and positive samples, respectively.

[0132] Of course, this is not the only option. In specific implementations, other methods can be used to obtain model training samples. The method for obtaining model training samples can be determined according to business needs, and this embodiment does not impose any restrictions on it.

[0133] It is understood that the above examples are merely examples listed to better understand the technical solutions of the embodiments of this application, and are not intended to be the only limitation on the embodiments.

[0134] After obtaining the model training samples, proceed to step 302.

[0135] Step 302: Input the training samples of the model into the signal detection model to be trained; the signal detection model to be trained includes: an attention mechanism layer and a DRNN network layer.

[0136] The signal detection model to be trained refers to the constructed, untrained signal detection model used for speech signal denoising and enhancement. In this example, the signal detection model to be trained may include an attention mechanism layer and a DRNN network layer.

[0137] After obtaining the model training samples, the model training samples can be input into the signal detection model to be trained.

[0138] After inputting the model training samples into the signal detection model to be trained, step 303 is executed.

[0139] Step 303: Call the attention mechanism layer to extract the temporal information of each speech frame in the model training samples, and concatenate the temporal information with the corresponding speech frame to generate concatenated speech information.

[0140] After inputting the model training samples into the signal detection model to be trained, the attention mechanism layer can be invoked to extract the temporal signal of each speech frame in the model training samples, and the temporal information can be concatenated with the corresponding speech frame to generate concatenated speech information. In this example, the attention mechanism layer may also include: a speech frame concatenation layer, a multilayer perceptron, a normalization layer, a temporal information acquisition layer, and a temporal information concatenation layer. By processing each speech frame in the model training samples through the speech frame concatenation layer, multilayer perceptron, normalization layer, temporal information acquisition layer, and temporal information concatenation layer, the concatenated speech information can be output. This implementation process can be combined with... Figure 4 The following is a detailed description.

[0141] Reference Figure 4 The flowchart illustrates the steps of a method for obtaining spliced ​​voice information according to an embodiment of this application. Figure 4 As shown, the method for obtaining spliced ​​voice information may include steps 401, 402, 403, 404 and 405.

[0142] Step 401: Call the speech frame splicing layer to splice each speech frame and the speech frames within the attention range in the model training samples to obtain spliced ​​speech frames.

[0143] In this embodiment, the attention mechanism layer may include: a speech frame splicing layer, a multilayer perceptron, a normalization layer, a temporal information acquisition layer, and a temporal information splicing layer.

[0144] After inputting the model training samples into the signal detection model to be trained, the speech frame splicing layer in the attention mechanism layer can be called to splice each speech frame in the model training samples and the speech frames within the attention range to obtain spliced ​​speech frames.

[0145] In the specific implementation, the model training samples are speech signal samples, and each speech signal sample contains multiple speech frames. The attention range refers to the range of speech frames in the speech signal sample. The specific numerical range of the attention range can be determined according to business requirements, and this embodiment does not impose any restrictions on it.

[0146] In practical applications, the current frame in the speech signal sample can be concatenated with all frames within a certain range. This range can be called the attention range, denoted as T. The attention range covers T / 2 frames before and T / 2 frames after the current frame.

[0147] After calling the speech frame splicing layer to splice each speech frame in the model training samples and the speech frames within the attention range to obtain spliced ​​speech frames, step 402 is executed.

[0148] Step 402: Invoke the multilayer perceptron to process the spliced ​​speech frames and obtain the attention score for each speech frame.

[0149] After the speech frame concatenation layer is called to concatenate each speech frame in the model training samples and the speech frames within the attention range to obtain the concatenated speech frame, the multilayer perceptron can be called to process the concatenated speech frame to obtain the attention score of each speech frame. Specifically, the calculation method is shown in the following formula (1):

[0150] e tj =G(x) t ,x j (1)

[0151] In the above formula (1), e tj For attention scoring, G() represents the computation of the multilayer perceptron, x t Indicates the current frame, x j This refers to all frames within the attention span, i.e., the frames before T / 2 and the frames after T / 2.

[0152] After calling the multilayer perceptron to process the spliced ​​speech frames and obtain the attention score for each speech frame, step 403 is executed.

[0153] Step 403: Call the normalization layer to normalize the attention score to obtain the attention weight of each speech frame.

[0154] After processing the concatenated speech frames using a multilayer perceptron to obtain the attention score for each frame, a normalization layer can be called to normalize the attention scores, thus obtaining the attention weight for each speech frame. Specifically, each attention score can be normalized to a pre-defined interval to obtain the attention weight corresponding to each speech frame. In the specific implementation, after obtaining all scores e... tjAfter that, all ratings can be e tj Normalization yields the attention weight α. tj It can be shown in the following formula (2):

[0155]

[0156] In the above formula (2), α tj Represents attention weight, e tk This indicates that attention scores are selected sequentially within the range of 1-T (T being the attention range), and exp() represents an exponential function with the natural constant e as the base.

[0157] After calling the normalization layer to normalize the attention score and obtain the attention weight for each speech frame, step 404 is executed.

[0158] Step 404: Call the timing information acquisition layer to process the attention weights and each speech frame to obtain the timing information corresponding to each speech frame.

[0159] After normalizing the attention scores using the normalization layer to obtain the attention weights for each speech frame, the temporal information acquisition layer can be called to process the attention weights and each speech frame to obtain the temporal information corresponding to each speech frame.

[0160] In the specific implementation, the previously obtained attention weights can be used as coefficients for the corresponding frames, and the summation can be performed to obtain the temporal information c. t The calculation method can be shown in the following formula (3):

[0161]

[0162] After the timing information acquisition layer is called to process the attention weights and each speech frame to obtain the timing information corresponding to each speech frame, step 405 is executed.

[0163] Step 405: Call the timing information splicing layer to splice each speech frame and its corresponding timing information to obtain the spliced ​​speech information.

[0164] After the timing information acquisition layer processes the attention weights and each speech frame to obtain the timing information corresponding to each speech frame, the timing information splicing layer can be called to splice each speech frame and the timing information corresponding to each speech frame, thereby obtaining the spliced ​​speech information.

[0165] In the specific implementation, after obtaining the timing information c t Then, the timing information c can be... tThe data is concatenated with the corresponding speech frame and then input into the DRNN network layer. During model training, the parameters in the multilayer perceptron can be optimized together with the parameters in the DRNN network layer to better enhance the speech.

[0166] After calling the attention mechanism layer to extract the temporal information of each speech frame in the model training samples, and concatenating the temporal information with the corresponding speech frame to generate concatenated speech information, step 304 is executed.

[0167] Step 304: Call the DRNN network layer to denoise the spliced ​​speech information to obtain the predicted speech signal.

[0168] The attention mechanism layer is called to extract the temporal information of each speech frame in the model training samples. After concatenating the temporal information with the corresponding speech frame to generate concatenated speech information, the DRNN network layer can be called to denoise the concatenated speech information, thereby obtaining the predicted speech signal.

[0169] In its implementation, the DRNN network model mainly consists of recurrent layers and fully connected layers. To improve the model's expressive power, multiple layers are stacked together. In the initial recurrent layer, at each time step t, the t-th frame x of the noisy amplitude spectrum is processed. t and the output h of the previous time step t-1 Input is given to the neuron, and h is output. t The neurons in the recurrent layer are transmitted to the next layer and the next time step. The computational formula for neurons in the recurrent layer is as follows:

[0170] h t =f(Wx t +Uh t-1 +b) (4)

[0171] In the above formula (4), h t h represents the output of the recurrent layer neurons. t-1 x represents the output of the previous time step. t Let f() represent the t-th frame of the noisy amplitude spectrum, f() is the activation function tanh, W and U are model parameters, and b is the bias term.

[0172] Following the recurrent layer are multiple feedforward layers, denoted as L. The depth of the model can be set by adjusting the value of L. When L is 0, there is only one recurrent layer and no fully connected layers. The formula for calculating the i-th fully connected layer at time step t is as follows:

[0173] h t (i) =f(W (i) h t (i-1) +b (i) (5)

[0174] In the above formula (5), h t (i-1) It is the output of the (i-1)th fully connected layer at time step t. When i equals 1, it is the output of the recurrent layer, and W... (i) and b (i) These are the parameters and biases of the feedforward layer.

[0175] Step 305: Calculate the loss value of the signal detection model to be trained based on the predicted speech signal and the processed speech signal corresponding to the model training sample.

[0176] Speech signal processing refers to the speech signal obtained after enhancing a speech signal sample, i.e., the real speech signal.

[0177] After obtaining the predicted speech signal corresponding to the model training sample, the loss value of the signal detection model to be trained can be calculated based on the predicted speech signal and the processed speech signal corresponding to the model training sample.

[0178] After calculating the loss value of the detection model for the signal to be trained based on the predicted speech signal and the processed speech signal corresponding to the model training sample, step 306 is executed.

[0179] Step 306: If the loss value is within a preset range, the trained signal detection model is used as the final signal detection model.

[0180] The preset range refers to a pre-defined range of loss values ​​used to determine the training level of the signal detection model. The specific values ​​of the preset range can be determined according to business needs, and this embodiment does not impose any restrictions on them.

[0181] After calculating the loss value of the detection model for the signal to be trained based on the predicted speech signal and the processed speech signal corresponding to the model training sample, it can be determined whether the loss value is within the preset range.

[0182] If the loss value is not within the preset range, the training of the signal detection model can continue until the signal detection model converges, that is, the calculated loss value of the signal detection model is within the preset range.

[0183] If the loss value is within a preset range, it indicates that the signal detection model to be trained has converged. At this point, the trained signal detection model can be used as the final signal detection model.

[0184] After training the signal detection model, the speech signal to be recognized can be denoised and enhanced based on the trained signal detection model to obtain the processed speech signal (i.e., clean speech signal). The specific processing process is similar to the model training process, and will not be described in detail here.

[0185] After the pre-trained signal detection model is used to denoise and enhance the speech signal to be recognized to obtain the processed speech signal, step 202 is executed.

[0186] Step 202: Preprocess the processed speech signal to obtain the spectrogram features.

[0187] After denoising and enhancing the speech signal to be recognized using a pre-trained signal detection model, the processed speech signal can be further preprocessed to obtain spectrogram features. Specifically, pre-emphasis, framing, windowing, and Fourier transform can be applied to the processed speech signal to obtain spectrogram features. This implementation process can be combined with… Figure 5 The following is a detailed description.

[0188] Reference Figure 5 The diagram illustrates a flowchart of the steps in a spectrogram feature determination method provided in an embodiment of this application. Figure 5 As shown, the spectrogram feature determination method may include steps 501, 502, 503, 504, 505 and 506.

[0189] Step 501: Perform pre-emphasis processing on the processed speech signal to obtain a pre-emphasized speech signal.

[0190] In this embodiment, pre-emphasis is a signal processing method that compensates for the high-frequency components of the input signal at the transmitting end. As the signal rate increases, the signal is significantly attenuated during transmission. To obtain a better signal waveform at the receiving end, it is necessary to compensate for the damaged signal. The idea behind pre-emphasis is to enhance the high-frequency components of the signal at the beginning of the transmission line to compensate for the excessive attenuation of high-frequency components during transmission. Pre-emphasis has no effect on noise, thus effectively improving the output signal-to-noise ratio.

[0191] After obtaining the processed speech signal, pre-emphasis processing can be performed on the processed speech signal to obtain a pre-emphasized speech signal.

[0192] In practical implementation, a first-order high-pass filter can be used to perform pre-emphasis processing, and the calculation formula is as follows:

[0193] H(z) = 1 - μz -1 ,0.9≤μ≤1.0 (6)

[0194] In the above formula (6), H(z) represents the signal after pre-emphasis processing, μ represents the pre-emphasis coefficient, and z -1 It represents the reciprocal of the original signal.

[0195] After pre-emphasizing the processed speech signal to obtain the pre-emphasized speech signal, step 502 is executed.

[0196] Step 502: Perform overlapping frame processing on the pre-emphasized speech signal to obtain a framed speech signal.

[0197] Overlapping framing refers to the framing of a speech signal in which two adjacent speech frames overlap.

[0198] After pre-emphasizing the processed speech signal to obtain the pre-emphasized speech signal, the pre-emphasized speech signal can be subjected to overlapping frame segmentation to obtain the framed speech signal. In specific implementation, when overlapping and segmenting the processed speech signal, each frame can be 25ms long, and the frame shift can be 10ms, resulting in an overlap of 15ms. At this point, a single frame of speech signal has a sufficiently long period without drastic changes, making it suitable for subsequent Fourier transform analysis.

[0199] After performing overlapping framing processing on the pre-emphasized speech signal to obtain the framed speech signal, step 503 is executed.

[0200] Step 503: Window the framed speech signal to obtain a windowed speech signal.

[0201] Windowing: In signal processing, windowing is an essential process. Typically, computers can only process signals of finite length. Therefore, the original signal X(t) must be truncated by T (sampling time), i.e., finiteened, to become XT(t) before further processing. This process is called windowing.

[0202] After obtaining framed speech signals by overlapping and framing the pre-emphasized speech signal, windowing can be applied to these framed speech signals to obtain windowed speech signals. In the specific implementation, windowing is performed on the framed speech signals by multiplying each frame of speech by a smooth window function (here, a Hamming window is chosen). The signals at both ends of the frame can smoothly decay to zero, and the non-periodic speech signal exhibits some characteristics of a periodic function. This operation can reduce the intensity of the sidelobes after the Fourier transform (the main lobe is the peak with the largest amplitude after the spectrum is transformed, and the sidelobes are the smaller peaks around it), obtain a higher quality spectrum, and also make the global speech more continuous, avoiding the Gibbs effect.

[0203] After windowing the framed audio signal to obtain the windowed audio signal, step 504 is executed.

[0204] Step 504: Perform Fourier transform on the windowed speech signal to obtain the speech spectrum.

[0205] The Fourier transform can represent a function that satisfies certain conditions as a linear combination of trigonometric functions (sine and / or cosine functions) or their integrals. In different research fields, the Fourier transform has various variants, such as the continuous Fourier transform and the discrete Fourier transform.

[0206] After windowing the framed speech signal to obtain the windowed speech signal, a Fourier transform can be performed on the windowed speech signal to obtain the speech spectrum.

[0207] After performing a Fourier transform on the windowed speech signal to obtain the speech spectrum, step 505 is executed.

[0208] Step 505: Obtain the logarithmic energy spectrum of each frame of speech signal based on the logarithm of the energy spectrum corresponding to the speech spectrum.

[0209] After performing a Fourier transform on the windowed speech signal to obtain the speech spectrum, the logarithm of the energy spectrum can be obtained to get the logarithmic energy spectrum of each frame of the speech signal.

[0210] After obtaining the logarithmic energy spectrum of each frame of speech signal based on the logarithm of the energy pavement corresponding to the speech spectrum, step 506 is executed.

[0211] Step 506: Rotate and concatenate the logarithmic energy spectrum into a feature vector matrix, and determine the spectrogram features corresponding to the processed speech signal based on the feature vector matrix.

[0212] After obtaining the logarithmic energy spectrum of each frame of the speech signal based on the logarithm of the energy spectrum, the logarithmic energy spectrum can be rotated and concatenated into a feature vector matrix. The spectrogram features corresponding to the processed speech signal can then be determined based on this feature vector matrix. In a specific implementation, a short-time Fourier transform can be performed on each windowed frame of the signal to obtain the spectrum. Then, the square of the modulus of the spectrum is taken to obtain the energy spectrum, and the logarithm is taken to obtain the logarithmic energy spectrum of each frame. Finally, the energy spectra corresponding to each frame of the speech signal are rotated and concatenated into a feature vector matrix to obtain the spectrogram features.

[0213] The implementation process can be Figure 10As shown, after inputting speech, the input speech can be pre-emphasized, framed, and windowed. Then, a fast Fourier transform is performed, followed by linear amplitude spectrum and logarithmic amplitude spectrum processing to obtain feature vectors. Finally, the feature vectors are concatenated to output the final feature vector. The spectrogram features can be obtained from the feature vector matrix formed by the feature vectors of multiple frames.

[0214] After preprocessing the speech signal to be recognized to obtain spectrogram features, step 103 is executed.

[0215] Step 103: Process the spectrogram features based on pre-set quality inspection rules to obtain speech keywords and their confidence scores.

[0216] Quality inspection rules refer to pre-defined rules used to detect voice messages that pose a risk of leakage. In this example, the quality inspection rules define rules such as keyword features that pose a risk of leakage. The specific limitations of the quality inspection rules can be determined according to business needs, and this embodiment does not impose any restrictions on them.

[0217] Confidence scores can be used to indicate the probability that a voice signal poses a risk of leakage. In this example, a higher confidence score indicates a lower probability of leakage, and vice versa.

[0218] After preprocessing the speech signal to be recognized to obtain spectrogram features, these features can be further processed based on pre-defined quality control rules to obtain speech keywords and their confidence scores. In a specific implementation, a bidirectional long short-term memory network model can be used to output the high-level speech features and keywords corresponding to the speech signal to be recognized, followed by the identification of risk keywords and the calculation of their confidence scores. This implementation process can be combined with... Figure 6 The following is a detailed description.

[0219] Reference Figure 6 The flowchart illustrates the steps of a confidence score determination method provided in an embodiment of this application, as shown below. Figure 6 As shown, the confidence score determination method may include steps 601, 602, 603, 604, 605, and 606.

[0220] Step 601: Process the keywords in the quality inspection rules to obtain the keyword embedding vector corresponding to the quality inspection rules.

[0221] In this embodiment, the voice keyword detection module can be used to detect and determine whether there are sensitive, illegal, or other leak-related keywords in the voice information, such as... Figure 11As shown, it mainly consists of a keyword embedding module, an acoustic module, an attention mechanism module, and a discriminator module. Among them,

[0222] The keyword embedding module can be used to obtain keyword embedding vectors. Specifically, it can vectorize the keywords in the quality inspection rules into N keywords k1, k2, ... k. N After embedding through the embedding matrix E, the corresponding embedding vectors e1, e2, ... e are obtained. N To ensure that the keyword embedding vector and the audio feature vector are in the same modal space for better attention operations, a linear transformation q = g(We) is used to process the embedding vector, where W is the linear layer and g() is the Leaky ReLU function.

[0223] The acoustic module is used to extract high-level features from speech features. It uses a bidirectional long short-term memory network (BiLSTM) and a linear layer to perform a linear transformation on the hidden layer output of the BiLSTM so that the final output speech high-level features and keyword embedding vectors are in the same embedding space.

[0224] The attention mechanism module uses an attention mechanism to obtain an attention score between the keyword embedding vector and the high-level speech feature vector. This score serves as an indicator of the correlation between the two, thereby extracting the important information needed for the current keyword recognition task from the high-level speech features, and then performing keyword recognition. Specifically: for the keyword embedding vector q... i i∈(1,2,...,N) and high-level speech features V=v1,v2,...,v T′ , use q i The inner product of each eigenvector in V yields the weight vector d = d1, d2, ..., dn. T′ The calculation formula is as follows:

[0225] d i′ =dot(v i′ ,q i ),i′∈(1,2...,T′) (7)

[0226] Where: d i′ Represents the keyword embedding vector q i With each high-level feature v i′ The correlation magnitude, dot() represents the inner product function, i′ represents the number of values ​​selected sequentially from 1 to T′, T′ represents the number of high-level speech features, d i′ After applying the softmax function, the corresponding weight vector s = s1, s2, ..., s is obtained. T′ The calculation formula is as follows:

[0227]

[0228] Where: s i′ Let d represent the weight vector. j This indicates that weight vectors within the range of 1 to T′ are selected sequentially;

[0229] Then, each weight in s is multiplied by its corresponding high-level feature v to obtain the weighted C = c1, c2, ..., c i′ ,...,c T′ , where c i′ =s i′ v i′ ,i′∈(1,2,...,T′),C represents the weighted high-level features.

[0230] Finally, the vector c in C will be... i′ Adding the vectors i′∈(1,2,...,T′) together yields a vector a that combines the keyword embedding vector and the high-level feature vector, i.e.:

[0231]

[0232] The discriminator module can identify keywords based on vector 'a'. It consists of two linear layers followed by a sigmoid activation function, where the activation function in the linear layers is Leaky ReLU. The final output is a confidence score in the range [0,1] to indicate whether the keyword is contained in the speech signal.

[0233] The detection process can be described in detail below using the following steps.

[0234] In this embodiment, after obtaining the quality inspection rules, the keywords in the quality inspection rules can be processed to obtain the keyword embedding vector corresponding to the quality inspection rules.

[0235] After processing the keywords in the quality inspection rules to obtain the keyword embedding vectors corresponding to the quality inspection rules, step 602 is executed.

[0236] Step 602: Determine the audio feature vector of the speech signal to be identified based on the feature vector matrix corresponding to the spectrogram features.

[0237] After obtaining the spectrogram features corresponding to the speech signal to be recognized, the feature vector matrix corresponding to the spectrogram features can be obtained, and the audio feature vector of the speech signal to be recognized can be determined based on the feature vector matrix.

[0238] After determining the audio feature vector of the speech signal to be recognized based on the feature vector matrix corresponding to the spectrogram features, step 603 is executed.

[0239] Step 603: Extract the high-level speech features from the audio feature vector.

[0240] After determining the audio feature vector of the speech signal to be identified based on the feature vector matrix corresponding to the spectrogram features, the high-level speech features in the audio feature vector can be extracted.

[0241] After extracting the high-level speech features from the audio feature vector, step 604 is executed.

[0242] Step 604: Call the bidirectional long short-term memory network model to process the high-level speech features and the keyword embedding vector to obtain the attention score between the high-level speech features and the keyword embedding vector.

[0243] Attention scores can be used to indicate the degree of association between high-level speech features and keyword embedding vectors. In this example, a higher attention score indicates a stronger association between the high-level speech features and keyword embedding vectors, and vice versa.

[0244] After extracting the high-level speech features from the audio feature vector, a bidirectional long short-term memory network model can be called to process the high-level speech features and keyword embedding vectors to obtain the attention score between the high-level speech features and keyword embedding vectors.

[0245] After calling the bidirectional long short-term memory network model to process the high-level speech features and keyword embedding vectors, and obtaining the attention score between the high-level speech features and keyword embedding vectors, step 605 is executed.

[0246] Step 605: Based on the attention score, fuse the high-level speech features and the keyword embedding vector to obtain the fused feature vector of the speech keywords.

[0247] After processing the high-level speech features and keyword embedding vectors using a bidirectional long short-term memory network model to obtain an attention score between them, the high-level speech features and keyword embedding vectors can be fused based on the attention score to obtain a fused feature vector of speech keywords. Specifically, target high-level speech features can be extracted from the high-level speech features based on the attention score. Then, a weight vector is calculated by combining the target high-level speech features and the corresponding target keyword embedding vector, and the high-level speech features and keyword embedding vectors are fused based on the weight vector to obtain the fused feature vector. This implementation process can be combined with... Figure 7 The following is a detailed description.

[0248] Reference Figure 7 The flowchart illustrates the steps of a method for obtaining fused feature vectors according to an embodiment of this application. Figure 7 As shown, the method for obtaining the fused feature vector may include steps 701, 702, 703 and 704.

[0249] Step 701: Extract the target speech high-level features from the speech high-level features based on the attention score.

[0250] In this embodiment, the target speech high-level feature refers to the speech high-level feature whose corresponding keyword has a high degree of risk of leakage.

[0251] After obtaining the attention score, the target high-level speech features can be extracted from the high-level speech features based on the attention score. In a specific implementation, an attention score threshold can be preset. Then, high-level speech features with attention scores greater than or equal to the attention score threshold can be obtained from the high-level speech features and used as the target high-level speech features.

[0252] After extracting the target speech high-level features from the speech high-level features based on the attention score, step 702 is performed.

[0253] Step 702: Process the target speech high-level features according to the target keyword embedding vector to obtain the initial weight vector corresponding to the target speech high-level features; the target keyword embedding vector is the keyword embedding vector that matches the target speech high-level features in the keyword embedding vector.

[0254] After extracting the target speech high-level features from the speech high-level features based on attention scores, the target keyword embedding vector matching the target speech high-level features can be obtained. Then, the target speech high-level features can be processed according to the target keyword embedding vector to obtain the initial weight vector corresponding to the target speech high-level features.

[0255] After obtaining the initial weight vector, proceed to step 703.

[0256] Step 703: Normalize the initial weight vector to obtain the weight vector corresponding to the initial weight vector.

[0257] After obtaining the initial weight vector, the initial weight vector can be normalized to obtain the weight vector corresponding to the initial weight vector. That is, the initial weight vector is normalized to a certain interval to obtain the weight vector.

[0258] After normalizing the initial weight vector to obtain the corresponding weight vector, step 704 is executed.

[0259] Step 704: Perform weighted processing on the high-level features of the target speech according to the weight vector to obtain the fusion feature vector of the speech keywords corresponding to the high-level features of the target speech.

[0260] After normalizing the initial weight vector to obtain the corresponding weight vector, the target speech high-level features can be weighted according to the weight vector to obtain the fusion feature vector of the speech keywords corresponding to the target speech high-level features.

[0261] After fusing the high-level speech features and keyword embedding vectors based on attention scores to obtain the fused feature vector of speech keywords, step 606 is executed.

[0262] Step 606: Determine the confidence score of the speech keyword based on the fused feature vector.

[0263] After fusing the high-level speech features and keyword embedding vectors based on attention scores to obtain the fused feature vector of the speech keywords, the confidence score of the speech keywords can be determined based on the fused feature vector.

[0264] After processing the spectrogram features based on pre-set quality control rules to obtain speech keywords and their confidence scores, step 104 is executed.

[0265] Step 104: Determine the risk identification result of the speech signal to be identified based on the confidence score and confidence threshold.

[0266] The confidence threshold refers to a pre-set confidence score used to determine whether a voice signal has a risk of leakage. The specific value of the confidence threshold can be determined according to business needs, and this embodiment does not impose any restrictions on it.

[0267] After processing the spectrogram features based on pre-defined quality control rules to obtain speech keywords and their confidence scores, the risk identification result of the speech signal to be identified can be determined based on the confidence scores and confidence thresholds. This implementation process can be combined with... Figure 8 The following is a detailed description.

[0268] Reference Figure 8 The flowchart illustrates the steps of a risk identification result acquisition method provided in an embodiment of this application, as shown below. Figure 8 As shown, the method for obtaining the risk identification result may include steps 801 and 802.

[0269] Step 801: If the confidence score is greater than or equal to the confidence threshold, a risk identification result indicating that the voice signal to be identified has a risk of leakage is obtained.

[0270] Step 802: If the confidence score is less than the confidence threshold, a risk identification result indicating that the voice signal to be identified does not pose a risk of leakage is obtained.

[0271] In this embodiment, after obtaining the speech keywords and their confidence scores in the speech signal to be recognized, the relationship between the confidence scores and the confidence threshold can be compared.

[0272] When the confidence score is greater than or equal to the confidence threshold, a risk identification result indicating that the voice signal to be identified has a risk of leakage can be obtained.

[0273] When the confidence score is less than the confidence threshold, a risk identification result indicating that the voice signal to be identified does not pose a risk of leakage can be obtained.

[0274] In this embodiment, after obtaining the risk identification result indicating that the voice signal to be identified has the risk of leakage, the voice signal to be identified can be recorded in the manual review queue. The voice content is manually reviewed to determine whether the voice signal to be identified is a voice signal that leaks secrets.

[0275] The speech recognition method provided in this application acquires a speech signal to be recognized, preprocesses the speech signal to obtain spectrogram features, performs speech keyword detection processing on the spectrogram features based on pre-set quality inspection rules, obtains the confidence score of the speech keywords, and determines the risk recognition result of the speech signal to be recognized based on the confidence score and confidence threshold. This application embodiment converts the speech signal to be recognized into spectrogram features and combines pre-set quality inspection rules to detect the spectrogram features, identifying the confidence score of the speech keywords in the speech signal to be recognized for risk recognition result determination, which can effectively improve the accuracy of risk speech recognition. At the same time, no human intervention is required in the speech recognition process, which can effectively improve the efficiency of risk speech recognition.

[0276] Reference Figure 12 The diagram shows a structural schematic of a voice recognition device provided in an embodiment of this application, such as... Figure 12 As shown, the voice recognition device 1200 may include the following modules:

[0277] The speech signal acquisition module 1210 is used to acquire the speech signal to be recognized.

[0278] The spectrogram acquisition module 1220 is used to preprocess the speech signal to be recognized to obtain the spectrogram features of the speech signal to be recognized.

[0279] The confidence score acquisition module 1230 is used to process the spectrogram features based on pre-set quality inspection rules to obtain speech keywords and their confidence scores.

[0280] The recognition result determination module 1240 is used to determine the risk recognition result of the speech signal to be recognized based on the confidence score and the confidence threshold.

[0281] Optionally, the spectrogram acquisition module includes:

[0282] The speech signal processing unit is used to perform noise reduction and enhancement processing on the speech signal to be recognized based on a pre-trained signal detection model to obtain the processed speech signal.

[0283] The spectrogram acquisition unit is used to preprocess the processed speech signal to obtain the spectrogram features.

[0284] Optionally, the device further includes:

[0285] The training sample acquisition module is used to acquire model training samples; the model training samples include positive samples and negative samples, the negative samples are speech signal samples with the risk of leakage, and the positive samples are speech signal samples without the risk of leakage.

[0286] The training sample input module is used to input the model training samples into the signal detection model to be trained; the signal detection model to be trained includes: an attention mechanism layer and a DRNN network layer;

[0287] The concatenated speech generation module is used to call the attention mechanism layer to extract the temporal information of each speech frame in the model training samples, and concatenate the temporal information with the corresponding speech frame to generate concatenated speech information;

[0288] The predicted speech acquisition module is used to call the DRNN network layer to perform noise reduction processing on the spliced ​​speech information to obtain the predicted speech signal;

[0289] The loss value calculation module is used to calculate the loss value of the signal detection model to be trained based on the predicted speech signal and the processed speech signal corresponding to the model training sample.

[0290] The signal detection model acquisition module is used to use the trained signal detection model as the final signal detection model when the loss value is within a preset range.

[0291] Optionally, the attention mechanism layer includes: a speech frame splicing layer, a multilayer perceptron, a normalization layer, a temporal information acquisition layer, and a temporal information splicing layer.

[0292] The speech generation module includes:

[0293] The speech acquisition unit is used to call the speech frame splicing layer to splice each speech frame and the speech frame within the attention range in the model training sample to obtain a spliced ​​speech frame.

[0294] The attention score acquisition unit is used to call the multilayer perceptron to process the spliced ​​speech frames and obtain the attention score of each speech frame.

[0295] The attention weight acquisition unit is used to call the normalization layer to normalize the attention score and obtain the attention weight of each speech frame.

[0296] The timing information acquisition unit is used to call the timing information acquisition layer to process the attention weight and each speech frame to obtain the timing information corresponding to each speech frame;

[0297] The speech acquisition unit is used to call the timing information splicing layer to splice each speech frame and the corresponding timing information to obtain the spliced ​​speech information.

[0298] Optionally, the spectrogram acquisition module includes:

[0299] A pre-emphasis speech acquisition unit is used to pre-emphasize the processed speech signal to obtain a pre-emphasis speech signal.

[0300] The framed speech acquisition unit is used to perform overlapping framed processing on the pre-emphasized speech signal to obtain a framed speech signal.

[0301] A windowed speech acquisition unit is used to perform windowing processing on the framed speech signal to obtain a windowed speech signal;

[0302] The speech spectrum acquisition unit is used to perform Fourier transform on the windowed speech signal to obtain the speech spectrum;

[0303] The logarithmic energy spectrum acquisition unit is used to acquire the logarithmic energy spectrum of each frame of speech signal based on the logarithm of the energy spectrum corresponding to the speech spectrum;

[0304] The spectrogram determination unit is used to rotate and concatenate the logarithmic energy spectrum into a feature vector matrix, and determine the spectrogram features corresponding to the processed speech signal based on the feature vector matrix.

[0305] Optionally, the confidence score acquisition module includes:

[0306] An embedding vector acquisition unit is used to process the keywords in the quality inspection rules to obtain the keyword embedding vectors corresponding to the quality inspection rules.

[0307] An audio feature determination unit is used to determine the audio feature vector of the speech signal to be identified based on the feature vector matrix corresponding to the spectrogram features.

[0308] A high-level speech feature extraction unit is used to extract high-level speech features from the audio feature vector;

[0309] The attention score acquisition unit is used to call a bidirectional long short-term memory network model to process the high-level speech features and the keyword embedding vector to obtain the attention score between the high-level speech features and the keyword embedding vector.

[0310] The fusion feature vector acquisition unit is used to fuse the high-level speech features and the keyword embedding vector based on the attention score to obtain the fusion feature vector of the speech keywords.

[0311] The confidence score determination unit is used to determine the confidence score of the speech keyword based on the fused feature vector.

[0312] Optionally, the fusion feature vector acquisition unit includes:

[0313] The target high-level feature extraction subunit is used to extract target high-level speech features from the high-level speech features based on the attention score;

[0314] An initial weight vector acquisition subunit is used to process the target speech high-level features based on the target keyword embedding vector to obtain the initial weight vector corresponding to the target speech high-level features; the target keyword embedding vector is the keyword embedding vector that matches the target speech high-level features in the keyword embedding vector;

[0315] The weight vector acquisition sub-unit is used to normalize the initial weight vector to obtain the weight vector corresponding to the initial weight vector.

[0316] The feature vector fusion acquisition subunit is used to perform weighted processing on the high-level features of the target speech according to the weight vector to obtain the fusion feature vector of the speech keywords corresponding to the high-level features of the target speech.

[0317] Optionally, the identification result determination module includes:

[0318] The first recognition result acquisition unit is used to obtain a risk recognition result indicating that the voice signal to be recognized has a risk of leakage when the confidence score is greater than or equal to the confidence threshold.

[0319] The second recognition result acquisition unit is used to obtain a risk recognition result indicating that the voice signal to be recognized does not pose a risk of leakage when the confidence score is less than the confidence threshold.

[0320] The speech recognition device provided in this application acquires a speech signal to be recognized, preprocesses the speech signal to be recognized to obtain spectrogram features of the speech signal to be recognized, performs speech keyword detection processing on the spectrogram features based on pre-set quality inspection rules, obtains the confidence score of the speech keywords, and determines the risk recognition result of the speech signal to be recognized based on the confidence score and confidence threshold. This application embodiment converts the speech signal to be recognized into spectrogram features and combines them with pre-set quality inspection rules to detect the spectrogram features, identifying the confidence score of the speech keywords in the speech signal to be recognized, and thus determining the risk recognition result, which can effectively improve the accuracy of risk speech recognition. At the same time, no human intervention is required in the speech recognition process, which can effectively improve the efficiency of risk speech recognition.

[0321] This application also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-described speech recognition method when executed by the processor.

[0322] Figure 13 A schematic diagram of the structure of an electronic device 1300 according to an embodiment of the present invention is shown. Figure 13 As shown, the electronic device 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 1302 or loaded from storage unit 1308 into random access memory (RAM) 1303. The RAM 1303 may also store various programs and data required for the operation of the electronic device 1300. The CPU 1301, ROM 1302, and RAM 1303 are interconnected via bus 1304. An input / output (I / O) interface 1305 is also connected to bus 1304.

[0323] Multiple components in electronic device 1300 are connected to I / O interface 1305, including: input unit 1306, such as keyboard, mouse, microphone, etc.; output unit 1307, such as various types of monitors, speakers, etc.; storage unit 1308, such as disk, optical disk, etc.; and communication unit 1309, such as network card, modem, wireless transceiver, etc. Communication unit 1309 allows electronic device 1300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0324] The various processes and handling described above can be executed by processing unit 1301. For example, the methods of any of the above embodiments can be implemented as computer software programs tangibly contained in a computer-readable medium, such as storage unit 1308. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 1300 via ROM 1302 and / or communication unit 1309. When the computer program is loaded into RAM 1303 and executed by CPU 1301, one or more actions of the methods described above can be performed.

[0325] Additionally, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned speech recognition method.

[0326] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0327] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0328] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminals (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0329] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0330] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal, causing a series of operational steps to be executed on the computer or other programmable terminal to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0331] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0332] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes said element.

[0333] The above provides a detailed description of a speech recognition method, a speech recognition device, an electronic device, and a computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A voice recognition method, characterized by, The method includes: Acquire the speech signal to be recognized; The speech signal to be identified is preprocessed to obtain the spectrogram features of the speech signal to be identified; The spectrogram features are processed based on pre-set quality inspection rules to obtain speech keywords and their confidence scores. The risk identification result of the speech signal to be identified is determined based on the confidence score and the confidence threshold. The process of processing the spectrogram features based on pre-defined quality control rules to obtain speech keywords and their confidence scores includes: The keywords in the quality inspection rules are processed to obtain the keyword embedding vectors corresponding to the quality inspection rules. Based on the feature vector matrix corresponding to the spectrogram features, the audio feature vector of the speech signal to be identified is determined; Extract the high-level speech features from the audio feature vector; The bidirectional long short-term memory network model is invoked to process the high-level speech features and the keyword embedding vector to obtain the attention score between the high-level speech features and the keyword embedding vector; The high-level speech features and the keyword embedding vector are fused based on the attention score to obtain the fused feature vector of the speech keywords. The confidence score of the speech keyword is determined based on the fused feature vector.

2. The method according to claim 1, characterized in that, The preprocessing of the speech signal to be recognized to obtain the spectrogram features of the speech signal to be recognized includes: The speech signal to be identified is denoised and enhanced based on a pre-trained signal detection model to obtain the processed speech signal. The processed speech signal is preprocessed to obtain the spectrogram features.

3. The method according to claim 2, characterized in that, Before the pre-trained signal detection model performs denoising and enhancement processing on the speech signal to be identified to obtain the processed speech signal, the method further includes: Obtain model training samples; the model training samples include positive samples and negative samples, the negative samples are speech signal samples with the risk of leakage, and the positive samples are speech signal samples without the risk of leakage; The training samples of the model are input into the signal detection model to be trained; the signal detection model to be trained includes: an attention mechanism layer and a DRNN network layer; The attention mechanism layer is invoked to extract the temporal information of each speech frame in the model training samples, and the temporal information is concatenated with the corresponding speech frame to generate concatenated speech information; The DRNN network layer is invoked to denoise the spliced ​​speech information to obtain a predicted speech signal. The loss value of the signal detection model to be trained is calculated based on the predicted speech signal and the processed speech signal corresponding to the model training sample. If the loss value is within a preset range, the trained signal detection model is used as the final signal detection model.

4. The method according to claim 3, characterized in that, The attention mechanism layer includes: a speech frame splicing layer, a multilayer perceptron, a normalization layer, a temporal information acquisition layer, and a temporal information splicing layer. The step of calling the attention mechanism layer to extract the temporal information of each speech frame in the model training samples, and concatenating the temporal information with the corresponding speech frame to generate concatenated speech information includes: The speech frame splicing layer is invoked to splice each speech frame in the model training samples and the speech frames within the attention range to obtain spliced ​​speech frames; The multilayer perceptron is invoked to process the spliced ​​speech frames to obtain an attention score for each speech frame; The attention score is normalized by calling the normalization layer to obtain the attention weight for each speech frame; The timing information acquisition layer is invoked to process the attention weights and each speech frame to obtain the timing information corresponding to each speech frame; The timing information splicing layer is invoked to splice each speech frame and its corresponding timing information to obtain the spliced ​​speech information.

5. The method according to claim 2, characterized in that, The preprocessing of the processed speech signal to obtain the spectrogram features includes: The processed speech signal is pre-emphasized to obtain a pre-emphasized speech signal; The pre-emphasized speech signal is subjected to overlapping frame segmentation processing to obtain a framed speech signal; The framed audio signal is windowed to obtain a windowed audio signal; Perform a Fourier transform on the windowed speech signal to obtain the speech spectrum; The logarithmic energy spectrum of each frame of speech signal is obtained based on the logarithm of the energy spectrum corresponding to the speech spectrum. The logarithmic energy spectrum is rotated and concatenated into a feature vector matrix, and the spectrogram features corresponding to the processed speech signal are determined based on the feature vector matrix.

6. The method according to claim 1, characterized in that, The process of fusing the high-level speech features and the keyword embedding vector based on the attention score to obtain the fused feature vector of the speech keywords includes: Based on the attention score, target speech high-level features are extracted from the speech high-level features; The target speech high-level features are processed based on the target keyword embedding vector to obtain the initial weight vector corresponding to the target speech high-level features; the target keyword embedding vector is the keyword embedding vector that matches the target speech high-level features in the keyword embedding vector; The initial weight vector is normalized to obtain the weight vector corresponding to the initial weight vector; The target speech high-level features are weighted according to the weight vector to obtain the fusion feature vector of the speech keywords corresponding to the target speech high-level features.

7. The method according to claim 1, characterized in that, The step of determining the risk identification result of the speech signal to be identified based on the confidence score and confidence threshold includes: If the confidence score is greater than or equal to the confidence threshold, a risk identification result indicating that the voice signal to be identified has a risk of leakage is obtained; If the confidence score is less than the confidence threshold, a risk identification result indicating that the voice signal to be identified does not pose a risk of leakage is obtained.

8. A voice recognition device, characterized in that, The device includes: The speech signal acquisition module is used to acquire the speech signal to be recognized; The spectrogram acquisition module is used to preprocess the speech signal to be recognized to obtain the spectrogram features of the speech signal to be recognized. The confidence score acquisition module is used to process the spectrogram features based on pre-set quality inspection rules to obtain speech keywords and their confidence scores. The recognition result determination module is used to determine the risk recognition result of the speech signal to be recognized based on the confidence score and the confidence threshold. The confidence score acquisition module includes: An embedding vector acquisition unit is used to process the keywords in the quality inspection rules to obtain the keyword embedding vectors corresponding to the quality inspection rules. An audio feature determination unit is used to determine the audio feature vector of the speech signal to be identified based on the feature vector matrix corresponding to the spectrogram features. A high-level speech feature extraction unit is used to extract high-level speech features from the audio feature vector; The attention score acquisition unit is used to call a bidirectional long short-term memory network model to process the high-level speech features and the keyword embedding vector to obtain the attention score between the high-level speech features and the keyword embedding vector. The fusion feature vector acquisition unit is used to fuse the high-level speech features and the keyword embedding vector based on the attention score to obtain the fusion feature vector of the speech keywords. The confidence score determination unit is used to determine the confidence score of the speech keyword based on the fused feature vector.

9. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the speech recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the speech recognition method according to any one of claims 1 to 7.

Images