Method and apparatus for voice quality inspection, electronic device and storage medium

Abstract

Embodiments of the present application provide a voice quality inspection method and device, electronic equipment and storage medium. The method comprises: obtaining an audio signal to be inspected; extracting first speech features and second speech features of the audio signal, wherein the second speech features comprise spectral information; encoding the first speech features to obtain a time domain representation of the first speech features; encoding the second speech features to obtain a frequency domain representation of the spectral information; merging the time domain representation and the frequency domain representation as encoded output data; decoding the encoded output data; and determining a score of the audio signal based on the decoding result. Thus, the interference of background noise on voice quality inspection is avoided, and the accuracy of voice quality inspection is significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of speech analysis and processing, and more particularly to a method, device, electronic device and storage medium for speech quality inspection. Background Art

[0002] In recent years, with advances in neural network technology, voice technology has gradually matured and been applied across various industries. For example, because neural networks are fundamentally data-driven, they have spawned numerous data companies. After hiring dedicated personnel to record audio, these companies need to conduct quality control on the audio. Relying on manual labor for voice quality control wastes enormous time, effort, and financial resources, making automated quality control based on technology increasingly urgent.

[0003] Current speech quality inspection techniques typically determine the accuracy of audio signals by comparing them to standard text. Specifically, a single acoustic feature is first extracted from the audio signal to be inspected. Based on this acoustic feature, the difference between the audio signal and the normal pronunciation of the standard text is determined, and the inspection result is then determined. However, the accuracy of these inspection results is difficult to guarantee.

[0004] Therefore, a new voice quality inspection technology is urgently needed to solve the above problems. Summary of the Invention

[0005] The present invention has been made in view of the above-mentioned problems.

[0006] According to one aspect of the present invention, a method for speech quality inspection is provided. The method comprises:

[0007] Obtain the audio signal to be inspected;

[0008] extracting a first speech feature and a second speech feature of the audio signal, wherein the second speech feature includes spectrum information;

[0009] encoding the first speech feature to obtain a time domain representation of the first speech feature;

[0010] encoding the second speech feature to obtain a frequency domain representation of the spectrum information;

[0011] Combining the time domain representation and the frequency domain representation to provide encoded output data;

[0012] Decoding the encoded output data;

[0013] Based on the decoding result, a score of the audio signal is determined.

[0014] Exemplarily, decoding the encoded output data includes:

[0015] Inputting the encoded output data into a decoder to determine, by the decoder, based on a phonetic dictionary, a probability that a speech frame in the audio signal is pronounced as each word in the phonetic dictionary, wherein the phonetic dictionary includes a plurality of groups of words, each group of words including different variants of the same word corresponding to a plurality of speech of different qualities;

[0016] Determining the score of the audio signal based on the decoding result includes:

[0017] Determining, based on the determined probability, the word to which the speech frame in the audio signal actually corresponds;

[0018] Determining a score corresponding to a speech frame based on a word actually corresponding to the speech frame in the audio signal;

[0019] A score of the audio signal is determined based on the score corresponding to the speech frame.

[0020] Exemplarily, the decoder comprises a first decoder and a second decoder connected in parallel,

[0021] Inputting the encoded output data into a decoder to determine, using the decoder based on the phonetic dictionary, the probability that a speech frame in the audio signal is pronounced as each word in the phonetic dictionary, comprises:

[0022] inputting the encoded output data into the first decoder to determine a first probability that the speech frame corresponds to each word in the speech dictionary;

[0023] inputting the encoded output data into the second decoder to determine a second probability that the speech frame corresponds to each word in the speech dictionary;

[0024] The determining, based on the determined probability, the word actually corresponding to the speech frame in the audio signal comprises:

[0025] The word to which the speech frame in the audio signal actually corresponds is determined based on the first probability and the second probability.

[0026] Exemplarily, determining the word actually corresponding to the speech frame in the audio signal based on the first probability and the second probability includes:

[0027] For each word in the speech dictionary, the word probability P corresponding to the speech frame is calculated based on the following formula: w , P w =a*P ctc +(1-a)*P att Where a represents the weight of the first probability corresponding to the speech frame and the word, P ctc represents the first probability that the speech frame corresponds to the word, P attA second probability indicating that the speech frame corresponds to the word;

[0028] The word probabilities of each word in the speech dictionary are compared to determine the word corresponding to the maximum word probability and the determined word is used as the word actually corresponding to the speech frame.

[0029] Exemplarily, different characters in each group of characters in the speech dictionary correspond to different score segments, and determining the score corresponding to the speech frame includes:

[0030] Determining the fractional segment corresponding to the speech frame based on the word actually corresponding to the speech frame in the audio signal; and

[0031] Based on the probability that the speech frame corresponds to the word to which the speech frame actually corresponds, a score corresponding to the speech frame is determined.

[0032] Exemplarily, the method further includes:

[0033] Inputting the pending audio signal and the corresponding standard text into a speech evaluation system to obtain an evaluation quality of the pending audio signal by the speech evaluation system;

[0034] Based on the evaluation quality, multiple groups of words in the speech dictionary are determined to obtain the speech dictionary.

[0035] Exemplarily, before determining the score of the audio signal based on the decoding result, the method further includes:

[0036] Obtaining the evaluation audio signal;

[0037] Extracting a first speech feature and a second speech feature of the evaluation audio signal, wherein the second speech feature includes spectrum information;

[0038] Encoding the first speech feature of the evaluation audio signal to obtain a time domain representation of the first speech feature of the evaluation audio signal;

[0039] Encoding the second speech feature of the evaluation audio signal to obtain a frequency domain representation of the spectrum information of the evaluation audio signal;

[0040] Combining the time domain representation of the first speech feature of the evaluation audio signal and the frequency domain representation of the spectrum information of the evaluation audio signal to serve as encoded output data of the evaluation audio signal;

[0041] Decoding the encoded output data of the evaluation audio signal;

[0042] Determining a basic score of the evaluation audio signal based on the decoding result;

[0043] receiving an ideal score of the evaluation audio signal;

[0044] Fitting a relationship between the basic score and the ideal score based on the basic score and the ideal score of the evaluation audio signal;

[0045] Determining the score of the audio signal based on the decoding result includes:

[0046] determining a base score of the audio signal based on the decoding result;

[0047] An actual score of the audio signal is determined based on a base score of the audio signal and the fitted relationship.

[0048] Exemplarily, extracting the first speech feature of the audio signal includes:

[0049] extracting acoustic features of the audio signal;

[0050] determining energy information of the audio signal;

[0051] A first speech feature of the audio signal is determined based on the acoustic feature and the energy information.

[0052] Exemplarily, for each frame of the audio signal, the acoustic features of the frame are represented by an N-dimensional feature vector, and the energy information is represented by an energy value.

[0053] The determining the first speech feature of the frame based on the acoustic feature and the energy information includes:

[0054] The energy value is added before the first element or after the last element of the N-dimensional feature vector, so that the obtained N+1-dimensional feature vector is used as the first speech feature of the frame.

[0055] Exemplarily, the time domain representation is represented by a first eigenvector, and the frequency domain representation is represented by a second eigenvector.

[0056] The combining the time domain representation and the frequency domain representation comprises:

[0057] The first eigenvector and the second eigenvector are connected to obtain a connection vector of the first eigenvector and the second eigenvector as the encoding output data.

[0058] According to another aspect of the present invention, there is also provided a device for speech quality inspection, comprising:

[0059] An acquisition module, used to acquire the audio signal to be inspected;

[0060] A feature extraction module, configured to extract a first speech feature and a second speech feature of the audio signal, wherein the second speech feature includes spectrum information;

[0061] A first encoding module, configured to encode the first speech feature to obtain a time domain representation of the first speech feature;

[0062] A second encoding module, configured to encode the second speech feature to obtain a frequency domain representation of the spectrum information;

[0063] a merging module, configured to merge the time domain representation and the frequency domain representation to serve as encoded output data;

[0064] A decoding module, configured to decode the encoded output data;

[0065] A scoring module is configured to determine a score of the audio signal based on a decoding result.

[0066] According to another aspect of the present invention, an electronic device is provided, comprising a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute the above-mentioned voice quality inspection method when executed.

[0067] According to yet another aspect of the present invention, a storage medium is provided on which program instructions are stored. The program instructions are used to execute the above-mentioned method for speech quality inspection when running.

[0068] In the above technical solution, speech quality inspection is performed based on the first and second speech features of the audio signal, where the second speech feature includes the audio signal's spectral information. This comprehensively considers all frequencies of the audio signal, including both valid speech and noise, and avoids background noise interference in speech quality inspection. This significantly improves the accuracy of speech quality inspection, especially in situations with high background noise and cluttered spectrograms.

[0069] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The above and other objects, features, and advantages of the present invention will become more apparent through a more detailed description of the embodiments of the present invention with reference to the accompanying drawings. The accompanying drawings are provided to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and are not intended to limit the present invention. In the drawings, the same reference numerals generally represent the same components or steps.

[0071] Figure 1A schematic flow chart of a speech quality inspection method according to an embodiment of the present invention is shown;

[0072] Figure 2 A schematic flow chart of extracting a first speech feature of an audio signal according to an embodiment of the present invention is shown;

[0073] Figure 3 A schematic flow chart of determining a score of an audio signal based on a decoding result according to an embodiment of the present invention is shown;

[0074] Figure 4 A schematic flow chart of modeling a speech dictionary according to an embodiment of the present invention is shown;

[0075] Figure 5 A schematic flow chart of determining a fitting relationship between a basic score and an ideal score for adjusting a basic score according to one embodiment of the present invention is shown;

[0076] Figure 6 A schematic block diagram of a device for speech quality inspection according to an embodiment of the present invention is shown;

[0077] Figure 7 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solutions and advantages of the present invention more apparent, exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present invention.

[0079] As mentioned above, the accuracy of existing speech quality inspection systems is difficult to guarantee. This is especially true in some application scenarios where the audio signal has high background noise, which can lead to a messy spectrogram. This can seriously interfere with the quality inspection process and result in low scores.

[0080] In order to solve the above problem, according to one embodiment of the present invention, a speech quality inspection method is provided. Figure 1 FIG1 is a schematic flow chart of a speech quality inspection method 100 according to an embodiment of the present invention. As shown in FIG1 , the speech quality inspection method 100 may include the following steps.

[0081] Step S110: Acquire an audio signal to be quality-checked.

[0082] Exemplarily, the audio signal to be inspected can be speech collected by a sound collection device (e.g., a microphone) of an electronic device, and converted by an analog / digital conversion circuit to convert the analog signal, i.e., the speech to be inspected, into a digital signal, i.e., an audio signal, that can be recognized and processed by the electronic device. Thus, the audio signal corresponds to the speech to be inspected. Alternatively, the audio signal to be inspected that is pre-stored in another device or storage medium can be obtained through data transmission technology.

[0083] Step S120 , extracting the first speech feature and the second speech feature of the audio signal acquired in step S110 .

[0084] Preferably, the audio signal can be preprocessed. Preprocessing may include operations such as denoising and framing. For example, an audio signal is first filtered and sampled to reduce interference from signals with frequencies other than human voice and / or 50 Hz current frequency. In addition, the audio signal can also be framed. Framing refers to the operation of dividing the audio signal into multiple small segments to obtain multiple speech frames, each of which is called a "frame" and the frame length is generally 10 to 30 ms. For example, 20 milliseconds. The audio signal of each frame after framing has the characteristic of short-term stability.

[0085] For example, a feature extraction operation can be performed on the preprocessed audio signal. The audio signal can be divided into many frames, and for each speech frame, the first speech feature and the second speech feature of the speech frame are extracted. The speech features of the speech frame can be represented by an N-dimensional vector, for example, 256 or 512 dimensions. The first speech feature may include acoustic information of the audio signal. For example, the first speech feature may include Mel-Frequency Cepstral Coefficient (MFCC) features, Mel-Scaled Filter Bank (Filter Bank) acoustic features, and Perceptual Linear Prediction Coefficient (PLP) features. The second speech feature includes spectral information of the audio signal. Each speech frame corresponds to a spectrum, and the spectrum represents the relationship between frequency and energy. Each frame in the audio signal can be considered to be extracted from different stationary signal waveforms, and the short-time spectrum of each frame is an approximation of the spectrum of each stationary signal waveform. The data of the speech frame can be subjected to a Fourier transform (FFT), i.e., a short-time Fourier transform, to extract the spectral information of the audio signal.

[0086] The embodiments of this application do not limit the two speech feature extraction methods. Any existing or future technology that can achieve speech feature extraction is within the scope of this application. By way of example and not limitation, the code for extracting the corresponding speech features can be directly called from the KALDI open source code.

[0087] Step S130 , encoding the first speech feature extracted in step S120 to obtain a time domain representation of the first speech feature.

[0088] Exemplarily, various suitable encoders can be used to encode the first speech feature to obtain its time domain representation. The time domain representation can capture the characteristics of the first speech feature along the time domain. Optionally, a recurrent neural network can be used to implement the encoder. The recurrent neural network can integrate the speech features of different frames in the audio signal, making it possible to estimate the information of other frames (previous frames and / or subsequent frames) based on the information of the current frame. The inter-neuron connections of the recurrent neural network constitute a directed graph, and the recurrent neural network can receive a wide range of time series inputs by cyclically transmitting the state in its own network. According to an embodiment of the present invention, the time series is the first speech feature of each speech frame in the audio signal. For each speech frame in a plurality of speech frames, the first speech feature of the speech frame is input into the recurrent neural network, and the recurrent neural network can establish an association between it and the previous speech frame and / or subsequent speech frame with different acquisition time, thereby outputting the time domain representation of the first speech feature.

[0089] Optionally, the encoding operation in step S130 can reduce the dimensionality of the first speech feature of the audio signal. As previously mentioned, the speech feature of a speech frame can be represented by a vector. Therefore, the first speech feature of the audio signal can be a matrix whose number of rows is the number of frames in the audio signal and whose number of columns is the dimension of the feature, for example, 256. Through this encoding operation, the first speech feature can be encoded into a vector, which is the time-domain representation of the first speech feature.

[0090] Step S140 , encoding the second speech feature extracted in step S120 to obtain a frequency domain representation of the spectrum information.

[0091] For example, various suitable encoders can be used to encode the second speech feature to obtain its frequency domain representation. This frequency domain representation can capture the characteristics of the second speech feature along the frequency domain. Alternatively, a convolutional neural network (CNN) can be used to implement the encoder. Convolutional neural networks have powerful learning capabilities. Through training operations, it can be enabled to encode the second speech feature to obtain a frequency domain representation of spectral information.

[0092] Similarly, through the encoding operation in step S140 , the second speech feature of the audio signal can be reduced in dimension. The matrix representing the second speech feature of the audio signal is encoded into a vector, that is, a frequency domain representation of the spectrum information.

[0093] Step S150: Combine the time-domain representation and the frequency-domain representation obtained in Step 130 and Step S140 as the encoded output data. Thus, the encoded output synthesizes the information of both the time-domain representation of the first speech feature and the frequency-domain representation of the spectral information. In other words, it can reflect both the acoustic characteristics of the audio signal and the frequency-domain characteristics of the audio signal.

[0094] Step S160: Decode the encoded output data obtained in Step S150.

[0095] As described above, the concatenated vector combining the time-domain representation of the first speech feature and the frequency-domain representation of the second speech feature is used as the encoded output data. In this step, the encoded output data is decoded to determine the probabilities that each speech frame of the audio signal corresponds to different words. For example, the encoded output data is fed into a connectionist temporal classification model (CTC) to output the probabilities that the speech frames correspond to each word. Assume that there are 25,000 words in the speech dictionary. Here, for a speech frame, a vector with a dimension of 25,000 can be output, and each element in the vector represents the probability that the speech frame corresponds to the corresponding word in the speech dictionary. For example, the first element in the vector represents the probability that the speech frame corresponds to the first word in the speech dictionary, the second element in the vector represents the probability that the speech frame corresponds to the second word in the speech dictionary, and so on.

[0096] Step S170: Determine the score of the audio signal based on the decoding result in Step S160.

[0097] In Step S160, the probabilities that each frame in the audio signal corresponds to different words are determined. In this Step S170, the score of the audio signal can be determined based on this probability. Exemplarily, assume that for a certain speech frame, compared with other words, the probability that it corresponds to the word "标" is the largest. The score of this speech frame can be determined according to the probability that it corresponds to the word "标". Similarly, the scores of each speech frame in the audio signal are determined. According to the scores of each speech frame, the score of the audio signal is determined. For example, the average value of the scores of all speech frames is taken as the score of the audio signal.

[0098] In the above technical solution, speech quality inspection is performed based on the first speech feature and the second speech feature of the audio signal, where the second speech feature includes the spectral information of the audio signal. Thus, the sounds of each frequency in the audio signal, including valid speech and noise, are comprehensively considered, avoiding the interference of background noise on speech quality inspection, and significantly improving the accuracy of speech quality inspection, especially for situations with high background noise and messy spectrograms.

[0099] As described above, the first speech feature and the second speech feature of the audio signal are extracted in Step S120. Figure 2FIG. 5 shows a schematic flow chart of extracting the first speech feature of an audio signal according to an embodiment of the present invention. Figure 2 As shown, extracting the first speech feature of the audio signal may specifically include the following steps.

[0100] Step S121: extracting acoustic features of the audio signal.

[0101] The acoustic feature may be a physical quantity representing an acoustic characteristic of speech, and may be, for example, an MFCC feature, a Mel-scale filter bank acoustic feature, a PLP feature, etc. Those skilled in the art will appreciate that step S210 may be implemented based on any existing or future developed feature extraction technology, and for the sake of brevity, details are not repeated here.

[0102] Step S122: determining energy information of the audio signal.

[0103] The energy information of a speech frame in an audio signal can represent the volume of the speech frame. In this embodiment, if the speaker's voice is too soft, it will be difficult to hear clearly, so the score should be lower. If the speaker's voice is too loud and clipped, the score should also be lower. For example, the energy information of each speech frame in the audio signal can be represented by an energy value.

[0104] Step S123: Determine a first speech feature of the audio signal based on the acoustic feature and the energy information.

[0105] In this step, the acoustic features of the audio signal extracted in step S121 and the energy information of the audio signal determined in step S122 are combined to obtain the first speech feature. Thus, the first speech feature combines the acoustic features and the energy information.

[0106] The above method of obtaining the first speech feature takes into account the influence of the energy information of the audio signal on its quality. The speech quality inspection of the audio signal based on the first speech feature is more accurate.

[0107] For example, for each frame of the audio signal, the acoustic features of the frame are represented by an N-dimensional feature vector, and the energy information is represented by an energy value. In step S123, the energy value can be appended before the first element or after the last element of the N-dimensional feature vector to obtain the N+1-dimensional feature vector as the first speech feature of the frame.

[0108] For example, the MFCC feature is described as an example. First, the audio signal is pre-emphasized, framed, windowed, and fast Fourier transformed to obtain the spectral information of each frame. Then the power spectrum is calculated, and the obtained power spectrum is passed through a triangular bandpass filter. The filtered output result is converted into a logarithmic form using the relationship between the Mel domain and the linear frequency. Finally, a discrete cosine transform is performed to obtain the MFCC feature. The MFCC feature can be represented by an N-dimensional vector. The energy value can be directly appended to the last element of the N-dimensional vector to form the first speech feature represented by an N+1-dimensional vector. Specifically, it is assumed that the MFCC feature is represented as M=[m1,m2,…,m N ], the energy value is expressed as E, then the first speech feature F can be expressed as F = [m1, m2, ..., m N ,E].

[0109] Alternatively, the acoustic features and energy information can be weighted separately, and then the weighted acoustic features and energy information can be fused to determine the first speech feature. For example, depending on different application scenarios, the first weight of the acoustic features can be made greater than the second weight of the energy information. After performing the corresponding weighting operation, the weighted energy information is appended to the beginning or end of the weighted acoustic features to determine the first speech feature.

[0110] This technical solution obtains the first speech feature by directly attaching the energy value to the head or tail of the acoustic feature, which not only ensures the accuracy of the quality inspection result but also is easy to implement.

[0111] Exemplarily, the time domain representation obtained in step S130 is represented by a first eigenvector, the frequency domain representation obtained in step S140 is represented by a second eigenvector, and step S150 combines the time domain representation and the frequency domain representation, including the following steps. The first eigenvector and the second eigenvector are concatenated to obtain a concatenated vector of the first eigenvector and the second eigenvector. The concatenated vector is used as the merged encoded output data. Specifically, assuming that the first eigenvector is represented by Ve1=[s11,s12,…,s1 k ], the second eigenvector is expressed as Ve2=[s21,s22,…,s2 l ], then the connection vector is Ve=[s11,s12,…,s1 k ,s21,s22,…,s2 l ]. Among them, the dimension of the first eigenvector is k, the dimension represented by the second eigenvector is l, and the dimension of the connection vector is k+l.

[0112] This technical solution combines the first eigenvector and the second eigenvector through a connection operation to obtain coded output data, which not only ensures the accuracy of the quality inspection result but also is easy to implement.

[0113] Exemplarily, the decoding of the encoded output data in step S160 may include the following steps: Step S161, input the encoded output data into an encoder to utilize the encoder to determine the probability that the pronunciation of the speech frame in the audio signal is each word in the speech dictionary. The speech dictionary includes multiple groups of words, and each group of words includes different variants of the same word corresponding to voices of multiple different qualities respectively. In other words, the speech dictionary may include many words, and these words appear in groups. Each group of words corresponds to different variants of the same word. Different variants of a word respectively correspond to a voice of a different quality of this word. In a specific embodiment, there are 5,000 different words in the speech dictionary, and each word corresponds to five variants. For example, for the word "标 (biāo)", it has different variants: "标1 (biāo1)", "标2 (biāo2)", "标3 (biāo3)", "标4 (biāo4)", and "标5 (biāo5)", and the variants of these words respectively correspond to voices that are increasingly close to the Mandarin pronunciation of the word "标 (biāo)", that is, the voice quality is getting better and better. Thus, the actual number of words included in the speech dictionary is 5,000 * 5 = 25,000.

[0114] It can be understood that the number of variants of the words in each group in the above speech dictionary can be set differently according to specific application scenarios. For example, for a scenario with relatively loose requirements for the quality inspection granularity, each group may include fewer variants; otherwise, vice versa. Thus, the granularity of the speech quality inspection can be flexibly adjusted to improve the user experience.

[0115] As described in step S161, use the decoder to decode the connection vector Ve = [s11, s12, …, s1 k , s21, s22, …, s2 l obtained in the above step S150, and the decoding result will obtain the probability of each word corresponding to the pronunciation of the speech frame in the audio signal. Specifically, in the case where there are 25,000 words in the above speech dictionary, for the speech frame in the audio signal, the decoder can output a 25,000-dimensional vector. Each element in this vector respectively represents the probability that this speech frame corresponds to the word corresponding to this element. For example, if the 1,000th word in the speech dictionary is "标2 (biāo2)", then the 1,000th element in the 25,000-dimensional vector represents the probability that the pronunciation of this speech frame is the voice corresponding to "标2 (biāo2)".

[0116] After the probability that the pronunciation of the speech frame in the audio signal is each word in the speech dictionary has been determined, step S170 may be executed to determine the score of the audio signal based on the decoding result. Figure 3 FIG. shows a schematic flowchart of step S170 for determining the score of the audio signal based on the decoding result according to an embodiment of the present invention. As Figure 3 shown, step S170 may include the following steps.

[0117] Step S171: Based on the determined probability, determine the actual word corresponding to the speech frame in the audio signal.

[0118] Based on the probability of each word in the speech dictionary determined in step S161, it can be considered that the word corresponding to the maximum probability is the actual word corresponding to the speech frame in the audio signal. Still taking the above speech dictionary with 25,000 words as an example, assume that in the 25,000-dimensional vector output by the decoder, the value of the 1000th element is the largest, which is 0.8. As mentioned above, the 1000th word in the speech dictionary is "Standard 2", so the actual word corresponding to this speech frame is "Standard 2".

[0119] Step S172: Based on the actual word corresponding to the speech frame in the audio signal, determine the score corresponding to the speech frame. In the above speech dictionary, each group of words in the speech dictionary includes different variants of the same word corresponding to multiple different qualities of speech. Therefore, the score of the speech frame can be determined according to the actual word corresponding to the determined speech frame.

[0120] In one example, for each group of words, since each word corresponds to a different speech quality respectively, each word can correspond to a different score respectively. Specifically, the above "Standard 1", "Standard 2", "Standard 3", "Standard 4" and "Standard 5" can correspond to scores 20, 40, 60, 80, 100 respectively. In the example where the actual word corresponding to the above speech frame is "Standard 2", the score corresponding to this speech frame can be determined as 40 according to "Standard 2".

[0121] In an alternative example, for each group of words, each word can correspond to a different score range respectively. Specifically, the above "Standard 1", "Standard 2", "Standard 3", "Standard 4" and "Standard 5" can correspond to score ranges 0 - 20, 21 - 40, 41 - 60, 61 - 80, 81 - 100 respectively. This step S172 may include: First, based on the actual word corresponding to the speech frame in the audio signal, determine the score range corresponding to the speech frame. In the example where the actual word corresponding to the above speech frame is "Standard 2", the score range corresponding to this speech frame can be determined as 21 - 40 according to "Standard 2". Then, based on the probability that the speech frame corresponds to the word corresponding to this speech frame, determine the score corresponding to this speech frame. The higher the probability that the speech frame corresponds to the word, the higher the score of the speech frame; otherwise, vice versa. Exemplarily, assume that the probability that the speech frame corresponds to the word is 100%, then the score of the speech frame can take the highest score in the score range. If the probability that the speech frame corresponds to the word is less than 4%, then the score of the speech frame can take the lowest score in the score range. In the above example, the probability that the speech frame corresponds to "Standard 2" is 80%, then the score of the speech frame can take 35 points.

[0122] In this solution, the score of a speech frame is determined based on both the word actually corresponding to the speech frame in the audio signal and the probability corresponding to the word, resulting in a more accurate result. Furthermore, the score of the audio signal determined by this speech quality inspection method can be made more accurate.

[0123] Step S173: Determine the score of the audio signal based on the score corresponding to the speech frame.

[0124] Exemplarily, the scores respectively corresponding to all the speech frames in the audio signal can be added and averaged, and the calculated average value is used as the score of the audio signal.

[0125] Existing speech quality inspections are basically achieved in two ways: 1. relying on speech recognition; 2. relying on speech evaluation. Both simply determine the accuracy of speech through text comparison and have a strong dependence on the text. This technology has limitations in detecting speech quality problems. In addition, the above existing speech quality inspection methods not only require the use of multiple complex models and are difficult to optimize uniformly, but also require forced alignment operations, with a large amount of calculation and long time consumption. Finally, in the absence of a standard text, an auxiliary speech recognition system is required, further increasing the workload. Finally, because the architecture of the entire speech quality inspection system is so complex, it is difficult to guarantee the accuracy of speech quality inspection. However, the above technical solution does not require forced alignment operations, does not require many complex module combinations such as language models and acoustic models, is relatively simple, and is easier to optimize when problems occur. In addition, the quality result can be given without a reference text, combining the original process of recognizing the text and then scoring in a free evaluation scenario into one stage, making the tool more lightweight and reducing the coupling between tools. Finally, the above solution can perfectly solve the problem of repeated readings. If the text read by the user contains reduplicated characters, such as "vivid", the above solution will give a quality score for each character in the reduplicated characters. This avoids the problem that multiple repeated results directly affect the final score and cause a decline in the quality of quality inspection.

[0126] Exemplarily, the above decoder includes a first decoder and a second decoder in parallel. The first decoder and the second decoder can be different decoders. Step S161 inputs the encoded output data into the decoder to determine the probability that each word in the speech dictionary is pronounced for the speech frame in the audio signal by using the decoder, including: inputting the encoded output data into the first decoder to determine the first probability corresponding to each word in the speech dictionary for the speech frame respectively; inputting the encoded output data into the second decoder to determine the second probability corresponding to each word in the speech dictionary for the speech frame respectively. The first decoder can be a CTC model, and the second decoder can be an Attention model. The Attention model realizes the function of adopting different degrees of attention to different parts of the input sequence by calculating the weights of different parts of the input sequence. In the case where there are 25,000 words in the foregoing speech dictionary, for the speech frame in the audio signal, the first decoder and the second decoder can respectively output a 25,000-dimensional vector as the first probability and the second probability respectively. The CTC model has an independence assumption, while the Attention model has a characteristic of front-back dependence; at the same time, the Attention model cannot guarantee the order in time, but the CTC model can. Thus, overall, the CTC model and the Attention model complement each other, which is a relatively perfect combination.

[0127] The above step S171 determines the word actually corresponding to the speech frame in the audio signal based on the determined probabilities, including: determining the word actually corresponding to the speech frame in the audio signal based on the first probability and the second probability.

[0128] Exemplarily, the probability of the word corresponding to the element can be determined based on the corresponding elements in the vector representing the first probability and the vector representing the second probability. In the example where the 1000th word in the above speech dictionary is "Standard 2", based on the 1000th element 0.8 in the vector representing the first probability and the 1000th element 0.7 in the vector representing the second probability, the probability that the speech frame corresponds to "Standard 2" can be determined. In one example, the two can be averaged, and it is determined that the probability that the speech frame corresponds to "Standard 2" is 0.75. After determining the probabilities that the speech frame in the audio signal corresponds to each word in the speech dictionary based on the first probability and the second probability, the word actually corresponding to the speech frame can be determined therefrom, such as the word corresponding to the maximum probability.

[0129] In another example, the above step determines the word actually corresponding to the speech frame in the audio signal based on the first probability and the second probability, including: First, for each word in the speech dictionary, calculate the word probability P that the speech frame corresponds to the word based on the following formula w , P w =a*P ctc +(1 - a)*Patt Where a represents the weight of the first probability corresponding to the speech frame and the word, P ctc represents the first probability that the speech frame corresponds to the word, P att Represents the second probability that the speech frame corresponds to the word. The value of a can be determined according to the importance of the first encoder and the second encoder. In the above example of combining the CTC model and the Attention model, a can take any value between 0.3 and 0.5. For the above example, the word probability corresponding to "label 2" is: 0.3*0.8+0.7*0.7=0.73. Then, compare the word probability of each word in the speech dictionary to determine the word corresponding to the maximum word probability and use the determined word as the word that actually corresponds to the speech frame. Similar to "label 2", the word probability corresponding to the speech frame and each word in the speech dictionary can be calculated based on the vectors output by the two decoders respectively, and then compare the word probabilities of all words, so that the word corresponding to the maximum word probability is used as the word that actually corresponds to the speech frame.

[0130] Two different decoders are used to decode the encoded output data. Different encoders have their own strengths, resulting in more accurate decoded data and, in turn, more accurate speech quality inspection results. Furthermore, the aforementioned formula-based approach for calculating the probability of each word in the speech dictionary provides more accurate word probabilities, leading to a higher degree of confidence in the quality inspection results.

[0131] It is understood that the various models in the above scheme can be trained under a unified architecture. The entire speech quality inspection system is trained using labeled training data. The training data can be labeled using an existing speech quality inspection system.

[0132] Illustratively, the speech quality inspection method provided by the embodiment of the present invention may further include the step of obtaining the above-mentioned speech dictionary by using a speech evaluation system. Figure 4 FIG. 4 shows a schematic flow chart of modeling a speech dictionary according to an embodiment of the present invention. Figure 4As shown, first, the pending audio signal and the corresponding standard text are input into the speech evaluation system to obtain the evaluation quality of the speech evaluation system for the pending audio signal. The speech evaluation system can be any traditional speech evaluation system. The speech evaluation system can be used to obtain the evaluation quality of the audio signal by comparing the pronunciation of the pending audio signal with the correct pronunciation of the standard text of the audio signal. Specifically, the speech evaluation system can perform four operations on the audio signal: feature extraction, forward calculation, forced alignment, and scoring, to determine the score of each word in the audio signal. Then, based on the obtained evaluation quality, multiple groups of words in the speech dictionary are determined to obtain a speech dictionary. In the previous step, by evaluating audio signals of different qualities, the evaluation quality of each of them can be obtained. Based on these audio signals of different qualities, the words in the speech dictionary can be determined, wherein the speech quality of different variants in each group of words is different.

[0133] In the above solution, a speech evaluation system is used to obtain a speech dictionary and archive speech of various qualities. This greatly reduces the consumption of manpower and financial resources while ensuring the quality of speech evaluation.

[0134] For example, in the step S170 , the score of the audio signal may be determined by adjusting the score determined based on the decoding result of step S160 , thereby obtaining a more accurate actual score of the audio signal. Figure 5 FIG. 1 is a schematic flow chart of determining a fitting relationship between a basic score and an ideal score for adjusting a basic score according to an embodiment of the present invention. Figure 5 As shown, determining the fitting relationship between the basic score and the ideal score may include the following steps.

[0135] Step S510: Acquire an evaluation audio signal.

[0136] Step S520: extracting a first speech feature and a second speech feature of the evaluation audio signal, wherein the second speech feature includes spectrum information.

[0137] Step S530: Encode the first speech feature of the evaluation audio signal to obtain a time domain representation of the first speech feature of the evaluation audio signal.

[0138] Step S540: Encode the second speech feature of the evaluation audio signal to obtain a frequency domain representation of the spectrum information of the evaluation audio signal.

[0139] Step S550: Combine the time domain representation of the first speech feature of the evaluation audio signal and the frequency domain representation of the spectrum information of the evaluation audio signal to serve as encoded output data of the evaluation audio signal.

[0140] Step S560: Decode the encoded output data of the evaluation audio signal.

[0141] Step S570: Determine a basic score for evaluating the audio signal based on the decoding result.

[0142] The above steps S510 to S570 are similar to steps S110 to S170 respectively. Steps S110 to S170 have been described in detail above and will not be repeated here for the sake of brevity.

[0143] Step S580: Receive the ideal score of the evaluation audio signal.

[0144] For example, a speech expert may manually score the evaluation audio signal to serve as an ideal score for the audio signal.

[0145] Step S590: fitting the relationship between the basic score and the ideal score based on the basic score and the ideal score of the evaluation audio signal.

[0146] In the case of a large number of evaluation audio signals, parameter fitting can be performed on the base score and the ideal score to determine the functional relationship between the base score and the ideal score. This fitting operation can be implemented using machine learning.

[0147] After determining the relationship between the base score and the ideal score, step S170 may include the following steps: First, determining the base score of the audio signal based on the decoding result. This process is described in detail in the previous example and will not be repeated here for the sake of brevity. Then, determining the actual score of the audio signal based on the base score of the audio signal and the relationship between the base score and the ideal score obtained by fitting in step S590. Based on the determined base score of the audio signal, the actual score of the audio signal can be determined based on the fitting relationship.

[0148] As a result, the determined actual score is closer to the ideal score of the audio signal, thereby improving the accuracy of speech evaluation.

[0149] According to another aspect of the present invention, a device for speech quality inspection is provided. Figure 6 FIG. 6 shows a schematic block diagram of a device 600 for speech quality inspection according to an embodiment of the present invention. Figure 6 As shown, the speech evaluation apparatus 600 may include the following modules.

[0150] The acquisition module 610 is configured to acquire an audio signal to be quality-checked.

[0151] The feature extraction module 620 is configured to extract a first speech feature and a second speech feature of the audio signal, wherein the second speech feature includes spectrum information.

[0152] The first encoding module 630 is configured to encode the first speech feature to obtain a time domain representation of the first speech feature.

[0153] The second encoding module 640 is configured to encode the second speech feature to obtain a frequency domain representation of the spectrum information.

[0154] The merging module 650 is configured to merge the time domain representation and the frequency domain representation to produce encoded output data.

[0155] The decoding module 660 is configured to decode the encoded output data.

[0156] The scoring module 670 is configured to determine a score of the audio signal based on the decoding result.

[0157] It should be noted that the components of the device should be understood as functional modules established to implement each step of the program flow or method, and the individual functional modules are not defined by actual functional division or separation. A device defined by such a set of functional modules should be understood as a functional module architecture that primarily implements the solution through the computer program described in the specification, and should not be understood as a physical device that primarily implements the solution through hardware.

[0158] According to yet another aspect of the present invention, an electronic device is provided. Figure 7 FIG. 1 shows a schematic block diagram of an electronic device 700 according to an embodiment of the present invention. Figure 7 As shown, the electronic device 700 may include a processor 710 and a memory 720. The memory 720 stores computer program instructions, which are used by the processor 710 to execute the voice quality inspection method as described above when the computer program instructions are executed.

[0159] According to another aspect of the present invention, a storage medium is also provided. Program instructions are stored on the storage medium, and when the program instructions are executed by a computer or processor, the computer or processor is caused to perform the corresponding steps of the voice quality inspection method according to the embodiment of the present invention, and is used to implement the corresponding modules in the voice wake-up device and equipment according to the embodiment of the present invention. The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0160] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0161] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the description of exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this approach to the present invention should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More precisely, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with fewer features than all the features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0162] Those skilled in the art will understand that, except where mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus disclosed herein may be combined in any combination. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that provides the same, equivalent, or similar purpose.

[0163] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of the present invention and to form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

[0164] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that in practice, a microprocessor or digital signal processor (DSP) can be used to implement some or all of the functions of some modules in the apparatus for speech quality inspection according to an embodiment of the present invention. The present invention can also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0165] It should be noted that the above embodiments illustrate rather than limit the present invention, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbols placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0166] The foregoing description is merely a specific embodiment of the present invention or an illustration of a specific embodiment. The scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present invention are intended to be encompassed by the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for speech quality inspection, characterized in that: include: Obtain the audio signal to be inspected; extracting a first speech feature and a second speech feature of the audio signal, wherein the second speech feature includes spectrum information; encoding the first speech feature to obtain a time domain representation of the first speech feature; encoding the second speech feature to obtain a frequency domain representation of the spectrum information; Combining the time domain representation and the frequency domain representation to provide encoded output data; Decoding the encoded output data; determining a score of the audio signal based on a decoding result; Wherein, decoding the encoded output data comprises: Inputting the encoded output data into a decoder to determine, by the decoder, based on a phonetic dictionary, a probability that a speech frame in the audio signal is pronounced as each word in the phonetic dictionary, wherein the phonetic dictionary includes multiple groups of words, each group of words includes different variants of the same word corresponding to multiple speech qualities, and different scores are assigned to different variants of the same word; Determining the score of the audio signal based on the decoding result includes: Determining, based on the determined probability, the word to which the speech frame in the audio signal actually corresponds; Determining a score corresponding to a speech frame based on a word actually corresponding to the speech frame in the audio signal; A score of the audio signal is determined based on the score corresponding to the speech frame.

2. The method according to claim 1, wherein The decoder comprises a first decoder and a second decoder connected in parallel, Inputting the encoded output data into a decoder to determine, using the decoder based on the phonetic dictionary, the probability that a speech frame in the audio signal is pronounced as each word in the phonetic dictionary, comprises: inputting the encoded output data into the first decoder to determine a first probability that the speech frame corresponds to each word in the speech dictionary; inputting the encoded output data into the second decoder to determine a second probability that the speech frame corresponds to each word in the speech dictionary; The determining, based on the determined probability, the word actually corresponding to the speech frame in the audio signal comprises: The word to which the speech frame in the audio signal actually corresponds is determined based on the first probability and the second probability.

3. The method according to claim 2, wherein: The determining, based on the first probability and the second probability, the word actually corresponding to the speech frame in the audio signal comprises: For each word in the speech dictionary, the word probability P corresponding to the speech frame is calculated based on the following formula: w , P w =a*P ctc +(1-a)*P att Where a represents the weight of the first probability corresponding to the speech frame and the word, P ctc represents the first probability that the speech frame corresponds to the word, P att A second probability indicating that the speech frame corresponds to the word; The word probabilities of each word in the speech dictionary are compared to determine the word corresponding to the maximum word probability and the determined word is used as the word actually corresponding to the speech frame.

4. The method according to claim 1, wherein Different characters in each group of characters in the speech dictionary correspond to different score segments, and determining the score corresponding to the speech frame includes: Determining the fractional segment corresponding to the speech frame based on the word actually corresponding to the speech frame in the audio signal; and Based on the probability that the speech frame corresponds to the word to which the speech frame actually corresponds, a score corresponding to the speech frame is determined.

5. The method according to claim 1, wherein The method further comprises: Inputting the pending audio signal and the corresponding standard text into a speech evaluation system to obtain an evaluation quality of the pending audio signal by the speech evaluation system; Based on the evaluation quality, multiple groups of words in the speech dictionary are determined to obtain the speech dictionary.

6. The method of claim 1, wherein: Before determining the score of the audio signal based on the decoding result, the method further includes: Obtaining the evaluation audio signal; Extracting a first speech feature and a second speech feature of the evaluation audio signal, wherein the second speech feature includes spectrum information; Encoding the first speech feature of the evaluation audio signal to obtain a time domain representation of the first speech feature of the evaluation audio signal; Encoding the second speech feature of the evaluation audio signal to obtain a frequency domain representation of the spectrum information of the evaluation audio signal; Combining the time domain representation of the first speech feature of the evaluation audio signal and the frequency domain representation of the spectrum information of the evaluation audio signal to serve as encoded output data of the evaluation audio signal; Decoding the encoded output data of the evaluation audio signal; Determining a basic score of the evaluation audio signal based on the decoding result; receiving an ideal score of the evaluation audio signal; Fitting a relationship between the basic score and the ideal score based on the basic score and the ideal score of the evaluation audio signal; Determining the score of the audio signal based on the decoding result includes: determining a base score of the audio signal based on the decoding result; An actual score of the audio signal is determined based on a base score of the audio signal and the fitted relationship.

7. The method of claim 1, wherein: Extracting the first speech feature of the audio signal includes: extracting acoustic features of the audio signal; determining energy information of the audio signal; A first speech feature of the audio signal is determined based on the acoustic feature and the energy information.

8. The method of claim 7, wherein: For each frame of the audio signal, the acoustic features of the frame are represented by an N-dimensional feature vector, and the energy information is represented by an energy value. The determining the first speech feature of the frame based on the acoustic feature and the energy information includes: The energy value is added before the first element or after the last element of the N-dimensional feature vector, so that the obtained N+1-dimensional feature vector is used as the first speech feature of the frame.

9. The method according to any one of claims 1 to 8, wherein: The time domain representation is represented by a first eigenvector, and the frequency domain representation is represented by a second eigenvector. The combining the time domain representation and the frequency domain representation comprises: The first eigenvector and the second eigenvector are connected to obtain a connection vector of the first eigenvector and the second eigenvector as the encoding output data.

10. A device for voice quality inspection, comprising: An acquisition module, used to acquire the audio signal to be inspected; A feature extraction module, configured to extract a first speech feature and a second speech feature of the audio signal, wherein the second speech feature includes spectrum information; A first encoding module, configured to encode the first speech feature to obtain a time domain representation of the first speech feature; A second encoding module, configured to encode the second speech feature to obtain a frequency domain representation of the spectrum information; a merging module, configured to merge the time domain representation and the frequency domain representation to serve as encoded output data; A decoding module, configured to decode the encoded output data; a scoring module, configured to determine a score of the audio signal based on a decoding result; Decoding the encoded output data comprises: Inputting the encoded output data into a decoder to determine, by the decoder, based on a phonetic dictionary, a probability that a speech frame in the audio signal is pronounced as each word in the phonetic dictionary, wherein the phonetic dictionary includes multiple groups of words, each group of words includes different variants of the same word corresponding to multiple speech qualities, and different scores are assigned to different variants of the same word; Determining the score of the audio signal based on the decoding result includes: Determining, based on the determined probability, the word to which the speech frame in the audio signal actually corresponds; Determining a score corresponding to a speech frame based on a word actually corresponding to the speech frame in the audio signal; A score of the audio signal is determined based on the score corresponding to the speech frame.

11. An electronic device comprising a processor and a memory, wherein: The memory stores computer program instructions, which are used by the processor to execute the method for speech quality inspection according to any one of claims 1 to 9 when the processor runs the computer program instructions.

12. A storage medium having program instructions stored thereon, wherein the program instructions are used to execute the method for speech quality inspection according to any one of claims 1 to 9 when run.

Images