Audio comparison method, device and equipment, computer readable storage medium and product

Abstract

The invention provides an audio comparison method, device and equipment, a computer readable storage medium and a product. The method comprises the following steps: acquiring a first path of audio and a second path of audio; respectively sampling the first path of audio and the second path of audio to obtain a first frame signal set and a second frame signal set; according to the second frame signal set and a preset critical bandwidth, determining a spectrum envelope window of each frame signal in the second frame signal set; translating the spectrum envelope window of each frame of signal in the second frame signal set in the spectrum envelope window of the corresponding frame of signal in the first frame signal set to obtain the audio similarity between each frame of signal in the second frame signal set and the corresponding frame of signal in the first frame signal set; determining an audio comparison result of the first path of audio and the second path of audio according to the audio similarity; according to the embodiment of the invention, a tedious audio test task is changed into a full-process automatic comparison test, so that the test efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to an audio comparison method, device, equipment, computer-readable storage medium and product. Background Art

[0002] As speech synthesis technology becomes more and more mature, many applications on the market provide audiobook functions. As for how to check the audiobook function, or whether the audio output content is consistent with expectations; the existing testing methods mainly rely on manual operation, that is, relying on the tester to make subjective judgments on the audio signal. Specifically, the tester looks at the text, listens to the played audio, and manually judges whether the played content is consistent with the current text.

[0003] This kind of manual testing not only takes up a lot of time and costs, but also its reliability and accuracy cannot be guaranteed due to the existence of human factors. Especially in terms of audio freezes, voice and text synchronization, and multi-voice package difference evaluation, manual testing relies on subjective judgment and lacks scientific standard definitions, making it difficult to ensure the objectivity and consistency of test results. Summary of the invention

[0004] The purpose of the present application is to provide an audio comparison method, device, equipment, computer-readable storage medium and product to solve the problem in the prior art that the reliability and accuracy cannot be guaranteed by manually testing whether the audio output content is consistent with the text content.

[0005] In order to solve the above problem, the embodiment of the present application provides an audio comparison method, including:

[0006] Acquire a first audio channel and a second audio channel; the first audio channel is obtained by character recognition and audio encoding of a target text, and the second audio channel is obtained by audio playback of the target text;

[0007] The first audio channel and the second audio channel are sampled respectively to obtain a first frame signal set and a second frame signal set; the number of frame signals included in the first frame signal set is the same as the number of frame signals included in the second frame signal set;

[0008] Determining a spectrum envelope window of each frame signal in the second frame signal set according to the second frame signal set and a preset critical bandwidth;

[0009] Shifting the spectrum envelope window of each frame signal in the second frame signal set to the spectrum envelope window of the corresponding frame signal in the first frame signal set, respectively, to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set;

[0010] An audio comparison result of the first audio channel and the second audio channel is determined according to the audio similarity.

[0011] Wherein, determining the audio comparison result of the first audio channel and the second audio channel according to the audio similarity includes:

[0012] Comparing the audio similarity corresponding to each frame signal in the second frame signal set with the first threshold value respectively;

[0013] When the audio similarity corresponding to the first frame signal is greater than or equal to the first threshold, the first frame signal is determined to be a matching frame signal; or, when the audio similarity corresponding to the first frame signal is less than the first threshold, the first frame signal is determined to be an unmatched frame signal; wherein the first frame signal is any frame signal in the second frame signal set.

[0014] Wherein, the method further comprises:

[0015] According to the unmatched frame signal, an audio timestamp corresponding to the unmatched frame signal is determined, and / or a position of the unmatched frame signal in the target text is determined and marked.

[0016] The method of translating the spectrum envelope window of each frame signal in the second frame signal set to the spectrum envelope window of the corresponding frame signal in the first frame signal set to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set includes:

[0017] A first operation is performed on the spectrum envelope window of each frame signal in the second frame signal set, where the first operation includes:

[0018] translating the spectrum envelope window of a frame signal in the second frame signal set along the vertical axis in the spectrum envelope window of the corresponding frame signal in the first frame signal set; wherein the translation interval is from the minimum value to the maximum value of the vertical axis of the first frame signal set, and the translation step is an integer multiple of the difference between the minimum value and the maximum value of the vertical axis of the first frame signal set; the vertical axis is used to indicate the loudness of the frame signal;

[0019] Obtaining the audio similarity index of the center frequency of the critical frequency band obtained by each step of translation;

[0020] The maximum value of the audio similarity index is used as the audio similarity between a frame signal in the second frame signal set and a corresponding frame signal in the first frame signal set.

[0021] Before translating the spectrum envelope window of a frame signal in the second frame signal set along the vertical axis in the spectrum envelope window of the corresponding frame signal in the first frame signal set, the method further includes:

[0022] Performing a framing operation on the first frame signal set;

[0023] After performing feature extraction, windowing, and fast Fourier transform on each frame of signal obtained by framing, the spectrum envelope of each frame of signal is obtained;

[0024] According to the spectrum envelope of each frame signal, a spectrum envelope window of each frame signal of the first frame signal set is determined.

[0025] The step of determining the spectrum envelope window of each frame signal in the second frame signal set according to the second frame signal set and a preset critical bandwidth includes:

[0026] performing a framing operation on the second frame signal set;

[0027] After performing feature extraction, windowing, and fast Fourier transformation on each frame of signal obtained by framing, the spectrum components of each frame of signal and the spectrum envelope of each frame of signal are obtained;

[0028] According to a preset critical bandwidth, the frequency domain range is divided into a plurality of critical frequency bands;

[0029] According to the values ​​of the central frequencies of the spectrum components of each frame signal in the critical frequency band, a bandpass threshold sequence is obtained;

[0030] The upper limit and the lower limit are respectively set with each value in the bandpass threshold sequence as the center, and the spectrum envelope window is determined by using the obtained bandpass window sequence.

[0031] Wherein, before respectively sampling the first audio channel and the second audio channel to obtain a first frame signal set and a second frame signal set, the method includes:

[0032] Align the start and end time points of the second audio channel with the start and end time points of the first audio channel;

[0033] The first audio channel and the second audio channel are sampled respectively to obtain a first frame signal set and a second frame signal set, including:

[0034] The first audio channel and the second audio channel whose start and end time points are aligned are sampled for a preset number of sampling frames respectively to obtain a first frame signal set and a second frame signal set.

[0035] The present application also provides an audio comparison device, including:

[0036] An acquisition module, used to acquire a first audio channel and a second audio channel; the first audio channel is acquired by character recognition and audio encoding of a target text, and the second audio channel is acquired by audio playback of the target text;

[0037] A sampling module, used to sample the first audio channel and the second audio channel respectively to obtain a first frame signal set and a second frame signal set; the number of frame signals included in the first frame signal set is the same as the number of frame signals included in the second frame signal set;

[0038] A first determining module, configured to determine a spectrum envelope window of each frame signal in the second frame signal set according to the second frame signal set and a preset critical bandwidth;

[0039] A second determination module is used to translate the spectrum envelope window of each frame signal in the second frame signal set to the spectrum envelope window of the corresponding frame signal in the first frame signal set, so as to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set;

[0040] The third determination module is used to determine the audio comparison result of the first audio and the second audio according to the audio similarity.

[0041] An embodiment of the present application also provides an audio comparison device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor implements the audio comparison method as described above when executing the program.

[0042] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the audio comparison method as described above are implemented.

[0043] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the audio comparison method as described above.

[0044] The above technical solution of the present application has at least the following beneficial effects:

[0045] In the audio comparison method, apparatus, equipment, computer-readable storage medium and product of the embodiments of the present application, an automatic comparison is performed between a first audio channel obtained by text recognition and audio encoding of a target text and a second audio channel obtained by audio playback of the target text, and the audio comparison result is directly obtained based on audio similarity comparison, thereby changing the tedious audio testing task into a full-process automated comparison test, thereby improving the testing efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1A flowchart showing the steps of the audio comparison method provided in an embodiment of the present application;

[0047] Figure 2 One of the application example diagrams of the audio comparison method provided in the embodiment of the present application is shown;

[0048] Figure 3 A second diagram showing an application example of the audio comparison method provided in an embodiment of the present application;

[0049] Figure 4 FIG3 shows an application example of the audio comparison method provided in an embodiment of the present application;

[0050] Figure 5 A fourth diagram showing an application example of the audio comparison method provided in an embodiment of the present application;

[0051] Figure 6 Figure 5 shows an application example of the audio comparison method provided in an embodiment of the present application;

[0052] Figure 7 Figure 6 showing an application example of the audio comparison method provided in an embodiment of the present application;

[0053] Figure 8 FIG7 is a diagram showing an application example of the audio comparison method provided in an embodiment of the present application;

[0054] Fig. 9 FIG8 is an example of an application of the audio comparison method provided in an embodiment of the present application;

[0055] Fig.10 FIG9 is a ninth example of an application of the audio comparison method provided in an embodiment of the present application;

[0056] Fig.11 FIG10 is a diagram showing an application example of the audio comparison method provided in an embodiment of the present application;

[0057] Fig.12 A schematic diagram showing the structure of an audio comparison device provided in an embodiment of the present application;

[0058] Fig.13 A schematic diagram showing the structure of an audio comparison device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0060] like Figure 1 As shown, the embodiment of the present application also provides an audio comparison method, including:

[0061] Step 101, obtaining a first audio channel and a second audio channel; the first audio channel is obtained by character recognition and audio encoding of a target text, and the second audio channel is obtained by audio playback of the target text;

[0062] Step 102: Sample the first audio channel and the second audio channel respectively to obtain a first frame signal set and a second frame signal set; the first frame signal set includes the same number of frame signals as the second frame signal set;

[0063] Step 103, determining a spectrum envelope window of each frame signal in the second frame signal set according to the second frame signal set and a preset critical bandwidth;

[0064] Step 104: translating the spectrum envelope window of each frame signal in the second frame signal set to the spectrum envelope window of the corresponding frame signal in the first frame signal set, to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set;

[0065] Step 105: Determine an audio comparison result between the first audio channel and the second audio channel according to the audio similarity.

[0066] The embodiment of the present application automatically compares the first audio obtained by text recognition and audio encoding of the target text and the second audio obtained by audio playback of the target text, and directly obtains the audio comparison result based on audio similarity comparison, thereby changing the tedious audio testing task into a full-process automated comparison test, thereby improving the testing efficiency and accuracy.

[0067] For example, Figure 2 The automated testing device for the audio comparison method provided in the embodiment of the present application is shown, which is used to solve the problem of low efficiency of audio playback testing. The automated testing device includes: an automated testing module, a book source text module, a book source audio module, and a reader corpus module. In this example, obtaining the first audio and the second audio in step 101 specifically includes:

[0068] When an electronic device is connected to the above-mentioned automated testing device, and the application program (APP) to be tested is opened to enter the electronic book reading page to select audiobook playback, the automated testing module simultaneously collects the screen image of the electronic device and the currently playing audio signal.

[0069] The automated test module uses OCR (Optical Character Recognition) technology to recognize the e-book text in the screen image and transmits the recognized target text to the book source text module for processing. For example, referring to the normal speaking speed of 240 words per minute, that is, the time for each word is 250 milliseconds, and considering the reserved margin, the interval time of the screenshot is set to 200 milliseconds.

[0070] like Figure 3 As shown, a screenshot is taken every 200 milliseconds, and the text recognized in each screenshot (i.e., the e-book text of the current page) is compared with the previous text; if they are consistent, it is determined that the e-book reading page has not been turned, and the text data recognized in this screenshot is discarded; if they are inconsistent, it is determined that the e-book reading page has been turned, and data is transmitted until the end of the test.

[0071] The book source text module receives the e-book text sent by the automated test module, processes the received target text, including text normalization, punctuation processing, conversion of numbers and dates, etc., uses the language model to analyze the text, determines the most appropriate voice output, and translates each word, punctuation, and segment into the code in the corpus voice dictionary. For example, a comma corresponds to a blank voice (pause) of the length of one word, and then transmits it to the reader corpus module in sequence; then waits to receive the next page of e-book text from the automated test module until the test is completed. It should be noted that the last voice code after the test is completed is non-blank voice.

[0072] The reader corpus module synthesizes the audio files in sequence according to the received speech dictionary codes until the test is finished, generates the first audio from the target text, and sends it back to the automated test module; the first audio is characterized by word-by-word synthesis, and the waveform characteristics of the time domain are obvious; the waveform of the time domain is intercepted as follows Figure 4 shown.

[0073] The book source audio module receives the currently playing audio signal sent by the automated test module, performs noise reduction processing after the test reception, generates a second audio channel derived from the audio playback, and sends it to the automated test module. The second audio channel is characterized by recording generation, which is the real scene perceived by the user; the waveform of its time domain is intercepted as follows Figure 5 shown.

[0074] In at least one embodiment of the present application, before step 102, the method includes:

[0075] Align the start and end time points of the second audio channel with the start and end time points of the first audio channel;

[0076] Accordingly, step 102 includes:

[0077] The first audio channel and the second audio channel whose start and end time points are aligned are sampled for a preset number of sampling frames respectively to obtain a first frame signal set and a second frame signal set.

[0078] In the actual test (i.e., audio comparison test), it is impossible to start the test when a voice just starts, or to end the test when a voice just ends. Therefore, there will inevitably be blank parts in the second audio. Therefore, in order to improve the accuracy of audio comparison, it is necessary to align the time points of the second audio with the first audio.

[0079] Optionally, “aligning the start and end time points of the second audio channel with the start and end time points of the first audio channel” can be understood as: trimming the audio so that the two audio channels start at the same word and end at the same word. Figure 6 As shown, taking the start time points of two audio channels as an example, the implementation process is as follows:

[0080] (1) Sampling and quantizing the second audio channel: To digitize the two audio channels, the second audio channel is sampled and quantized first, with a sampling frequency of f2;

[0081] (2) Framing: In order to reduce the computational complexity, the densely sampled signal is divided into relatively sparse signal frames. The sampling point sequence of the second audio channel is framed, with a frame length of 25 milliseconds and a frame interval of 10 milliseconds.

[0082] (3) Calculate the linear relationship strength: Calculate the linear relationship strength between the first frame of the first audio channel and each frame of the second audio channel. Since the first audio channel is encoded and synthesized, the speech dictionary library in the reader corpus contains the frame features of all speech encodings corresponding to the speech. Take the first frame (FirstFrame) of the first speech encoding of the first audio channel and calculate the linear relationship strength with each frame of the second audio channel. For example, the linear relationship strength is calculated by the following formula:

[0083]

[0084] Among them, x i and i are the sample values ​​of the two audio signals at time i, μ x and μ y are the means of the two signals respectively, and n is the number of samples.

[0085] (4) Find the first frame that meets the threshold: Usually, there are multiple related frames that meet the threshold in the second audio channel. The first frame that meets the threshold is taken as the starting time point of the second audio channel;

[0086] (5) Data cropping before the frame, that is, discarding the sampling points before the first frame that meets the threshold;

[0087] (6) The starting time points are aligned, that is, after cropping, the two audio channels start at the same frame.

[0088] In the same way, according to the linear relationship strength between the last frame (LastFrame) of the last speech encoding of the first audio channel and each frame of the second audio channel, the last frame that meets the threshold is obtained as the end time point of the second audio channel, and the data after the last frame that meets the threshold is cropped, so as to align the end time points of the two audio channels.

[0089] Continuing with the above example, after the start and end time points of the first audio channel are aligned with the start and end time points of the second audio channel, the number of sampling points of the two audio channels also needs to be aligned, that is, the two audio channels are resampled at different frequencies to make the total number of sampling points consistent; in other words, the number of frame signals contained in the first frame signal set is the same as the number of frame signals contained in the second frame signal set, for example, the number of frame signals is the above-mentioned preset sampling frame number.

[0090] For example, set the total duration of the first audio channel to t 1 , the total duration of the second audio after alignment is t 2 , alignment factor N = round(t 1 / t 2 ) 。 Since the first audio is not connected and there are pauses (blanks) with punctuation marks, generally speaking, t 1 >t 2， It can be ensured that N>1. Then, the first audio channel is sampled and quantized to obtain a sampling sequence S1 (i.e., the first frame signal set), whose sampling frequency is F and the number of sampling points is Ft 1 ; Sampling and quantizing the second audio channel to obtain a sampling sequence S2 (ie, a second frame signal set), whose sampling frequency is NF and the number of sampling points is NFt 2 Therefore, the sampling sequences of the two audio channels are S1 and S2 respectively, and the total number of sampling points of the two sequences is equal, that is, Ft 1 =NFt 2 .

[0091] In at least one embodiment of the present application, step 105 includes:

[0092] Comparing the audio similarity corresponding to each frame signal in the second frame signal set with the first threshold value respectively;

[0093] When the audio similarity corresponding to the first frame signal is greater than or equal to the first threshold, the first frame signal is determined to be a matching frame signal; or, when the audio similarity corresponding to the first frame signal is less than the first threshold, the first frame signal is determined to be an unmatched frame signal; wherein the first frame signal is any frame signal in the second frame signal set.

[0094] Accordingly, the method further includes:

[0095] According to the unmatched frame signal, an audio timestamp corresponding to the unmatched frame signal is determined, and / or a position of the unmatched frame signal in the target text is determined and marked.

[0096] In at least one embodiment of the present application, step 104 includes:

[0097] A first operation is performed on the spectrum envelope window of each frame signal in the second frame signal set, where the first operation includes:

[0098] translating the spectrum envelope window of a frame signal in the second frame signal set along the vertical axis in the spectrum envelope window of the corresponding frame signal in the first frame signal set; wherein the translation interval is from the minimum value to the maximum value of the vertical axis of the first frame signal set, and the translation step is an integer multiple of the difference between the minimum value and the maximum value of the vertical axis of the first frame signal set; the vertical axis is used to indicate the loudness of the frame signal;

[0099] Obtaining the audio similarity index of the center frequency of the critical frequency band obtained by each step of translation;

[0100] The maximum value of the audio similarity index is used as the audio similarity between a frame signal in the second frame signal set and a corresponding frame signal in the first frame signal set.

[0101] Accordingly, before step 104, the method further includes:

[0102] Performing a framing operation on the first frame signal set;

[0103] After performing feature extraction, windowing, and fast Fourier transform on each frame of signal obtained by framing, the spectrum envelope of each frame of signal is obtained;

[0104] According to the spectrum envelope of each frame signal, a spectrum envelope window of each frame signal of the first frame signal set is determined.

[0105] In at least one embodiment of the present application, step 103 includes:

[0106] performing a framing operation on the second frame signal set;

[0107] After performing feature extraction, windowing, and fast Fourier transformation on each frame of signal obtained by framing, the spectrum components of each frame of signal and the spectrum envelope of each frame of signal are obtained;

[0108] According to a preset critical bandwidth, the frequency domain range is divided into a plurality of critical frequency bands;

[0109] According to the values ​​of the central frequencies of the spectrum components of each frame signal in the critical frequency band, a bandpass threshold sequence is obtained;

[0110] The upper limit and the lower limit are respectively set with each value in the bandpass threshold sequence as the center, and the spectrum envelope window is determined by using the obtained bandpass window sequence.

[0111] The embodiment of the present application converts two channels of audio to the frequency domain for feature extraction to obtain a corresponding spectrum envelope, and intuitively displays problems such as playback freezes and voice and text asynchrony through the spectrum envelope.

[0112] For example, Figure 7 As shown, the embodiment of the present application converts the two-channel audio to the frequency domain for feature extraction (spectrum envelope window), and uses the spectrum envelope window translation method to compare whether the two-channel audio is similar. The implementation process is as follows:

[0113] (1) Two-channel audio framing: The first frame signal set S1 and the second frame signal set S2 are framed, for example, using a frame length of 25 milliseconds and a frame interval of 10 milliseconds.

[0114] (2) Perform frame feature extraction. Perform feature extraction on each frame signal. Since it is usually difficult to see the characteristics of the signal from the transformation of the frame signal in the time domain, it is usually converted into energy distribution in the frequency domain for observation. Different energy distributions can represent the characteristics of different speech. Therefore, after multiplying by the Hamming window to reduce sidelobe leakage, each frame signal is subjected to a fast Fourier transform (FFT) to obtain the energy distribution on the spectrum. For example, the spectrum components of each frame signal can be obtained by the following formula:

[0115]

[0116] Among them, x(n) is the input frame signal, N represents the number of sampling signal points, and K represents the index in the spectrum (different frequency components).

[0117] (3) After FFT, the frame spectrum envelopes of the first frame signal set S1 and the second frame signal set S2 are obtained respectively. For example, the frame spectrum envelope of a frame signal is as follows: Figure 8 shown.

[0118] (4) Divide critical frequency bands. In the frequency domain, critical frequency bands are divided based on the sensitivity of the human ear to sounds of different frequencies and the masking effect. In this system, 24 critical frequency bands are divided in the frequency range of 20 Hz to 16 kHz. The 24 critical frequency bands are distributed in the frequency range of 20 Hz to 16 kHz, with center frequencies ranging from 50 Hz to 13.5 kHz, and include commonly used measurement frequencies such as 250 Hz, 1 kHz, and 4 kHz. The specific division method is as follows:

[0119] When the center frequency is less than 500 Hz, the preset critical bandwidth is about 50 Hz.

[0120] When the center frequency is greater than 500 Hz, the preset critical bandwidth is approximately 20% of the center frequency.

[0121] (5) Extract the spectrum components of each frame signal in the second frame signal set S2, and the values ​​of each center frequency in the critical bandwidth to obtain the passband threshold sequence B(fn) of each center frequency.

[0122] (6) For the center frequency value of the sequence B(fn) in each critical frequency band, an upper and lower floating range is set to generate a bandpass envelope frame W(fn), that is, a spectrum envelope window, which can also be called a bandpass window sequence W(fn), for example, Fig. 9 As shown, Fig. 9 The horizontal axis represents frequency Hz, and the vertical axis represents loudness db.

[0123] (7) The passband window sequence W(fn) is shifted along the vertical axis in the same frame window of the first frame signal set S1. The shift interval is from the minimum value to the maximum value of the vertical axis of S1. The step length is an integer multiple of the difference between the minimum value and the maximum value. Fig.10 As shown, Fig.10 The horizontal axis represents frequency Hz, and the vertical axis represents loudness db.

[0124] (8) Each step of translation can obtain a set of audio similarity indexes J at the critical center frequency point. After the translation is completed, the maximum value of the similarity index J is taken as the audio similarity Jmax of the frame. For example, the calculation formula of the similarity index J is:

[0125]

[0126] Among them, i represents the critical frequency band, and its value range is 1 to 24; x i is the loudness value on critical frequency band i;

[0127] bi _上 is the maximum value of the bandpass window sequence W(i) on critical band i, b i_下 is the minimum value of the bandpass window sequence W(i) on critical band i.

[0128] It should be noted that (b i_上 -x i ) is negative and is judged as 0; (x i -b i_下 ) is negative, it is judged as 0.

[0129] (9) Compare Jmax with the preset audio similarity decision threshold J;

[0130] (10) When Jmax>=J, it is determined that the audio feature of S1 is similar to S2, that is, the current frame signal is a matching frame signal; otherwise, Jmax<J, it is determined that the audio feature of S1 is not similar to S2, that is, the current frame signal is an unmatched frame signal.

[0131] Furthermore, in the embodiment of the present application, frames that fail the audio similarity test (i.e., mismatched frame signals) can be marked, and the timestamp corresponding to the audio of the mismatched content can be traced back according to the corresponding sampling points to mark the error points; the effect diagram is shown in FIG. Fig.11 shown.

[0132] In summary, in the embodiments of the present application, the first audio obtained by text recognition and audio encoding of the target text and the second audio obtained by audio playback of the target text are automatically compared, and the audio comparison result is directly obtained based on the audio similarity comparison, thereby changing the cumbersome audio testing task into a full-process automated comparison test, and being able to trace and quickly locate mismatched content, thereby improving test efficiency and accuracy.

[0133] like Fig.12 As shown, the embodiment of the present application also provides an audio comparison device, including:

[0134] The acquisition module 1201 is used to acquire a first audio channel and a second audio channel; the first audio channel is acquired by character recognition and audio encoding of a target text, and the second audio channel is acquired by audio playback of the target text;

[0135] The sampling module 1202 is used to sample the first audio channel and the second audio channel respectively to obtain a first frame signal set and a second frame signal set; the number of frame signals included in the first frame signal set is the same as the number of frame signals included in the second frame signal set;

[0136] A first determining module 1203 is used to determine the spectrum envelope window of each frame signal in the second frame signal set according to the second frame signal set and a preset critical bandwidth;

[0137] The second determination module 1204 is used to translate the spectrum envelope window of each frame signal in the second frame signal set to the spectrum envelope window of the corresponding frame signal in the first frame signal set, so as to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set;

[0138] The third determination module 1205 is used to determine the audio comparison result of the first audio channel and the second audio channel according to the audio similarity.

[0139] As an optional embodiment, the third determining module includes:

[0140] A comparison submodule, used for comparing the audio similarity corresponding to each frame signal in the second frame signal set with the first threshold value respectively;

[0141] The first determination submodule is used to determine that the first frame signal is a matching frame signal when the audio similarity corresponding to the first frame signal is greater than or equal to the first threshold; or to determine that the first frame signal is an unmatched frame signal when the audio similarity corresponding to the first frame signal is less than the first threshold; wherein the first frame signal is any frame signal in the second frame signal set.

[0142] As an optional embodiment, the device further includes:

[0143] The fourth determination module is used to determine the audio timestamp corresponding to the non-matching frame signal according to the non-matching frame signal, and / or determine the position of the non-matching frame signal in the target text and mark it.

[0144] As an optional embodiment, the second determining module includes:

[0145] The second determining submodule is configured to perform a first operation on the spectrum envelope window of each frame signal in the second frame signal set, wherein the first operation includes:

[0146] translating the spectrum envelope window of a frame signal in the second frame signal set along the vertical axis in the spectrum envelope window of the corresponding frame signal in the first frame signal set; wherein the translation interval is from the minimum value to the maximum value of the vertical axis of the first frame signal set, and the translation step is an integer multiple of the difference between the minimum value and the maximum value of the vertical axis of the first frame signal set; the vertical axis is used to indicate the loudness of the frame signal;

[0147] Obtaining the audio similarity index of the center frequency of the critical frequency band obtained by each step of translation;

[0148] The maximum value of the audio similarity index is used as the audio similarity between a frame signal in the second frame signal set and a corresponding frame signal in the first frame signal set.

[0149] As an optional embodiment, the device further includes:

[0150] A first operation module, configured to perform a frame operation on the first frame signal set;

[0151] The first processing module is used to extract features, add windows, and perform fast Fourier transform on each frame of the signal obtained by framing, so as to obtain the spectrum envelope of each frame of the signal;

[0152] The fifth determination module is used to determine the spectrum envelope window of each frame signal of the first frame signal set according to the spectrum envelope of each frame signal.

[0153] As an optional embodiment, the first determining module includes:

[0154] A second operation submodule, used for performing a frame operation on the second frame signal set;

[0155] The second processing submodule is used to extract features, add windows, and perform fast Fourier transform on each frame of the signal obtained by framing, so as to obtain the spectrum components of each frame of the signal and the spectrum envelope of each frame of the signal;

[0156] A division submodule, used for dividing the frequency domain range into a plurality of critical frequency bands according to a preset critical bandwidth;

[0157] The fourth determination submodule is used to obtain a bandpass threshold sequence according to the values ​​of the central frequencies of the spectrum components of each frame signal in the critical frequency band;

[0158] The fifth determination submodule is used to set an upper limit value and a lower limit value respectively with each value in the bandpass threshold sequence as the center, and determine the spectrum envelope window using the obtained bandpass window sequence.

[0159] As an optional embodiment, the device includes:

[0160] An alignment module, used to align the start and end time points of the second audio channel with the start and end time points of the first audio channel;

[0161] The sampling module comprises:

[0162] The sampling submodule is used to sample the first audio channel and the second audio channel after the start and end time points are aligned, respectively, by a preset number of sampling frames, to obtain a first frame signal set and a second frame signal set.

[0163] In an embodiment of the present application, an automatic comparison is performed between a first audio channel obtained by text recognition and audio encoding of a target text and a second audio channel obtained by audio playback of the target text, and an audio comparison result is directly obtained based on an audio similarity comparison, thereby converting the cumbersome audio testing task into a full-process automated comparison test, and being able to trace and quickly locate mismatched content, thereby improving test efficiency and accuracy.

[0164] It should be noted that the audio comparison device provided in the embodiment of the present application is a device capable of executing the above-mentioned audio comparison method. All embodiments of the above-mentioned audio comparison method are applicable to the device and can achieve the same or similar beneficial effects, and no repeated limitations are made here.

[0165] like Fig.13 As shown, an embodiment of the present application also provides an audio comparison device, including a memory 1310, a processor 1300, and a computer program stored on the memory 1310 and executable on the processor 1300. When the processor 1300 executes the program, each process in the audio comparison method embodiment as described above is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0166] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, each process in the audio comparison method embodiment described above is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0167] The embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the audio comparison method embodiment described above are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0168] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0169] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A device that specifies functions in one or more processes and / or one or more blocks.

[0170] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable storage medium produce a paper product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that the computer or other programmable device executes a series of operating steps to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0172] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An audio comparison method, characterized in that: include: Acquire a first audio channel and a second audio channel; the first audio channel is obtained by character recognition and audio encoding of a target text, and the second audio channel is obtained by audio playback of the target text; The first audio channel and the second audio channel are sampled respectively to obtain a first frame signal set and a second frame signal set; the number of frame signals included in the first frame signal set is the same as the number of frame signals included in the second frame signal set; Determining a spectrum envelope window of each frame signal in the second frame signal set according to the second frame signal set and a preset critical bandwidth; Shifting the spectrum envelope window of each frame signal in the second frame signal set to the spectrum envelope window of the corresponding frame signal in the first frame signal set, respectively, to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set; An audio comparison result of the first audio channel and the second audio channel is determined according to the audio similarity.

2. The method according to claim 1, characterized in that Determining, according to the audio similarity, an audio comparison result of the first audio channel and the second audio channel, including: Comparing the audio similarity corresponding to each frame signal in the second frame signal set with the first threshold value respectively; When the audio similarity corresponding to the first frame signal is greater than or equal to the first threshold, the first frame signal is determined to be a matching frame signal; or, when the audio similarity corresponding to the first frame signal is less than the first threshold, the first frame signal is determined to be an unmatched frame signal; wherein the first frame signal is any frame signal in the second frame signal set.

3. The method according to claim 2, characterized in that The method further comprises: According to the unmatched frame signal, an audio timestamp corresponding to the unmatched frame signal is determined, and / or a position of the unmatched frame signal in the target text is determined and marked.

4. The method according to claim 1, characterized in that The frequency spectrum envelope window of each frame signal in the second frame signal set is respectively translated in the frequency spectrum envelope window of the corresponding frame signal in the first frame signal set to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set, including: Performing a first operation on the spectrum envelope window of each frame signal in the second frame signal set, respectively, the first operation comprising: translating the spectrum envelope window of a frame signal in the second frame signal set along the vertical axis in the spectrum envelope window of the corresponding frame signal in the first frame signal set; wherein the translation interval is from the minimum value to the maximum value of the vertical axis of the first frame signal set, and the translation step is an integer multiple of the difference between the minimum value and the maximum value of the vertical axis of the first frame signal set; the vertical axis is used to indicate the loudness of the frame signal; Obtaining the audio similarity index of the center frequency of the critical frequency band obtained by each step of translation; The maximum value of the audio similarity index is used as the audio similarity between a frame signal in the second frame signal set and a corresponding frame signal in the first frame signal set.

5. The method according to claim 1 or 4, characterized in that: Before translating the spectrum envelope window of a frame signal in the second frame signal set along the vertical axis in the spectrum envelope window of the corresponding frame signal in the first frame signal set, the method further includes: Performing a framing operation on the first frame signal set; After performing feature extraction, windowing, and fast Fourier transform on each frame of signal obtained by framing, the spectrum envelope of each frame of signal is obtained; According to the spectrum envelope of each frame signal, a spectrum envelope window of each frame signal of the first frame signal set is determined.

6. The method according to claim 1, characterized in that The determining, according to the second frame signal set and the preset critical bandwidth, the spectrum envelope window of each frame signal in the second frame signal set includes: performing a framing operation on the second frame signal set; After feature extraction, windowing, and fast Fourier transformation are performed on each frame of the signal obtained by framing, the spectrum components of each frame of the signal and the spectrum envelope of each frame of the signal are obtained; According to a preset critical bandwidth, the frequency domain range is divided into a plurality of critical frequency bands; According to the values ​​of the central frequencies of the spectrum components of each frame signal in the critical frequency band, a bandpass threshold sequence is obtained; The upper limit and the lower limit are respectively set with each value in the bandpass threshold sequence as the center, and the spectrum envelope window is determined by using the obtained bandpass window sequence.

7. The method according to any one of claims 1 to 6, characterized in that: Before respectively sampling the first audio channel and the second audio channel to obtain a first frame signal set and a second frame signal set, the method includes: Align the start and end time points of the second audio channel with the start and end time points of the first audio channel; The first audio channel and the second audio channel are sampled respectively to obtain a first frame signal set and a second frame signal set, including: The first audio channel and the second audio channel whose start and end time points are aligned are sampled for a preset number of sampling frames respectively to obtain a first frame signal set and a second frame signal set.

8. An audio comparison device, characterized in that: include: An acquisition module, used to acquire a first audio channel and a second audio channel; the first audio channel is acquired by character recognition and audio encoding of a target text, and the second audio channel is acquired by audio playback of the target text; A sampling module, used to sample the first audio channel and the second audio channel respectively to obtain a first frame signal set and a second frame signal set; the number of frame signals included in the first frame signal set is the same as the number of frame signals included in the second frame signal set; A first determining module, configured to determine a spectrum envelope window of each frame signal in the second frame signal set according to the second frame signal set and a preset critical bandwidth; A second determination module is used to translate the spectrum envelope window of each frame signal in the second frame signal set to the spectrum envelope window of the corresponding frame signal in the first frame signal set, so as to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set; The third determination module is used to determine the audio comparison result of the first audio and the second audio according to the audio similarity.

9. An audio comparison device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the audio comparison method as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the audio comparison method as described in any one of claims 1 to 7 are implemented.

11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the audio comparison method according to any one of claims 1 to 7.

Images