Audio comparison methods, apparatus, devices, computer-readable storage media and products

Abstract

This application provides an audio comparison method, apparatus, device, computer-readable storage medium, and product. The method includes: acquiring a first audio channel and a second audio channel; sampling the first and second audio channels respectively to obtain a first frame signal set and a second frame signal set; determining the spectral envelope window of each frame signal in the second frame signal set based on the second frame signal set and a preset critical bandwidth; shifting the spectral envelope window of each frame signal in the second frame signal set to the spectral envelope window of the corresponding frame signal in the first frame signal set to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set; and determining the audio comparison result of the first and second audio channels based on the audio similarity. This application's embodiments transform the tedious audio testing task into a fully automated comparison test, improving testing efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an audio comparison method, apparatus, device, computer-readable storage medium, and product. Background Technology

[0002] With the increasing maturity of speech synthesis technology, many applications on the market offer audiobook functions. However, existing testing methods mainly rely on manual operation to check these functions or whether the audio output content matches expectations. This means that testers subjectively judge the audio signal, specifically by looking at the text, listening to the audio, and manually judging whether the content played matches the current text.

[0003] Such manual testing is not only time-consuming and costly, but also often lacks reliability and accuracy due to human factors. Especially in areas such as audio stuttering, speech-text synchronization, and multi-speech packet difference assessment, manual testing relies on subjective judgment and lacks scientific standard definitions, making it difficult to ensure the objectivity and consistency of test results. Summary of the Invention

[0004] The purpose of this application is to provide an audio comparison method, apparatus, device, computer-readable storage medium, and product to solve the problem that the existing technology of manually testing whether the audio output content is consistent with the text content cannot guarantee reliability and accuracy.

[0005] To address the aforementioned problems, embodiments of this application provide an audio comparison method, including:

[0006] Acquire the first audio stream and the second audio stream; the first audio stream is obtained through text recognition and audio encoding of the target text, and the second audio stream is obtained by playing the audio of the target text.

[0007] The first audio channel and the second audio channel are sampled respectively to obtain a first frame signal set and a second frame signal set; the first frame signal set contains the same number of frame signals as the second frame signal set.

[0008] Based on the second frame signal set and the preset critical bandwidth, determine the spectral envelope window of each frame signal in the second frame signal set;

[0009] The spectral envelope window of each frame signal in the second frame signal set is translated into the spectral envelope window of the corresponding frame signal in the first frame signal set to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set.

[0010] Based on the audio similarity, the audio comparison results of the first audio stream and the second audio stream are determined.

[0011] The audio comparison result between the first audio stream and the second audio stream is determined based on the audio similarity, including:

[0012] The audio similarity of each frame signal in the second frame signal set is compared with the first threshold.

[0013] If the audio similarity corresponding to the first frame signal is greater than or equal to the first threshold, the first frame signal is determined to be a matching frame signal; or, if the audio similarity corresponding to the first frame signal is less than the first threshold, the first frame signal is determined to be a non-matching frame signal; wherein, the first frame signal is any frame signal in the second set of frame signals.

[0014] The method further includes:

[0015] Based on the mismatched frame signal, determine the audio timestamp corresponding to the mismatched frame signal, and / or determine and mark the position of the mismatched frame signal in the target text.

[0016] Specifically, the spectral envelope window of each frame signal in the second frame signal set is shifted relative to the spectral envelope window of the corresponding frame signal in the first frame signal set to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set, including:

[0017] For each frame signal in the second set of signal frames, a first operation is performed, the first operation including:

[0018] The spectral envelope window of one frame signal in the second frame signal set is shifted along the vertical axis within the spectral envelope window of the corresponding frame signal in the first frame signal set; wherein the shift interval is from the minimum to the maximum value of the vertical axis of the first frame signal set, and the shift step size is an integer multiple of the difference between the minimum and maximum values ​​of the vertical axis of the first frame signal set; the vertical axis is used to indicate the loudness of the frame signal;

[0019] Obtain the audio similarity index at the center frequency of the critical frequency band obtained from each translation step;

[0020] The maximum value of the audio similarity index is used as the audio similarity between a frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set.

[0021] The method further includes, before shifting the spectral envelope window of one frame signal in the second frame signal set along the vertical axis within the spectral envelope window of the corresponding frame signal in the first frame signal set:

[0022] Perform a frame division operation on the first frame signal set;

[0023] After feature extraction, windowing, and fast Fourier transform of each frame of signal obtained by framing, the spectral envelope of each frame of signal is obtained.

[0024] Based on the spectral envelope of each frame of the signal, determine the spectral envelope window of each frame of the first frame signal set.

[0025] The step of determining the spectral envelope window of each frame signal in the second frame signal set based on the second frame signal set and the preset critical bandwidth includes:

[0026] Perform a frame division operation on the second frame signal set;

[0027] After feature extraction, windowing, and fast Fourier transform of each frame of signal obtained by framing, the spectral components and spectral envelope of each frame of signal are obtained.

[0028] Based on the preset critical bandwidth, the frequency domain is divided into multiple critical frequency bands;

[0029] Based on the values ​​of the spectral components of each frame of signal at each center frequency in the critical band, a bandpass threshold sequence is obtained;

[0030] The upper and lower limits are set centered on each value in the bandpass threshold sequence, and the obtained bandpass window sequence is used to determine the spectral envelope window.

[0031] Before sampling the first audio stream and the second audio stream to obtain the first frame signal set and the second frame signal set, the method includes:

[0032] Align the start and end times of the second audio stream with the start and end times of the first audio stream;

[0033] The first audio stream and the second audio stream are sampled respectively to obtain a first frame signal set and a second frame signal set, including:

[0034] The first and second audio channels, after being aligned with the start and end times, are sampled for a preset number of frames to obtain the first frame signal set and the second frame signal set.

[0035] This application also provides an audio comparison device, including:

[0036] The acquisition module is used to acquire a first audio source and a second audio source; the first audio source is obtained by character recognition and audio encoding of the target text, and the second audio source is obtained by playing the audio of the target text.

[0037] The sampling module is used to sample the first audio channel and the second audio channel respectively to obtain a first frame signal set and a second frame signal set; the number of frame signals contained in the first frame signal set is the same as the number of frame signals contained in the second frame signal set;

[0038] The first determining module is used to determine the spectral envelope window of each frame signal in the second frame signal set based on the second frame signal set and the preset critical bandwidth.

[0039] The second determining module is used to translate the spectral envelope window of each frame signal in the second frame signal set into the spectral envelope window of the corresponding frame signal in the first frame signal set, respectively, to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set.

[0040] The third determining module is used to determine the audio comparison result between the first audio channel and the second audio channel based on the audio similarity.

[0041] This application also provides an audio comparison device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the audio comparison method described above.

[0042] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the audio comparison method described above.

[0043] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the audio comparison method described above.

[0044] The above-mentioned technical solution of this application has at least the following beneficial effects:

[0045] In the audio comparison method, apparatus, device, computer-readable storage medium, and product of this application embodiment, the automatic comparison is performed based on the first audio obtained by character recognition and audio encoding of the target text and the second audio obtained by playing the audio of the target text, and the audio comparison result is directly obtained based on the audio similarity comparison, thereby changing the tedious audio testing task into a fully automated comparison test, improving the testing efficiency and accuracy. Attached Figure Description

[0046] Figure 1This is a flowchart illustrating the steps of the audio comparison method provided in the embodiments of this application;

[0047] Figure 2 This diagram illustrates one application example of the audio comparison method provided in the embodiments of this application.

[0048] Figure 3 This is the second example diagram illustrating the application of the audio comparison method provided in the embodiments of this application;

[0049] Figure 4 Figure 3 illustrates an application example of the audio comparison method provided in this application embodiment;

[0050] Figure 5 Figure 4 illustrates an application example of the audio comparison method provided in this application embodiment;

[0051] Figure 6 Figure 5 illustrates an application example of the audio comparison method provided in this application embodiment;

[0052] Figure 7 Figure 6 illustrates an application example of the audio comparison method provided in this application embodiment;

[0053] Figure 8 Figure 7 illustrates an application example of the audio comparison method provided in this application.

[0054] Figure 9 Figure 8 illustrates an application example of the audio comparison method provided in this application embodiment;

[0055] Figure 10 Figure 9 illustrates an application example of the audio comparison method provided in this application embodiment;

[0056] Figure 11 Figure 10 illustrates an application example of the audio comparison method provided in this application embodiment;

[0057] Figure 12 This is a schematic diagram of the structure of the audio comparison device provided in the embodiments of this application;

[0058] Figure 13 This is a schematic diagram showing the structure of the audio comparison device provided in the embodiments of this application. Detailed Implementation

[0059] To make the technical problems, technical solutions and advantages of this application clearer, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments.

[0060] like Figure 1 As shown in the embodiments of this application, an audio comparison method is also provided, including:

[0061] Step 101: Obtain the first audio channel and the second audio channel; the first audio channel is obtained by character recognition and audio encoding of the target text, and the second audio channel is obtained by playing the audio of the target text.

[0062] Step 102: Sample the first audio channel and the second audio channel respectively to obtain a first frame signal set and a second frame signal set; the first frame signal set contains the same number of frame signals as the second frame signal set.

[0063] Step 103: Determine the spectral envelope window of each frame signal in the second frame signal set based on the second frame signal set and the preset critical bandwidth;

[0064] Step 104: Shift the spectral envelope window of each frame signal in the second frame signal set to the spectral envelope window of the corresponding frame signal in the first frame signal set to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set.

[0065] Step 105: Determine the audio comparison result between the first audio stream and the second audio stream based on the audio similarity.

[0066] This application embodiment automatically compares the first audio channel obtained by character recognition and audio encoding of the target text with the second audio channel obtained by playing the audio of the target text, and directly obtains the audio comparison result based on the audio similarity comparison, thereby changing the tedious audio testing task into a fully automated comparison test, improving testing efficiency and accuracy.

[0067] For example, such as Figure 2 The diagram shows an automated testing device for the audio comparison method provided in this application embodiment, used to solve the problem of low efficiency in audio playback testing. The automated testing device includes: an automated testing module, a book source text module, a book source audio module, and a reader corpus module. In this example, step 101, obtaining the first and second audio streams, specifically includes:

[0068] When an electronic device is connected to the aforementioned automated testing device and the application (APP) to be tested is opened and the e-book reading page is accessed to select audiobook playback, the automated testing module simultaneously acquires the screen image of the electronic device and the currently playing audio signal.

[0069] The automated testing module uses OCR (Optical Character Recognition) technology to identify e-book text in screen images and transmits the identified target text to the book source text module for processing. For example, based on a normal speaking speed of 240 words per minute, which means each word takes 250 milliseconds, and considering a margin for error, the interval for taking screenshots is set to 200 milliseconds.

[0070] like Figure 3 As shown, a screenshot is taken every 200 milliseconds. The text recognized in each screenshot (i.e., the text of the e-book on the current page) is compared with the text of the previous screenshot. If they match, it is determined that the e-book reading page has not been turned, and the text data recognized in this screenshot is discarded. If they do not match, it is determined that the e-book reading page has been turned, and the data is transmitted until the test ends.

[0071] The book source text module receives the ebook text from the automated testing module. It processes the received target text, including text normalization, punctuation handling, and conversion of numbers and dates. It uses a language model for text analysis to determine the most suitable speech output, translating each character, punctuation mark, and segment into an encoding from the corpus's speech dictionary. For example, a comma corresponds to a pause of one character length. This is then sequentially transmitted to the reader corpus module. The module then waits to receive the next page of ebook text from the automated testing module until the test ends. It's important to note that the last speech encoding after the test is non-blank.

[0072] The reader corpus module synthesizes audio files sequentially according to the received speech dictionary encodings until the test ends, generating the first audio stream derived from the target text, which is then sent back to the automated testing module. The first audio stream is characterized by word-by-word synthesis and distinct time-domain waveform features. A portion of the time-domain waveform is shown below. Figure 4 As shown.

[0073] The audio source module receives the currently playing audio signal from the automated testing module. After receiving the signal, it performs noise reduction processing to generate a second audio stream originating from the audio playback, which is then sent to the automated testing module. This second audio stream is characterized by being a recorded audio stream, representing a real-world scenario as perceived by the user; its time-domain waveform is shown in the image below. Figure 5 As shown.

[0074] In at least one embodiment of this application, prior to step 102, the method includes:

[0075] Align the start and end times of the second audio stream with the start and end times of the first audio stream;

[0076] Accordingly, step 102 includes:

[0077] The first and second audio channels, after being aligned with the start and end times, are sampled for a preset number of frames to obtain the first frame signal set and the second frame signal set.

[0078] Since in actual testing (i.e., audio comparison testing), it is impossible to start testing at the very beginning of a voice recording or end testing immediately after a voice recording ends, there will inevitably be blank parts in the second audio recording. Therefore, in order to improve the accuracy of audio comparison, it is necessary to align the second audio recording with the first audio recording in terms of time points.

[0079] Optionally, "aligning the start and end times of the second audio stream with the start and end times of the first audio stream" can be understood as: trimming the audio so that both audio streams begin and end with the same word. For example, as... Figure 6 As shown, taking the alignment of the start times of two audio streams as an example, the implementation process is as follows:

[0080] (1) Second audio sampling quantization: Digitize the two audio channels, first sample and quantize the second audio channel, with a sampling frequency of f2;

[0081] (2) Framing: In order to reduce the computational complexity, the dense sampled signal is divided into sparser signal frames. The sampling point sequence of the second audio channel is framed with a frame length of 25 milliseconds and a frame interval of 10 milliseconds.

[0082] (3) Calculate the linear relationship strength: Calculate the linear relationship strength between the first frame of the first audio stream and each frame of the second audio stream. Since the first audio stream is encoded and synthesized, the speech dictionary in the reader's corpus contains the frame features of all speech codes corresponding to the speech. Take the first frame of the first speech code of the first audio stream and calculate the linear relationship strength with each frame of the second audio stream. The linear relationship strength is calculated using the following formula:

[0083]

[0084] Where, x i and y i These are the sample values ​​of the two audio signals at time i, μ x and μ y These are the means of the two signals, and n is the number of samples.

[0085] (4) Find the first frame that meets the threshold: Usually, there are multiple related frames that meet the threshold in the second audio path. Take the first frame that meets the threshold as the start time point of the second audio path.

[0086] (5) Data pruning before this frame, i.e. discarding the sampling points before the first frame that meets the threshold;

[0087] (6) Start time alignment, that is, after trimming, both audio streams start at the same frame.

[0088] Similarly, based on the linear relationship strength between the last frame of the last speech encoding of the first audio stream and each frame of the second audio stream, the last frame that meets the threshold is obtained as the end time point of the second audio stream, and the data after the last frame that meets the threshold is clipped, thereby aligning the end time points of the two audio streams.

[0089] Following the previous example, after aligning the start and end times of the first audio path and the second audio path, it is also necessary to align the number of sampling points of the two audio paths. That is, the two audio paths are sampled twice at different frequencies to make the total number of sampling points consistent. In other words, the number of frame signals contained in the first frame signal set is the same as the number of frame signals contained in the second frame signal set. For example, the number of frame signals is the preset number of sampling frames mentioned above.

[0090] For example, let the total duration of the first audio stream be t1, and the total duration of the aligned second audio stream be t2. The alignment coefficient N = round(t1 / t2). 。 Since the first audio stream has no connected speech and includes pauses (blanks) with punctuation marks, generally t1 > t2. 2， It can be guaranteed that N>1. Then, the first audio stream is sampled and quantized to obtain the sampled sequence S1 (i.e., the first frame signal set), with a sampling frequency of F and a number of sampling points of Ft1; the second audio stream is sampled and quantized to obtain the sampled sequence S2 (i.e., the second frame signal set), with a sampling frequency of NF and a number of sampling points of NFt2. Therefore, the sampled sequences of the two audio streams are S1 and S2, and the total number of sampling points of the two sequences is equal, i.e., Ft1 = NFt2.

[0091] In at least one embodiment of this application, step 105 includes:

[0092] The audio similarity of each frame signal in the second frame signal set is compared with the first threshold.

[0093] If the audio similarity corresponding to the first frame signal is greater than or equal to the first threshold, the first frame signal is determined to be a matching frame signal; or, if the audio similarity corresponding to the first frame signal is less than the first threshold, the first frame signal is determined to be a non-matching frame signal; wherein, the first frame signal is any frame signal in the second set of frame signals.

[0094] Accordingly, the method further includes:

[0095] Based on the mismatched frame signal, determine the audio timestamp corresponding to the mismatched frame signal, and / or determine and mark the position of the mismatched frame signal in the target text.

[0096] In at least one embodiment of this application, step 104 includes:

[0097] For each frame signal in the second set of signal frames, a first operation is performed, the first operation including:

[0098] The spectral envelope window of one frame signal in the second frame signal set is shifted along the vertical axis within the spectral envelope window of the corresponding frame signal in the first frame signal set; wherein the shift interval is from the minimum to the maximum value of the vertical axis of the first frame signal set, and the shift step size is an integer multiple of the difference between the minimum and maximum values ​​of the vertical axis of the first frame signal set; the vertical axis is used to indicate the loudness of the frame signal;

[0099] Obtain the audio similarity index at the center frequency of the critical frequency band obtained from each translation step;

[0100] The maximum value of the audio similarity index is used as the audio similarity between a frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set.

[0101] Accordingly, prior to step 104, the method further includes:

[0102] Perform a frame division operation on the first frame signal set;

[0103] After feature extraction, windowing, and fast Fourier transform of each frame of signal obtained by framing, the spectral envelope of each frame of signal is obtained.

[0104] Based on the spectral envelope of each frame of the signal, determine the spectral envelope window of each frame of the first frame signal set.

[0105] In at least one embodiment of this application, step 103 includes:

[0106] Perform a frame division operation on the second frame signal set;

[0107] After feature extraction, windowing, and fast Fourier transform of each frame of signal obtained by framing, the spectral components and spectral envelope of each frame of signal are obtained.

[0108] Based on the preset critical bandwidth, the frequency domain is divided into multiple critical frequency bands;

[0109] Based on the values ​​of the spectral components of each frame of signal at each center frequency in the critical band, a bandpass threshold sequence is obtained;

[0110] The upper and lower limits are set centered on each value in the bandpass threshold sequence, and the obtained bandpass window sequence is used to determine the spectral envelope window.

[0111] In this embodiment, the two audio streams are converted to the frequency domain for feature extraction to obtain the corresponding spectral envelope. The spectral envelope is used to visually demonstrate problems such as playback stuttering and asynchrony between voice and text.

[0112] For example, such as Figure 7 As shown, in this embodiment, the two audio streams are converted to the frequency domain for feature extraction (spectral envelope window). The spectral envelope window shifting method is used to compare whether the two audio streams are similar. The implementation process is as follows:

[0113] (1) Two-channel audio framing. The first frame signal set S1 and the second frame signal set S2 are framed, for example, using a frame length of 25 milliseconds and a frame interval of 10 milliseconds.

[0114] (2) Frame feature extraction. Feature extraction is performed on each frame of the signal. Since it is usually difficult to discern the characteristics of a signal from its transformation in the time domain, it is typically converted into an energy distribution in the frequency domain for observation. Different energy distributions can represent the characteristics of different speech sounds. Therefore, after multiplying by a Hamming window to reduce sidelobe leakage, each frame of the signal undergoes a Fast Fourier Transform (FFT) to obtain its energy distribution in the spectrum. The spectral components of each frame of the signal are obtained using the following formula:

[0115]

[0116] Where x(n) is the input frame signal, N represents the number of sampled signal points, and K represents the index in the spectrum (different frequency components).

[0117] (3) After FFT, the frame spectrum envelopes of the first frame signal set S1 and the second frame signal set S2 are obtained respectively. For example, the frame spectrum envelope of a certain frame signal is as follows: Figure 8 As shown.

[0118] (4) Critical frequency band division. Within the frequency domain, critical frequency bands are divided based on the human ear's sensitivity to different frequencies and the masking effect. In this system, 24 critical frequency bands are divided within the frequency range of 20Hz to 16kHz. These 24 critical frequency bands are distributed within the 20Hz to 16kHz frequency range, with center frequencies ranging from 50Hz to 13.5kHz, and include commonly used measurement frequencies such as 250Hz, 1kHz, and 4kHz. The specific division method is as follows:

[0119] When the center frequency is less than 500Hz, the preset critical bandwidth is approximately 50Hz.

[0120] When the center frequency is greater than 500Hz, the preset critical bandwidth is approximately 20% of the center frequency.

[0121] (5) Extract the spectral components of each frame signal in the second frame signal set S2, and obtain the values ​​of each center frequency in the critical bandwidth to obtain the bandpass threshold sequence B(fn) of each center frequency.

[0122] (6) Set the upper and lower floating ranges for the center frequency values ​​of sequence B(fn) in each critical frequency band to generate the bandpass envelope W(fn), i.e., the spectral envelope window, which can also be called the bandpass window sequence W(fn), for example... Figure 9 As shown, Figure 9 In the coordinate system, the horizontal axis represents frequency (Hz) and the vertical axis represents loudness (dB).

[0123] (7) Shift the bandpass window sequence W(fn) along the vertical axis within the same frame window of the first frame signal set S1. The shift interval is from the minimum to the maximum value of the vertical axis of S1, and the step size is an integer multiple of the difference between the minimum and maximum values. For example... Figure 10 As shown, where, Figure 10 In the coordinate system, the horizontal axis represents frequency (Hz) and the vertical axis represents loudness (dB).

[0124] (8) Each translation step yields a set of audio similarity indices J at the critical center frequency point. After the translation is complete, the maximum value of the similarity indices J is taken as the audio similarity Jmax of that frame. For example, the formula for calculating the similarity index J is:

[0125]

[0126] Where i represents the critical frequency band, and its value ranges from 1 to 24; x i The loudness value at the critical frequency band i;

[0127] bi _上 b is the maximum value of the bandpass window sequence W(i) on the critical frequency band i. i_下 It is the minimum value of the bandpass window sequence W(i) on the critical frequency band i.

[0128] It should be noted that (b) i_上 -x i When x is negative, the value is judged as 0; i -b i_下 When the value is negative, the judgment is 0.

[0129] (9) Compare Jmax with the preset audio similarity decision threshold J;

[0130] (10) When Jmax>=J, it is determined that the audio features of S1 are similar to those of S2, that is, the current frame signal is a matching frame signal; otherwise, Jmax<J, it is determined that the audio features of S1 are not similar to those of S2, that is, the current frame signal is a non-matching frame signal.

[0131] Furthermore, in this embodiment, frames with audio similarity discrepancies (i.e., mismatched frame signals) can be marked, and the timestamp corresponding to the audio of the mismatched content can be traced back to the corresponding sampling point to mark the error point; the effect diagram is as follows. Figure 11 As shown.

[0132] In summary, in this embodiment of the application, the first audio channel obtained by character recognition and audio encoding of the target text and the second audio channel obtained by playing the audio of the target text are automatically compared, and the audio comparison result is directly obtained based on the audio similarity comparison. This transforms the tedious audio testing task into a fully automated comparison test, and can trace and quickly locate mismatched content, thereby improving testing efficiency and accuracy.

[0133] like Figure 12 As shown in the figure, this application embodiment also provides an audio comparison device, including:

[0134] The acquisition module 1201 is used to acquire a first audio channel and a second audio channel; the first audio channel is obtained by character recognition and audio encoding of the target text, and the second audio channel is obtained by playing the audio of the target text.

[0135] The sampling module 1202 is used to sample the first audio channel and the second audio channel respectively to obtain a first frame signal set and a second frame signal set; the number of frame signals contained in the first frame signal set is the same as the number of frame signals contained in the second frame signal set;

[0136] The first determining module 1203 is used to determine the spectral envelope window of each frame signal in the second frame signal set based on the second frame signal set and the preset critical bandwidth.

[0137] The second determining module 1204 is used to translate the spectral envelope window of each frame signal in the second frame signal set into the spectral envelope window of the corresponding frame signal in the first frame signal set, so as to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set.

[0138] The third determining module 1205 is used to determine the audio comparison result between the first audio channel and the second audio channel based on the audio similarity.

[0139] As an optional embodiment, the third determining module includes:

[0140] The comparison submodule is used to compare the audio similarity corresponding to each frame signal in the second frame signal set with the first threshold respectively;

[0141] The first determining submodule is configured to determine the first frame signal as a matching frame signal if the audio similarity corresponding to the first frame signal is greater than or equal to the first threshold; or, if the audio similarity corresponding to the first frame signal is less than the first threshold, determine the first frame signal as a non-matching frame signal; wherein the first frame signal is any frame signal in the second set of frame signals.

[0142] As an optional embodiment, the apparatus further includes:

[0143] The fourth determining module is used to determine the audio timestamp corresponding to the mismatched frame signal based on the mismatched frame signal, and / or to determine and mark the position of the mismatched frame signal in the target text.

[0144] As an optional embodiment, the second determining module includes:

[0145] The second determining submodule is configured to perform a first operation on the spectral envelope window of each frame signal in the second frame signal set, the first operation including:

[0146] The spectral envelope window of one frame signal in the second frame signal set is shifted along the vertical axis within the spectral envelope window of the corresponding frame signal in the first frame signal set; wherein the shift interval is from the minimum to the maximum value of the vertical axis of the first frame signal set, and the shift step size is an integer multiple of the difference between the minimum and maximum values ​​of the vertical axis of the first frame signal set; the vertical axis is used to indicate the loudness of the frame signal;

[0147] Obtain the audio similarity index at the center frequency of the critical frequency band obtained from each translation step;

[0148] The maximum value of the audio similarity index is used as the audio similarity between a frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set.

[0149] As an optional embodiment, the apparatus further includes:

[0150] The first operation module is used to perform frame division operation on the first frame signal set;

[0151] The first processing module is used to extract features, window, and perform fast Fourier transform on each frame of signal obtained by framing to obtain the spectral envelope of each frame of signal.

[0152] The fifth determining module is used to determine the spectral envelope window of each frame signal in the first frame signal set based on the spectral envelope of each frame signal.

[0153] As an optional embodiment, the first determining module includes:

[0154] The second operation submodule is used to perform frame division operation on the second frame signal set;

[0155] The second processing submodule is used to extract features, window, and perform fast Fourier transform on each frame of signal obtained by framing, so as to obtain the spectral components and spectral envelope of each frame of signal.

[0156] The partitioning submodule is used to divide the frequency domain range into multiple critical frequency bands according to a preset critical bandwidth;

[0157] The fourth determining submodule is used to obtain the bandpass threshold sequence based on the values ​​of the spectral components of each frame signal at each center frequency in the critical frequency band;

[0158] The fifth determining submodule is used to set upper and lower limits centered on each value in the bandpass threshold sequence, and to determine the spectral envelope window using the obtained bandpass window sequence.

[0159] As an optional embodiment, the device includes:

[0160] The alignment module is used to align the start and end times of the second audio stream with the start and end times of the first audio stream.

[0161] The sampling module includes:

[0162] The sampling submodule is used to sample the first and second audio channels after the start and end time points are aligned, respectively, to obtain the first frame signal set and the second frame signal set.

[0163] In this embodiment, the first audio channel obtained by character recognition and audio encoding of the target text and the second audio channel obtained by playing the audio of the target text are automatically compared, and the audio comparison result is directly obtained based on the audio similarity comparison. This transforms the tedious audio testing task into a fully automated comparison test, and can trace and quickly locate mismatched content, thereby improving testing efficiency and accuracy.

[0164] It should be noted that the audio comparison device provided in this application embodiment is a device capable of performing the above-described audio comparison method. Therefore, all embodiments of the above-described audio comparison method are applicable to this device and can achieve the same or similar beneficial effects. No further limitation is made here.

[0165] like Figure 13As shown, this application embodiment also provides an audio comparison device, including a memory 1310, a processor 1300, and a computer program stored in the memory 1310 and executable on the processor 1300. When the processor 1300 executes the program, it implements the various processes in the audio comparison method embodiment described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0166] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the various processes described above in the audio comparison method embodiments, achieving the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0167] This application also provides a computer program product, including computer instructions. When executed by a processor, the computer instructions implement the various processes of the audio comparison method embodiment described above and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0168] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0169] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 A device for one or more processes and / or the functions specified in one or more boxes.

[0170] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce a paper article including an instruction means, the instruction means being implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment, causing the computer or other programmable equipment to perform a series of operational steps to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0172] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An audio comparison method, characterized in that, include: Acquire the first audio stream and the second audio stream; the first audio stream is obtained through text recognition and audio encoding of the target text, and the second audio stream is obtained by playing the audio of the target text. The first audio channel and the second audio channel are sampled respectively to obtain a first frame signal set and a second frame signal set; the first frame signal set contains the same number of frame signals as the second frame signal set. Based on the second frame signal set and the preset critical bandwidth, determine the spectral envelope window of each frame signal in the second frame signal set; The spectral envelope window of each frame signal in the second frame signal set is translated into the spectral envelope window of the corresponding frame signal in the first frame signal set to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set. Based on the audio similarity, the audio comparison results of the first audio stream and the second audio stream are determined.

2. The method according to claim 1, characterized in that, Based on the audio similarity, the audio comparison results of the first audio stream and the second audio stream are determined, including: The audio similarity of each frame signal in the second frame signal set is compared with the first threshold. If the audio similarity corresponding to the first frame signal is greater than or equal to the first threshold, the first frame signal is determined to be a matching frame signal; or, if the audio similarity corresponding to the first frame signal is less than the first threshold, the first frame signal is determined to be a non-matching frame signal; wherein, the first frame signal is any frame signal in the second set of frame signals.

3. The method according to claim 2, characterized in that, The method further includes: Based on the mismatched frame signal, determine the audio timestamp corresponding to the mismatched frame signal, and / or determine and mark the position of the mismatched frame signal in the target text.

4. The method according to claim 1, characterized in that, The spectral envelope window of each frame signal in the second frame signal set is shifted relative to the spectral envelope window of the corresponding frame signal in the first frame signal set to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set, including: For each frame signal in the second set of signal frames, a first operation is performed, the first operation including: The spectral envelope window of one frame signal in the second frame signal set is shifted along the vertical axis within the spectral envelope window of the corresponding frame signal in the first frame signal set; wherein the shift interval is from the minimum to the maximum value of the vertical axis of the first frame signal set, and the shift step size is an integer multiple of the difference between the minimum and maximum values ​​of the vertical axis of the first frame signal set; the vertical axis is used to indicate the loudness of the frame signal; Obtain the audio similarity index at the center frequency of the critical frequency band obtained from each translation step; The maximum value of the audio similarity index is used as the audio similarity between a frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set.

5. The method according to claim 1 or 4, characterized in that, Before shifting the spectral envelope window of one frame signal in the second frame signal set along the vertical axis within the spectral envelope window of the corresponding frame signal in the first frame signal set, the method further includes: Perform a frame division operation on the first frame signal set; After feature extraction, windowing, and fast Fourier transform of each frame of signal obtained by framing, the spectral envelope of each frame of signal is obtained. Based on the spectral envelope of each frame of the signal, determine the spectral envelope window of each frame of the first frame signal set.

6. The method according to claim 1, characterized in that, The step of determining the spectral envelope window of each frame signal in the second frame signal set based on the second frame signal set and the preset critical bandwidth includes: Perform a frame division operation on the second frame signal set; After feature extraction, windowing, and fast Fourier transform of each frame of signal obtained by framing, the spectral components and spectral envelope of each frame of signal are obtained. Based on the preset critical bandwidth, the frequency domain is divided into multiple critical frequency bands; Based on the values ​​of the spectral components of each frame of signal at each center frequency in the critical band, a bandpass threshold sequence is obtained; The upper and lower limits are set centered on each value in the bandpass threshold sequence, and the obtained bandpass window sequence is used to determine the spectral envelope window.

7. The method according to any one of claims 1-6, characterized in that, Before sampling the first audio channel and the second audio channel respectively to obtain the first frame signal set and the second frame signal set, the method includes: Align the start and end times of the second audio stream with the start and end times of the first audio stream; The first audio stream and the second audio stream are sampled respectively to obtain a first frame signal set and a second frame signal set, including: The first and second audio channels, after being aligned with the start and end times, are sampled for a preset number of frames to obtain the first frame signal set and the second frame signal set.

8. An audio comparison device, characterized in that, include: The acquisition module is used to acquire a first audio source and a second audio source; the first audio source is obtained by character recognition and audio encoding of the target text, and the second audio source is obtained by playing the audio of the target text. The sampling module is used to sample the first audio channel and the second audio channel respectively to obtain a first frame signal set and a second frame signal set; the number of frame signals contained in the first frame signal set is the same as the number of frame signals contained in the second frame signal set; The first determining module is used to determine the spectral envelope window of each frame signal in the second frame signal set based on the second frame signal set and the preset critical bandwidth. The second determining module is used to translate the spectral envelope window of each frame signal in the second frame signal set into the spectral envelope window of the corresponding frame signal in the first frame signal set, respectively, to obtain the audio similarity between each frame signal in the second frame signal set and the corresponding frame signal in the first frame signal set. The third determining module is used to determine the audio comparison result between the first audio channel and the second audio channel based on the audio similarity.

9. An audio comparison device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that, When the processor executes the program, it implements the audio comparison method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the audio comparison method as described in any one of claims 1-7.

11. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the audio comparison method as described in any one of claims 1-7.

Images