Audio quality detection method and device, medium and product

Abstract

The invention relates to an audio quality detection method and device, a medium and a product. The method comprises the following steps: acquiring an original audio, and extracting a time domain feature and a frequency domain feature of the original audio; performing anomaly detection of different anomaly types on the original audio according to the time domain feature and the frequency domain feature to obtain multiple anomaly detection results of the original audio; inputting the original audio into a pre-trained deep learning model for reconstruction to obtain a plurality of reconstructed audios; performing reconstruction performance detection on each reconstructed audio to obtain a reconstruction performance detection result of each reconstructed audio; wherein different reconstruction performance detection results are used for representing different global qualities of the original audio; obtaining one or more groups of associated detection results from each anomaly detection result and each reconstruction performance detection result; and outputting a quality detection result of the original audio according to the correlation detection result. By adopting the method, high-efficiency and high-accuracy audio quality detection can be realized.

Description

Technical Field

[0001] This application relates to the field of audio detection technology, and in particular to an audio quality detection method, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of generative AI (Artificial Intelligence) technology, various generative audio formats have emerged, leading to an increasingly widespread demand for audio quality testing. Audio quality testing aims to comprehensively analyze and evaluate key attributes of audio, such as noise, loudness, and timbre, to reflect the quality of the audio.

[0003] Among the relevant audio quality testing technologies, one approach is to reflect audio quality through the subjective evaluation of experts, but this is easily influenced by individual preferences, difficult to standardize, and inefficient. Another approach is to analyze and test objective audio indicators through signal processing, but this approach has a one-sided testing dimension, strong threshold dependence, and is difficult to accurately reflect audio quality. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, computer device, computer-readable storage medium, and computer program product that can achieve high efficiency and high accuracy in audio quality detection, addressing the aforementioned technical problems.

[0005] Firstly, this application provides an audio quality detection method, including:

[0006] Obtain the original audio and extract its time-domain and frequency-domain features.

[0007] Based on the time-domain features and the frequency-domain features, different types of anomaly detection are performed on the original audio to obtain multiple anomaly detection results for the original audio;

[0008] The original audio input is reconstructed using a pre-trained deep learning model to obtain multiple reconstructed audio files;

[0009] Reconstruction performance testing is performed on each of the reconstructed audio files to obtain reconstruction performance testing results for each of the reconstructed audio files; wherein, different reconstruction performance testing results are used to characterize different global quality of the original audio files;

[0010] Obtain one or more sets of associated detection results from each of the anomaly detection results and each of the reconstruction performance detection results;

[0011] Based on the correlation detection results, output the quality detection results of the original audio.

[0012] In one embodiment, the deep learning model includes an encoder and a decoder, and the reconstructing of the pre-trained deep learning model of the original audio input includes:

[0013] The original audio is input into the deep learning model, and the encoder encodes the original audio according to different bandwidths to obtain multiple encoded audios;

[0014] The decoder decodes each of the encoded audio files to obtain the reconstructed audio files.

[0015] In one embodiment, the reconstruction performance test result includes a signal-to-noise ratio (SNR) improvement, the magnitude of which is negatively correlated with the global quality of the original audio. The reconstruction performance test for each of the reconstructed audio samples includes:

[0016] Determine the original signal power of the original audio and the reconstructed signal power of each of the reconstructed audios;

[0017] Determine the power difference between the reconstructed signal power and the original signal power;

[0018] The power difference is logarithmically calculated with the original signal power to obtain the signal-to-noise ratio improvement of each reconstructed audio.

[0019] In one embodiment, the reconstruction performance detection result further includes a reconstruction error index value, the magnitude of which is negatively correlated with the global quality of the original audio. The reconstruction performance detection for each of the reconstructed audio segments further includes:

[0020] The original audio samples are obtained by sampling the original audio.

[0021] Each of the reconstructed audio samples is sampled to obtain the reconstructed audio samples of each of the reconstructed audio samples;

[0022] By comparing the differences between each reconstructed audio sample and the original audio sample, the reconstruction error index value of each reconstructed audio is determined based on the differences.

[0023] In one embodiment, the anomaly types include multiple different time-domain anomalies and multiple different frequency-domain anomalies. The anomaly detection of the original audio based on the time-domain features and the frequency-domain features for different anomaly types includes:

[0024] Based on the time-domain characteristics, determine the time-domain anomaly index value for each type of time-domain anomaly;

[0025] Based on the frequency domain characteristics, determine the frequency domain anomaly index value for each type of frequency domain anomaly;

[0026] Determine the threshold adjustment index value for each of the aforementioned anomalies, update the initial anomaly threshold for each of the aforementioned anomalies based on the threshold adjustment index value, and obtain the target anomaly threshold for each of the aforementioned anomalies.

[0027] The time-domain anomaly index value and the frequency-domain anomaly index value are compared with the corresponding target anomaly threshold to detect different types of anomalies in the original audio.

[0028] In one embodiment, each threshold adjustment index value is any one of the time-domain anomaly index value, the frequency-domain anomaly index value, the correlation index value of the time-domain anomaly index value, and the correlation index value of the frequency-domain anomaly index value. The step of determining the threshold adjustment index value for each anomaly type, updating the initial anomaly threshold for each anomaly type based on the threshold adjustment index value, and obtaining the target anomaly threshold for each anomaly type includes:

[0029] Determine the threshold adjustment index value for each of the aforementioned anomaly types;

[0030] Based on the preset normalization factor corresponding to each threshold adjustment index value, the threshold adjustment index values ​​are normalized according to the preset normalization function to obtain the normalized threshold adjustment index values.

[0031] The initial anomaly threshold for each anomaly type is updated based on the normalized threshold adjustment index value to obtain the target anomaly threshold for each anomaly type.

[0032] In one embodiment, the frequency domain features include the average energy value of each of the multiple frequency bands corresponding to the original audio; the frequency domain anomalies include energy drop anomalies; and based on the frequency domain features, a frequency domain anomaly index value is determined for each type of frequency domain anomaly, including:

[0033] The energy difference between the average energy values ​​of adjacent frequency bands is determined to obtain the frequency domain anomaly index value for the energy drop anomaly.

[0034] In one embodiment, the temporal feature includes the amplitude value of each temporal sampling point among multiple temporal sampling points corresponding to the original audio; the temporal anomaly includes a clicking sound anomaly; determining the temporal anomaly index value for each type of temporal anomaly based on the temporal feature includes:

[0035] Determine the amplitude difference between the amplitude values ​​of adjacent time-domain sampling points, and determine the signal abrupt change point in each time-domain sampling point based on each amplitude difference;

[0036] The signal mutation point density of the signal mutation point is determined to obtain the time-domain anomaly index value for the clicking sound anomaly.

[0037] In one embodiment, the step of outputting a quality detection result for the original audio based on the correlation detection result includes:

[0038] Among multiple preset conditions, the target conditions that each group of associated detection results meets are determined; wherein, each preset condition has associated quality detection results;

[0039] Output the quality detection results associated with each of the target conditions to obtain the quality detection results of the original audio.

[0040] Secondly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0041] Obtain the original audio and extract its time-domain and frequency-domain features.

[0042] Based on the time-domain features and the frequency-domain features, different types of anomaly detection are performed on the original audio to obtain multiple anomaly detection results for the original audio;

[0043] The original audio input is reconstructed using a pre-trained deep learning model to obtain multiple reconstructed audio files;

[0044] Reconstruction performance testing is performed on each of the reconstructed audio files to obtain reconstruction performance testing results for each of the reconstructed audio files; wherein, different reconstruction performance testing results are used to characterize different global quality of the original audio files;

[0045] Obtain one or more sets of associated detection results from each of the anomaly detection results and each of the reconstruction performance detection results;

[0046] Based on the correlation detection results, output the quality detection results of the original audio.

[0047] Thirdly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0048] Obtain the original audio and extract its time-domain and frequency-domain features.

[0049] Based on the time-domain features and the frequency-domain features, different types of anomaly detection are performed on the original audio to obtain multiple anomaly detection results for the original audio;

[0050] The original audio input is reconstructed using a pre-trained deep learning model to obtain multiple reconstructed audio files;

[0051] Reconstruction performance testing is performed on each of the reconstructed audio files to obtain reconstruction performance testing results for each of the reconstructed audio files; wherein, different reconstruction performance testing results are used to characterize different global quality of the original audio files;

[0052] Obtain one or more sets of associated detection results from each of the anomaly detection results and each of the reconstruction performance detection results;

[0053] Based on the correlation detection results, output the quality detection results of the original audio.

[0054] Fourthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0055] Obtain the original audio and extract its time-domain and frequency-domain features.

[0056] Based on the time-domain features and the frequency-domain features, different types of anomaly detection are performed on the original audio to obtain multiple anomaly detection results for the original audio;

[0057] The original audio input is reconstructed using a pre-trained deep learning model to obtain multiple reconstructed audio files;

[0058] Reconstruction performance testing is performed on each of the reconstructed audio files to obtain reconstruction performance testing results for each of the reconstructed audio files; wherein, different reconstruction performance testing results are used to characterize different global quality of the original audio files;

[0059] Obtain one or more sets of associated detection results from each of the anomaly detection results and each of the reconstruction performance detection results;

[0060] Based on the correlation detection results, output the quality detection results of the original audio.

[0061] The aforementioned audio quality detection methods, computer equipment, computer-readable storage media, and computer program products perform anomaly detection on the original audio using time-domain and frequency-domain features. This allows multiple anomaly detection results to reflect the local quality of the original audio across multiple dimensions, enabling detailed perception of the original audio. By reconstructing the original audio using a pre-trained deep learning model, multiple reconstructed audio samples are obtained. Reconstruction performance detection is then performed on each reconstructed audio sample, allowing multiple reconstruction performance detection results to reflect different global qualities of the original audio. This eliminates reliance on subjective evaluation and enables a holistic assessment of the original audio. Furthermore, by correlating detection results, multiple local and global qualities are modally correlated, achieving cross-validation of the original audio quality issues across multiple dimensions. Consequently, the audio quality detection results for the original audio can be efficiently and accurately output without relying on subjective evaluation. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a schematic diagram of the application environment provided in an embodiment of this application;

[0064] Figure 2 A flowchart illustrating the steps of an audio quality detection method provided in an embodiment of this application;

[0065] Figure 3 A diagram illustrating the anomaly detection process of the original audio provided in an embodiment of this application;

[0066] Figure 4 A diagram illustrating the bandwidth adaptability scoring process provided in one embodiment of this application;

[0067] Figure 5 A flowchart of an audio quality detection system provided in an embodiment of this application;

[0068] Figure 6 This is a structural block diagram of an audio quality detection device provided in an embodiment of this application;

[0069] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0071] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0072] The audio quality detection method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on another network server. Terminal 102 initiates an audio quality request to server 104, sending the original audio to server 104. Server 104 receives the original audio from terminal 102 and extracts its temporal and frequency domain features. Server 104 performs anomaly detection on the original audio based on the temporal and frequency domain features, obtaining multiple anomaly detection results. Server 104 inputs the original audio into a pre-trained deep learning model for reconstruction, obtaining reconstructed audio. Server 104 performs reconstruction performance detection on the reconstructed audio, obtaining reconstruction performance detection results. These results characterize the overall quality of the original audio. Server 104 obtains one or more sets of associated detection results from each anomaly detection result and each reconstruction performance detection result. Based on the associated detection results, server 104 outputs the quality detection result for the original audio and returns it to terminal 102.

[0073] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart in-vehicle systems, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0074] In one exemplary embodiment, such as Figure 2 As shown, an audio quality detection method is provided, which is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 202 to 212. Wherein:

[0075] Step 202: Obtain the original audio and extract the time-domain and frequency-domain features of the original audio.

[0076] In practical implementation, the original audio is the audio that needs quality inspection. This can include recorded audio, such as studio-recorded speech or music, or generative audio generated using computer technology, such as TTS (Text-To-Speech) audio. Temporal features are used to reflect the characteristics of the audio in the time domain, such as the energy of the original audio signal at each time point in the time-domain spectrum. Frequency domain features are used to reflect the characteristics of the audio in the frequency domain, such as the energy of the original audio signal at each frequency point in the frequency-domain spectrum.

[0077] By performing a series of processing steps on the time-domain signal of the original audio, such as frame segmentation and windowing, the time-domain features of the original audio can be extracted; by converting the time-domain signal into a frequency-domain signal, the frequency-domain features can be extracted from the frequency-domain signal.

[0078] Step 204: Perform anomaly detection on the original audio based on the time domain features and the frequency domain features to obtain multiple anomaly detection results for the original audio;

[0079] In practical implementation, different anomaly types can include clipping distortion, long silences, pops, loudness anomalies, signal-to-noise ratio (SNR) anomalies, roll-off point anomalies, spectral flatness anomalies, and harmonic-to-noise ratio (HNR) anomalies. Corresponding detection methods are then applied for each anomaly type, such as clipping distortion detection, long silence detection, pop detection, loudness anomaly detection, SNR anomaly detection, roll-off point anomaly detection, spectral flatness anomaly detection, and HNR anomaly detection. The detection results reflect the specific anomaly of the original audio for each anomaly type. For example, the audio energy percentile of the original audio is determined through time-domain features. When the audio energy percentile is greater than a threshold, a pop anomaly is identified, resulting in a pop anomaly detection result. This result can include the specific audio energy percentile value, the specific threshold used for anomaly judgment, and an anomaly flag (e.g., 1 for anomaly, 0 for normal).

[0080] In some embodiments, the anomaly types include multiple different time-domain anomalies and multiple different frequency-domain anomalies, and the anomaly detection of the original audio based on the time-domain features and the frequency-domain features for different anomaly types includes:

[0081] Based on the time-domain characteristics, determine the time-domain anomaly index value for each type of time-domain anomaly;

[0082] Based on the frequency domain characteristics, determine the frequency domain anomaly index value for each type of frequency domain anomaly;

[0083] Determine the threshold adjustment index value for each of the aforementioned anomalies, update the initial anomaly threshold for each of the aforementioned anomalies based on the threshold adjustment index value, and obtain the target anomaly threshold for each of the aforementioned anomalies.

[0084] The time-domain anomaly index value and the frequency-domain anomaly index value are compared with the corresponding target anomaly threshold to detect different types of anomalies in the original audio.

[0085] In practical implementation, temporal anomalies refer to anomalies occurring in the temporal signal of the original audio, which may include clipping distortion anomalies, long silence anomalies, popping anomalies, signal-to-noise ratio (SNR) anomalies, loudness anomalies, etc. Temporal anomaly index values ​​are index values ​​used to determine whether the original audio has the corresponding temporal anomaly. For example, the temporal anomaly index value for clipping distortion anomalies can be the audio signal amplitude; the temporal anomaly index value for long silence anomalies can be the number or proportion of silence segments with a duration exceeding a preset duration; the temporal anomaly index value for click anomalies can be the density of signal abrupt change points; the temporal anomaly index value for popping anomalies can be the audio energy percentile; the temporal anomaly index value for SNR anomalies can be the energy ratio (SNR) of the temporal signal to the noise signal; and the temporal anomaly index value for loudness anomalies can be the audio loudness (which can be quantified by an approximation of the original audio using LUFS (Loudness Units Full Scale)).

[0086] Frequency domain anomalies refer to abnormalities in the frequency domain signal of the original audio, which may include anomalies in the roll-off point, spectral flatness, and harmonic-to-noise ratio (HNR). Frequency domain anomaly indicators are values ​​used to determine whether the original audio has a corresponding frequency domain anomaly. For example, the frequency domain anomaly indicator for a roll-off point anomaly can be the frequency range of the energy concentration points (e.g., the frequency range of the 85% energy concentration points is 0-2Hz); the frequency domain anomaly indicator for a spectral flatness anomaly can be an indicator reflecting the uniformity of spectral energy distribution (values ​​ranging from 0-1, with values ​​closer to 1 indicating more noise); and the frequency domain anomaly indicator for a harmonic-to-noise ratio (HNR) anomaly is the ratio of harmonic energy to noise energy.

[0087] By comparing the anomaly values ​​in each time domain and frequency domain with their corresponding anomaly thresholds, it can be determined whether the original audio has anomalies of the corresponding type. Static configuration of anomaly thresholds is difficult to adapt to various situations. Setting the threshold too strictly can easily lead to false positives; setting it too leniently can easily lead to missed detections. Therefore, adaptive adjustment of the anomaly thresholds is necessary.

[0088] The threshold adjustment index values ​​for each anomaly type serve as the basis for adaptive adjustment of the anomaly threshold. These index values ​​determine the magnitude and direction of the threshold adjustment. The initial anomaly threshold can be a preset threshold, such as one based on prior knowledge and preset through a configuration file.

[0089] Furthermore, the abnormal index values ​​in each time domain and each abnormal index value in each frequency domain are compared with the corresponding target abnormal thresholds to obtain the comparison results. If the comparison results do not meet the preset conditions (such as being too large or too small relative to the target abnormal threshold), it can be determined that the original audio has an anomaly corresponding to the abnormal type, so as to perform anomaly detection for different types of anomalies on the original audio.

[0090] In practical applications, safety threshold boundaries can also be set to prevent the updated target anomaly threshold from being too large or too small. For example, if the updated target anomaly threshold is less than the lower limit of the safety threshold boundary, the target anomaly threshold is adjusted to that lower limit; if the updated target anomaly threshold is greater than the upper limit of the safety threshold boundary, the target anomaly threshold is adjusted to that upper limit.

[0091] For example, for clipping distortion anomaly, the audio signal amplitude is the time-domain anomaly index value, the initial anomaly threshold can be 0.99, and the safe threshold boundary can be (0.9,1); for long silence anomaly, the proportion of long silence segments is the time-domain anomaly index value, the initial anomaly threshold can be 0.01, and the safe threshold boundary can be (0.005,0.05); for popping anomaly, the audio energy percentile is the frequency-domain anomaly index value, the initial anomaly threshold can be 0.8, and the safe threshold boundary can be (0.6,1).

[0092] In some examples, the process of adaptively adjusting the outlier threshold is as follows: Figure 3 As shown, the original audio undergoes preprocessing such as frame segmentation and sampling, followed by the extraction of temporal and frequency domain features. Temporal and frequency domain anomaly index values ​​are then determined based on these features, and threshold adjustment index values ​​for each anomaly type are determined. The initial anomaly thresholds for each anomaly type are updated according to these threshold adjustment index values, achieving adaptive calculation of the target anomaly threshold. By managing the target anomaly thresholds for each anomaly type, the corresponding target anomaly thresholds are applied during anomaly detection, outputting the corresponding anomaly detection results for each anomaly type.

[0093] In this embodiment, the initial anomaly threshold is updated to the target anomaly threshold based on the threshold adjustment index value. This allows for adaptive adjustment of the anomaly threshold without relying on historical data, reducing false detections and missed detections, and making each anomaly detection result more accurate and better reflect the true anomaly situation of the original audio.

[0094] In some embodiments, each threshold adjustment index value is any one of the time-domain anomaly index value, the frequency-domain anomaly index value, the associated index value of the time-domain anomaly index value, and the associated index value of the frequency-domain anomaly index value. Determining the threshold adjustment index value for each anomaly type, updating the initial anomaly threshold for each anomaly type based on the threshold adjustment index value, and obtaining the target anomaly threshold for each anomaly type includes:

[0095] Determine the threshold adjustment index value for each of the aforementioned anomaly types;

[0096] Based on the preset normalization factor corresponding to each threshold adjustment index value, the threshold adjustment index values ​​are normalized according to the preset normalization function to obtain the normalized threshold adjustment index values.

[0097] The initial anomaly threshold for each anomaly type is updated based on the normalized threshold adjustment index value to obtain the target anomaly threshold for each anomaly type.

[0098] In this embodiment, each threshold adjustment index value is any one of the following: time-domain anomaly index value, frequency-domain anomaly index value, correlation index value of time-domain anomaly index value, and correlation index value of frequency-domain anomaly index value.

[0099] For example, time-domain anomaly indicators and frequency-domain anomaly indicators can be directly used as threshold adjustment indicators. For instance, for popping anomalies, the energy percentile is the time-domain anomaly indicator, so the energy percentile can be directly used as the threshold adjustment indicator. Alternatively, the associated indicators of the time-domain anomaly indicator and the associated indicators of the frequency-domain anomaly indicator can be used as threshold adjustment indicators. For example, for clipping distortion anomalies, the audio signal amplitude is the time-domain anomaly indicator, so the audio signal kurtosis associated with the audio signal amplitude can be used as the threshold adjustment indicator. For long silence anomalies, the proportion of long silence segments is the time-domain anomaly indicator; the presence of long silence segments significantly reduces the energy variance of the original audio, so the energy variance associated with the long silence segments can be used as the threshold adjustment indicator.

[0100] In this embodiment, a preset normalization factor is used to uniformly map the threshold adjustment index values ​​to the numerical range of [0, 1] through a preset normalization function, thereby eliminating the dimensional deviation between different threshold adjustment index values. The normalization function can be a hyperbolic tangent function, etc. By using the hyperbolic tangent function, the smooth adjustment of abnormal thresholds can be ensured, preventing abrupt changes.

[0101] Furthermore, the initial anomaly thresholds are updated based on the normalized threshold values ​​to obtain the target anomaly thresholds for each anomaly type, thus achieving adaptive adjustment of the anomaly thresholds.

[0102] In some examples, the initial anomaly threshold can be set using the following formula (1). Adjustments are made to obtain the target anomaly threshold. :

[0103] (1)

[0104] In equation (1), is the initial anomaly threshold. To adjust the strength coefficient (value range 0-1). To preset the normalization factor, Adjust the indicator value to the threshold. It is the hyperbolic tangent function.

[0105] For example, for clipping distortion anomalies, the audio signal amplitude is the time-domain anomaly index value, the initial anomaly threshold can be 0.99, the adjustment intensity coefficient can be 0.05, and the preset normalization factor can be 10; for long silence anomalies, the proportion of long silence segments is the time-domain anomaly index value, the initial anomaly threshold can be 0.01, the adjustment intensity coefficient can be 0.03, and the preset normalization factor can be 0.1; for popping anomalies, the audio energy percentile is the frequency-domain anomaly index value, the initial anomaly threshold can be 0.8, the adjustment intensity coefficient can be 0.06, and the preset normalization factor can be 0.5; for clicking anomalies, the density of signal abrupt change points is the time-domain anomaly index, the initial anomaly threshold can be 0.2, the adjustment intensity coefficient can be 0.04, and the preset normalization factor can be 0.05.

[0106] In this embodiment, by determining any one of the real-time indicators among the time-domain anomaly indicator value, the frequency-domain anomaly indicator value, the associated indicator value of the time-domain anomaly indicator value, and the associated indicator value of the frequency-domain anomaly indicator value as the threshold adjustment indicator value, the adjustment direction of adaptive threshold adjustment can be provided in multiple dimensions. Then, the threshold adjustment indicator values ​​are normalized to eliminate the deviation of the dimensions between different threshold adjustment indicator values ​​and improve the accuracy of anomaly detection results.

[0107] In some embodiments, the frequency domain features include the average energy value of each of the multiple frequency bands corresponding to the original audio; the frequency domain anomaly includes an energy drop anomaly; and based on the frequency domain features, a frequency domain anomaly index value is determined for each type of frequency domain anomaly, including:

[0108] The energy difference between the average energy values ​​of adjacent frequency bands is determined to obtain the frequency domain anomaly index value for the energy drop anomaly.

[0109] In the specific implementation, after preprocessing the original audio signal such as pre-emphasis, framing, and windowing, a spectral transform (e.g., short-time Fourier transform) is applied to each frame of the signal to obtain the frequency domain spectrum; then, the frequency domain spectrum is divided into multiple frequency bands corresponding to the original audio according to a preset step size. For example, if the preset step size is 100Hz, then the frequency range of each frequency band obtained by each division is also 100Hz.

[0110] In practical applications, the frequency range can be further divided according to specific needs. For example, for the 4kHz-15kHz frequency band sensitive to human hearing, interference from extremely low frequencies (<100Hz) and extremely high frequencies (>15kHz) is ignored, and the frequency band is divided using a fine preset step size of 500Hz, which is twice the resolution of the traditional 1kHz band. This is especially suitable for TTS audio quality testing scenarios, as it is specifically optimized for the high-frequency characteristics of TTS audio, achieving accurate detection of abnormal energy drops in the key frequency band of 4kHz-15kHz.

[0111] The average energy of each frequency band can be determined by the following formula (2):

[0112] (2)

[0113] in, For frequency To frequency The frequency band energy, It represents the amplitude of a single frequency point within the frequency band.

[0114] By determining the energy difference between the average energies of adjacent frequency bands, a frequency domain anomaly index value for energy drop anomalies can be obtained.

[0115] For example, the energy difference between the average energies of adjacent frequency bands can be expressed by the following equation (3):

[0116] (3)

[0117] in, Let i be the average energy of frequency band i. It represents the average energy of the next frequency band adjacent to frequency band i.

[0118] By comparing each energy difference with a target anomaly threshold, it can be determined whether the original audio contains an energy drop anomaly. For example, if all or most of the energy differences are greater than the target anomaly threshold, it can be determined that the original audio contains an energy drop anomaly.

[0119] In the case of updating the initial abnormal threshold of the energy drop anomaly to the target abnormal threshold, since the energy difference is too large, it will affect the spectral flatness. Therefore, the spectral flatness associated with the energy difference between adjacent frequency bands can be used as the threshold adjustment index value. Using this threshold adjustment index value, the initial abnormal threshold of the energy drop anomaly is updated by Equation (1) to obtain the target abnormal threshold.

[0120] For example, for the energy drop anomaly, in equation (1), the initial anomaly threshold can be 20, the adjustment intensity coefficient can be 0.08, and the preset normalization factor can be 0.5. At the same time, the safety threshold boundary can be set to (15, 25) to prevent the target anomaly threshold of the energy drop anomaly from becoming extreme.

[0121] In this embodiment, by determining the energy difference between adjacent frequency bands, a frequency domain anomaly index value for energy drop anomalies is obtained. This allows for targeted anomaly detection of energy loss or imbalance in the original audio within a specific frequency range. It is particularly suitable for anomaly detection of energy drop anomalies in generative audio (such as TTS audio) caused by factors such as insufficient acoustic model training and vocoder generation defects. This provides a basis for optimizing acoustic parameter adjustments and improving model structure, thereby enhancing the naturalness and fidelity of generative audio.

[0122] In some embodiments, the temporal features include the amplitude value of each temporal sampling point among a plurality of temporal sampling points corresponding to the original audio; the temporal anomaly includes a clicking sound anomaly; determining a temporal anomaly index value for each type of temporal anomaly based on the temporal features includes:

[0123] Determine the amplitude difference between the amplitude values ​​of adjacent time-domain sampling points, and determine the signal abrupt change point in each time-domain sampling point based on each amplitude difference;

[0124] The signal mutation point density of the signal mutation point is determined to obtain the time-domain anomaly index value for the clicking sound anomaly.

[0125] In this embodiment, the clicks anomaly refers to an unexpected, brief, pulse-like popping sound in the audio. Clicks disrupt the continuity and listening experience of the audio.

[0126] In practical implementation, the original audio signal can be converted into a time-domain spectrum, and samples can be taken in the time-domain spectrum at preset steps (e.g., 1ms) to obtain multiple time-domain sampling points. Each time-domain sampling point has a corresponding time and amplitude value, and the amplitude value reflects the energy intensity of the audio signal. By signal differential analysis, that is, determining the amplitude difference between the amplitude values ​​of adjacent time-domain sampling points, the signal abrupt change points in each time-domain sampling point can be determined based on the amplitude difference. For example, if the amplitude difference between the amplitude values ​​of the i-th time-domain sampling point and the adjacent (i+1)-th time-domain sampling point is greater than or equal to a preset threshold, the i-th time-domain sampling point can be determined as a signal abrupt change point.

[0127] Furthermore, the signal mutation point density is determined, which is the time-domain anomaly index value for the clicking sound anomaly. When the signal mutation point density is greater than or equal to the target anomaly index value for the clicking sound anomaly, it is determined that the original audio has a clicking sound anomaly.

[0128] Specifically, the signal mutation density can be the ratio of the number of signal mutation points to the number of all time-domain sampling points, or the ratio of the duration of signal mutation points to the total duration of all time-domain sampling points. Alternatively, the signal mutation density can be obtained through preset density calculation algorithms (such as mutation point interval distribution calculation, local density estimation, etc.).

[0129] In some examples, for the clicking sound anomaly, the time-domain anomaly index is the signal mutation point density, which is obtained based on the signal difference. Therefore, the greater the density of signal mutation points, the greater the variance of the signal difference. Thus, the variance of the signal difference associated with the signal mutation point density can be used as the threshold adjustment index value. Using this threshold adjustment index value, the initial anomaly threshold of the clicking sound anomaly is updated using Equation (1) to obtain the target anomaly threshold.

[0130] For example, for the clicking sound anomaly, the signal mutation point density is the time-domain anomaly index. In equation (1), the initial anomaly threshold can be 0.2, the intensity adjustment coefficient can be 0.04, and the preset normalization factor can be 0.05. At the same time, the safety threshold boundary can be set to (0.1, 0.3) to prevent the target anomaly threshold of the clicking sound anomaly from becoming extreme.

[0131] In this embodiment, by determining the amplitude difference between adjacent time-domain sampling points, signal abrupt change points are identified at each time-domain sampling point based on each amplitude difference. This fully utilizes the temporal characteristics of the original audio to capture instantaneous abrupt changes in the original audio, accurately identifying clicking sound anomalies. It is particularly suitable for detecting clicking sound anomalies in generated audio (such as TTS audio) caused by factors such as parameter conversion errors and discontinuous audio splicing points, thereby enabling targeted optimization of the generated audio.

[0132] Step 206: Reconstruct the pre-trained deep learning model of the original audio input to obtain multiple reconstructed audio files;

[0133] In practical implementations, deep learning models can be generative framework-based models (such as WaveNet and WaveGlow) or encoder-decoder framework-based models (such as autoencoder models and EnCodec). Different parameters (such as bandwidth, compression ratio, and number of channels) or different models can be used to reconstruct the original audio into multiple reconstructed audio files. Compared to signal processing for audio reconstruction, using deep learning models to reconstruct the original audio can be more efficient and accurate, providing reliable support for reconstruction performance testing.

[0134] Step 208: Perform reconstruction performance testing on each of the reconstructed audio files to obtain reconstruction performance testing results for each of the reconstructed audio files; wherein, different reconstruction performance testing results are used to characterize the different global quality of the original audio files;

[0135] In practical implementation, reconstruction performance testing can include saving relevant data during the reconstruction of the original audio, evaluating and testing the reconstruction performance of the reconstructed audio using the saved data, or testing the reconstruction performance of the reconstructed audio by comparing the differences between the reconstructed audio and the original audio. The reconstruction performance testing results can specifically include relevant index values ​​for various reconstruction performance aspects of the reconstructed audio. The reconstruction performance testing results of different reconstructed audios are used to characterize the different global quality of the original audio, for example, characterizing the global quality of the original audio at different bandwidths, compression rates, and channels. The better the global quality of the original audio, such as less distortion and noise, the easier it is for the deep learning model to parse and infer the structure of the original audio and output the reconstructed audio; therefore, the better the reconstruction performance of each reconstructed audio.

[0136] In some embodiments, the deep learning model includes an encoder and a decoder, and the reconstructing of the pre-trained deep learning model of the original audio input includes:

[0137] The original audio is input into the deep learning model, and the encoder encodes the original audio according to different bandwidths to obtain multiple encoded audios;

[0138] The decoder decodes each of the encoded audio files to obtain the reconstructed audio files.

[0139] In practice, the original audio can first be resampled to an audio format acceptable to the deep learning model (e.g., EnCodec requires resampling to 24Hz). Then, the deep learning model's encoder encodes the original audio at different bandwidths (e.g., 3kbps, 6kbps, and 12kbps), resulting in multiple encoded audio files. Finally, the deep learning model's encoder applies a predefined decoding algorithm to each encoded audio file to obtain the reconstructed audio.

[0140] In this embodiment, the deep learning model first understands the original audio through an encoder, that is, it analyzes the signal structure of the original audio and encodes it to obtain encoded audio. Then, based on the understanding, the decoder reconstructs the encoded audio. The reconstruction process is structurally decoupled, avoiding the "black box" nature of performance detection and making the reconstruction performance detection more accurate. At the same time, the reconstructed audio corresponding to different bandwidths can also reflect the global quality of the original audio at different bandwidths, thus reflecting the bandwidth adaptability of the original audio. It can accurately reflect the global quality of the original audio without relying on subjective evaluation results.

[0141] In some embodiments, the reconstruction performance test result includes a signal-to-noise ratio (SNR) improvement, the magnitude of which is negatively correlated with the global quality of the original audio. The reconstruction performance test for each of the reconstructed audio samples includes:

[0142] Determine the original signal power of the original audio and the reconstructed signal power of each of the reconstructed audios;

[0143] Determine the power difference between the reconstructed signal power and the original signal power;

[0144] The power difference is logarithmically calculated with the original signal power to obtain the signal-to-noise ratio improvement of each reconstructed audio.

[0145] In this embodiment, the signal-to-noise ratio improvement refers to the improvement of the signal-to-noise ratio of the reconstructed audio relative to the signal-to-noise ratio of the original audio. The magnitude of the signal-to-noise ratio improvement is negatively correlated with the global quality of the original audio. That is, the smaller the signal-to-noise ratio improvement, the richer the acoustic features contained in the original audio, the more natural the listening experience, and the better the global quality of the original audio.

[0146] In the specific implementation, the original signal power of the original audio is The power of the reconstructed audio signal is decode The power difference between the reconstructed signal power and the original signal power is:

[0147] =| decode - |(4)

[0148] Furthermore, the signal-to-noise ratio improvement for each reconstructed audio is determined by the logarithmic operation of the following equation (5):

[0149] (5)

[0150] The improvement in signal-to-noise ratio for each reconstructed audio segment can reflect the overall quality of the original audio across different bandwidths.

[0151] In practical applications, if the original audio is generative audio (such as TTS audio), the improvement in signal-to-noise ratio (SNR) of each reconstructed audio can also reflect the coding friendliness of generative audio on different bandwidths. The lower the SNR improvement, the higher the coding friendliness, the easier it is for the generative audio to be compressed efficiently and with high quality by the audio encoder, and the better the generation quality of the generative audio, thus providing direction for the optimization of generative audio algorithms.

[0152] In some examples, the adaptability of the original audio across different bandwidths can also be scored using quantitative metrics to reflect the overall quality of the original audio across different bandwidths, as follows: Figure 4 As shown:

[0153] The original audio is resampled and input into a deep learning model (such as EnCodec). The deep learning model encodes and decodes the resampled original audio at 3kps, 6kps, and 12kps using an encoder and decoder to obtain the reconstructed audio. The reconstruction performance of the reconstructed audio is then evaluated to determine the improvement in signal-to-noise ratio for each reconstructed audio. The bandwidth adaptability of the original audio is scored according to the following rules:

[0154] If the signal-to-noise ratio (SNR) improvement of the reconstructed audio is less than 3 dB, the original audio performs well in the corresponding bandwidth. If the SNR improvement of the reconstructed audio is greater than or equal to 3 dB and less than 10 dB, the original audio performs well in the corresponding bandwidth. If the SNR improvement of the reconstructed audio is greater than or equal to 10 dB and less than 20 dB, the original audio performs moderately well in the corresponding bandwidth. If the SNR improvement of the reconstructed audio is greater than or equal to 20 dB, the original audio performs well in the corresponding bandwidth.

[0155] Among them, the bandwidth adaptability performance can be represented by numerical indicators of different sizes, such as excellent, good, average, and poor, to reflect the corresponding global quality of the original audio on different bandwidths.

[0156] In this embodiment, the signal-to-noise ratio (SNR) improvement of each reconstructed audio is determined. This SNR improvement transforms subjective listening experience into an objective indicator, thereby improving the accuracy of the overall quality assessment of the original audio. Furthermore, the SNR improvement of reconstructed audio at different bandwidths can also reflect the adaptability of the original audio to different bandwidths, making it particularly suitable for quality detection of generated audio.

[0157] In some embodiments, the reconstruction performance test result further includes a reconstruction error index value, the magnitude of which is negatively correlated with the global quality of the original audio, and the reconstruction performance test for each of the reconstructed audios further includes:

[0158] The original audio samples are obtained by sampling the original audio.

[0159] Each of the reconstructed audio samples is sampled to obtain the reconstructed audio samples of each of the reconstructed audio samples;

[0160] By comparing the differences between each reconstructed audio sample and the original audio sample, the reconstruction error index value of each reconstructed audio is determined based on the differences.

[0161] In this embodiment, the magnitude of the reconstruction error index is negatively correlated with the global quality of the original audio. That is, the smaller the reconstruction error index, the smaller the difference between the reconstructed audio and the original audio, the easier it is to reconstruct the original audio, and the better the global quality of the original audio. The reconstruction error index of the reconstructed audio at different bandwidths can also reflect the adaptability of the original audio at different bandwidths.

[0162] In practical implementation, the original audio is sampled to obtain original audio samples, which can be composed of multiple sampling points of the original audio signal. Correspondingly, the same sampling method (such as the same sampling frequency and step size) can be applied to each reconstructed audio sample to obtain reconstructed audio samples for each reconstructed audio. These reconstructed audio samples can also be composed of multiple sampling points of the reconstructed audio signal. Further, the differences between each reconstructed audio sample and the original audio sample are compared. For example, the amplitude of each sampling point corresponding to each reconstructed audio sample is subtracted from the amplitude of the original audio sample to obtain the difference between each reconstructed audio sample and the original audio sample. Then, based on the difference, the reconstruction error index value for each reconstructed audio is determined. The reconstruction error index value can include mean square error, root mean square error, coefficient of determination, etc.

[0163] In some examples, the mean square error of each reconstructed audio is determined by the following equation (6):

[0164] (6)

[0165] in, For the original audio sample, To reconstruct audio samples;

[0166] Then through The root mean square error of each reconstructed audio is determined, and the reconstruction error index value of each reconstructed audio is obtained.

[0167] In practical applications, if the original audio is generative audio (such as TTS audio), the reconstruction error metric can also reflect the degree of optimization of the generative audio. The smaller the reconstruction error metric value, the better the quality of the generated audio. However, if the reconstruction error metric value is too small, it indicates that the generative audio has been over-optimized, resulting in distortion and unnaturalness. This provides guidance for the optimization direction of the generative audio.

[0168] In this embodiment, the differences between each reconstructed audio sample and the original audio sample are compared, and the reconstruction error index value is determined based on the differences. The reconstruction performance of the reconstructed audio is quantified by the reconstruction error index value, which reflects the robustness and inherent complexity of the original audio, and thus reflects the global quality of the original audio. It is especially suitable for accurately reflecting the global quality of generative audio.

[0169] In some embodiments, a multi-bandwidth encoding reconstruction function can be constructed to perform multi-bandwidth encoding and decoding. The multi-bandwidth encoding reconstruction function converts the input raw audio into a floating-point tensor, and then iterates through different bandwidth values. For each bandwidth, a configuration model (such as the EncodeC model) applies the target bandwidth. Next, in a mode where gradient computation is disabled, the function encodes the raw audio and then immediately decodes it to reconstruct the audio, obtaining the reconstructed audio. Afterwards, the mean squared error and signal-to-noise ratio (SNR) improvement between the reconstructed audio and the original audio are calculated. Finally, the function returns a dictionary containing the mean squared error and SNR improvement for each bandwidth to reflect the global quality of the original audio across different bandwidths.

[0170] Step 210: Obtain one or more sets of associated detection results from each of the anomaly detection results and each of the reconstruction performance detection results;

[0171] In the specific implementation, the correlation detection result is the detection result that is related to the quality detection of the original audio. Each group of correlation detection results may include one type of anomaly detection result, multiple reconstruction performance detection results, or a combination of multiple anomaly detection results and multiple anomaly detection results.

[0172] The correlation can be a causal correlation. For example, if an anomaly detection result is a cause of the low global quality of an original audio, then the anomaly detection result and the reconstruction performance detection result that characterizes the global quality of the original audio are a set of correlated detection results.

[0173] In some examples, if one anomaly detection result indicates that the original audio has a loudness anomaly, and one reconstruction performance detection result indicates that the overall listening fatigue of the original audio is high, then the anomaly detection result and the reconstruction performance detection result are a set of associated detection results.

[0174] Correlation can also be a relationship where both factors are considered influencing factors. For example, if multiple reconstruction performance test results can jointly characterize the global quality of the original audio in a certain dimension, then these reconstruction performance test results are correlated test results. Another example is that there is a correlation between factors influencing audio quality detection in the time domain and the frequency domain.

[0175] In some examples, multiple reconstruction performance test results can reflect the bandwidth adaptability of the original audio at different bandwidths; these reconstruction performance test results are considered correlated test results.

[0176] Step 212: Based on the correlation detection results, output the quality detection results of the original audio.

[0177] In practical implementation, the quality detection results of the original audio can be output by aggregating and analyzing the results of each group of associated detections; alternatively, a rule engine, decision tree, etc., can be used to make comprehensive decisions on the results of each group of associated detections to output the quality detection results of the original audio; or the results of each associated detection can be input into a pre-trained audio detection model, which will generate quality detection results for visualization of the original audio, such as quality detection results for graphics and text.

[0178] In practical applications, the quality inspection results of the original audio can be output through a structured report. This structured report can include detailed results of various anomaly detections, results of various reconstruction performance tests, various quality problems existing in the original audio, numerical quality inspection index values, and confidence scores for each test.

[0179] In some embodiments, outputting a quality detection result for the original audio based on the correlation detection result includes:

[0180] Among multiple preset conditions, the target conditions that each group of associated detection results meets are determined; wherein, each preset condition has associated quality detection results;

[0181] Output the quality detection results associated with each of the target conditions to obtain the quality detection results of the original audio.

[0182] In the specific implementation, each preset condition can be a numerical judgment condition or a complex logical judgment condition for the correlation detection result. When it is determined that the correlation detection result meets the corresponding target condition, it means that the original audio meets the situation corresponding to the correlation quality detection result. Then, the quality detection result of the target condition correlation is output, and the quality detection result of the original audio is obtained.

[0183] In practical applications, each preset condition can be configured as a corresponding preset rule. By establishing an association rule library to store multiple preset rules, and storing the audio quality detection results associated with each preset rule, the associated quality detection result can be determined by matching the preset rule that the associated detection result conforms to in the association rule library.

[0184] In some examples, for generated audio (such as TTS audio), optimization suggestions for the generated audio can be provided based on the associated quality detection results. The specific association rule base is shown in Table 1:

[0185] Table 1: Audio Quality Detection Association Rule Base

[0186] In this embodiment, by pre-configuring various preset conditions with associated quality detection results, and then determining the target conditions that the associated detection results meet, the quality detection results associated with each target condition are output. This enables rapid cross-validation of the original audio in terms of local anomaly detection and global quality detection, widely covering various scenarios of audio quality detection, and thus efficiently locating quality problems in the original audio.

[0187] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0188] In some embodiments, an audio detection system is also provided, which includes a four-dimensional analysis engine, a multimodal analysis engine, and a comprehensive decision engine. The workflow of this system is as follows: Figure 5As shown, it specifically includes:

[0189] The four-dimensional analysis engine includes a time-domain analysis module, a frequency-domain analysis module, a spatial analysis module, and a reconstruction analysis module. After the original audio is input into the engine, the time-domain analysis module detects anomalies such as clipping distortion, long silences, pops, signal-to-noise ratio (SNR), loudness, and clicks based on various time-domain features. The frequency-domain analysis module detects anomalies such as spectral roll-off, spectral flatness, and harmonic-to-noise ratio (HNR) based on various frequency-domain features. The spatial analysis module detects energy drop anomalies (either time-domain or frequency-domain) in the original audio to identify spatial anomalies in the energy space. The reconstruction analysis module reconstructs the original audio into a reconstructed audio file and performs reconstruction performance testing to reflect various global qualities of the original audio.

[0190] Furthermore, the anomaly detection results and reconstruction performance detection results are input into the multimodal analysis engine for correlation analysis, such as correlation between time-domain anomalies and frequency-domain anomalies, and correlation between frequency-domain anomalies and spatial anomalies, to obtain correlation detection results. The correlation detection results are then input into the comprehensive decision engine for comprehensive decision-making, and the quality detection results of the original audio are output, such as outputting a structured detection report and audio optimization suggestions.

[0191] The embodiments of this application have the following advantages: By performing anomaly detection of different anomaly types on the original audio using time-domain and frequency-domain features, multiple anomaly detection results can reflect the local quality of the original audio in multiple dimensions, achieving detailed perception of the original audio; by inputting the original audio into a pre-trained deep learning model for reconstruction, multiple reconstructed audios are obtained, and reconstruction performance detection is performed on each reconstructed audio, so that multiple reconstruction performance detection results can reflect different global qualities of the original audio, eliminating dependence on subjective evaluation and achieving overall evaluation of the original audio; furthermore, by associating detection results, multiple local qualities and multiple global qualities are multimodally correlated, achieving cross-validation of the original audio quality problem in multiple dimensions, thus efficiently and accurately outputting the audio quality detection results of the original audio without relying on subjective evaluation.

[0192] Based on the same inventive concept, this application also provides an audio quality detection apparatus for implementing the audio quality detection method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more audio quality detection apparatus embodiments provided below can be found in the limitations of the audio quality detection method described above, and will not be repeated here.

[0193] In one embodiment, such as Figure 6 As shown, an audio quality detection device 60 is provided, comprising:

[0194] Extraction module 601 is used to acquire the original audio and extract the time-domain and frequency-domain features of the original audio.

[0195] Anomaly detection module 602 is used to perform anomaly detection of different anomaly types on the original audio based on the time domain features and the frequency domain features, and obtain multiple anomaly detection results of the original audio;

[0196] The reconstruction module 603 is used to reconstruct the original audio input pre-trained deep learning model to obtain multiple reconstructed audio files;

[0197] The reconstruction performance detection module 604 is used to perform reconstruction performance detection on each of the reconstructed audios and obtain reconstruction performance detection results for each of the reconstructed audios; wherein, different reconstruction performance detection results are used to characterize the different global quality of the original audio.

[0198] The association module 605 is used to obtain one or more sets of association detection results from each of the anomaly detection results and each of the reconstruction performance detection results;

[0199] The quality detection result output module 606 outputs the quality detection result of the original audio based on the associated detection result.

[0200] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data including, but not limited to, raw audio, and related data such as the time-domain and frequency-domain characteristics of the raw audio. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements an audio quality detection method.

[0201] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0202] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the audio quality detection method described above.

[0203] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the audio quality detection method described above.

[0204] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the audio quality detection method described above.

[0205] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0206] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0207] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0208] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An audio quality detection method, characterized in that, The method includes: Obtain the original audio and extract its time-domain and frequency-domain features. Based on the time-domain features and the frequency-domain features, different types of anomaly detection are performed on the original audio to obtain multiple anomaly detection results for the original audio; The original audio input is reconstructed using a pre-trained deep learning model to obtain multiple reconstructed audio files; Reconstruction performance testing is performed on each of the reconstructed audio files to obtain reconstruction performance testing results for each of the reconstructed audio files; wherein, different reconstruction performance testing results are used to characterize different global quality of the original audio files; Obtain one or more sets of associated detection results from each of the anomaly detection results and each of the reconstruction performance detection results; Based on the correlation detection results, output the quality detection results of the original audio.

2. The method according to claim 1, characterized in that, The deep learning model includes an encoder and a decoder. The process of reconstructing the pre-trained deep learning model from the original audio input includes: The original audio is input into the deep learning model, and the encoder encodes the original audio according to different bandwidths to obtain multiple encoded audios; The decoder decodes each of the encoded audio files to obtain the reconstructed audio files.

3. The method according to claim 2, characterized in that, The reconstruction performance test results include the signal-to-noise ratio (SNR) improvement, the magnitude of which is negatively correlated with the global quality of the original audio. The reconstruction performance test for each of the reconstructed audio samples includes: Determine the original signal power of the original audio and the reconstructed signal power of each of the reconstructed audios; Determine the power difference between the reconstructed signal power and the original signal power; The power difference is logarithmically calculated with the original signal power to obtain the signal-to-noise ratio improvement of each reconstructed audio.

4. The method according to claim 2, characterized in that, The reconstruction performance test results include a reconstruction error index value, the magnitude of which is negatively correlated with the global quality of the original audio. The reconstruction performance test for each of the reconstructed audio samples includes: The original audio samples are sampled to obtain the original audio samples; the reconstructed audio samples are sampled to obtain the reconstructed audio samples of each of the reconstructed audio samples. By comparing the differences between each reconstructed audio sample and the original audio sample, the reconstruction error index value of each reconstructed audio is determined based on the differences.

5. The method according to claim 1, characterized in that, The anomaly types include various time-domain anomalies and various frequency-domain anomalies. The anomaly detection of the original audio based on the time-domain features and the frequency-domain features for different anomaly types includes: Based on the time-domain characteristics, determine the time-domain anomaly index value for each type of time-domain anomaly; Based on the frequency domain characteristics, determine the frequency domain anomaly index value for each type of frequency domain anomaly; Determine the threshold adjustment index value for each of the aforementioned anomalies, update the initial anomaly threshold for each of the aforementioned anomalies based on the threshold adjustment index value, and obtain the target anomaly threshold for each of the aforementioned anomalies. The time-domain anomaly index value and the frequency-domain anomaly index value are compared with the corresponding target anomaly threshold to detect different types of anomalies in the original audio.

6. The method according to claim 5, characterized in that, Each threshold adjustment index value is any one of the time-domain anomaly index value, the frequency-domain anomaly index value, the correlation index value of the time-domain anomaly index value, and the correlation index value of the frequency-domain anomaly index value. Determining the threshold adjustment index value for each anomaly type, updating the initial anomaly threshold for each anomaly type based on the threshold adjustment index value, and obtaining the target anomaly threshold for each anomaly type includes: Determine the threshold adjustment index value for each of the aforementioned anomaly types; Based on the preset normalization factor corresponding to each threshold adjustment index value, the threshold adjustment index values ​​are normalized according to the preset normalization function to obtain the normalized threshold adjustment index values. The initial anomaly threshold for each anomaly type is updated based on the normalized threshold adjustment index value to obtain the target anomaly threshold for each anomaly type.

7. The method according to claim 5, characterized in that, The frequency domain features include the average energy value of each frequency band in the multiple frequency bands corresponding to the original audio; the frequency domain anomalies include energy drop anomalies; Based on the frequency domain characteristics, determine the frequency domain anomaly index value for each type of frequency domain anomaly, including: The energy difference between the average energy values ​​of adjacent frequency bands is determined to obtain the frequency domain anomaly index value for the energy drop anomaly.

8. The method according to claim 5, characterized in that, The temporal features include the amplitude value of each temporal sampling point among multiple temporal sampling points corresponding to the original audio; the temporal anomalies include clicking sound anomalies; determining the temporal anomaly index value for each type of temporal anomaly based on the temporal features includes: Determine the amplitude difference between the amplitude values ​​of adjacent time-domain sampling points, and determine the signal abrupt change point in each time-domain sampling point based on each amplitude difference; The signal mutation point density of the signal mutation point is determined to obtain the time-domain anomaly index value for the clicking sound anomaly.

9. The method according to any one of claims 1 to 8, characterized in that, The step of outputting a quality detection result for the original audio based on the correlation detection result includes: Among multiple preset conditions, the target conditions that each group of associated detection results meets are determined; wherein, each preset condition has associated quality detection results; Output the quality detection results associated with each of the target conditions to obtain the quality detection results of the original audio.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images