Methods, apparatus, systems, devices, and storage media for measuring audio output delay
By sampling and analyzing audio signals using measurement equipment, the audio output delay is accurately measured, solving the problem of poor audio-visual synchronization in audio and video playback and improving audio-visual synchronization and user experience.
Patent Information
- Application Number
- CN202211275512.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-18
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-10-18
AI Technical Summary
During audio and video playback, audio output delay leads to poor audio-visual synchronization, affecting user experience. Existing technologies make it difficult to accurately measure and calibrate audio output delay.
The audio signal from the terminal under test is received and sampled by the measuring device. The audio sampling data and time are recorded. The audio output delay is determined by combining the data with the terminal under test. The analog signal is converted into a digital signal by an analog-to-digital converter for analysis. The main control chip is used for data processing to accurately measure the audio output delay.
It enables precise measurement of audio output latency, improves audio-visual synchronization during playback, and enhances the user experience.
Smart Images

Figure CN117915255B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of multimedia testing, and in particular to a method, apparatus, system, device, and storage medium for measuring audio output delay. Background Technology
[0002] In scenarios such as audio and video playback, after receiving an audio playback command, the audio player in the terminal still needs to perform processing such as obtaining and decoding the audio file. Furthermore, the processed audio signal needs to be transmitted to the speaker or headphone jack to complete the audio output process. This makes it impossible for the audio signal to be output in real time in response to the playback command, which means there is an unavoidable audio output delay.
[0003] In practical applications, audio output latency is an important factor affecting user experience. Therefore, it is often necessary to measure audio latency during the development and testing of terminals in order to calibrate the audio and video playback process and improve audio-visual synchronization. Summary of the Invention
[0004] This application provides a method, apparatus, system, device, and storage medium for measuring audio output delay, which can obtain accurate audio playback delay data. Based on this data, the accuracy of audio-visual synchronization during audio and video playback can be further improved. The technical solution is as follows:
[0005] On one hand, embodiments of this application provide a method for measuring audio output delay, the method being used in a terminal under test, the method comprising:
[0006] In response to an audio playback command, play the test audio;
[0007] The device receives audio sampling data and sampling time from a measuring device, wherein the audio sampling data is obtained by the measuring device sampling the received test audio according to the sampling frequency;
[0008] The start playback time of the test audio is determined based on the audio sampling data and the sampling time.
[0009] The audio output delay of the terminal under test is determined based on the start playback time and the instruction trigger time of the audio playback command.
[0010] On the other hand, embodiments of this application provide a method for measuring audio output delay, the method being used in a measuring device, the method comprising:
[0011] The audio signal output by the terminal under test is sampled according to the sampling frequency to obtain audio sampling data. The terminal under test is used to play test audio in response to received audio playback instructions.
[0012] The audio sampling data and sampling time are sent to the terminal under test so that the terminal under test can determine the start playback time of the test audio based on the audio sampling data and the sampling time, and determine the audio output delay based on the start playback time and the instruction trigger time of the audio playback command.
[0013] On the other hand, embodiments of this application provide an audio output delay measurement device, the device comprising:
[0014] The playback module is used to play test audio in response to audio playback commands;
[0015] The receiving module is used to receive audio sampling data and sampling time sent by the measuring device. The audio sampling data is obtained by the measuring device sampling the received test audio according to the sampling frequency.
[0016] The first determining module is used to determine the start playback time of the test audio based on the audio sampling data and the sampling time;
[0017] The second determining module is used to determine the audio output delay of the terminal under test based on the start playback time and the instruction trigger time of the audio playback instruction.
[0018] On the other hand, embodiments of this application provide an audio output delay measurement device, the device comprising:
[0019] The sampling module is used to sample the audio signal output by the terminal under test according to the sampling frequency to obtain audio sampling data. The terminal under test is used to play test audio in response to the received audio playback command.
[0020] The sending module is used to send the audio sampling data and sampling time to the terminal under test, so that the terminal under test can determine the start playback time of the test audio based on the audio sampling data and the sampling time, and determine the audio output delay based on the start playback time and the instruction trigger time of the audio playback command.
[0021] On the other hand, embodiments of this application provide an audio output delay measurement system, the system including a terminal under test and a measuring device; the measuring device is connected to the terminal under test through the audio output interface of the terminal under test;
[0022] The terminal under test is used to implement the audio output delay measurement method for the terminal under test as described above, and the measuring device is used to implement the audio output delay measurement method for the measuring device as described above.
[0023] On the other hand, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores at least one program, which is loaded and executed by the processor to implement the audio output delay measurement method as described above.
[0024] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one program, wherein the at least one instruction is loaded and executed by a processor to implement the audio output delay measurement method as described above.
[0025] On the other hand, embodiments of this application provide a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the audio output delay measurement method described above.
[0026] In this embodiment, an audio signal played by a terminal under test (DUT) is received by a measuring device, and the measuring device sends the sampled audio data to the DUT for data analysis. This determines the start time of the audio signal corresponding to the actual output test audio of the DUT, and further determines the delay between the trigger time of the audio playback command and the start time, i.e., the audio output delay. Based on the audio output delay measurement system composed of the measuring device and the DUT, this application can accurately measure the audio output delay corresponding to the DUT. Therefore, in scenarios such as audio and video playback, the audio output delay can be used to align the playback process of audio and video, and improve audio-visual synchronization. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 A schematic diagram of an audio output delay measurement system provided in an exemplary embodiment of this application is shown;
[0029] Figure 2 A flowchart illustrating a method for measuring audio output delay provided in an exemplary embodiment of this application is shown.
[0030] Figure 3A schematic diagram of the built-in chip of a measuring device provided in an exemplary embodiment of this application is shown;
[0031] Figure 4 A timing diagram of an illustrative analog-to-digital converter provided in an exemplary embodiment of this application is shown;
[0032] Figure 5 This illustration shows a schematic diagram of time synchronization processing provided by an exemplary embodiment of this application;
[0033] Figure 6 A flowchart illustrating a method for determining the start playback time provided in an exemplary embodiment of this application is shown;
[0034] Figure 7 A schematic diagram of a reference sampling point provided in an exemplary embodiment of this application is shown;
[0035] Figure 8 A schematic diagram of a reference sampling point provided by another exemplary embodiment of this application is shown;
[0036] Figure 9 A schematic diagram of a reference sampling point provided by another exemplary embodiment of this application is shown;
[0037] Figure 10 A structural block diagram of an audio output delay measurement device provided in an exemplary embodiment of this application is shown;
[0038] Figure 11 A structural block diagram of an audio output delay measurement device provided in another exemplary embodiment of this application is shown;
[0039] Figure 12 This invention provides a structural block diagram of a terminal under test according to an exemplary embodiment of the present application.
[0040] Figure 13 A structural block diagram of a measuring device provided in an exemplary embodiment of this application is shown. Detailed Implementation
[0041] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0042] For ease of understanding, the terms used in the embodiments of this application will be explained below.
[0043] An analog-to-digital converter (ADC) is an electronic component that converts analog signals into digital signals. Typically, an ADC converts an input voltage signal into an output digital signal. Since digital signals themselves do not have inherent meaning and only represent a relative magnitude, any ADC needs a reference analog signal as a standard for conversion. A common reference standard is the magnitude of the largest convertible signal, while the output digital signal represents the magnitude of the input signal relative to the reference signal.
[0044] Audio output latency, also known as audio playback latency, refers to the time elapsed between when an application with audio playback capabilities executes an audio playback command and when the audio file is played through the headphone jack or built-in speaker. In audio and video playback scenarios, devices need to calibrate the audio and video playback process in terms of timing based on audio playback latency to avoid audio-visual desynchronization that would negatively impact the user experience.
[0045] Please refer to Figure 1 This illustration shows a schematic diagram of an audio output delay measurement system provided in an exemplary embodiment of this application. The system includes a terminal under test (DUT) 110 and a measuring device 120, wherein the measuring device 120 is connected to the DUT 110 via an audio output interface 130.
[0046] The terminal under test 110 can be a smart TV, smartphone, tablet computer, or other device with audio playback functionality. This audio playback functionality can be a native feature of a native application on the terminal under test, or it can be a feature of a third-party application. In this embodiment, the terminal under test is equipped with a test program 111 for measuring audio output latency. This test program has audio playback functionality and can be used to play test audio.
[0047] The measuring device 120 is an electronic device with audio signal analysis function. The measuring device 120 includes an analog-to-digital converter 122, which is used to convert the sampled audio analog signal into audio data so as to analyze the audio signal based on the obtained audio data.
[0048] In this embodiment, the terminal under test (DUT) 110 plays test audio through test program 111 and records the command trigger time 112. The DUT then outputs an audio signal 113 corresponding to the test audio. This audio signal 113 is an analog signal, and there is an audio output delay between the command trigger time 112 and the output time of the audio signal 113. Further, the measuring device 120 samples the audio signal 113 at the audio output interface 130 and records the sampling time 121. Then, the measuring device 120 performs analog-to-digital conversion on the sampled audio signal 113 using an analog-to-digital converter 122 to obtain audio sampling data 123. Upon obtaining the above data, the measuring device 120 transmits the sampling time 121 and the audio sampling data 123 to the DUT 110 so that the DUT 110 can process the data to obtain the audio output delay.
[0049] In one possible implementation, the measuring device 120 and the terminal under test 110 can transmit data via Wi-Fi or other means. Furthermore, the measuring device 120 can simultaneously establish Wi-Fi connections with multiple terminals under test 110 that have audio playback capabilities. This embodiment is only illustrated by the example of the measuring device 120 measuring the audio output delay of only one terminal under test 110, but it does not constitute a limitation.
[0050] Please refer to Figure 2 This document illustrates a flowchart of an audio output delay measurement method proposed in an exemplary embodiment of this application. This application uses this method for... Figure 1 The terminal under test and the measuring equipment shown are described.
[0051] Step 201: The terminal under test responds to the audio playback command and plays the test audio.
[0052] The measurement system for audio output delay is based on the device under test (DUT) and the measuring equipment. This system is used to measure the delay between when the DUT receives and responds to an audio playback command and when it actually outputs the audio signal corresponding to the test audio. This delay refers to the audio output delay, which is the delay caused by the DUT's operations such as acquiring the audio file and decoding and transmitting the audio file after the audio playback command triggers the DUT to play the audio.
[0053] In one possible implementation, the terminal under test (DUT) is equipped with a measurement program for measuring audio output delay. This program has an audio playback function and can play test audio in response to an audio playback command. Correspondingly, while playing the test audio, the DUT can record the command trigger time, i.e., the moment the audio playback command triggers the DUT to play the test audio. After receiving the audio playback command and completing operations such as acquiring, decoding, and transmitting the audio file, the DUT outputs the audio signal corresponding to the test audio from its audio output interface at the terminal's audio output power. The duration of the signal output is the audio playback duration. For example, the DUT can output an audio signal with a duration of 50ms at an audio output power of 50mW.
[0054] Step 202: The measuring device samples the audio signal output by the terminal under test according to the sampling frequency to obtain audio sampling data. The terminal under test is used to play the test audio when it receives an audio playback command.
[0055] In the aforementioned audio output delay measurement system, the measuring device samples the output audio signal at the audio output interface of the terminal under test (DUT) and records the audio signal voltage corresponding to each sampling point based on the sampling frequency. Illustratively, the sampling frequency of the measuring device can be 10kHz, and a high-precision 32Ω sampling resistor can be used. Through actual measurement data, it can be found that when the DUT outputs an audio signal with a 50mW audio output, and the measuring device uses a 32Ω sampling resistor for sampling, the audio signal voltage range is -1.3V to +1.3V. Furthermore, at a sampling frequency of 10kHz, the maximum error introduced by the sampling delay is less than 0.05ms, as measured in practice.
[0056] In one possible implementation, since the audio signal output by the terminal under test is an analog signal, in order to further determine the audio output delay by analyzing the test audio, the measuring device needs to convert the acquired audio signal into a digital signal through an analog-to-digital converter. Furthermore, the analog-to-digital converter transmits the digital signal to the main control chip for data processing and storage, that is, to obtain audio sampling data containing the audio signal voltage.
[0057] As an illustration, a MAX19191 Flash analog-to-digital converter can be used in the measuring equipment to convert the acquired analog signal into a digital signal, and then transmit the digital signal to the main control chip for further data processing. Correspondingly, the main control chip in the measuring equipment can be an ESP32-DOWD-V3 (hereinafter referred to as ESP32). For example... Figure 3As shown, the ADC and the main control chip are connected via GPIO (General Purpose Input Output) to D0-D7 and DVAL (Data Valid). During the sampling of the audio signal, as... Figure 4 As shown, the ESP32 outputs a PWM (Pulse Width Modulation) signal via its built-in timer and connects to the clock. The ADC starts analog-to-digital conversion on the rising edge of the clock (CLK), and after the conversion is complete, it sets the data corresponding to the obtained digital signal to D0-D7 (data pins). It further sets the high level to DVAL so that the ESP32 can read the 8-bit data corresponding to D0-D7 when DVAL is high and store it in memory.
[0058] Step 203: The measuring device sends audio sampling data and sampling time to the terminal under test, so that the terminal under test can determine the start playback time of the test audio based on the audio sampling data and sampling time, and determine the audio output delay based on the start playback time and the instruction trigger time of the audio playback command.
[0059] Since the main control chip of the measuring device stores audio sampling data, the measuring device can send the audio sampling data to the terminal under test (DUT) via Wi-Fi, Bluetooth, or other means, so that the DUT can process the audio sampling data. It should be noted that the above data transmission methods are for illustrative purposes only and are not intended to limit the scope of this application.
[0060] Optionally, the measuring device can record the sampling time corresponding to each sampling point and send the sampling time to the terminal under test, so that the terminal under test can determine the audio output delay based on the audio sampling data and the sampling time; the measuring device can also send the start sampling time and sampling frequency to the terminal under test, so that the terminal under test can determine the sampling time corresponding to each sampling point.
[0061] Indicative, such as Figure 3 As shown, the main control chip in the measuring device can be ESP32-D0WD-V3. The ESP32 has a built-in Wi-Fi function, which can be used to send the audio sampling data and sampling time stored in the memory to the device under test. Furthermore, according to actual measurements, based on the timer frequency of the ESP32 being 80MHz, the error introduced during the sampling process due to the accuracy limitation of the measuring device is less than 1 / 80µs.
[0062] Step 204: The terminal under test receives the audio sampling data and sampling time sent by the measuring device. The audio sampling data is obtained by the measuring device sampling the received test audio according to the sampling frequency.
[0063] The audio sampling data contains the audio signal voltage corresponding to each sampling point. Accordingly, the terminal under test can match the received sampling time with the audio signal voltage to obtain complete sampling point information.
[0064] Step 205: The terminal under test determines the start playback time of the test audio based on the audio sampling data and the sampling time.
[0065] Based on the audio signal voltage contained in the audio sampling data, which reflects the audio characteristics corresponding to the test audio, the terminal under test can determine the start time of the test audio playback in the terminal under test by combining the sampling time, that is, determine the time when the terminal under test actually starts to output the audio signal corresponding to the test audio.
[0066] Step 206: The terminal under test determines the audio output delay based on the start playback time and the instruction trigger time of the audio playback command.
[0067] Since the audio output time indicates the time when the terminal under test responds to the audio playback command and the time when it actually outputs the audio signal, the terminal under test can determine that the time difference between the above-mentioned start playback time and command trigger time is the audio output delay.
[0068] It should be noted that in calculating the audio output delay, the terminal under test (DUT) uses both the start playback time determined by the clock of the measuring device and the command trigger time determined by the clock of the DUT. An unavoidable time difference exists between these two clocks. To eliminate the calculation error introduced by this time difference, the DUT can synchronize its time with the measuring device to determine the time difference between them.
[0069] like Figure 5 As shown, in one possible implementation, the test program set in the terminal under test can transmit the current time T corresponding to the clock in the terminal under test. a The data packet is packaged and sent to the measuring device via the time data packet P1. Upon receiving the data packet P1, the measuring device synchronously records the receiving time T. b And the current time T of the measuring device c With T a And T b The data is packaged and further processed. The measuring device sends the measuring device time data packet P2 to the terminal under test. Upon receiving the data packet P2, the terminal under test synchronously records the reception time T. d At each of the above time points, the corresponding T for the terminal under test a Time and the corresponding T of the measuring device bThe time difference between moments indicates the time required for the device under test (DUT) to package data, send the data packet to the measuring device (TMD), and for it to be received. This time difference also includes the clock error between the two devices. Correspondingly, the T value for the measuring device is... c Time and T corresponding to the terminal under test d The time difference between two points indicates the time required for the measuring device to package data, send the data packet to the terminal under test (DUT), and for the DUT to receive it, as well as the clock error between the two. Furthermore, averaging these two time differences determines the device time difference between the DUT and the measuring device. This calculation can be expressed by the following formula:
[0070] ΔT=[(T b -T a )+(T d -T c )] / 2
[0071] Furthermore, the terminal under test can correct the start playback time based on the device time difference, and then determine the audio output delay of the terminal under test based on the corrected start playback time and the instruction trigger time of the audio playback command.
[0072] The process of correcting the start playback time based on the device time difference can be expressed by the following mathematical formula:
[0073] T2′=T2+ΔT
[0074] Where T2 is the start playback time, ΔT is the device time difference calculated above, and T2′ is the corrected start playback time. Then, given the corrected start playback time, the audio output delay of the terminal under test is calculated using the following formula:
[0075] Audio output delay = T2′ - T1
[0076] Where T1 is the command trigger time, and T2′ is the corrected start playback time. Data packets are transmitted between the terminal under test and the measuring device via a direct Wi-Fi connection. The error introduced in this process is very small; based on actual testing, the time synchronization error between the two devices is less than ±1ms.
[0077] In summary, in this embodiment, the measuring device receives the audio played by the terminal under test (DUT), and sends the sampled audio data to the DUT for data analysis. This allows for the determination of the start time of the audio signal corresponding to the actual output test audio from the DUT, and further determines the delay between the trigger time of the audio playback command and the start time, i.e., the audio output delay. Based on the audio output delay measurement system composed of the measuring device and the DUT, this application can accurately measure the audio output delay corresponding to the DUT. Therefore, in scenarios such as audio and video playback, the audio output delay can be used to align the playback process of audio and video, improving audio-visual synchronization.
[0078] Audio sampling data contains the audio signal voltage corresponding to each sampling point. Based on the audio signal voltage corresponding to each sampling point and the sampling time, the terminal under test can obtain discrete audio sampling data characterizing the test audio. For example... Figure 6 As shown, the method for determining the start playback time of the terminal under test based on audio sampling data may include the following steps:
[0079] Step 601: Determine reference sampling points based on audio sampling data.
[0080] In actual measurement processes, due to factors such as hardware equipment, a certain amount of sampling noise is unavoidable in audio sampling data. Experimental data shows that the overall sampling noise follows a normal distribution, and schematically, the measured noise voltage range is between -0.015V and +0.015V. Because of the presence of sampling noise, the moment the measured device first acquires the audio signal voltage from the audio sampling data obtained by the terminal under test cannot accurately indicate the actual start time of the audio signal output. For example, as... Figure 6 As shown, when the test audio is a sine wave signal, the terminal under test starts playing the test audio from the point where the audio signal voltage is 0. When the measuring device collects the audio signal, the audio sampling data is severely affected by sampling noise at the point where the audio signal voltage is low, making it difficult to determine the start playback time based on the audio sampling data.
[0081] Therefore, in one possible implementation, the terminal under test (DUT) determines a reference sampling point that can characterize the audio features of the test audio. This reference sampling point can be a sampling point with a relatively high corresponding audio signal voltage, in order to reduce the impact of sampling noise on the calculation process and results. When the audio curve of the test audio is a sine or cosine curve, the DUT can determine the sampling point where the audio signal voltage reaches its extreme value in the audio sampling data as the reference sampling point, and further calculate the time interval between the sampling reference point and the starting playback point, thus determining the starting playback time of the test audio. Figure 7As shown, based on the premise that the reference sampling point should be a sampling point with a higher voltage for the same audio signal, when the audio curve is a sine curve, the reference sampling point determined by the terminal under test based on the audio sampling data can be the first maximum point (at 1 / 4 of the cycle) or the first minimum point (at 3 / 4 of the cycle) of the audio curve in the first cycle.
[0082] Step 602: Determine the time interval between the reference sampling point and the starting playback point of the test audio.
[0083] Since the reference sampling point often differs from the sampling point corresponding to the start playback time, the device under test (DUT) needs to further determine the time interval between the reference sampling point and the start playback time in order to determine the start playback time based on the sampling time corresponding to the reference sampling point. Specifically, the DUT can determine this time interval based on the audio characteristics of the test audio represented by the audio sampling data.
[0084] In one possible implementation, if the reference sampling point determined by the terminal under test based on the audio sampling data is the sampling point when the audio signal voltage reaches an extreme value, the terminal under test determines the time interval between the reference sampling point and the starting playback point of the test audio based on the audio frequency of the test audio and the periodic position of the extreme point corresponding to the extreme value in the audio curve.
[0085] Indicatively, the test audio is a sine wave with a frequency of 880Hz. The sampling point corresponding to the maximum value of the audio signal voltage in the first sine cycle of the test audio is determined by the terminal under test as the reference sampling point, which is the sampling point corresponding to 1 / 4 of the first cycle of the test audio. Therefore, the time interval between the reference sampling point and the starting playback point is the time corresponding to 1 / 4 of the cycle. Since the audio frequency is 880Hz, the duration of one cycle can be calculated as 1 / 880s. Therefore, the time interval between the reference sampling point and the starting playback point of the test audio is (1 / 880)*(1 / 4)s.
[0086] It should be noted that, in this embodiment, since the terminal under test does not perform function fitting on the audio sampling data, that is, the above-mentioned reference sampling points are the actual sampling points corresponding to the audio sampling data, rather than the fitted sampling points obtained by filtering through the function fitting process, the audio signal voltage corresponding to the above-mentioned reference sampling points contains sampling noise. Consequently, there is a large error in the time interval between the reference sampling points calculated in the above embodiment and the starting playback point of the test audio. To reduce the impact of sampling noise on the calculation results, the terminal under test can determine at least two reference starting playback times based on the sampling times and time intervals of at least two reference sampling points to participate in subsequent calculations.
[0087] Step 603: Determine the start playback time of the test audio based on the sampling time of the reference sampling point and the time interval.
[0088] The time interval characterizes the time interval between the reference sampling point and the starting playback point. This time interval is the difference between the sampling time of the reference sampling point and the starting playback time. The starting playback time of the test audio can then be determined by subtraction.
[0089] Indicatively, the test audio is a sine wave with an audio frequency of 880Hz. Assuming the reference sampling point determined by the terminal under test based on the audio sampling data is the maximum value of the audio signal voltage within the first cycle, as shown in the above embodiment, the time interval is (1 / 880)*(1 / 4)s. Therefore, the formula for calculating the start playback time can be expressed as:
[0090]
[0091] Where T2 is the start playback time, T max This is the sampling time corresponding to the reference sampling point at the peak.
[0092] It should be noted that if the terminal under test determines at least two reference start times, the terminal under test can determine the average of the at least two reference start times as the start time of the test audio.
[0093] The audio sampling data contains the audio signal voltage corresponding to each sampling point. The audio signal voltage corresponding to each sampling point and the sampling time constitute a discrete data graph characterizing the test audio. Given the unavoidable presence of sampling noise in the audio sampling data, when the test audio played by the terminal under test (DUT) is a sine or cosine wave, the DUT can perform function fitting based on the audio signal voltage corresponding to each sampling point and the sampling time to reduce the impact of sampling noise on the audio output delay calculation. In other words, the DUT can achieve a noise reduction effect similar to filtering through function fitting. Furthermore, the DUT can construct the corresponding audio curve based on the obtained fitted function, that is, a sine or cosine curve representing the test audio waveform.
[0094] Based on the trigonometric function expression: Here, b represents the offset of the curve corresponding to the function along the vertical axis. In this embodiment, the sine or cosine wave corresponding to the test audio has no offset along the vertical axis, and b is set to 0 when performing function fitting. In actual measurement, during the sampling of the audio signal corresponding to the test audio output by the terminal under test, sampling noise is unavoidable due to hardware limitations of the measurement equipment, affecting the accuracy of the audio signal voltage. Since the true value of b is a constant of 0, the terminal under test can use this feature to correct the audio sampling data.
[0095] In one possible implementation, the terminal under test determines the sum of the audio signal voltages of the audio sampling data of the reference sampling point and the sampling points after the reference sampling point.
[0096] During the sampling process of the measuring device, the sampling period corresponding to the fitted sampling point is an integer multiple of the sine or cosine period. For example, based on T = 1 / f, when the audio frequency is 880Hz, the sine or cosine period corresponding to the audio curve is 1 / 880s. Therefore, a sampling period of 25ms can be selected, which is 22 times 1 / 880s. That is, an audio sampling data with a sampling period of 25ms contains 22 complete sine or cosine periods. Correspondingly, if there is no offset of the audio curve in the vertical direction, the sum of the audio signal voltages corresponding to all sampling points should be 0.
[0097] Furthermore, when the audio sampling data is affected by sampling noise, i.e., when the sum of the audio signal voltages is not zero, the terminal under test determines the correction voltage based on the sum of the audio signal voltages and the number of fitted sampling points. This calculation process can be represented by the following mathematical expression:
[0098]
[0099] Where dc is the correction voltage, n is the number of fitted sampling points, and di is the audio signal voltage corresponding to the i-th sampling point. Furthermore, the terminal under test corrects the audio sampling data of the fitted sampling points based on the correction voltage, that is, letting di′=di+dc, where di′ is the corrected audio sampling data.
[0100] Based on the correction of the audio sampling data, the method for fitting a function to the discrete audio sampling data by the terminal under test to obtain the fitting function may include the following steps:
[0101] 1. Based on the audio frequency of the test audio, the audio sampling data of the fitting sampling points, and the sampling frequency, a function is fitted to obtain the fitting function. The fitting sampling points include the reference sampling point and the sampling points after the reference sampling point. The fitting function is used to characterize the audio curve of the fitted test audio.
[0102] In one possible implementation, when the audio curve corresponding to the test audio is a sine function or a cosine function, the function expression corresponding to the fitting function can be:
[0103]
[0104] Where ω = 2πf, f is the audio frequency, and f can be taken as 880Hz for illustration.
[0105] In order to determine the start playback time by fitting a function to the audio sampling data, in this embodiment of the application, the reference sampling point is set as the starting point of the fitting function, that is, the x value at the reference sampling point is 0. Accordingly, the initial phase in the above function expression represents the difference between the reference time corresponding to the reference sampling point and the start playback time. That is, the initial phase is numerically equal to the time interval between the reference sampling point and the start playback point.
[0106] The method for determining the reference sampling point can be any of the following:
[0107] A. When the audio sampling data is recorded by the measuring device when the audio signal voltage is sampled, the terminal under test determines the sampling point where the audio signal voltage first reaches the voltage threshold in the audio sampling data as the reference sampling point.
[0108] like Figure 8 As shown, taking the audio curve corresponding to the test audio as a sine function curve as an example, the measuring device starts sampling as soon as it receives the audio signal. When the audio signal voltage corresponding to the early sampling point of the first audio cycle is small, the audio sampling data is greatly affected by sampling noise and has a large error. Therefore, the terminal under test can determine the sampling point corresponding to the first time the voltage threshold is reached in the fitted sampling points as the reference sampling point based on the set voltage threshold.
[0109] It should be noted that the voltage threshold should be as large as possible to reduce the impact of sampling noise on the audio sampling data. However, in order to ensure that the reference sampling point and the sampling points after the reference sampling point can represent a complete audio cycle, the voltage threshold needs to maintain a certain distance from the voltage peak. For example, when the voltage peak is +1.3V, the voltage threshold can be +1V.
[0110] B. When the audio sampling data is recorded by the measuring device when the audio signal voltage first reaches the voltage threshold, the terminal under test determines the sampling point corresponding to the first audio sampling data as the reference sampling point.
[0111] like Figure 9As shown, the measuring device can set a voltage threshold as a sampling threshold, and start sampling the audio signal only when the audio signal voltage reaches the voltage threshold. The first sampling point in the audio sampling data is the reference sampling point, and the audio sampling data only includes the reference sampling point and the sampling points after the reference sampling point.
[0112] Therefore, given the starting point of the x-axis, we can determine the value of x = i / F for each sampling point, where F is the sampling frequency of the measuring device and i refers to the i-th sampling point. For illustration, if the sampling frequency is 10kHz, then with the reference sampling point as the starting point of the x-axis, the sampling time corresponding to each sampling point can be expressed as x = i / 10000.
[0113] Furthermore, in this embodiment, the audio sampling data is fitted using the least squares method. Based on the corrected actual audio sampling data di′, the terminal under test calculates the sum of squared errors E:
[0114]
[0115] Based on the fact that a function reaches its extreme value when its derivative is 0, the terminal under test can calculate A and B in the function expression by combining the following formulas. The value of .
[0116]
[0117]
[0118] in, This represents the initial phase of the corresponding time interval, and A is the amplitude in the fitting function, corresponding to the voltage range in the audio data. Furthermore, the values of each parameter in the above fitting function's expression can be determined, meaning the terminal under test has obtained the fitting function.
[0119] 2. Determine the time interval between the reference sampling point and the starting playback point of the test audio based on the fitting function.
[0120] In one possible implementation, the terminal under test can determine the initial phase in the fitting function as the time interval between the reference sampling point and the starting playback point of the test audio, which is expressed by the following mathematical expression:
[0121]
[0122] Where T3 is the sampling time corresponding to the reference sampling point, and T2 is the start playback time.
[0123] In the process of calculating the audio output delay through the above embodiments, that is, when the terminal under test performs function fitting on the audio sampling data and determines the audio output delay based on the fitting function, the function fitting calculation process will introduce a certain error, and the error estimation formula can be expressed as:
[0124]
[0125] Where f is the audio frequency. Indicatively, with a noise voltage peak of 0.015V, an audio signal voltage peak of 1.3V, and an audio frequency of 880Hz, the fitting error is approximately 6.5µs.
[0126] In summary, in this embodiment, when the audio curve corresponding to the test audio is a sine or cosine function, the terminal under test reduces the calculation error caused by sampling noise by performing function fitting on the obtained audio sampling data, achieving an effect similar to data filtering. During the function fitting process, by controlling the abscissa value of the reference sampling point to be 0, the initial phase value in the fitting function is the time interval between the reference sampling point and the starting playback point. Thus, while completing the function fitting, the time interval used to calculate the audio output delay can be obtained.
[0127] Please refer to Figure 10 This illustration shows a structural block diagram of an audio output delay measurement device proposed in an exemplary embodiment of this application. This application uses this device for... Figure 1 The device shown is a terminal under test and includes:
[0128] Playback module 1001 is used to play test audio in response to audio playback commands;
[0129] The receiving module 1002 is used to receive audio sampling data and sampling time sent by the measuring device. The audio sampling data is obtained by the measuring device sampling the received test audio according to the sampling frequency.
[0130] The first determining module 1003 is used to determine the start playback time of the test audio based on the audio sampling data and the sampling time;
[0131] The second determining module 1004 is used to determine the audio output delay of the terminal under test based on the start playback time and the instruction trigger time of the audio playback instruction.
[0132] Optionally, the first determining module 1003 is further configured to:
[0133] Reference sampling points are determined based on the audio sampling data;
[0134] Determine the time interval between the reference sampling point and the starting playback point of the test audio;
[0135] Based on the sampling time of the reference sampling point and the time interval, the start playback time of the test audio is determined.
[0136] Optionally, if the audio curve of the test audio is a sine curve or a cosine curve, the first determining module 1003 is further configured to:
[0137] Based on the audio frequency of the test audio, the audio sampling data of the fitting sampling points, and the sampling frequency, a function is fitted to obtain a fitting function. The fitting sampling points include the reference sampling point and the sampling points after the reference sampling point. The fitting function is used to characterize the audio curve of the fitted test audio.
[0138] The time interval between the reference sampling point and the starting playback point of the test audio is determined based on the fitting function.
[0139] Optionally, if the audio curve of the test audio is a sine curve or a cosine curve, the first determining module 1003 is further configured to:
[0140] When the audio sampling data is started being recorded by the measuring device when the audio signal voltage is sampled, the sampling point in the audio sampling data where the audio signal voltage first reaches the voltage threshold is determined as the reference sampling point;
[0141] or,
[0142] When the audio sampling data is recorded by the measuring device when the audio signal voltage first reaches the voltage threshold, the sampling point corresponding to the first audio sampling data is determined as the reference sampling point.
[0143] Optionally, if the audio curve of the test audio is a sine curve or a cosine curve, the first determining module 1003 is further configured to:
[0144] The initial phase in the fitting function is determined as the time interval between the reference sampling point and the starting playback point of the test audio.
[0145] Optionally, when the audio curve of the test audio is a sine curve or a cosine curve, and the sampling period corresponding to the fitted sampling point is an integer multiple of the sine period or the cosine period, the first determining module 1003 is further configured to:
[0146] Determine the sum of the audio signal voltages of the audio sampling data at the reference sampling point and the sampling points after the reference sampling point;
[0147] If the sum of the audio signal voltages is not zero, a correction voltage is determined based on the sum of the audio signal voltages and the number of fitted sampling points;
[0148] The audio sampling data of the fitted sampling points is corrected based on the corrected voltage.
[0149] Optionally, if the audio curve of the test audio is a sine curve or a cosine curve, the first determining module 1003 is further configured to:
[0150] The sampling point in the audio sampling data where the audio signal voltage reaches an extreme value is determined as the reference sampling point;
[0151] Determining the time interval between the reference sampling point and the starting playback point of the test audio includes:
[0152] Based on the audio frequency of the test audio and the periodic position of the extreme point corresponding to the extreme value in the audio curve, the time interval between the reference sampling point and the starting playback point of the test audio is determined.
[0153] Optionally, the device further includes a time synchronization module for synchronizing with the measuring device to determine the device time difference between the terminal under test and the measuring device;
[0154] After determining the start playback time of the test audio based on the audio sampling data and the sampling time, the first determining module 1003 is further configured to:
[0155] The start playback time is corrected based on the device time difference;
[0156] The second determining module 1004 is further configured to:
[0157] The audio output delay of the terminal under test is determined based on the corrected start playback time and the instruction trigger time of the audio playback command.
[0158] In summary, in this embodiment, the terminal under test plays test audio through a playback module and receives audio sampling data and sampling time sent by a measuring device through a receiving module. This allows the first determining module to determine the start playback time of the audio signal corresponding to the actual output test audio from the terminal under test, and the second determining module to determine the delay between the instruction trigger time and the start playback time when the terminal under test plays audio from an audio playback instruction—that is, the audio output delay. Based on the audio output delay measurement system composed of the measuring device and the terminal under test, this application can accurately measure the audio output delay corresponding to the terminal under test. Furthermore, in scenarios such as audio and video playback, the audio output delay can be used to align the playback process of audio and video, improving audio-visual synchronization.
[0159] Please refer to Figure 11 This illustration shows a structural block diagram of an audio output delay measurement device proposed in an exemplary embodiment of this application. This application uses this device for... Figure 1 The measuring device shown is described below. The device includes:
[0160] The sampling module 1101 is used to sample the audio signal output by the terminal under test according to the sampling frequency to obtain audio sampling data. The terminal under test is used to play test audio in response to a received audio playback command.
[0161] The sending module 1102 is used to send the audio sampling data and sampling time to the terminal under test, so that the terminal under test can determine the start playback time of the test audio based on the audio sampling data and the sampling time, and determine the audio output delay based on the start playback time and the instruction trigger time of the audio playback instruction.
[0162] Please refer to Figure 12 This illustration shows a structural block diagram of a terminal under test (DUT) 1200 provided in an exemplary embodiment of this application. The DUT 1200 can be a portable mobile terminal, such as a smartphone, tablet computer, Moving Picture Experts Group Audio Layer III (MP3) player, or Moving Picture Experts Group Audio Layer IV (MP4) player. The DUT 1200 may also be referred to as a user equipment, portable terminal, or other names.
[0163] Typically, the terminal under test 1200 includes a processor 1201 and a memory 1202.
[0164] Processor 1201 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1201 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1201 may also include a main processor and a coprocessor. The main processor, also known as a central processing unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1201 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1201 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.
[0165] The memory 1202 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1202 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1202 are used to store at least one instruction, which is executed by the processor 1201 to implement the audio output delay measurement method provided in this application embodiment.
[0166] Optionally, the terminal under test 1200 may also include a communication component 1203.
[0167] The communication component 1203 is used to establish a communication connection with the terminal under test 1200 and the measuring equipment.
[0168] Please refer to Figure 13 The diagram shows a structural block diagram of a measuring device 1300 provided in an exemplary embodiment of this application.
[0169] The measuring device 1300 includes: a processor 1301 and a memory 1302.
[0170] Processor 1301 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 1301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 1301 may include a main processor and a coprocessor. The main processor is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.
[0171] Memory 1302 may include one or more computer-readable storage media, which may be tangible and non-transitory. Memory 1302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 1302 is used to store at least one instruction, which is executed by processor 1301 to implement the audio output delay measurement method provided in the above-described embodiments.
[0172] Optionally, the measuring device 1300 may also include: an analog-to-digital converter 1303 and a communication component 1304.
[0173] The analog-to-digital converter 1303 is used to perform analog-to-digital conversion on the acquired analog signal, i.e., the audio signal corresponding to the test audio, to obtain audio sampling data. The communication component 1304 can be a Bluetooth component, a serial communication component, etc., or it can be the communication function of the main control chip built into the measuring device 1300. The measuring device 1300 can establish a data communication connection with the terminal under test through the communication component 1304.
[0174] This application also provides a computer-readable storage medium storing at least one program that is executed by a processor to implement the audio output delay measurement method as described in the above embodiments.
[0175] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the audio output delay measurement method provided in the above embodiments.
[0176] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.
[0177] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A method of measuring audio output delay, characterized by, The method is used for a terminal under test, and the method includes: In response to an audio playback command, test audio of a measurement program is played, the measurement program being set in the terminal under test; The measuring device receives audio sampling data and sampling time sent by the measuring device. The measuring device is connected to the terminal under test through the audio output interface of the terminal under test. The audio sampling data is obtained by the measuring device sampling the received test audio according to the sampling frequency. The audio sampling data includes audio signal voltage that reflects the audio characteristics corresponding to the test audio. Based on the audio sampling data, a reference sampling point is determined. The reference sampling point is the sampling point in the audio sampling data where the audio signal voltage reaches an extreme value, which is used to reduce the influence of sampling noise. Based on the audio frequency of the test audio, the audio sampling data of the fitting sampling points, and the sampling frequency, a function is fitted to obtain a fitting function. The fitting sampling points include the reference sampling point and the sampling points after the reference sampling point. The fitting function is used to characterize the audio curve of the fitted test audio, and the audio curve of the test audio is a sine curve or a cosine curve. The time interval between the reference sampling point and the starting playback point of the test audio is determined based on the fitting function. Based on the sampling time of the reference sampling point and the time interval, the start playback time of the test audio is determined; The device time difference between the terminal under test and the measuring device is determined by synchronizing with the measuring device; the starting playback time is corrected based on the device time difference. The audio output delay of the terminal under test is determined based on the corrected start playback time and the instruction trigger time of the audio playback command.
2. The method of claim 1, wherein, Determining the reference sampling point based on the audio sampling data includes: When the audio sampling data is recorded by the measuring device when the audio signal voltage is sampled, the sampling point where the audio signal voltage first reaches the voltage threshold in the audio sampling data is determined as the reference sampling point.
3. The method according to claim 1, characterized in that, The step of determining the time interval between the reference sampling point and the starting playback point of the test audio based on the fitting function includes: Based on the initial phase in the fitting function, the time interval between the reference sampling point and the starting playback point of the test audio is determined.
4. The method according to claim 1, characterized in that, The sampling period corresponding to the fitted sampling point is an integer multiple of the sine period or the cosine period; Before performing function fitting based on the audio frequency of the test audio, the audio sampling data of the fitting sampling points, and the sampling frequency to obtain the fitting function, the method further includes: Determine the sum of the audio signal voltages of the audio sampling data at the reference sampling point and the sampling points after the reference sampling point; If the sum of the audio signal voltages is not zero, a correction voltage is determined based on the sum of the audio signal voltages and the number of fitted sampling points; The audio sampling data of the fitted sampling points is corrected based on the corrected voltage.
5. The method according to claim 1, characterized in that, The method further includes: Based on the audio frequency of the test audio and the periodic position of the extreme point corresponding to the extreme value in the audio curve, the time interval between the reference sampling point and the starting playback point of the test audio is determined.
6. A method for measuring audio output delay, characterized in that, The method is used for a measuring device, and the method includes: The audio signal output by the terminal under test is sampled according to the sampling frequency to obtain audio sampling data. The terminal under test is used to play the test audio of the measurement program when the audio playback instruction is received. The measurement program is set in the terminal under test. The audio sampling data includes an audio signal voltage that reflects the audio characteristics corresponding to the test audio. The measuring device is connected to the terminal under test through the audio output interface of the terminal under test. The audio sampling data and sampling time are sent to the terminal under test (DUT) so that the DUT can determine a reference sampling point based on the audio sampling data. A function is fitted based on the audio frequency of the test audio, the audio sampling data of the fitted sampling point, and the sampling frequency to obtain a fitting function. The time interval between the reference sampling point and the starting playback point of the test audio is determined based on the fitting function. The starting playback time of the test audio is determined based on the sampling time of the reference sampling point and the time interval. The time difference between the DUT and the measuring device is then determined by synchronizing with the measuring device. The starting playback time is corrected based on the device time difference. The audio output delay of the DUT is determined based on the corrected starting playback time and the trigger time of the audio playback command. The reference sampling point is the sampling point where the audio signal voltage in the audio sampling data reaches an extreme value, used to reduce the influence of sampling noise. The fitted sampling point includes the reference sampling point and the sampling points after the reference sampling point. The fitting function is used to characterize the audio curve of the fitted test audio, and the audio curve of the test audio is a sine curve or a cosine curve.
7. A device for measuring audio output delay, characterized in that, The device includes: A playback module is used to play test audio of a measurement program in response to an audio playback command, the measurement program being set in the terminal under test; The receiving module is used to receive audio sampling data and sampling time sent by the measuring device. The measuring device is connected to the terminal under test through the audio output interface of the terminal under test. The audio sampling data is obtained by the measuring device sampling the received test audio according to the sampling frequency. The audio sampling data includes audio signal voltage that reflects the audio characteristics corresponding to the test audio. The first determining module is used to determine a reference sampling point based on the audio sampling data. The reference sampling point is a sampling point where the audio signal voltage in the audio sampling data reaches an extreme value, which is used to reduce the influence of sampling noise. Based on the audio frequency of the test audio, the audio sampling data of the fitting sampling point, and the sampling frequency, a function is fitted to obtain a fitting function. The fitting sampling point includes the reference sampling point and sampling points after the reference sampling point. The fitting function is used to characterize the audio curve of the fitted test audio, which is a sine curve or a cosine curve. Based on the fitting function, the time interval between the reference sampling point and the starting playback point of the test audio is determined. Based on the sampling time of the reference sampling point and the time interval, the starting playback time of the test audio is determined. The second determining module is used to synchronize with the measuring device to determine the device time difference between the terminal under test and the measuring device; to correct the start playback time based on the device time difference; and to determine the audio output delay of the terminal under test based on the corrected start playback time and the instruction trigger time of the audio playback command.
8. A device for measuring audio output delay, characterized in that, The device includes: The sampling module is used to sample the audio signal output by the terminal under test according to the sampling frequency to obtain audio sampling data. The terminal under test is used to play the test audio of the measurement program when it receives an audio playback command. The measurement program is set in the terminal under test. The audio sampling data includes an audio signal voltage that reflects the audio characteristics corresponding to the test audio. The measuring device is connected to the terminal under test through the audio output interface of the terminal under test. The sending module is used to send the audio sampling data and sampling time to the terminal under test, so that the terminal under test can determine a reference sampling point based on the audio sampling data, perform function fitting based on the audio frequency of the test audio, the audio sampling data of the fitting sampling point, and the sampling frequency to obtain a fitting function, determine the time interval between the reference sampling point and the starting playback point of the test audio based on the fitting function, determine the starting playback time of the test audio based on the sampling time of the reference sampling point and the time interval, and synchronize with the measuring device to determine the device time difference between the terminal under test and the measuring device; correct the starting playback time based on the device time difference; determine the audio output delay of the terminal under test based on the corrected starting playback time and the instruction triggering time of the audio playback command. The reference sampling point is the sampling point where the audio signal voltage in the audio sampling data reaches an extreme value, which is used to reduce the influence of sampling noise. The fitting sampling point includes the reference sampling point and the sampling points after the reference sampling point. The fitting function is used to characterize the audio curve of the fitted test audio, and the audio curve of the test audio is a sine curve or a cosine curve.
9. A system for measuring audio output delay, characterized in that, The system includes a terminal under test and a measuring device; the measuring device is connected to the terminal under test through the audio output interface of the terminal under test. The terminal under test is used to implement the audio output delay measurement method as described in any one of claims 1 to 5, and the measuring device is used to implement the audio output delay measurement method as described in claim 6.
10. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one program, the at least one program being loaded and executed by the processor to implement the audio output delay measurement method as described in any one of claims 1 to 5, or to implement the audio output delay measurement method as described in claim 6.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one program, which is loaded and executed by a processor to implement the method for measuring audio output delay as described in any one of claims 1 to 5, or to implement the method for measuring audio output delay as described in claim 6.
12. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the audio output delay measurement method as described in any one of claims 1 to 5, or to perform the audio output delay measurement method as described in claim 6.
Citation Information
Patent Citations
Audio equipment, signal delay testing method, system and storage medium
CN109688528A
Delay measurement method, device, equipment, system and storage medium
CN112039612A
Method for testing audio time delay of Bluetooth headset
CN112866893A
