Audio end-to-end delay measurement method, system, and media

Abstract

This application provides a method, system, and medium for measuring end-to-end audio delay, relating to the field of delay measurement technology. The method includes: a transmitting end generating an audio test sequence based on service audio transmitted in an audio processing link; injecting the audio test sequence into the service audio; and determining the transmission time of the audio test sequence; a receiving end decoding and extracting features from the received audio stream to obtain an audio feature sequence; performing time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence; and determining the end-to-end delay of the service audio based on the transmission and reception times. The solution of this application can improve the measurement accuracy and reliability of end-to-end audio delay without increasing hardware costs.

Description

Technical Field

[0001] This application relates to the field of delay measurement technology, and in particular to an audio end-to-end delay measurement method, system and medium. Background Technology

[0002] The audio and video terminal experiences end-to-end delay from acquisition and encoding to transmission via the platform and decoding and playback on the client. This delay directly impacts the experience of real-time services such as intercom and monitoring, and needs to be quantified during the R&D and acceptance phases.

[0003] In related technologies, latency can be measured using dedicated hardware such as time synchronizers, or custom time stamp information can be inserted into the audio transmission protocol, and latency can be calculated by comparing the time stamps at the transmitting and receiving ends. Both of these methods are often paired with single-point feature detection methods, which calculate latency by identifying the time difference of single-point features at the transmitting and receiving ends. However, hardware or protocol-intrusive measurement methods are costly to modify and lack deployment flexibility, while single-point feature detection methods have weak anti-interference capabilities, only reflecting the instantaneous state at a specific moment and having poor global characterization capabilities, resulting in low accuracy and poor reliability in end-to-end latency measurement.

[0004] Therefore, there is a need for an audio end-to-end delay measurement solution that can improve the measurement accuracy and reliability of audio end-to-end delay without increasing hardware costs. Summary of the Invention

[0005] This application provides an audio end-to-end delay measurement method, system, and medium, which can improve the measurement accuracy and reliability of audio end-to-end delay without increasing hardware costs.

[0006] In a first aspect, embodiments of this application provide an audio end-to-end delay measurement method, applied to an audio end-to-end delay measurement system, the audio end-to-end delay measurement system including a transmitting end and a receiving end, the method comprising:

[0007] The transmitting end generates an audio test sequence based on the service audio transmitted in the audio processing link, injects the audio test sequence into the service audio in the same path, and determines the transmission time of the audio test sequence.

[0008] The receiving end decodes and extracts features from the received audio stream to obtain an audio feature sequence; performs time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence; and determines the end-to-end delay of the service audio based on the transmission time and the reception time.

[0009] In one possible implementation, generating the audio test sequence based on the service audio transmitted in the audio processing link includes:

[0010] The service audio transmitted in the audio processing link is sampled and statistically analyzed to determine the energy statistics of the service audio.

[0011] The sequence amplitude is determined based on the energy statistics of the service audio and a preset coefficient, wherein the preset coefficient is greater than 1;

[0012] An audio test sequence is generated based on a preset audio duration, a preset center frequency, and the sequence amplitude, wherein the preset center frequency is greater than the ambient noise frequency.

[0013] In one possible implementation, the step of co-channel injection of the audio test sequence into the service audio and determining the transmission time of the audio test sequence includes:

[0014] When the service audio reaches the preset injection trigger point, the audio test sequence is injected into the service audio in the same path;

[0015] The data after the injection trigger point is processed to equalize the length so that the duration of the output stream of the service audio remains unchanged before and after the injection. The equalization process includes randomly discarding equal-length data or time axis scaling compensation.

[0016] The transmission time of the audio test sequence is determined based on the trigger time of the injection trigger point.

[0017] In one possible implementation, the step of decoding and feature extraction of the received audio stream to obtain an audio feature sequence includes:

[0018] The received audio stream is decoded, and the decoded audio is sampled at fixed time intervals to obtain audio samples;

[0019] Energy features are extracted from each audio sample to obtain the audio features of each audio sample;

[0020] Based on the audio features of each audio sample and the sampling timestamp of each audio sample, an audio feature sequence arranged in chronological order is generated.

[0021] In one possible implementation, the step of performing time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence includes:

[0022] The audio feature sequence is analyzed according to the chronological order of the sampling timestamps to determine the target continuous interval that matches the features of the audio test sequence.

[0023] The reception time of the audio test sequence is determined based on the target continuous interval;

[0024] Wherein, the duration of the target continuous interval is greater than or equal to the preset audio duration, and the proportion of audio features with energy greater than an energy threshold in the target continuous interval is greater than or equal to a proportion threshold, wherein the energy threshold is determined by the sequence amplitude.

[0025] In one possible implementation, determining the reception time of the audio test sequence based on the target continuous interval includes:

[0026] The reception time of the audio test sequence is determined based on the first sampling timestamp of the target continuous interval.

[0027] In one possible implementation, it also includes:

[0028] The steps of generating an audio test sequence based on the service audio transmitted in the audio processing link, injecting the audio test sequence into the service audio in the same path, and determining the single delay for each execution are performed periodically.

[0029] A delay sample set is generated based on a preset number of single delays, and statistical calculations are performed on the delay sample set to obtain the end-to-end delay index of the service audio.

[0030] Secondly, embodiments of this application provide an audio end-to-end delay measurement system, including:

[0031] The transmitting module is used to generate an audio test sequence based on the service audio transmitted in the audio processing link, inject the audio test sequence into the service audio in the same path, and determine the transmission time of the audio test sequence;

[0032] The receiving module is used to decode and extract features from the received audio stream to obtain an audio feature sequence; perform time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence; and determine the end-to-end delay of the service audio based on the transmission time and the reception time.

[0033] Thirdly, embodiments of this application provide another audio end-to-end delay measurement system, including:

[0034] The processor, and the memory that is in communication with the processor;

[0035] Memory is used to store instructions that the computer executes;

[0036] The processor is configured to execute computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect described above.

[0038] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, is used to implement the first aspect and / or various possible implementations of the first aspect as described above.

[0039] This application provides an audio end-to-end delay measurement method, system, and medium. The transmitting end can inject a generated audio test sequence into the actual transmitted service audio, allowing the audio test sequence to undergo the same encoding, transmission, and decoding processes as the service audio, thus accurately reflecting the transmission delay of the user-perceived path. The receiving end can decode and extract features from the received audio stream to obtain an audio feature sequence, and perform time-domain analysis and feature matching on the audio feature sequence. Based on the full-process time-series matching of the feature sequence, the embedded audio test sequence can be accurately identified, effectively avoiding signal distortion and noise interference caused by audio encoding / decoding and channel transmission, significantly reducing the probability of false detection, and improving the accuracy of audio test sequence identification, thereby improving the accuracy and reliability of audio end-to-end delay measurement. With this setup, the audio test sequence can be directly injected into the standard audio link without deploying dedicated hardware or modifying the protocol, achieving high-precision and high-reliability measurement of audio end-to-end delay without increasing hardware costs. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] Figure 1 This is a system architecture diagram of an embodiment of this application;

[0042] Figure 2 This is a flowchart of an audio end-to-end delay measurement method according to an embodiment of this application;

[0043] Figure 3 This is a schematic diagram of the receiving end waveform according to an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of the structure of an audio end-to-end delay measurement system according to an embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the structure of an audio end-to-end delay measurement system according to another embodiment of this application.

[0046] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0047] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0048] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0049] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0050] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0051] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0052] The audio end-to-end delay measurement method, system, and medium of this application can be used in the field of delay measurement, or in any field other than delay measurement, such as audio processing. The application fields of the audio end-to-end delay measurement method, system, and medium of this application are not limited.

[0053] The audio end-to-end delay measurement method, system, and medium of this application can be applied to audio and video terminals such as in-vehicle audio systems, in scenarios where audio end-to-end delay measurement is performed during the research and development and acceptance phases. The audio end-to-end delay measurement method, system, and medium of this application can be applied to any scenario involving audio end-to-end delay measurement.

[0054] The audio and video terminal experiences end-to-end delay from acquisition and encoding to transmission via the platform and decoding and playback on the client. This delay directly impacts the experience of real-time services such as intercom and monitoring, and needs to be quantified during the R&D and acceptance phases.

[0055] In related technologies, latency can be measured using dedicated hardware such as time synchronizers, or custom time stamp information can be inserted into the audio transmission protocol, and latency can be calculated by comparing the time stamps at the transmitting and receiving ends. Both of these methods are often paired with single-point feature detection methods, which complete the latency calculation by identifying the time difference of single-point features at the transmitting and receiving ends.

[0056] However, measurement methods that involve hardware or protocol intrusion are costly to modify and have poor deployment flexibility, while single-point feature detection methods have weak anti-interference capabilities, can only reflect the instantaneous state at a specific moment, and have poor global characterization capabilities, resulting in low accuracy and poor reliability of end-to-end delay measurement.

[0057] Based on the above-mentioned technical problems, the inventive concept of this application is to provide an audio end-to-end delay measurement scheme that can improve the measurement accuracy and reliability of audio end-to-end delay without increasing hardware costs.

[0058] This application provides an audio end-to-end delay measurement method, system, and medium. The transmitting end can inject a generated audio test sequence into the actual transmitted service audio, allowing the audio test sequence to undergo the same encoding, transmission, and decoding processes as the service audio, thus accurately reflecting the transmission delay of the user-perceived path. The receiving end can decode and extract features from the received audio stream to obtain an audio feature sequence, and perform time-domain analysis and feature matching on the audio feature sequence. Based on the full-process time-series matching of the feature sequence, the embedded audio test sequence can be accurately identified, effectively avoiding signal distortion and noise interference caused by audio encoding / decoding and channel transmission, significantly reducing the probability of false detection, and improving the accuracy of audio test sequence identification, thereby improving the accuracy and reliability of audio end-to-end delay measurement. With this setup, the audio test sequence can be directly injected into the standard audio link without deploying dedicated hardware or modifying the protocol, achieving high-precision and high-reliability measurement of audio end-to-end delay without increasing hardware costs.

[0059] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0060] Figure 1 This is a system architecture diagram of an embodiment of this application, such as... Figure 1 As shown, an audio end-to-end delay measurement system may include a transmitter and a receiver. The transmitter may include an injection compensation module and an encoder, while the receiver may include a decoder, an acquisition and analysis module, a matching detection module, and a delay calculation module. The injection compensation module at the transmitter generates an audio test sequence based on the service audio transmitted in the audio processing link, injects the audio test sequence into the service audio, and determines the transmission time of the audio test sequence. The encoder encodes the data stream output by the injection compensation module and transmits it to the platform / network, which then transmits the data to the receiver. The decoder at the receiver decodes the received audio stream, and the acquisition and analysis module extracts the audio features from the decoded audio to obtain an audio feature sequence. The matching detection module performs time-domain analysis and feature matching on the audio feature sequence to determine the target continuous interval where the audio test sequence is located. The delay calculation module determines the reception time of the audio test sequence based on the timestamp of the target continuous interval and, based on the transmission and reception times, determines the end-to-end delay of the service audio.

[0061] Figure 2This is a flowchart of an audio end-to-end delay measurement method according to an embodiment of this application. This embodiment describes the audio end-to-end delay measurement method with an audio end-to-end delay measurement system as the executing entity. The audio end-to-end delay measurement system may include a transmitting end and a receiving end. For example... Figure 2 As shown, the audio end-to-end delay measurement method may include the following steps:

[0062] S201: The sending end generates an audio test sequence based on the service audio transmitted in the audio processing link.

[0063] In this embodiment, the sending end can be a simulated terminal of the audio and video terminal to be developed and accepted, that is, an audio sending simulated terminal.

[0064] In this embodiment, the audio processing link is the link that encodes and transmits the audio stream.

[0065] In this embodiment, the service audio can be the source audio during acceptance testing, and those skilled in the art can flexibly set it according to actual needs.

[0066] In this embodiment, in order to ensure accurate identification of subsequent audio test sequences, the audio test sequences can be significantly different from the business audio in terms of characteristics such as amplitude when they are generated.

[0067] For example, the business audio can be audio data with relatively flat (small) amplitude, and the audio test sequence can be a sequence in which the amplitude of all values ​​is greater than that of the business audio, such as the amplitude of the audio test sequence plus a preset amplitude threshold > the amplitude of the business audio.

[0068] S202: The sending end injects the audio test sequence into the service audio and determines the sending time of the audio test sequence.

[0069] In this embodiment, the transmitting end can inject the audio test sequence into the transmitted service audio at fixed intervals. When the audio test sequence is injected into the service audio, the transmission time of the audio test sequence can be determined.

[0070] In this embodiment, the audio test sequence is co-injected into the service audio, so that the audio test sequence undergoes the same encoding, transmission and decoding process as the service audio, which truly reflects the transmission delay of the user's perceived path, ensures the complete consistency between the measured path and the user's actual perceived path, truly reflects the end-to-end audio delay, and improves the accuracy of subsequent delay measurements.

[0071] S203: The receiving end decodes and extracts features from the received audio stream to obtain an audio feature sequence.

[0072] In this embodiment, the receiving end can be a simulated client of the audio / video terminal to be developed and accepted, or an automated test script, i.e., an audio receiving simulation end. The receiving end pulls the media stream and decodes it into data through a standard protocol. This process completely simulates the playback path of a real user, without any secondary processing by a proxy server, ensuring that the measured audio end-to-end latency is the actual latency.

[0073] In this embodiment, the decoder at the receiving end can decode the received audio stream and sample the decoded data at fixed time intervals, calculate the energy characteristics of the sample, and form an audio feature sequence arranged in chronological order together with the timestamp corresponding to the sample, that is, a sequence of energy changes over time.

[0074] S204: The receiving end performs time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence.

[0075] In this embodiment, temporal analysis can be performed by scanning the audio feature sequence in chronological order, and feature matching can be performed by matching the scanned audio features with the features of the audio feature sequence.

[0076] In this embodiment, by performing temporal analysis and feature matching on the audio feature sequence, the entire continuous process of the audio test sequence can be time-matched, that is, the temporal form and continuity of the entire continuous process can be constrained, rather than relying solely on a single instantaneous peak. This can effectively suppress transient interference such as short impact sounds and human voice fragments, improve the reliability of audio test sequence recognition and detection, and thus improve the accuracy and reliability of the audio test sequence reception time.

[0077] In this embodiment, after the audio test sequence is identified through time-domain analysis and feature matching, the reception time of the audio test sequence can be determined based on the sampling timestamp of the audio test sequence.

[0078] S205: The receiving end determines the end-to-end delay of the service audio based on the sending and receiving times.

[0079] In this embodiment, the end-to-end delay of the service audio is equal to the reception time minus the transmission time.

[0080] In this embodiment, in order to improve the accuracy of the delay measurement, the above steps S201-S205 can be repeated multiple times to determine the final delay measurement result using the results of multiple measurements.

[0081] In this embodiment, the transmitting end can inject the generated audio test sequence into the actual transmitted service audio, allowing the audio test sequence to undergo the same encoding, transmission, and decoding process as the service audio, thus accurately reflecting the transmission delay of the user-perceived path. The receiving end can decode and extract features from the received audio stream to obtain an audio feature sequence, and perform time-domain analysis and feature matching on the audio feature sequence. Based on the full-process time-series matching of the feature sequence, the embedded audio test sequence can be accurately identified, effectively avoiding signal distortion and noise interference caused by audio encoding / decoding and channel transmission, significantly reducing the probability of false detection, and improving the accuracy of audio test sequence identification, thereby improving the accuracy and reliability of audio end-to-end delay measurement. With this setup, the audio test sequence can be directly injected into the standard audio link without deploying dedicated hardware or modifying the protocol, achieving high-precision and high-reliability measurement of audio end-to-end delay without increasing hardware costs.

[0082] In one possible implementation, the step S201 above, generating the audio test sequence based on the service audio transmitted in the audio processing link, may include:

[0083] S11: Sample and statistically analyze the service audio transmitted in the audio processing link to determine the energy statistics of the service audio.

[0084] S12: Determine the sequence amplitude based on the energy statistics of the business audio and the preset coefficient, where the preset coefficient is greater than 1.

[0085] S13: Generate an audio test sequence based on the preset audio duration, preset center frequency, and sequence amplitude. The preset center frequency is greater than the ambient noise frequency.

[0086] In this embodiment, a bypass statistical path can be set up in parallel with the main transmission path in the audio processing link at the transmitting end. The bypass statistical path can perform real-time mirror sampling of the service audio frames "about to be encoded and transmitted" in the main path without changing the timing and data content of the main path. Within the pre-sampling window (e.g., 200ms to 2s, which can be flexibly set) before the injection trigger point (which can be flexibly set), short segments are extracted at fixed sampling intervals (e.g., every 20ms or every frame, which can be flexibly set), and the energy statistics of each segment (such as root mean square value, peak value, or average absolute value, etc.) are calculated.

[0087] For example, the sequence amplitude of each audio test sequence can be dynamically set according to preset rules based on the above energy statistics:

[0088] The sequence amplitude A satisfies: A = clamp(k × E_base, A_min, A_max), where E_base is the energy statistic mentioned above, k is a preset coefficient (such as 1.5 to 2.0, which can be set flexibly), A_min and A_max are the minimum and maximum amplitude limits pre-calibrated according to the encoding format, quantization bits, etc., respectively, and clamp means that the result is restricted to [A_min, A_max].

[0089] In this embodiment, energy statistics may include, but are not limited to, root mean square value, peak value, and average absolute value, as long as they are obtained by statistically analyzing the energy characteristics of the service audio.

[0090] In this embodiment, the preset coefficient can be a value greater than 1, so that the sequence amplitude is greater than the amplitude of the service audio, so that the audio test sequence can be quickly and accurately detected from the service audio. Those skilled in the art can flexibly set the specific preset coefficient according to actual needs, and no restrictions are imposed here.

[0091] In this embodiment, when dynamically setting the sequence amplitude, a maximum amplitude limit can also be introduced, that is, the sequence amplitude needs to be less than or equal to the maximum amplitude limit.

[0092] In this embodiment, after determining the sequence amplitude, the transmitting end can generate an audio test sequence of duration T (e.g., 200ms-1000ms, which can be flexibly set) in real time. This sequence adopts the same encoding format as the service audio, and the center frequency is set to a frequency band with low environmental noise (e.g., a single frequency of about 2kHz, which can be flexibly set), preferably a sine wave, to facilitate identification and detection.

[0093] In this embodiment, the preset audio duration and preset center frequency can be flexibly set by those skilled in the art according to actual conditions, and no restrictions are imposed here.

[0094] In this embodiment, by sampling and energy statistics of the actual transmitted service audio within the link, the amplitude of the audio test sequence can be determined based on the energy statistics of the real service audio. By limiting the sequence amplitude using energy statistics and preset coefficients, the energy characteristics of the audio test sequence can be significantly different from the service audio, facilitating subsequent identification and detection of the audio test sequence. By limiting the center frequency of the audio test sequence to be higher than the ambient noise frequency, the interference of ambient noise on the testing process can be effectively avoided, further improving the accuracy of audio test sequence detection.

[0095] In one possible implementation, step S202 above, which involves co-injecting the audio test sequence into the service audio and determining the transmission time of the audio test sequence, may include:

[0096] S21: When the service audio reaches the preset injection trigger point, the audio test sequence is injected into the service audio in the same path.

[0097] S22: Perform equal-length processing on the data after the injection trigger point to keep the duration of the business audio output stream unchanged before and after the injection. The equal-length processing includes randomly discarding equal-length data or time axis scaling compensation.

[0098] S23: Determine the sending time of the audio test sequence based on the triggering time of the injection trigger point.

[0099] In this embodiment, during the data reading process of the audio encoder at the transmitting end, when a preset injection trigger point is reached, the original service audio is no longer read. Instead, the generated audio test sequence is directly read and filled into the encoding buffer, and the duration is strictly controlled within T (the duration of the audio test sequence). This process ensures that the test signal is treated as normal audio for encoding and compression.

[0100] In this embodiment, the data after the injection trigger point is processed to equalize the length. This can be done on the data of the service audio after the injection trigger point (excluding the audio test sequence). That is, after the audio test sequence is injected, the remaining service audio is processed to equalize the length, so that the output stream duration of the entire service audio remains unchanged before and after the injection, thus eliminating the streaming media asynchrony caused by the insertion of the test signal.

[0101] In this embodiment, the random discarding of equal-length data can mean randomly discarding any segment of audio stream after the injection trigger point that has the same duration as the audio test sequence; that is, the same amount is injected and discarded. Time axis scaling compensation can compress the audio stream after the injection trigger point. This setting ensures the consistency between the audio-visual synchronization of the non-replaced portion and the measurement path with the user's actual perceived path.

[0102] For example, the original business audio (duration T0) = AB (duration T1) + BC (duration T2), point B is the injection trigger point, and the audio test sequence is a (duration T). After the same-path injection, the business audio = AB + a + BC. It is necessary to randomly discard audio data of duration T in BC, or adjust the time axis of BC so that the duration of BC changes from T2 to T2-T.

[0103] In this embodiment, the audio test sequence and service audio are injected into the same path based on a preset injection trigger point. No additional independent transmission channel is required, allowing testing to be conducted within a real service transmission link, making the test scenario more closely resemble actual application conditions. By randomly discarding data or using time axis scaling compensation for equal-length processing, the total duration of the audio output stream before and after injection is ensured to be consistent, avoiding problems such as audio stream timing errors, playback stuttering, and audio-visual asynchrony caused by signal injection, thus guaranteeing normal audio data transmission and playback. The injection trigger point time is used as the basis for calibrating the transmission time of the audio test sequence, achieving precise and controllable control over the test signal transmission timing, facilitating subsequent measurement of end-to-end audio delay.

[0104] In one possible implementation, the step S203 above, which involves decoding and extracting features from the received audio stream to obtain an audio feature sequence, may include:

[0105] S31: Decode the received audio stream and sample the decoded audio at fixed time intervals to obtain audio samples.

[0106] S32: Extract energy features from each audio sample to obtain the audio features of each audio sample.

[0107] S33: Generate an audio feature sequence arranged in chronological order based on the audio features of each audio sample and the sampling timestamp of each audio sample.

[0108] In this embodiment, an audio sample can be obtained after a certain period of time (e.g., 50ms, which can be flexibly set) after the audio test sequence is sent (Tsend).

[0109] In this embodiment, the sampling time interval can be flexibly set by those skilled in the art according to actual conditions, and no restrictions are imposed here.

[0110] In this embodiment, energy feature extraction of audio samples can be performed by statistically analyzing the root mean square value, peak value, and average absolute value of the audio samples. The method of energy feature extraction can be the same as the method of calculating the energy statistics of the service audio in step S11 above (that is, the energy statistics are calculated by calculating the root mean square value, and the energy feature extraction is also calculated by calculating the root mean square value).

[0111] In this embodiment, sampling the decoded audio at fixed time intervals standardizes data acquisition, yielding regular and time-sequential audio samples, laying the foundation for subsequent feature analysis. Extracting the energy features of each audio sample accurately captures the energy changes of the audio signal, intuitively reflecting the signal characteristics after audio link transmission. Combining audio features with sampling timestamps to generate an audio feature sequence enables the correlation and matching of feature data with the time dimension, facilitating subsequent tracing and analysis of audio features along timelines and performing feature matching, thus improving the reliability and accuracy of audio test sequence detection and analysis.

[0112] In one possible implementation, the step S204 above, which involves performing time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence, may include:

[0113] S41: Analyze the audio feature sequence according to the sampling timestamps to determine the target continuous interval that matches the features of the audio test sequence.

[0114] S42: Determine the reception time of the audio test sequence based on the target continuous interval.

[0115] Among them, the duration of the target continuous interval is greater than or equal to the preset audio duration, and the proportion of audio features with energy greater than the energy threshold in the target continuous interval is greater than or equal to the proportion threshold, which is determined by the sequence amplitude.

[0116] In this embodiment, the ratio threshold can be flexibly set by those skilled in the art according to actual conditions. For example, the ratio threshold can be 90% or 80%, and no restrictions are imposed here.

[0117] In this implementation, the target continuous interval must simultaneously satisfy the following two conditions:

[0118] ① Temporal duration condition: The duration of the target continuous interval is greater than or equal to the designed duration of the audio test sequence (i.e., the preset audio duration).

[0119] ② The energy (audio features) of most sampling points within the interval is higher than the energy threshold (determined by the sequence amplitude of the audio test sequence), that is, the proportion of the number of sampling points with energy higher than the energy threshold to the total number of sampling points in the target continuous interval is greater than or equal to the proportion threshold.

[0120] In this embodiment, the energy threshold = threshold coefficient × sequence amplitude. The threshold coefficient can be a value close to 1 (it can be obtained through multiple measurements and calibrations to make the energy threshold close to the sequence amplitude). Those skilled in the art can set it flexibly according to actual conditions.

[0121] For example, Figure 3 This is a schematic diagram of the receiving end waveform according to an embodiment of this application, as shown below. Figure 3 As shown, "send" represents the time when the service audio is sent, and "detect" represents the time when the audio test sequence is detected (i.e., the time when it is received). The time difference between the two is the end-to-end delay.

[0122] In this embodiment, time-domain analysis is performed sequentially according to timestamps. The target continuous interval is selected by combining the dual conditions of duration and energy proportion. This can constrain the temporal form and continuity of the entire continuous process, rather than relying solely on a single instantaneous peak. It can effectively suppress transient interference such as short impact sounds and human voice fragments, effectively distinguish audio test sequences from business audio and environmental noise, improve the reliability of audio test sequence identification and detection, and thus improve the accuracy and reliability of the audio test sequence reception time.

[0123] In one possible implementation, step S42, which determines the reception time of the audio test sequence based on the target continuous interval, may include:

[0124] The reception time of the audio test sequence is determined based on the first sampling timestamp of the target continuous interval.

[0125] In this embodiment, the timestamp of the first sampling frame of the target continuous interval can be used as the receiving time Trecv of the audio test sequence. The transmitting end has recorded the precise sending time Tsend when the audio test sequence is injected. Then the audio end-to-end delay Δt = Trecv − Tsend.

[0126] In this embodiment, the first sampling timestamp of the target continuous interval is used as the reception time of the audio test sequence. This can accurately capture the starting node of the test sequence, ensure the accuracy of the reception time determination, and provide an accurate time reference for subsequent delay measurements.

[0127] In one possible implementation, the method may further include:

[0128] S51: Periodically execute the steps of generating an audio test sequence based on the service audio transmitted in the audio processing link, injecting the audio test sequence into the service audio in the same path, and determining the single delay for each execution.

[0129] S52: Generate a delay sample set based on a preset number of single delays, and perform statistical calculations on the delay sample set to obtain the end-to-end delay index of the service audio.

[0130] In this embodiment, the sending end can initiate a test sequence injection once at a preset period (e.g., every N seconds or every fixed number of service frames); or the user / automation script can trigger the injection multiple times within a specified time interval. Each injection is executed according to the above steps S201-S202, and the sending time T_send_i (i is the sample sequence number) is recorded.

[0131] After each injection, the receiver executes steps S203-S205 as described above, records the reception time T_recv_i of the test signal, and calculates the end-to-end delay Δt_i = T_recv_i − T_send_i for that sample. If no valid signal (audio test sequence) is detected in a given test, the sample can be marked as invalid and discarded or retried, and will not be included in subsequent statistics.

[0132] After completing a set number of valid samplings (e.g., K times), perform scientific statistics on all valid delayed samples {Δt_1, Δt_2, ..., Δt_K} (delay sample set), and output end-to-end delay metrics including but not limited to the following:

[0133] ① Maximum and minimum values: These reflect the upper and lower bounds of delayed fluctuations;

[0134] ② Average value (mean): Reflects the overall level;

[0135] ③Median (50th percentile): Anti-outlier, reflects typical delay;

[0136] ④ Quantiles: such as 90th percentile (P90), 95th percentile (P95), 99th percentile (P99), etc., are used to evaluate latency performance in "most" or "extreme" scenarios to meet the needs of engineering acceptance and evaluation.

[0137] For example, the latency measurement results of an audio terminal are shown in Table 1 below:

[0138] Table 1

[0139]

[0140] The end-to-end latency metrics obtained from Table 1 above are: minimum latency: 644ms; maximum latency: 991ms; average latency: 802ms; 90th percentile: 772ms.

[0141] In this implementation, the system can summarize the above statistical results into a delay report (table or chart), and can mark each sampling result with timestamps or serial numbers to facilitate reproduction and problem localization. Through periodic injection and multi-sample statistics, stable, repeatable, and quantifiable end-to-end audio delay evaluation results can be provided based on a single measurement.

[0142] In this embodiment, the audio test sequence is periodically generated and injected repeatedly, and multiple sets of single-time delay data are continuously collected. This avoids the randomness of single measurements and makes the data samples more representative. Statistical analysis based on a sufficient delay sample set can effectively eliminate abnormal data interference, accurately obtain end-to-end delay indicators, and the results are more consistent with the actual transmission characteristics of the link, further improving the reliability and accuracy of delay measurement.

[0143] The audio end-to-end delay measurement method of this application is described below with a specific embodiment.

[0144] In one specific embodiment, the technician measures the end-to-end latency of the in-vehicle audio and video terminal, and the specific process is as follows:

[0145] The first step is for the sending end to sample and statistically analyze the service audio transmitted in the audio processing link to determine the energy statistics of the service audio; determine the sequence amplitude based on the energy statistics of the service audio and preset coefficients; and generate an audio test sequence based on the preset audio duration, preset center frequency, and sequence amplitude.

[0146] The second step is that when the service audio reaches the preset injection trigger point, the sending end injects the audio test sequence into the service audio in the same path.

[0147] The third step involves the sending end randomly discarding data of equal length after the injection trigger point to keep the duration of the service audio output stream unchanged before and after the injection; and determining the sending time of the audio test sequence based on the trigger time of the injection trigger point.

[0148] The fourth step involves the receiving end decoding the received audio stream and sampling the decoded audio at fixed time intervals to obtain audio samples; extracting energy features from each audio sample to obtain the audio features of each audio sample; and generating an audio feature sequence arranged in chronological order based on the audio features of each audio sample and the sampling timestamp of each audio sample.

[0149] The fifth step involves the receiver analyzing the audio feature sequence according to the sampling timestamps to determine the target continuous interval that matches the features of the audio test sequence; and determining the reception time of the audio test sequence based on the first sampling timestamp of the target continuous interval.

[0150] The sixth step is for the receiving end to determine the end-to-end delay of a single sampling based on the sending and receiving times.

[0151] Step 7: Repeat steps 1-7 above to perform multiple measurements, determine the single latency for each execution, and perform statistical calculations based on the latency sample sets of multiple single latency measurements to obtain the end-to-end latency index of the business audio.

[0152] Figure 4 This is a schematic diagram of the structure of an audio end-to-end delay measurement system according to an embodiment of this application, as shown below. Figure 4As shown, the audio end-to-end delay measurement system includes: a transmitting module 41, used to generate an audio test sequence based on the service audio transmitted in the audio processing link, inject the audio test sequence into the service audio, and determine the transmission time of the audio test sequence; a receiving module 42, used to decode and extract features from the received audio stream to obtain an audio feature sequence; perform time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence; and determine the end-to-end delay of the service audio based on the transmission time and the reception time.

[0153] The audio end-to-end delay measurement system provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be repeated here.

[0154] Figure 5 This is a schematic diagram of the structure of an audio end-to-end delay measurement system according to another embodiment of this application, as shown below. Figure 5 As shown, the audio end-to-end delay measurement system includes: a processor 501 and a memory 502 communicatively connected to the processor 501; the memory 502 stores computer execution instructions; the processor 501 executes the computer execution instructions stored in the memory 502 to implement the steps of the audio end-to-end delay measurement method in the above-described method embodiments.

[0155] In the aforementioned audio end-to-end delay measurement system, the memory 502 and the processor 501 are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines, such as a bus connection. The memory 502 stores computer-executable instructions that implement data access control methods, including at least one software functional module that can be stored in the memory 502 in the form of software or firmware. The processor 501 executes various functional applications and data processing by running the software programs and modules stored in the memory 502.

[0156] The memory 502 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 502 stores programs, which are then executed by the processor 501 upon receiving execution instructions. Furthermore, the software programs and modules within the memory 502 may include an operating system, which may include various software components and / or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.) and can communicate with various hardware or software components to provide an operating environment for other software components.

[0157] Processor 501 can be an integrated circuit chip with signal processing capabilities. The aforementioned processor 501 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0158] An embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the steps of the various method embodiments of this application.

[0159] An embodiment of this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the various method embodiments of this application.

[0160] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0161] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0162] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0163] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0164] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0165] Finally, it should be noted that other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from its scope.

Claims

1. A method of audio end-to-end delay measurement, characterized by, The method, applied to an audio end-to-end delay measurement system, comprising a transmitter and a receiver, includes: The transmitting end generates an audio test sequence based on the service audio transmitted in the audio processing link, injects the audio test sequence into the service audio in the same path, and determines the transmission time of the audio test sequence; The receiving end decodes and extracts features from the received audio stream to obtain an audio feature sequence; performs time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence; and determines the end-to-end delay of the service audio based on the transmission time and the reception time.

2. The audio end-to-end delay measurement method of claim 1, wherein, The step of generating an audio test sequence based on the service audio transmitted in the audio processing link includes: The service audio transmitted in the audio processing link is sampled and statistically analyzed to determine the energy statistics of the service audio. The sequence amplitude is determined based on the energy statistics of the service audio and a preset coefficient, wherein the preset coefficient is greater than 1; An audio test sequence is generated based on a preset audio duration, a preset center frequency, and the sequence amplitude, wherein the preset center frequency is greater than the ambient noise frequency.

3. The audio end-to-end delay measurement method of claim 2, wherein, The step of injecting the audio test sequence into the service audio and determining the transmission time of the audio test sequence includes: When the service audio reaches the preset injection trigger point, the audio test sequence is injected into the service audio in the same path; The data after the injection trigger point is processed to equalize the length so that the duration of the output stream of the service audio remains unchanged before and after the injection. The equalization process includes randomly discarding equal-length data or time axis scaling compensation. The transmission time of the audio test sequence is determined based on the trigger time of the injection trigger point.

4. The audio end-to-end delay measurement method of claim 3, wherein, The process of decoding and feature extraction of the received audio stream to obtain an audio feature sequence includes: The received audio stream is decoded, and the decoded audio is sampled at fixed time intervals to obtain audio samples; Energy features are extracted from each audio sample to obtain the audio features of each audio sample; Based on the audio features of each audio sample and the sampling timestamp of each audio sample, an audio feature sequence arranged in chronological order is generated.

5. The audio end-to-end delay measurement method of claim 4, wherein, The step of performing time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence includes: The audio feature sequence is analyzed according to the chronological order of the sampling timestamps to determine the target continuous interval that matches the features of the audio test sequence; The reception time of the audio test sequence is determined based on the target continuous interval; Wherein, the duration of the target continuous interval is greater than or equal to the preset audio duration, and the proportion of audio features with energy greater than an energy threshold in the target continuous interval is greater than or equal to a proportion threshold, wherein the energy threshold is determined by the sequence amplitude.

6. The audio end-to-end delay measurement method of claim 5, wherein, Determining the reception time of the audio test sequence based on the target continuous interval includes: The reception time of the audio test sequence is determined based on the first sampling timestamp of the target continuous interval.

7. The audio end-to-end delay measurement method according to any of claims 1 to 6, characterized in that, Also includes: The steps of generating an audio test sequence based on the service audio transmitted in the audio processing link, injecting the audio test sequence into the service audio in the same path, and determining the single delay for each execution are performed periodically. A delay sample set is generated based on a preset number of single delays, and statistical calculations are performed on the delay sample set to obtain the end-to-end delay index of the service audio.

8. An audio end-to-end delay measurement system characterized by, include: The transmitting module is used to generate an audio test sequence based on the service audio transmitted in the audio processing link, inject the audio test sequence into the service audio in the same path, and determine the transmission time of the audio test sequence; The receiving module is used to decode and extract features from the received audio stream to obtain an audio feature sequence; perform time-domain analysis and feature matching on the audio feature sequence to determine the reception time of the audio test sequence; and determine the end-to-end delay of the service audio based on the transmission time and the reception time.

9. An audio end-to-end delay measurement system characterized by, include: A processor, and a memory communicatively connected to the processor; The memory is used to store computer-executed instructions; The processor is configured to execute computer execution instructions stored in the memory, causing the processor to perform the audio end-to-end delay measurement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the audio end-to-end delay measurement method according to any one of claims 1 to 7.

Images