Audio delay detection method and apparatus, computing device, and storage medium

Abstract

Embodiments of the present application disclose an audio delay detection method and device, a computing device and a storage medium. The method comprises: a first user terminal generates a basic audio and a feature audio, and obtains a mixed audio after inserting the feature audio in the basic audio; the first user terminal sends the mixed audio and records a sending time of the feature audio in the mixed audio; a second user terminal receives the mixed audio, identifies the feature audio in the mixed audio, and records a receiving time of the feature audio; and the audio delay between the first user terminal and the second user terminal is determined according to the sending time and the receiving time. The scheme can reduce the detection cost, improve the similarity between the detection process and the real network voice call environment, guarantee the stability of the code rate and bandwidth demand in the detection process, and improve the detection efficiency and precision of the audio delay.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, in particular to an audio delay detection method and device, a computing device and a storage medium. BACKGROUND

[0002] With the continuous development of science and technology and society, various Internet services have greatly enriched people's work and life. Among them, network voice services can enable users in different locations to realize voice interaction through the network, and thus are widely used in instant messaging, live broadcast and other Internet products.

[0003] Due to the influence of service architecture, network environment and other factors, network voice services usually have corresponding audio delay, and the size of the audio delay directly affects the user experience, thus being an important indicator in network voice services.

[0004] The inventors have found in the implementation process that the prior art has the following defects: the existing audio delay detection method is: when detecting the audio delay of user terminal A and user terminal B, the user terminals A and B and a third-party device C are set in the same experimental environment, the signal generator of the third-party device C generates an audio signal, the user terminal A collects the audio signal and transmits it to the user terminal B, the user terminal B plays the audio signal through its loudspeaker, the signal receiver of the third-party device C receives the audio signal played by the user terminal B, and then the audio delay of the user terminal A and the user terminal B is determined according to the signal generation time of the signal generator and the signal receiving time of the signal receiver. However, the implementation of this method requires the use of an additional third-party device, resulting in high detection cost of this detection method. Moreover, the devices to be detected need to be set in the same experimental environment, so that the audio delay of the real network voice interaction parties cannot be obtained, and the detected audio delay includes the propagation time of the audio signal between the signal generator and the user terminal A, and between the user terminal B and the signal receiver, resulting in low detection accuracy of this implementation method. Moreover, this implementation method is complex and inefficient. SUMMARY

[0005] In view of the technical problems of low detection efficiency, low detection accuracy and high detection cost of the audio delay in the prior art, the embodiments of the present application are proposed to provide an audio delay detection method, device, computing device and storage medium that overcome the above problems or at least partially solve the above problems.

[0006] According to a first aspect of the embodiments of the present application, an audio delay detection method is provided, comprising:

[0007] The first user terminal generates a basic audio and a feature audio, and obtains a mixed audio by inserting the feature audio into the basic audio;

[0008] The first user terminal sends the mixed audio, and records a sending time of the characteristic audio in the mixed audio;

[0009] The second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records a receiving time of the characteristic audio;

[0010] According to the sending time and the receiving time, an audio delay between the first user terminal and the second user terminal is determined.

[0011] In an optional implementation, the first user terminal generating the base audio further includes: the first user terminal generating the base audio by using a random algorithm.

[0012] In an optional implementation, the first user terminal generating the base audio by using a random algorithm further includes: the first user terminal obtaining a sampling value range of a sampling point of the base audio, and generating a sampling value of the sampling point of the base audio by using a random algorithm based on the sampling value range.

[0013] In an optional implementation, the first user terminal generating the base audio by using a random algorithm based on the sampling value range further includes: generating a random number by using a random algorithm, determining a boundary value of the sampling value range, calculating a modulus of the random number and the boundary value, and generating the sampling value of the sampling point of the base audio according to the modulus; and / or, taking the sampling value range as a random range of a random algorithm, and generating a random number by using a random algorithm based on the random range, and generating the sampling value of the sampling point of the base audio according to the random number.

[0014] In an optional implementation, the first user terminal obtaining the sampling value range of the sampling point of the base audio further includes: obtaining historical audio stored by the first user terminal, and determining a sampling value range of a sampling point in the historical audio; and determining the sampling value range of the sampling point of the base audio according to the sampling value range of the sampling point in the historical audio.

[0015] In an optional implementation, the characteristic audio is mute audio; or, the characteristic audio is audio of a preset waveform.

[0016] In an optional implementation, after the characteristic audio is inserted into the base audio to obtain the mixed audio, the method further includes: recording an insertion position of the characteristic audio in the mixed audio.

[0017] The first user terminal sends the mixed audio, and recording the sending time of the characteristic audio in the mixed audio further comprises: the first user terminal divides the mixed audio into multiple audio segments, and records the starting position and the ending position of each audio segment in the mixed audio; the first user terminal sends an audio segment every preset time length, and judges whether the insertion position of the characteristic audio is within the starting position and the ending position of the audio segment when sending the audio segment; if yes, the sending time of the audio segment is determined as the sending time of the characteristic audio in the audio segment.

[0018] In an alternative embodiment, the first user terminal sends the mixed audio, and recording the sending time of the characteristic audio in the mixed audio further comprises: the first user terminal divides the mixed audio into multiple audio segments, and assigns a preset label to the audio segment containing the characteristic audio; the first user terminal sends an audio segment every preset time length, and judges whether the audio segment has the preset label when sending the audio segment; if yes, the sending time of the audio segment is determined as the sending time of the characteristic audio in the audio segment.

[0019] In an alternative embodiment, the second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records the receiving time of the characteristic audio further comprises: the second user terminal receives the audio segment, and identifies whether the audio segment contains the characteristic audio; if yes, the receiving time of the audio segment is determined as the receiving time of the characteristic audio.

[0020] In an alternative embodiment, if the characteristic audio is a mute audio; the identifying the characteristic audio in the mixed audio further comprises: performing windowing processing on the mixed audio; wherein the window length matches the audio length of the characteristic audio; for any window, calculating the short-time energy corresponding to the window, if the short-time energy is less than a preset energy threshold, determining that the audio segment corresponding to the window is the characteristic audio; and / or, for any window, calculating the zero-crossing rate corresponding to the window, if the zero-crossing rate is less than a preset zero-crossing rate threshold, determining that the audio segment corresponding to the window is the characteristic audio.

[0021] In an alternative embodiment, if the characteristic audio is an audio of a preset waveform; the identifying the characteristic audio in the mixed audio further comprises: performing windowing processing on the mixed audio; wherein the window length matches the audio length of the characteristic audio; for each window, judging whether the waveform of the audio corresponding to the window is the preset waveform; if yes, determining that the audio corresponding to the window is the characteristic audio.

[0022] In an alternative embodiment, the first user terminal generates a single feature audio; the inserting the feature audio into the base audio further comprises inserting the single feature audio into at least one position of the base audio; the recording the sending time of the feature audio in the mixed audio further comprises recording the sending time and sending order of the feature audio in each position of the mixed audio; the recording the receiving time of the feature audio further comprises recording the receiving time and receiving order of the feature audio; and the determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time further comprises determining the feature audio with matching sending order and receiving order, and determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of the matching feature audio.

[0023] In an alternative embodiment, the first user terminal generates a plurality of feature audios with different audio lengths; the inserting the feature audio into the base audio further comprises inserting the plurality of feature audios with different audio lengths into different positions of the base audio respectively; the recording the sending time of the feature audio in the mixed audio further comprises recording the sending time and audio length of the feature audio in the mixed audio; the recording the receiving time of the feature audio further comprises recording the receiving time and audio length of each feature audio; and the determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time further comprises determining the sending time and receiving time of the feature audio with same audio length, and determining the audio delay between the first user terminal and the second user terminal according to the sending time and receiving time of each feature audio with same audio length.

[0024] In an alternative embodiment, the recording the sending time of the feature audio in the mixed audio further comprises determining the system to which the first user terminal and the second user terminal belong, and recording the sending time of the feature audio based on the system time of the system; and the recording the receiving time of the feature audio further comprises recording the receiving time of the feature audio based on the system time of the system.

[0025] In an alternative embodiment, the first user terminal generates the base audio and the feature audio, and obtains the mixed audio after inserting the feature audio into the base audio further comprises: a first client in the first user terminal generates the base audio and the feature audio, and obtains the mixed audio after inserting the feature audio into the base audio; the first user terminal sends the mixed audio, and records the sending time of the feature audio in the mixed audio further comprises: the first client sends the mixed audio to a first audio processor in the first user terminal, and the first client records the sending time of the feature audio sent to the first audio processor in the mixed audio; the second user terminal receives the mixed audio, identifies the feature audio in the mixed audio, and records the receiving time of the feature audio further comprises: a second client in the second user terminal receives the mixed audio transmitted from a second audio processor in the second user terminal, identifies the feature audio in the mixed audio, and records the receiving time of the feature audio received from the second audio processor; and the audio delay between the first user terminal and the second user terminal is determined according to the sending time and the receiving time further comprises: the first audio delay between the first user terminal and the second user terminal is determined according to the sending time and the receiving time.

[0026] In an alternative embodiment, the method further comprises: the first audio processor transmits the mixed audio to an Ethernet after performing audio preprocessing, audio encoding and / or sending buffer on the mixed audio; and the second audio processor sends the mixed audio to the second client after performing buffer obtaining, audio decoding and / or audio post-processing on the mixed audio.

[0027] In an alternative embodiment, the method further comprises: obtaining a second audio delay of an audio collector in the first user terminal, and obtaining a third audio delay of an audio player in the second user terminal; and determining the total audio delay between the first user terminal and the second user terminal according to the first audio delay, the second audio delay and the third audio delay.

[0028] In an alternative embodiment, the method further comprises: the first user terminal monitors a preset event; and the first user terminal performs the steps of generating the base audio and the feature audio after monitoring the preset event; wherein the preset event comprises at least one of the following events: a version update event, a triggering event of user audio delay detection, and a network switching event.

[0029] According to a second aspect of the embodiments of the present application, an audio delay detection method is provided, comprising: generating a base audio and a feature audio, and obtaining a mixed audio after inserting the feature audio into the base audio; sending the mixed audio; recording a sending time of the feature audio in the mixed audio, so that a second user terminal receives the mixed audio, identifies the feature audio in the mixed audio, records a receiving time of the feature audio, and determines an audio delay between the second user terminal according to the sending time and the receiving time.

[0030] In an optional implementation, the generating the base audio further comprises: generating the base audio by using a random algorithm.

[0031] In an optional implementation, the generating the base audio by using the random algorithm further comprises: obtaining a sampling value range of a sampling point of the base audio, and generating a sampling value of a sampling point of the base audio by using a random algorithm based on the sampling value range.

[0032] In an optional implementation, the generating the sampling value of the sampling point of the base audio based on the sampling value range by using the random algorithm further comprises: generating a random number by using a random algorithm, determining a boundary value of the sampling value range, calculating a modulus of the random number and the boundary value, and generating the sampling value of the sampling point of the base audio according to the modulus; and / or, taking the sampling value range as a random range of the random algorithm, generating a random number based on the random range by using the random algorithm, and generating the sampling value of the sampling point of the base audio according to the random number.

[0033] In an optional implementation, the obtaining the sampling value range of the sampling point of the base audio further comprises: obtaining a stored historical audio, and determining a sampling value range of a sampling point in the historical audio; and determining the sampling value range of the sampling point of the base audio according to the sampling value range of the sampling point in the historical audio.

[0034] In an optional implementation, the feature audio is a mute audio; or, the feature audio is an audio with a preset waveform.

[0035] In an alternative embodiment, after the mixed audio is obtained by inserting the feature audio into the base audio, the method further comprises: recording the insertion position of the feature audio in the mixed audio; the sending the mixed audio; recording the sending time of the feature audio in the mixed audio further comprises: dividing the mixed audio into a plurality of audio segments, and recording the starting position and the ending position of each audio segment in the mixed audio; sending one audio segment every preset time length, and determining whether the insertion position of the feature audio is within the starting position and the ending position of the audio segment when the audio segment is sent; if yes, determining the sending time of the audio segment as the sending time of the feature audio in the audio segment.

[0036] In an alternative embodiment, the sending the mixed audio; recording the sending time of the feature audio in the mixed audio further comprises: dividing the mixed audio into a plurality of audio segments, and assigning a preset label to the audio segment containing the feature audio; sending one audio segment every preset time length, and determining whether the audio segment has the preset label when the audio segment is sent; if yes, determining the sending time of the audio segment as the sending time of the feature audio in the audio segment.

[0037] In an alternative embodiment, the generating the feature audio further comprises: generating a single feature audio; the inserting the feature audio into the base audio further comprises: inserting the single feature audio into at least one position of the base audio; the recording the sending time of the feature audio in the mixed audio further comprises: recording the sending time and the sending order of the feature audio in each position of the mixed audio; the recording the receiving time of the feature audio further comprises: recording the receiving time and the receiving order of the feature audio; the determining the audio delay between the second user terminal according to the sending time and the receiving time further comprises: determining the feature audio with the matching sending order and receiving order, and determining the audio delay between the second user terminal according to the sending time and the receiving time of the matching feature audio.

[0038] In an alternative implementation, the generating the feature audio further comprises: generating a plurality of feature audios with different audio lengths; the inserting the feature audio into the base audio further comprises: inserting the plurality of feature audios with different audio lengths into different positions of the base audio respectively; the recording the sending time of the feature audio in the mixed audio further comprises: recording the sending time and the audio length of the feature audio in the mixed audio; the recording the receiving time of the feature audio further comprises: recording the receiving time and the audio length of each feature audio; and the determining the audio delay between the second user terminal based on the sending time and the receiving time further comprises: determining the sending time and the receiving time of the feature audio with the same audio length, and determining the audio delay between the second user terminal based on the sending time and the receiving time of each feature audio with the same audio length.

[0039] In an alternative implementation, the recording the sending time of the feature audio in the mixed audio further comprises: determining a system to which the second user terminal belongs, and recording the sending time of the feature audio based on the system time of the system.

[0040] In an alternative implementation, the generating the base audio and the feature audio, and obtaining the mixed audio after inserting the feature audio into the base audio further comprises: a first client generates the base audio and the feature audio, and obtains the mixed audio after inserting the feature audio into the base audio; and the sending the mixed audio and recording the sending time of the feature audio in the mixed audio further comprises: sending the mixed audio to a first audio processor, and recording, by the first client, the sending time of the feature audio sent to the first audio processor.

[0041] In an alternative implementation, the method further comprises: performing, by the first audio processor, audio pre-processing, audio encoding and / or sending buffer on the mixed audio, and transmitting the mixed audio to an Ethernet.

[0042] In an alternative implementation, the method further comprises: obtaining a second audio delay of an audio collector, and determining a total audio delay between the second user terminal based on the first audio delay, the second audio delay and a third audio delay of an audio player in the second user terminal.

[0043] In an alternative implementation, the method further comprises: monitoring a preset event; and performing the generating the base audio and the feature audio after monitoring the preset event; wherein the preset event comprises at least one of the following events: a version update event, a triggering event of audio delay detection by a user, and a network switching event.

[0044] According to a third aspect of the embodiments of the present application, an audio delay detection method is provided, comprising: receiving mixed audio; the mixed audio is composed of base audio generated by a first user terminal and feature audio; identifying the feature audio in the mixed audio; recording the receiving time of the feature audio, so as to determine the audio delay between the first user terminals according to the sending time of the feature audio recorded by the first user terminal and the receiving time.

[0045] In an optional implementation, the recording of the receiving time of the feature audio further comprises: determining a system to which the first user terminal belongs, and recording the receiving time of the feature audio based on the system time of the system.

[0046] In an optional implementation, the receiving of the mixed audio further comprises: receiving, by a second client, the mixed audio transmitted from a second audio processor; the identifying of the feature audio in the mixed audio further comprises: identifying, by the second client, the feature audio in the mixed audio; and the recording of the receiving time of the feature audio further comprises: recording, by the second client, the receiving time of the feature audio received from the second audio processor.

[0047] In an optional implementation, if the feature audio is mute audio, the identifying of the feature audio in the mixed audio further comprises: performing windowing processing on the mixed audio; wherein the window length matches the audio length of the feature audio; for any window, calculating the short-time energy corresponding to the window, and if the short-time energy is less than a preset energy threshold, determining that the audio segment corresponding to the window is the feature audio; and / or, for any window, calculating the zero-crossing rate corresponding to the window, and if the zero-crossing rate is less than a preset zero-crossing rate threshold, determining that the audio segment corresponding to the window is the feature audio.

[0048] In an optional implementation, if the feature audio is audio of a preset waveform, the identifying of the feature audio in the mixed audio further comprises: performing windowing processing on the mixed audio; wherein the window length matches the audio length of the feature audio; for each window, judging whether the waveform of the audio corresponding to the window is the preset waveform; if yes, determining the audio corresponding to the window as the feature audio.

[0049] In an optional implementation, a third audio delay of an audio player is obtained, so as to determine the total audio delay between the first user terminals according to the first audio delay between the first user terminals determined according to the sending time of the feature audio recorded by the first user terminal and the receiving time, the second audio delay of the audio collector in the first user terminal, and the third audio delay.

[0050] According to a fourth aspect of the embodiments of the present application, an audio delay detection method is provided, comprising: receiving a sending time of a characteristic audio sent by a first user terminal; wherein the first user terminal generates a basic audio and a characteristic audio, and obtains a mixed audio by inserting the characteristic audio into the basic audio; receiving a receiving time of the characteristic audio sent by a second user terminal; wherein the second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records the receiving time of the characteristic audio; and determining an audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time.

[0051] In an optional implementation, the determining of the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time further comprises: determining a first audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time; obtaining a second audio delay of an audio collector in the first user terminal, and obtaining a third audio delay of an audio player in the second user terminal; and determining a total audio delay between the first user terminal and the second user terminal according to the first audio delay, the second audio delay and the third audio delay.

[0052] In an optional implementation, the determining of the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time further comprises: determining a characteristic audio with a sending sequence matching a receiving sequence, and determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of the matching characteristic audio; or determining the sending time and the receiving time of a characteristic audio with a consistent audio length, and determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of each characteristic audio with a consistent audio length.

[0053] According to a fifth aspect of the embodiments of the present application, an audio delay detection device is provided, comprising: a generating module for generating a basic audio and a characteristic audio; an inserting module for obtaining a mixed audio by inserting the characteristic audio into the basic audio; and a sending module for sending the mixed audio.

[0054] a recording module for recording a sending time of the characteristic audio in the mixed audio; so as to enable a second user terminal to receive the mixed audio, identify the characteristic audio in the mixed audio, record a receiving time of the characteristic audio, and determine an audio delay between the second user terminal according to the sending time and the receiving time.

[0055] In an alternative implementation, the generating module is configured to generate the base audio by using a random algorithm.

[0056] In an alternative implementation, the generating module is configured to obtain a sampling value range of the sampling points of the base audio, and generate the sampling value of the sampling points of the base audio by using a random algorithm based on the sampling value range.

[0057] In an alternative implementation, the generating module is configured to generate a random number by using a random algorithm, determine a boundary value of the sampling value range, calculate a modulus of the random number and the boundary value, and generate the sampling value of the sampling points of the base audio according to the modulus.

[0058] In an alternative implementation, the generating module is configured to obtain a sampling value range of the sampling points of the base audio, and generate the sampling value of the sampling points of the base audio by using a random algorithm based on the sampling value range.

[0059] In an alternative implementation, the generating module is configured to obtain a stored historical audio, and determine a sampling value range of the sampling points of the historical audio; and determine the sampling value range of the sampling points of the base audio according to the sampling value range of the sampling points of the historical audio.

[0060] In an alternative implementation, the characteristic audio is a mute audio; or the characteristic audio is an audio with a preset waveform.

[0061] In an alternative implementation, the sending module is configured to divide the mixed audio into a plurality of audio segments, and send one audio segment every predetermined time length; the recording module is configured to record the insertion position of the characteristic audio in the mixed audio, and record the starting position and the ending position of each audio segment in the mixed audio; and determine the sending time of the audio segment as the sending time of the characteristic audio in the audio segment if the insertion position of the characteristic audio is within the starting position and the ending position of the audio segment.

[0062] In an alternative implementation, the sending module is configured to divide the mixed audio into a plurality of audio segments, and send one audio segment every predetermined time length, and assign a preset label to the audio segment containing the characteristic audio; the recording module is configured to determine whether the audio segment has the preset label when the audio segment is sent, and determine the sending time of the audio segment as the sending time of the characteristic audio in the audio segment if the audio segment has the preset label.

[0063] In an alternative implementation, the generating module is configured to generate a single characteristic audio; the inserting module is configured to insert the single characteristic audio into at least one position of the base audio; and the recording module is configured to record the sending time and the sending order of the characteristic audio at each position in the mixed audio.

[0064] In an optional implementation, the generating module is configured to generate a plurality of feature audios with different audio lengths; the inserting module is configured to insert the plurality of feature audios with different audio lengths into different positions of the base audio respectively; and the recording module is configured to record the sending time and the audio length of the feature audio in the mixed audio.

[0065] In an optional implementation, the recording module is configured to determine a system to which the second user terminal belongs, and record the sending time of the feature audio based on the system time of the system.

[0066] In an optional implementation, the generating module, the inserting module, the sending module and the recording module are located in the first client.

[0067] The sending module is configured to send the mixed audio to a first audio processor.

[0068] The recording module is configured to record the sending time of the feature audio sent to the first audio processor in the mixed audio.

[0069] In an optional implementation, the first audio processor transmits the mixed audio to an Ethernet after performing audio preprocessing, audio encoding and / or sending buffer on the mixed audio.

[0070] In an optional implementation, the apparatus further comprises an obtaining module configured to obtain a second audio delay of an audio collector, so as to determine the total audio delay between the second user terminal based on the first audio delay, the second audio delay and a third audio delay of an audio player in the second user terminal.

[0071] In an optional implementation, the apparatus further comprises a monitoring module configured to monitor a preset event.

[0072] After the preset event is monitored, the steps of generating the base audio and the feature audio are performed.

[0073] The preset event comprises at least one of the following events:

[0074] a version update event, a triggering event of user audio delay detection and a network switching event.

[0075] According to a sixth aspect of an embodiment of the present application, an audio delay detection apparatus is provided, comprising:

[0076] a receiving module configured to receive a mixed audio; the mixed audio is composed of a base audio and a feature audio generated by a first user terminal;

[0077] an identifying module configured to identify a characteristic audio in the mixed audio;

[0078] a recording module configured to record a receiving time of the characteristic audio, so as to determine an audio delay between the first user terminal according to a sending time of the characteristic audio recorded by the first user terminal and the receiving time.

[0079] In an optional implementation, the recording module is configured to determine a system to which the first user terminal belongs, and record the receiving time of the characteristic audio based on a system time of the system.

[0080] In an optional implementation, the characteristic audio is a mute audio.

[0081] The identifying module is configured to perform windowing processing on the mixed audio, wherein a window length matches an audio length of the characteristic audio.

[0082] For any window, a short-time energy corresponding to the window is calculated, and if the short-time energy is less than a preset energy threshold, an audio segment corresponding to the window is determined as the characteristic audio.

[0083] For any window, a zero-crossing rate corresponding to the window is calculated, and if the zero-crossing rate is less than a preset zero-crossing rate threshold, an audio segment corresponding to the window is determined as the characteristic audio.

[0084] In an optional implementation, the characteristic audio is an audio of a preset waveform.

[0085] The identifying module is configured to perform windowing processing on the mixed audio, wherein a window length matches an audio length of the characteristic audio; for each window, it is judged whether a waveform of audio corresponding to the window is a preset waveform; if yes, the audio corresponding to the window is determined as the characteristic audio.

[0086] In an optional implementation, the receiving module, the identifying module and the recording module are located in the second client.

[0087] The receiving module is configured to receive the mixed audio transmitted from a second audio processor.

[0088] The recording module is configured to record a receiving time of the characteristic audio received from the second audio processor.

[0089] In an alternative implementation, the apparatus further comprises an obtaining module configured to obtain a third audio delay of an audio player, so as to determine a total audio delay between the first user terminal and the second user terminal based on a first audio delay between the first user terminal and the second user terminal determined according to a sending time of the characteristic audio recorded by the first user terminal and the receiving time, a second audio delay of an audio collector in the first user terminal, and the third audio delay.

[0090] According to a seventh aspect of an embodiment of the present application, an audio delay detection apparatus is provided, comprising:

[0091] a receiving module configured to receive a sending time of the characteristic audio sent by the first user terminal; wherein the first user terminal generates a base audio and a characteristic audio, and obtains a mixed audio by inserting the characteristic audio into the base audio; and receive a receiving time of the characteristic audio sent by the second user terminal; wherein the second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records the receiving time of the characteristic audio;

[0092] a determining module configured to determine an audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time.

[0093] In an alternative implementation, the determining module is configured to:

[0094] determine a first audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time;

[0095] obtain a second audio delay of an audio collector in the first user terminal, and obtain a third audio delay of an audio player in the second user terminal;

[0096] determine a total audio delay between the first user terminal and the second user terminal based on the first audio delay, the second audio delay, and the third audio delay.

[0097] According to an eighth aspect of an embodiment of the present application, a computing device is provided, comprising a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0098] the memory is configured to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the above audio delay detection method.

[0099] According to a ninth aspect of an embodiment of the present application, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction causes a processor to perform operations corresponding to the above audio delay detection method.

[0100] In the embodiment of the present application, the first user terminal generates the basic audio and the characteristic audio, obtains the mixed audio after inserting the characteristic audio in the basic audio, sends the mixed audio, records the sending time of the characteristic audio in the mixed audio, the second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, records the receiving time of the characteristic audio, and determines the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time. The scheme can reduce the detection cost, improve the similarity between the detection process and the real network voice call environment, guarantee the stability of the code rate and bandwidth demand in the detection process, and improve the detection efficiency and detection accuracy of the audio delay.

[0101] In the embodiment of the present application, the first user terminal generates the basic audio by using a random algorithm, so as to improve the adaptation of the basic audio to the actual network voice call environment and improve the accuracy of the audio delay detection.

[0102] In the embodiment of the present application, the first user terminal obtains the sampling value range of the sampling point of the basic audio, and generates the sampling value of the sampling point of the basic audio by using a random algorithm based on the sampling value range, so as to further guarantee that the sampling value of the sampling point of the basic audio is located in the corresponding sampling value range and improve the adaptation degree of the basic audio to the actual network voice call environment.

[0103] In the embodiment of the present application, the first user terminal generates a random number by using a random algorithm, and determines the boundary value of the sampling value range, the first user terminal calculates the modulus of the random number and the boundary value, and generates the sampling value of the sampling point of the basic audio according to the modulus, so as to improve the randomness of the sampling value; or the first user terminal takes the sampling value range as the random range of the random algorithm, generates a random number based on the random range by using a random algorithm, and generates the sampling value of the sampling point of the basic audio according to the random number, so as to improve the generation efficiency of the basic audio.

[0104] In the embodiment of the present application, the historical audio stored in the first user terminal is obtained, and the sampling value range of the sampling point in the historical audio is determined; the sampling value range of the sampling point of the basic audio is determined according to the sampling value range of the sampling point in the historical audio, so that the generated basic audio is adapted to the user voice characteristics in the first user terminal, the user-differentiated basic audio is realized, and the adaptation degree of the basic audio to the actual network voice call environment is further improved.

[0105] In the embodiment of the present application, the characteristic audio is a mute audio; or the characteristic audio is an audio with a preset waveform, so as to facilitate the identification of the characteristic audio, improve the identification efficiency of the characteristic audio, and improve the detection efficiency of the audio delay.

[0106] In the embodiment of the present application, the first user terminal divides the mixed audio into multiple audio segments, records the start position and the end position of each audio segment in the mixed audio, sends an audio segment every preset time length, and judges whether the insertion position of the characteristic audio is within the start position and the end position of the audio segment when sending the audio segment; if yes, the sending time of the audio segment is determined as the sending time of the characteristic audio in the audio segment, thereby the sending time of the characteristic audio can be accurately determined.

[0107] In the embodiment of the present application, a preset label is assigned to the audio segment containing the characteristic audio, and the first user terminal judges whether the audio segment has the preset label when sending the audio segment; if yes, the sending time of the audio segment is determined as the sending time of the characteristic audio in the audio segment, thereby the sending time of the characteristic audio can be quickly determined.

[0108] In the embodiment of the present application, if the characteristic audio is a mute audio, the mixed audio is subjected to windowing processing; wherein the window length matches the audio length of the characteristic audio, and the characteristic audio is identified according to the short-time energy and / or zero-crossing rate corresponding to the window, thereby the identification accuracy of the characteristic audio is improved.

[0109] In the embodiment of the present application, if the characteristic audio is an audio of a preset waveform, the mixed audio is subjected to windowing processing; wherein the window length matches the audio length of the characteristic audio, and for each window, it is judged whether the waveform of the audio corresponding to the window is the preset waveform; if yes, the audio corresponding to the window is determined as the characteristic audio, thereby the identification accuracy of the characteristic audio is improved.

[0110] In the embodiment of the present application, the first user terminal generates a single characteristic audio, inserts the single characteristic audio into at least one position of the base audio, records the sending time and the sending order of the characteristic audio at each position in the mixed audio, records the receiving time and the receiving order of the characteristic audio, determines the characteristic audio with the matching sending order and receiving order, and determines the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of the matched characteristic audio, thereby the number of generated characteristic audios can be reduced, and the overall execution efficiency is improved.

[0111] In the embodiment of the present application, the first user terminal generates a plurality of characteristic audios with different audio lengths, inserts the plurality of characteristic audios with different audio lengths into different positions of the base audio, records the sending time and the audio length of the characteristic audio in the mixed audio, records the receiving time and the audio length of each characteristic audio, determines the sending time and the receiving time of the characteristic audio with the same audio length, determines the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of each characteristic audio with the same audio length, and adopts the above-mentioned manner, so that the same characteristic audio can be accurately positioned through the audio length even in the case of abnormal conditions such as sending packet loss, thereby improving the detection accuracy of the audio delay.

[0112] In the embodiment of the present application, the system to which the first user terminal and the second user terminal belong is determined, the sending time of the characteristic audio is recorded based on the system time of the system, and the receiving time of the characteristic audio is recorded based on the system time of the system, so that the same system clock can be used for the sending time and the receiving time, and the detection accuracy of the audio delay is improved.

[0113] In the embodiment of the present application, the first client in the first user terminal generates the base audio and the characteristic audio, and obtains the mixed audio after inserting the characteristic audio into the base audio; the first client sends the mixed audio to the first audio processor in the first user terminal, and records the sending time of the characteristic audio sent to the first audio processor in the mixed audio; the second client in the second user terminal receives the mixed audio transmitted from the second audio processor in the second user terminal, identifies the characteristic audio in the mixed audio, and records the receiving time of the characteristic audio received from the second audio processor; and the first audio delay between the first user terminal and the second user terminal is determined according to the sending time and the receiving time. By adopting the above-mentioned implementation manner, the audio delay of the time period from when the audio collector in the first user terminal starts to send the audio to when the audio is transmitted to the audio player can be accurately determined.

[0114] In the embodiment of the present application, the first audio processor transmits the mixed audio to the Ethernet after performing audio preprocessing, audio encoding and / or sending buffer area on the mixed audio; and the second audio processor sends the mixed audio to the second client after performing audio decoding and / or audio post-processing from the buffer area. In this way, the audio delay detection process can be matched with the actual audio processing flow in the network voice call environment, and the detection accuracy of the audio delay is improved.

[0115] In the embodiment of the present application, the second audio delay of the audio collector in the first user terminal is acquired, and the third audio delay of the audio player in the second user terminal is acquired; the total audio delay between the first user terminal and the second user terminal is determined according to the first audio delay, the second audio delay and the third audio delay. By using the embodiment, the audio delay in the whole process from audio collection to audio playing can be accurately determined.

[0116] In the embodiment of the present application, the first user terminal monitors a preset event; the first user terminal executes the steps of generating the basic audio and the characteristic audio after monitoring the preset event; wherein the preset event includes at least one of the following events: a version update event, a trigger event of audio delay detection of a user, and a network switching event. By using the embodiment, the automatic detection of the audio delay can be realized.

[0117] The above description is only a summary of the technical scheme of the embodiment of the present application, in order to more clearly understand the technical means of the embodiment of the present application, the content of the specification can be implemented, and in order to make the above and other purposes, characteristics and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation of the embodiment of the present application is described below. BRIEF DESCRIPTION OF DRAWINGS

[0118] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of illustrating the preferred embodiment and are not considered to be a limitation on the embodiments of the present application. Moreover, the same reference symbols are used to represent the same parts throughout the drawings. In the drawings:

[0119] Figure 1 A flowchart of an audio delay detection method provided by the embodiment of the present application is shown;

[0120] Figure 2 A flowchart of another audio delay detection method provided by the embodiment of the present application is shown;

[0121] Figure 3 An audio transmission path provided by the embodiment of the present application is shown;

[0122] Figure 4 Another audio transmission path provided by the embodiment of the present application is shown;

[0123] Figure 5 A flowchart of another audio delay detection method provided by the embodiment of the present application is shown;

[0124] Figure 6 A flowchart of another audio delay detection method provided by the embodiment of the present application is shown;

[0125] Figure 7 Fig. 1 shows a flow diagram of another audio delay detection method provided by an embodiment of the present application;

[0126] Figure 8 Fig. 2 shows a structural diagram of an audio delay detection device provided by an embodiment of the present application;

[0127] Figure 9 Fig. 3 shows a structural diagram of another audio delay detection device provided by an embodiment of the present application;

[0128] Figure 10 Fig. 4 shows a structural diagram of another audio delay detection device provided by an embodiment of the present application;

[0129] Figure 11 Fig. 5 shows a structural diagram of a computing device provided by an embodiment of the present application; DETAILED DESCRIPTION

[0130] Exemplary embodiments of the present application will be described in detail with reference to the drawings. Although exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0131] Figure 1 Fig. 1 shows a flow diagram of an audio delay detection method provided by an embodiment of the present application. The flowchart in the present embodiment is not used to limit the order of execution of the steps. According to the needs, some steps in the flowchart can be added or deleted.

[0132] The present embodiment takes detecting the audio delay between a first user terminal and a second user terminal as an example to elaborate the audio delay detection method. The first user terminal and the second user terminal can perform voice interaction through a network. The execution subject of the present embodiment can be an audio delay detection system. The audio delay detection scheme provided by the present embodiment can be applied to the voice call function of an instant messaging application, and / or the microphone function of a live broadcast application, etc.

[0133] Specifically, as shown in Fig. 1, the method comprises the following steps: Figure 1

[0134] In step S110, the first user terminal generates a base audio and a feature audio, and obtains a mixed audio after inserting the feature audio into the base audio.

[0135] ​The prior art generates an audio signal by a third-party device, and the audio signal is collected by an audio collector in the first user terminal. Different from the prior art, the embodiment of the present application generates a basic audio by the first user terminal. The basic audio is an audio used to simulate a real voice interaction environment. The basic audio can guarantee the stability of the code rate and bandwidth demand of the first user terminal and the second user terminal when performing audio delay detection, so that the detected audio delay is matched with the audio delay in the real environment, and the detection accuracy of the audio delay is improved.

[0136] In an optional embodiment, the first user terminal generates the basic audio by using a random algorithm. Since the voice audio sent by the user in the real network voice interaction process has a certain randomness, the matching degree of the basic audio and the real voice interaction environment can be improved by using this method, which is beneficial to improve the detection accuracy of the audio delay. Specifically, the audio data generally includes the sampling value of each sampling point in the audio. In this embodiment, the sampling value of each sampling point is determined based on the random number generated by the random algorithm, and then the basic audio is generated.

[0137] Further optionally, in order to further improve the matching degree of the basic audio and the actual network voice interaction environment, and facilitate the processing of the audio data, the basic audio has a sampling value range of the corresponding sampling point in this embodiment. The sampling value of the sampling point of the generated basic audio is located in the sampling value range, that is, the first user terminal obtains the sampling value range of the sampling point of the basic audio, and generates the sampling value of the sampling point of the basic audio based on the sampling value range by using a random algorithm. Further, the way of generating the sampling value in the embodiment of the present application includes but is not limited to the following ways:

[0138] The first user terminal generates a random number by using a random algorithm, and determines the boundary value of the sampling value range. The first user terminal calculates the modulus of the random number and the boundary value, and generates the sampling value of the sampling point of the basic audio according to the modulus. Specifically, the maximum boundary value of the sampling value range is determined, and the value obtained by taking the modulus of each random number on the maximum boundary value is used as the sampling value of each sampling point. For example, the sampling value range is -255-255, and the maximum boundary value of the sampling value range is 255. rand() % 255 can be used as the sampling value of the sampling point of the basic audio, where rand() is a generated random number, and % is a modulus operator. By using this method, the obtained sampling value is located in the sampling value range, and this method can improve the randomness of the sampling value and improve the matching degree of the basic audio and the real scene audio.

[0139] The second sampling value generation manner is that the first user terminal takes the sampling value range as a random range of a random algorithm, and generates a random number based on the random range by using the random algorithm, and generates the sampling value of the sampling point of the basic audio according to the random number. In this generation manner, the sampling value range is directly taken as the value condition of the random algorithm, and the random number generated is in the sampling value range, and thus this manner can improve the generation efficiency of the basic audio.

[0140] Further, the embodiment of the present application does not limit the specific acquisition manner of the sampling value range of the sampling point of the basic audio. For example, a fixed sampling value range can be generated, thereby improving the acquisition efficiency of the sampling value range; or, the historical audio stored by the first user terminal is acquired, and the sampling value range of the sampling point in the historical audio is determined, and the sampling value range of the sampling point of the basic audio is determined according to the sampling value range of the sampling point in the historical audio. In this acquisition manner, the matching sampling value range can be generated according to the audio characteristics of each user terminal itself, thereby realizing differentiated sampling value range and improving the matching degree of the generated basic audio and the audio characteristics of the user corresponding to the first user terminal. It should be understood herein that the historical audio is acquired only in the case of user authorization, and the historical audio is not transmitted by the embodiment of the present application, thereby ensuring the security of user data.

[0141] In addition to generating the basic audio, the first user terminal also generates the characteristic audio. Specifically, since the basic audio has high similarity with the audio in the real network voice interaction environment, if only the basic audio is sent for audio delay detection, the recognition difficulty of the basic audio will be greatly increased, and the audio delay detection efficiency will be reduced. Therefore, the embodiment of the present application also generates the characteristic audio, which is different from the basic audio and can be quickly recognized. The embodiment of the present application does not limit the specific form of the characteristic audio, for example, the characteristic audio can be a mute audio, or a preset waveform audio, etc.

[0142] In an optional embodiment, the first user terminal generates a single characteristic audio, and inserts the single characteristic audio into at least one position of the basic audio. In this way, the generation process of the mixed audio can be simplified, and the generation efficiency of the mixed audio is improved, thereby improving the audio delay detection efficiency.

[0143] In another optional embodiment, the first user terminal generates a plurality of characteristic audios with different audio lengths, and inserts the plurality of characteristic audios with different audio lengths into different positions of the basic audio. In this way, the corresponding characteristic audio can be recognized by the audio length, thereby improving the determination accuracy of the subsequent audio delay.

[0144] In addition, embodiments of the present application do not limit the specific insertion manner. For example, the characteristic audio can be inserted into the base audio at a preset length interval, or the characteristic audio can be randomly inserted into the base audio, and so on.

[0145] In step S120, the first user terminal sends the mixed audio and records the sending time of the characteristic audio in the mixed audio.

[0146] In order to simulate a real network voice interaction scenario, the first user terminal specifically adopts a streaming transmission manner to send the mixed audio. Specifically, the transmission manner is to send a partial length of the mixed audio according to a corresponding sending period, that is, the first user terminal divides the mixed audio into multiple audio segments and sends an audio segment at a preset time interval.

[0147] Therefore, step S110 and step S120 can be sequentially executed or asynchronously executed. For example, the mixed audio can be sent through step S120 after the complete mixed audio is generated; or a partial mixed audio can be sent through step S120 after the partial mixed audio is generated. For example, step S110 can generate a base audio segment and send the base audio segment through step S120, and then step S120 can generate a characteristic audio and send the characteristic audio through step S120.

[0148] The first user terminal records the sending time of the characteristic audio in the mixed audio. Optionally, the system to which the first user terminal and the second user terminal belong is determined, and the sending time of the characteristic audio is recorded based on the system time of the system. Subsequently, when recording the receiving time, the receiving time of the characteristic audio is recorded based on the system time of the system. In this way, the same system time is used to record the receiving time and the sending time, thereby improving the accuracy of the audio delay.

[0149] In an optional sending time recording manner, after the mixed audio is obtained by inserting the characteristic audio into the base audio, the insertion position of the characteristic audio in the mixed audio is recorded; the first user terminal records the starting position and the ending position of each audio segment in the mixed audio; and the first user terminal determines whether the insertion position of the characteristic audio is within the starting position and the ending position of the audio segment when sending the audio segment, and if so, the sending time of the audio segment is determined as the sending time of the characteristic audio in the audio segment. In this recording manner, by comparing the insertion position of the characteristic audio with the starting position and the ending position of the currently sent audio segment, it can be accurately determined whether the characteristic audio is in the currently sent audio segment, and then the time when the audio segment is currently sent is determined as the sending time of the characteristic audio, thereby improving the accuracy of the sending time of the characteristic audio.

[0150] In another optional sending time recording mode, the first user terminal assigns a preset label to the audio segment containing the characteristic audio, so as to identify the audio segment containing the characteristic audio; then the first user terminal judges whether the audio segment has the preset label when sending the audio segment; if yes, the sending time of the audio segment is determined as the sending time of the characteristic audio in the audio segment. In this mode, the audio segment containing the characteristic audio is assigned with the preset label in advance, and when the audio segment is sent, it can be quickly determined based on the label whether the audio segment contains the characteristic audio, thereby improving the recording accuracy of the sending time of the characteristic audio.

[0151] In an optional embodiment, if the step S110 generates a single characteristic audio, the sending time and sending order of the characteristic audio at each position in the mixed audio are recorded. For example, the sending order of the first sent characteristic audio is 1, and the sent characteristic audio is identified by the sending order, which simplifies the implementation process of the embodiment of the application and improves the execution efficiency of the method.

[0152] In another optional embodiment, if the step S110 generates a plurality of characteristic audios with different audio lengths, the sending time and audio length of the characteristic audio in the mixed audio are recorded, the characteristic audio is identified by the audio length, the identification accuracy of the characteristic audio is improved, and then the determination accuracy of the audio delay is improved.

[0153] In step S130, the second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records the receiving time of the characteristic audio.

[0154] The second user terminal identifies the received mixed audio, so as to identify the characteristic audio in the mixed audio. The embodiment of the application does not limit the specific identification mode.

[0155] If the characteristic audio is mute audio, the characteristic audio is identified by a corresponding mute identification algorithm. For example, the mixed audio can be subjected to windowing processing, in which the window length matches the audio length of the characteristic audio. The mixed audio can be divided into multiple audio segments by the windowing processing, each audio segment corresponds to a window, and the length of each audio segment is consistent with the length of the characteristic audio. The window can be a rectangular window or a Hamming window, etc. Then, for any window, the short-time energy corresponding to the window is calculated. If the short-time energy is less than a preset energy threshold, it is determined that the audio segment corresponding to the window is the characteristic audio. And / or, for any window, the zero-crossing rate corresponding to the window is calculated. If the zero-crossing rate is less than a preset zero-crossing rate threshold, it is determined that the audio segment corresponding to the window is the characteristic audio. The specific calculation process of the short-time energy and the zero-crossing rate can refer to the algorithm in the prior art, which will not be described here. Since the short-time energy and the zero-crossing rate of the mute audio are low, the mute audio can be accurately identified in this way. Further, if multiple characteristic audios with different lengths are generated in step S110, a characteristic audio length is first selected for windowing processing and characteristic audio identification. If the characteristic audio cannot be identified, other characteristic audio lengths are selected for windowing processing and characteristic audio identification. After the characteristic audio is identified or all characteristic audio lengths are subjected to windowing processing, the identification is ended.

[0156] If the characteristic audio is a preset waveform, the characteristic audio is identified by a corresponding audio waveform identification algorithm. For example, the mixed audio is subjected to windowing processing, in which the window length matches the audio length of the characteristic audio. For each window, it is determined whether the waveform of the audio corresponding to the window is a preset waveform. If yes, the audio corresponding to the window is determined as the characteristic audio.

[0157] The second user terminal further records the receiving time of the characteristic audio. Since the first user terminal adopts a streaming mode, the second user terminal detects whether the audio segment contains the characteristic audio after receiving each audio segment. After determining that the audio segment contains the characteristic audio, the receiving time of the audio segment is determined as the receiving time of the characteristic audio.

[0158] In an optional implementation, if a single characteristic audio is generated in step S110, the receiving time and the receiving order of the characteristic audio are recorded. The receiving order M indicates that the characteristic audio is the Mth characteristic audio received by the second user terminal.

[0159] In another optional implementation, if multiple characteristic audios with different audio lengths are generated in step S110, the receiving time and the audio length of each characteristic audio are recorded.

[0160] Step S140, according to the sending time and the receiving time, determine the audio delay between the first user terminal and the second user terminal.

[0161] This step can be performed by the first user terminal, and the second user terminal feeds back the recorded information to the first user terminal, and the first user terminal determines the audio delay between the first user terminal and the second user terminal; and / or, this step can be performed by the second user terminal, and the first user terminal feeds back the recorded information to the second user terminal, and the second user terminal determines the audio delay between the first user terminal and the second user terminal; and / or, this step can also be performed by the corresponding server of the first user terminal and the second user terminal, and the first user terminal and the second user terminal feed back the recorded information to the server, and the server determines the audio delay between the first user terminal and the second user terminal. Further, the determined audio delay can be sent to the first user terminal and / or the second user terminal, so that the corresponding user can know the corresponding audio delay.

[0162] In the process of determining the audio delay between the terminals, specifically for the same feature audio, according to the sending time and the receiving time of the feature audio recorded by the first user terminal, the audio delay between the first user terminal and the second user terminal is determined. Specifically, the audio delay is determined according to the difference between the receiving time and the sending time of the same feature audio. Optionally, if the feature audio received by the second user terminal is multiple, the sending time and the receiving time corresponding to each received feature audio can be calculated respectively, and then the audio delay corresponding to each feature audio is determined, and the audio delay corresponding to multiple feature audios is determined according to the statistical value (such as average or mode, etc.) of the audio delay.

[0163] In an optional embodiment, if the step S110 generates a single feature audio, the feature audio with matching sending order and receiving order is determined, and the audio delay between the first user terminal and the second user terminal is determined according to the sending time and the receiving time of the matching feature audio. Specifically, for any order i, the sending time T 1_i of the feature audio with sending order i is obtained, and the receiving time T 2_i of the feature audio with receiving order i is obtained, and T 2_i -T 1-_i is taken as an audio delay. If the mixed audio contains multiple feature audios, each T 2_i -T 1_iThe average value is used to determine the audio delay between the first user terminal and the second user terminal. In this embodiment, the first user terminal only needs to generate a single characteristic audio, and the mixed audio containing multiple characteristic audios can be obtained through multiple insertions of the single characteristic audio. The characteristic audios are marked according to the sending order and the receiving order, and the audio delay is determined according to the sending time and the receiving time of the characteristic audios matched in the order, thereby having high execution efficiency.

[0164] In another optional embodiment, if the step S110 generates multiple characteristic audios with different audio lengths, the sending time and the receiving time of the characteristic audios with the same audio length are determined, and the audio delay between the first user terminal and the second user terminal is determined according to the sending time and the receiving time of each characteristic audio with the same audio length. Specifically, the audio length can identify the characteristic audio, the difference between the receiving time and the sending time of the characteristic audio with the same audio length is taken as the corresponding audio delay, and the audio delay between the first user terminal and the second user terminal is determined according to the average value of the audio delays corresponding to the multiple characteristic audios. By using this embodiment, even if abnormal situations such as network packet loss occur, the same characteristic audio can still be located, thereby improving the accuracy of the determined audio delay.

[0165] Therefore, in the audio delay detection method provided by the embodiment of the present application, the audio delay can be detected by the user terminals of the two parties of the network voice call, without the aid of third-party equipment for audio delay detection, thereby reducing the detection cost and reducing the dependence of audio delay detection on the environment. Moreover, the user terminal generates the basic audio, and transmits the mixed audio containing the basic audio, thereby improving the similarity between the detection process and the real network voice call environment, and ensuring the stability of the code rate and bandwidth demand in the detection process. Moreover, the audio delay between the terminals is determined by the sending time and the receiving time of the characteristic audio, which can improve the detection efficiency and the detection accuracy of the audio delay.

[0166] Figure 2 A flowchart of another audio delay detection method provided by the embodiment of the present application is shown. The flowchart in this embodiment is not used to limit the order of execution steps. According to the needs, some steps in the flowchart can be added or deleted.

[0167] The embodiment of the present application takes the detection of the audio delay between the first user terminal and the second user terminal as an example to elaborate the audio delay detection method. The first user terminal and the second user terminal can perform voice interaction through the network. The execution subject of the embodiment of the present application can be an audio delay detection system.

[0168] Specifically, Figure 3A transmission path of audio in a network voice interaction process. Figure 3 For example, when user terminal A and user terminal B perform network voice interaction, the audio collector in user terminal A collects user audio, the user audio is transmitted to the audio processor in user terminal A through the audio collector for processing, and is transmitted to the Ethernet after processing; the audio processor in user terminal B processes the received user audio, and the processed audio is played through the audio player in user terminal B. Therefore, the technical scheme of the embodiment of the application can calculate the total audio delay from user terminal A to user terminal B, that is, the audio delay from audio collection to audio playback.

[0169] As shown in Figure 2 The method comprises the following steps:

[0170] In step S210, the first client in the first user terminal generates basic audio and feature audio, and obtains mixed audio by inserting the feature audio into the basic audio.

[0171] The first client can be an audio delay detection tool, and the client can be an APP, a mini program, a public number, etc. The first client in the first user terminal can generate basic audio and feature audio, and obtain corresponding mixed audio.

[0172] In step S220, the first client sends the mixed audio to the first audio processor in the first user terminal, and records the sending time of the feature audio in the mixed audio to the first audio processor.

[0173] The first user terminal comprises the first client and the first audio processor, and the first client sends the mixed audio to the first audio processor. Therefore, the sending time recorded by the first client is the time when the feature audio is sent from the first client to the first audio processor.

[0174] In an optional embodiment, the first audio processor transmits the mixed audio to the Ethernet after audio preprocessing, audio encoding and / or sending buffer of the mixed audio. The audio preprocessing includes but is not limited to noise reduction processing, enhancement processing, etc. The audio encoding is specifically encoding processing of the mixed audio. The sending buffer is specifically sending the mixed audio to the buffer for further sending to the Ethernet.

[0175] In step S230, the second client in the second user terminal receives the mixed audio transmitted from the second audio processor in the second user terminal, identifies the feature audio in the mixed audio, and records the receiving time of the feature audio.

[0176] The second user terminal comprises a second client and a second audio processor. The mixed audio received by the second user terminal is first processed by the second audio processor, and then transmitted to the second client after the processing is completed. The second client can be an audio delay detection tool, and the client can be an APP, a mini program, a public number, etc. Thus, the second client records the receiving time of the characteristic audio received from the second audio processor.

[0177] In an optional embodiment, after the second audio processor performs the operations of obtaining the mixed audio from the cache area, audio decoding, and / or audio post-processing on the mixed audio, the mixed audio is transmitted to the second client. The operation of obtaining the mixed audio from the cache area specifically refers to obtaining the mixed audio from the cache area in the second user terminal. The operation of audio decoding specifically refers to decoding processing on the audio. The operation of audio post-processing specifically includes audio enhancement and the like.

[0178] In step S240, the first audio delay between the first user terminal and the second user terminal is determined according to the sending time and the receiving time of the characteristic audio.

[0179] The first audio delay is a part of the total audio delay between the first user terminal and the second user terminal. The specific determination process of the first audio delay can refer to the determination method of the audio delay in the embodiment, which will not be repeated here. Figure 1

[0180] In step S250, the second audio delay of the audio collector in the first user terminal and the third audio delay of the audio player in the second user terminal are obtained, and the total audio delay between the first user terminal and the second user terminal is determined according to the first audio delay, the second audio delay, and the third audio delay.

[0181] The audio delay of the audio collector in the first user terminal is the second audio delay, which is the hardware device delay of the audio collector. Specifically, it is the time length between the time when the audio collector in the first user terminal collects the audio and the time when the audio is sent to the first audio processor. The first audio delay can be obtained according to the hardware parameters of the first user terminal.

[0182] The audio delay of the audio player in the second user terminal is the second audio delay, which is the hardware device delay of the audio player. Specifically, it is the time length between the time when the audio player in the second user terminal obtains the audio and the time when the audio is played. The second audio delay can be obtained according to the hardware parameters of the first user terminal.

[0183] ​In some optional embodiments, the present invention can be applied to testing scenarios for network voice call functions. In this scenario, a first user terminal and a second user terminal can be integrated into the same device, thereby allowing the audio collected by the audio collector in the first user terminal to be directly transmitted to the audio player in the second user terminal, and the time difference between audio collection and audio playback is recorded. This time difference is the sum of the second audio delay and the third audio delay.

[0184] The sum of the first audio delay, the second audio delay, and the third audio delay is the total audio delay in the first user terminal from audio acquisition to playback.

[0185] by Figure 4 For example, the time when the first client in the first user terminal sends the characteristic audio is T1. The audio is transmitted to the Ethernet after audio preprocessing, audio encoding, and transmission buffering. The second user terminal receives the audio, which is received in the buffer, audio decoded, and audio post-processed before being sent to the second client. The time when the second client receives the characteristic audio is T2. Then, the delay between the sending time of the previous node of the first audio processor and the receiving time of the next node of the second audio processor is ΔT1 = T2 - T1. The delay of the audio collector in the first user terminal is ΔT2, and the delay of the audio player in the second user terminal is ΔT3. Then, the total audio delay ΔT between the first user terminal and the second user terminal is ΔT1 + ΔT2 + ΔT3. This total audio delay ΔT is the time T when the audio collector of the first user terminal collects the audio. M The time T when the audio is played by the audio player on the second user terminal N The time difference between them.

[0186] Therefore, the embodiments of the present invention determine the total delay between the first user terminal and the second user terminal based on the first audio delay between the first user terminal and the second user terminal, the second audio delay of the audio acquisition device in the first user terminal, and the third audio delay of the audio player in the second user terminal, thereby improving the accuracy of the determination of the total delay.

[0187] In some optional embodiments, the first user terminal monitors a preset event, and after monitoring the preset event, the first user terminal performs the steps of generating the base audio and the feature audio. That is, after monitoring the preset event, the first user terminal triggers the detection of the audio delay between the first user terminal and the second user terminal. The preset event includes at least one of the following events: a version update event, a trigger event of user audio delay detection, and a network switching event. For example, when the version of the call software of the first user terminal is updated, the audio delay detection between the user terminals can be automatically triggered; or, the user can actively trigger the audio delay detection between the user terminals according to the needs through a corresponding trigger mode; or, when it is monitored that the network is switched (such as from a WIFI network to a mobile network), the audio delay detection between the user terminals is triggered. In this way, the audio delay detection between the user terminals can be automatically implemented.

[0188] The embodiments provided by the present application can be applied to audio delay in a live streaming mic-in-mic scenario or audio delay in instant messaging. The embodiments can be applied to actual user scenarios and test scenarios. For example, in a user scenario, the audio delay between the user and a voice interaction user can be obtained through the embodiments of the present application, in a test scenario, the overall audio delay of the mic-in-mic function or the instant messaging function can be detected through the embodiments of the present application, and the embodiments of the present application can be applied to CI / CD (software development continuous integration and continuous exchange construction link), so as to facilitate the detection of the influence of new functions on the audio delay in the development process.

[0189] Figure 5 A flowchart of another audio delay detection method provided by an embodiment of the present application is shown. The flowchart in the embodiment is not used to limit the order of the execution steps. According to the needs, some steps in the flowchart can be added or deleted. The execution subject of the embodiment of the present application can be a first user terminal. As shown in the figure, the method includes the following steps: Figure 5

[0190] Step S510, generating the base audio and the feature audio, and obtaining the mixed audio after inserting the feature audio into the base audio.

[0191] Step S520, sending the mixed audio.

[0192] Step S530, recording the sending time of the feature audio in the mixed audio, so that the second user terminal receives the mixed audio, identifies the feature audio in the mixed audio, records the receiving time of the feature audio, and determines the audio delay between the second user terminal according to the sending time and the receiving time.

[0193] ​In an alternative implementation, the generating the base audio further comprises: generating the base audio by using a random algorithm.

[0194] In an alternative implementation, the generating the base audio further comprises: obtaining a sampling value range of the sampling points of the base audio, and generating the sampling value of the sampling points of the base audio by using a random algorithm based on the sampling value range.

[0195] In an alternative implementation, the generating the sampling value of the sampling points of the base audio by using a random algorithm based on the sampling value range further comprises:

[0196] generating a random number by using a random algorithm, determining a boundary value of the sampling value range, calculating a modulus of the random number and the boundary value, and generating the sampling value of the sampling points of the base audio according to the modulus;

[0197] and / or, taking the sampling value range as a random range of a random algorithm, generating a random number by using a random algorithm based on the random range, and generating the sampling value of the sampling points of the base audio according to the random number.

[0198] In an alternative implementation, the obtaining the sampling value range of the sampling points of the base audio further comprises:

[0199] obtaining a stored historical audio, and determining a sampling value range of the sampling points of the historical audio;

[0200] determining the sampling value range of the sampling points of the base audio according to the sampling value range of the sampling points of the historical audio.

[0201] In an alternative implementation, the feature audio is a mute audio; or, the feature audio is an audio with a preset waveform.

[0202] In an alternative implementation, after the inserting the feature audio into the base audio to obtain a mixed audio, the method further comprises: recording an insertion position of the feature audio in the mixed audio.

[0203] the sending the mixed audio; and the recording the sending time of the feature audio in the mixed audio further comprises: dividing the mixed audio into a plurality of audio segments, and recording a starting position and an ending position of each audio segment in the mixed audio; sending one audio segment every preset time length, and determining whether the insertion position of the feature audio is within the starting position and the ending position of the audio segment when the audio segment is sent; if yes, determining the sending time of the audio segment as the sending time of the feature audio in the audio segment.

[0204] In an alternative implementation, the sending the mixed audio; and recording the sending time of the characteristic audio in the mixed audio further comprises: dividing the mixed audio into a plurality of audio segments, and assigning a preset label to an audio segment containing a characteristic audio; sending an audio segment every preset time interval, and determining whether the audio segment has the preset label when the audio segment is sent; and if yes, determining the sending time of the audio segment as the sending time of the characteristic audio in the audio segment.

[0205] In an alternative implementation, the generating the characteristic audio further comprises: generating a single characteristic audio.

[0206] The inserting the characteristic audio into the base audio further comprises: inserting the single characteristic audio into at least one position of the base audio.

[0207] The recording the sending time of the characteristic audio in the mixed audio further comprises: recording the sending time and sending order of the characteristic audio at each position in the mixed audio.

[0208] The recording the receiving time of the characteristic audio further comprises: recording the receiving time and receiving order of the characteristic audio.

[0209] The determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time further comprises: determining a characteristic audio whose sending order matches a receiving order, and determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of the matched characteristic audio.

[0210] In an alternative implementation, the generating the characteristic audio further comprises: generating a plurality of characteristic audios with different audio lengths.

[0211] The inserting the characteristic audio into the base audio further comprises: inserting the plurality of characteristic audios with different audio lengths into different positions of the base audio respectively.

[0212] The recording the sending time of the characteristic audio in the mixed audio further comprises: recording the sending time and audio length of the characteristic audio in the mixed audio.

[0213] The recording the receiving time of the characteristic audio further comprises: recording the receiving time and audio length of each characteristic audio.

[0214] The determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time further comprises: determining the sending time and the receiving time of the characteristic audio with the same audio length, and determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of each characteristic audio with the same audio length.

[0215] In an alternative embodiment, the recording the sending time of the characteristic audio in the mixed audio further comprises: determining a system to which the second user terminal belongs, and recording the sending time of the characteristic audio based on the system time of the system.

[0216] In an alternative embodiment, the generating the base audio and the characteristic audio, and obtaining the mixed audio after inserting the characteristic audio in the base audio further comprises: the first client generating the base audio and the characteristic audio, and obtaining the mixed audio after inserting the characteristic audio in the base audio.

[0217] The sending the mixed audio and recording the sending time of the characteristic audio in the mixed audio further comprises: sending the mixed audio to a first audio processor, and the first client recording the sending time of the characteristic audio sent to the first audio processor in the mixed audio.

[0218] In an alternative embodiment, the method further comprises:

[0219] The first audio processor transmits the mixed audio to an Ethernet after performing audio preprocessing, audio encoding and / or sending buffer on the mixed audio.

[0220] In an alternative embodiment, the method further comprises: obtaining a second audio delay of an audio collector, so as to determine the total audio delay between the second user terminal according to the first audio delay, the second audio delay and a third audio delay of an audio player in the second user terminal.

[0221] In an alternative embodiment, the method further comprises:

[0222] Monitoring a preset event.

[0223] After monitoring the preset event, performing the step of generating the base audio and the characteristic audio.

[0224] The preset event comprises at least one of the following events:

[0225] A version update event, a trigger event of user audio delay detection, and a network switching event.

[0226] The specific implementation process of each step of the embodiment can refer to the description in the other method embodiments, which will not be repeated here.

[0227] It can be seen that the embodiment of the application can reduce the detection cost, improve the similarity of the detection process and the real network voice call environment, guarantee the stability of the code rate and bandwidth requirement in the detection process, and improve the detection efficiency and detection precision of the audio delay.

[0228] Figure 6 Another flowchart of another audio delay detection method provided by the embodiment of the application is shown; wherein the flowchart in the embodiment is not used to limit the order of the execution steps. According to the needs, some steps in the flowchart can be added or deleted. The execution subject of the embodiment of the application can be the second user terminal. As shown in the figure, the method comprises the following steps: Figure 6

[0229] Step S610, receiving mixed audio; the mixed audio is composed of the base audio generated by the first user terminal and the feature audio.

[0230] Step S620, identifying the feature audio in the mixed audio.

[0231] Step S630, recording the receiving time of the feature audio, so as to determine the audio delay between the first user terminal according to the sending time and the receiving time of the feature audio recorded by the first user terminal.

[0232] In an optional implementation, the recording of the receiving time of the feature audio further comprises: determining a system to which the first user terminal belongs, and recording the receiving time of the feature audio based on the system time of the system.

[0233] In an optional implementation, if the feature audio is a mute audio;

[0234] Then the identifying of the feature audio in the mixed audio further comprises: performing windowing processing on the mixed audio; wherein the window length matches the audio length of the feature audio.

[0235] For any window, the short-time energy corresponding to the window is calculated, and if the short-time energy is less than a preset energy threshold, the audio segment corresponding to the window is determined as the feature audio.

[0236] And / or, for any window, the zero-crossing rate corresponding to the window is calculated, and if the zero-crossing rate is less than a preset zero-crossing rate threshold, the audio segment corresponding to the window is determined as the feature audio.

[0237] In an optional implementation, if the feature audio is an audio of a preset waveform;

[0238] ​The identifying the characteristic audio in the mixed audio further includes: performing windowing processing on the mixed audio; wherein a window length matches an audio length of the characteristic audio; for each window, judging whether a waveform of audio corresponding to the window is a preset waveform; if yes, determining the audio corresponding to the window as the characteristic audio.

[0239] In an optional implementation, the receiving the mixed audio further includes: the second client receiving the mixed audio transmitted from the second audio processor.

[0240] The identifying the characteristic audio in the mixed audio further includes: the second client identifying the characteristic audio in the mixed audio.

[0241] The recording the receiving time of the characteristic audio further includes: the second client recording the receiving time of the characteristic audio received from the second audio processor.

[0242] In an optional implementation, a third audio delay of an audio player is acquired, so as to determine a total audio delay between the first user terminal according to the first audio delay between the first user terminal determined according to the sending time and the receiving time of the characteristic audio recorded by the first user terminal, the second audio delay of the audio collector in the first user terminal, and the third audio delay.

[0243] The specific implementation process of each step of the embodiment can refer to the description in the other method embodiments, which will not be repeated here.

[0244] Therefore, the embodiment can reduce the detection cost, improve the similarity between the detection process and the real network voice call environment, guarantee the stability of the code rate and bandwidth requirement in the detection process, and improve the detection efficiency and detection precision of the audio delay.

[0245] Figure 7 A flowchart of another audio delay detection method provided by the embodiment is shown; wherein the flowchart in the embodiment is not used to limit the order of the execution steps. According to the need, some steps in the flowchart can be added or deleted. The execution subject of the embodiment can be a preset processing end, which can be located in the first user terminal, the second user terminal, or a service end corresponding to the first user terminal and the second user terminal. As shown in the figure, the method includes the following steps: Figure 7

[0246] Step S710, receiving the sending time of the characteristic audio sent by the first user terminal; wherein the first user terminal generates the basic audio and the characteristic audio, and obtains the mixed audio by inserting the characteristic audio in the basic audio.

[0247] ​Step S720, receiving the receiving time of the characteristic audio sent by the second user terminal; wherein the second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records the receiving time of the characteristic audio.

[0248] Step S730, determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time.

[0249] In an optional embodiment, the determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time further comprises:

[0250] determining a first audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time;

[0251] obtaining a second audio delay of an audio collector in the first user terminal, and obtaining a third audio delay of an audio player in the second user terminal;

[0252] determining a total audio delay between the first user terminal and the second user terminal according to the first audio delay, the second audio delay and the third audio delay.

[0253] In an optional embodiment, the determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time further comprises: determining the characteristic audio with the sending sequence matching the receiving sequence, and determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of the matched characteristic audio.

[0254] Or, determining the sending time and the receiving time of the characteristic audio with the same audio length, and determining the audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time of each characteristic audio with the same audio length.

[0255] Therefore, the embodiment of the present application can reduce the detection cost, improve the similarity between the detection process and the real network voice call environment, guarantee the stability of the code rate and bandwidth demand in the detection process, and improve the detection efficiency and detection precision of the audio delay.

[0256] Figure 8 Fig. 1 shows a structural schematic diagram of an audio delay detection device provided by an embodiment of the present application. The device is specifically located in a first user terminal.

[0257] As shown in Fig. 8, the device 800 comprises: Figure 8

[0258] ​The generating module 810 is configured to generate a base audio and a feature audio.

[0259] The inserting module 820 is configured to insert the feature audio into the base audio to obtain a mixed audio.

[0260] The sending module 830 is configured to send the mixed audio.

[0261] The recording module 840 is configured to record a sending time of the feature audio in the mixed audio, so that a second user terminal receives the mixed audio, identifies the feature audio in the mixed audio, records a receiving time of the feature audio, and determines an audio delay between the second user terminal and the first user terminal according to the sending time and the receiving time.

[0262] In an optional implementation, the generating module is configured to generate the base audio by using a random algorithm.

[0263] In an optional implementation, the generating module is configured to obtain a sampling value range of a sampling point of the base audio, and generate a sampling value of the sampling point of the base audio by using a random algorithm based on the sampling value range.

[0264] In an optional implementation, the generating module is configured to generate a random number by using a random algorithm, determine a boundary value of the sampling value range, calculate a modulus of the random number and the boundary value, and generate the sampling value of the sampling point of the base audio according to the modulus.

[0265] In an optional implementation, the generating module is configured to obtain the sampling value range as a random range of the random algorithm, generate a random number by using a random algorithm based on the random range, and generate the sampling value of the sampling point of the base audio according to the random number.

[0266] In an optional implementation, the generating module is configured to obtain a stored historical audio, and determine a sampling value range of a sampling point in the historical audio.

[0267] The generating module is configured to determine a sampling value range of a sampling point of the base audio according to the sampling value range of the sampling point in the historical audio.

[0268] In an optional implementation, the feature audio is a mute audio, or the feature audio is an audio with a preset waveform.

[0269] In an optional implementation, the sending module is configured to divide the mixed audio into a plurality of audio segments, and send one audio segment every preset time length.

[0270] The recording module is configured to record the insertion position of the characteristic audio in the mixed audio, and record the start position and the end position of each audio segment in the mixed audio, and determine whether the insertion position of the characteristic audio is within the start position and the end position of the audio segment when the audio segment is sent, and if so, determine the sending time of the audio segment as the sending time of the characteristic audio in the audio segment.

[0271] In an optional implementation, the sending module is configured to divide the mixed audio into a plurality of audio segments, send one audio segment every preset time length, and assign a preset label to an audio segment containing a characteristic audio.

[0272] The recording module is configured to determine whether the audio segment has the preset label when the audio segment is sent, and if so, determine the sending time of the audio segment as the sending time of the characteristic audio in the audio segment.

[0273] In an optional implementation, the generating module is configured to generate a single characteristic audio.

[0274] The inserting module is configured to insert the single characteristic audio into at least one position of the base audio.

[0275] The recording module is configured to record the sending time and the sending order of the characteristic audio at each position in the mixed audio.

[0276] In an optional implementation, the generating module is configured to generate a plurality of characteristic audios with different audio lengths.

[0277] The inserting module is configured to insert the plurality of characteristic audios with different audio lengths into different positions of the base audio respectively.

[0278] The recording module is configured to record the sending time and the audio length of the characteristic audio in the mixed audio.

[0279] In an optional implementation, the recording module is configured to determine a system to which the second user terminal belongs, and record the sending time of the characteristic audio based on the system time of the system.

[0280] In an optional implementation, the generating module, the inserting module, the sending module, and the recording module are located in the first client.

[0281] The sending module is configured to send the mixed audio to a first audio processor.

[0282] The recording module is configured to record the sending time of the characteristic audio in the mixed audio sent to the first audio processor.

[0283] In an alternative embodiment, the first audio processor transmits the mixed audio to an Ethernet after performing audio pre-processing, audio encoding and / or sending buffer on the mixed audio.

[0284] In an alternative embodiment, the apparatus further comprises an obtaining module configured to obtain a second audio delay of the audio collector, so as to determine a total audio delay between the second user terminal based on the first audio delay, the second audio delay and a third audio delay of an audio player in the second user terminal.

[0285] In an alternative embodiment, the apparatus further comprises a monitoring module configured to monitor a preset event.

[0286] After the preset event is monitored, the generating the base audio and the feature audio is performed.

[0287] The preset event comprises at least one of the following events:

[0288] a version update event, a trigger event of user audio delay detection and a network switching event.

[0289] Therefore, the embodiment of the present application can reduce the detection cost, improve the similarity between the detection process and the real network voice call environment, guarantee the stability of the code rate and bandwidth requirement in the detection process, and improve the detection efficiency and detection accuracy of the audio delay.

[0290] Figure 9 Fig. 9 shows a structure schematic diagram of another audio delay detection apparatus provided by the embodiment of the present application; the apparatus 900 is located in the second user terminal. As shown in Fig. 9, the apparatus 900 comprises: Figure 9

[0291] a receiving module 910 configured to receive mixed audio; the mixed audio is composed of base audio and feature audio generated by the first user terminal;

[0292] a recognizing module 920 configured to recognize the feature audio in the mixed audio;

[0293] a recording module 930 configured to record a receiving time of the feature audio, so as to determine an audio delay between the first user terminal based on a sending time of the feature audio recorded by the first user terminal and the receiving time.

[0294] In an alternative embodiment, the recording module is configured to determine a system to which the first user terminal belongs, and record the receiving time of the feature audio based on a system time of the system.

[0295] In an alternative embodiment, if the feature audio is mute audio, the recognizing module is configured to recognize the mute audio based on a preset time interval. ​

[0296] The identification module is configured to perform windowing processing on the mixed audio, wherein a window length matches an audio length of the characteristic audio.

[0297] For any window, a short-time energy corresponding to the window is calculated, and if the short-time energy is less than a preset energy threshold, an audio segment corresponding to the window is determined as the characteristic audio.

[0298] For any window, a zero-crossing rate corresponding to the window is calculated, and if the zero-crossing rate is less than a preset zero-crossing rate threshold, an audio segment corresponding to the window is determined as the characteristic audio.

[0299] In an optional embodiment, the characteristic audio is audio of a preset waveform.

[0300] The identification module is configured to perform windowing processing on the mixed audio, wherein a window length matches an audio length of the characteristic audio; for each window, it is determined whether a waveform of audio corresponding to the window is a preset waveform; if yes, the audio corresponding to the window is determined as the characteristic audio.

[0301] In an optional embodiment, the receiving module, the identification module, and the recording module are located in the second client.

[0302] The receiving module is configured to receive the mixed audio transmitted from the second audio processor.

[0303] The recording module is configured to record a receiving time of the characteristic audio received from the second audio processor.

[0304] In an optional embodiment, the apparatus further comprises an acquisition module configured to acquire a third audio delay of an audio player, so as to determine a total audio delay between the first user terminal based on the first audio delay between the first user terminal determined according to a sending time of the characteristic audio recorded by the first user terminal and the receiving time, the second audio delay of the audio collector in the first user terminal, and the third audio delay.

[0305] Therefore, the embodiment of the present application can reduce detection cost, improve similarity between a detection process and a real network voice call environment, guarantee stability of code rate and bandwidth demand in the detection process, and improve detection efficiency and detection precision of the audio delay.

[0306] Figure 10 FIG. 1 shows a structural schematic diagram of another audio delay detection apparatus provided by an embodiment of the present application; the apparatus 1000 is located in a preset processing terminal, which can be located in a first user terminal or a second user terminal, or in a service terminal corresponding to the first user terminal and the second user terminal. As shown in the figure, the apparatus 1000 comprises: Figure 10 a receiving module 1001, an identification module 1002, and a recording module 1003.

[0307] The receiving module 1010 is configured to receive a sending time of the characteristic audio sent by the first user terminal; wherein the first user terminal generates a basic audio and a characteristic audio, and obtains a mixed audio after inserting the characteristic audio into the basic audio; and receive a receiving time of the characteristic audio sent by the second user terminal; wherein the second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records the receiving time of the characteristic audio;

[0308] The determining module 1020 is configured to determine an audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time.

[0309] In an optional implementation, the determining module is configured to determine a first audio delay between the first user terminal and the second user terminal according to the sending time and the receiving time.

[0310] Obtain a second audio delay of an audio collector in the first user terminal, and obtain a third audio delay of an audio player in the second user terminal.

[0311] Determine a total audio delay between the first user terminal and the second user terminal according to the first audio delay, the second audio delay, and the third audio delay.

[0312] Therefore, the embodiment of the present application can reduce the detection cost, improve the similarity between the detection process and the real network voice call environment, guarantee the stability of the code rate and bandwidth demand in the detection process, and improve the detection efficiency and detection precision of the audio delay.

[0313] Figure 11 A structural schematic diagram of a computing device provided by an embodiment of the present application is shown. The embodiment of the present application does not limit the specific implementation of the computing device.

[0314] As shown in Figure 11 , the computing device can include a processor 1102, a communications interface 1104, a memory 1106, and a communications bus 1108.

[0315] The processor 1102, the communication interface 1104, and the memory 1106 communicate with each other through the communication bus 1108. The communication interface 1104 is configured to communicate with network elements such as clients or other servers. The processor 1102 is configured to execute the program 1110, and specifically can execute the related steps in the above-described embodiments of the method for audio delay detection. Specifically, the program 1110 can include program codes including computer operation instructions.

[0316] The processor 1102 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors included in the computing device can be of the same type or different types, such as one or more CPUs and one or more ASICs. The memory 1106 is configured to store the program 1110. The memory 1106 can include a high-speed RAM memory, and can also include a non-volatile memory such as at least one disk memory. The program 1110 can be specifically configured to cause the processor 1102 to execute the method in any of the above-described method embodiments.

[0317] The embodiments of the present application provide a non-volatile computer storage medium, which stores at least one executable instruction, and the computer executable instruction can execute the audio delay detection method in any of the above-described method embodiments.

[0318] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems can be used with these teachings, or with modifications that take into account the teachings herein. In accordance with the foregoing description, various aspects of the present application are implemented in the exemplary computing environment. Moreover, the present application is not necessarily limited to any particular programming language, and various embodiments of the present application can be implemented using a variety of programming languages. It will be understood that many implementation variations are possible based on the description herein.

[0319] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known methods, structures, and techniques are not described in detail in order not to obscure the understanding of the specification.

[0320] Similarly, it is to be understood that the embodiments of the application can be alternately or additionally employed in a wide variety of contexts and applications, and are not limited to the embodiments described above. For example, the embodiments of the application can be employed in connection with any type of wireless communication system, including but not limited to cellular telephone systems, wireless local area networks, wireless metropolitan area networks, satellite communication systems, and the like. Similarly, it should be appreciated that, for the purpose of simplifying the present embodiments of the application and helping with the understanding of one or more of the inventive aspects, in the above description of exemplary embodiments of the present embodiments of the application, various features of the present embodiments of the application are sometimes grouped together in a single embodiment, figure, or description of related features. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed embodiments of the application require more features than are explicitly recited in each claim. Rather, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following, in this application for patent, are hereby expressly incorporated into this detailed description of the embodiments of the present application, and they are meant to limit the scope of the claimed embodiments of the application to that of the claims.

[0321] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into more sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or apparatus so disclosed, can be taken in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features serving the same, equivalent or a similar purpose, unless expressly stated otherwise.

[0322] Furthermore, those skilled in the art will appreciate that the features of the different embodiments can be combined in any combination, and that the combinations of features of the embodiments are within the scope of the present embodiments of the application and form different embodiments of the present embodiments of the application. For example, in the following claims, any of the claims of the claimed embodiments can be used in any combination.

[0323] The various components of the embodiments of the present application can be implemented in hardware, or as software modules running in one or more processors, or combinations thereof. Skilled persons should appreciate that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present application. The embodiments of the present application can also be implemented as a program of apparatus or device (for example, a computer program and a computer program product) for performing part or all of the methods described herein. Such a program implementing the embodiments of the present application can be stored on a computer readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0324] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a conjunction like 'or', but it is to be understood that each of these devices can be implemented by its own hardware item. The use of the word 'at least' followed by a list of one or more members does not preclude the existence of additional such members. The word 'first' or'second' does not have any order of magnitude. The use of the words 'first' and'second' is not to be interpreted as indicating that the order of the steps is limited to the order in which the steps are recited.

Claims

1. An audio delay detection method, characterized in that, include: The first user terminal generates a base audio and multiple feature audios of different lengths. The multiple feature audios of different lengths are then inserted into different positions of the base audio to obtain a mixed audio. The base audio is used to simulate a real voice interaction environment. The first user terminal sends the mixed audio and records the sending time and length of the characteristic audio in the mixed audio; wherein, the first user terminal sends the mixed audio using a streaming transmission method; The second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records the reception time and length of the characteristic audio. The transmission and reception times of characteristic audios with consistent audio lengths are determined. Based on the transmission and reception times of each characteristic audio with consistent audio length, the audio delay corresponding to each characteristic audio is determined. Based on the statistical values ​​of the audio delays corresponding to multiple characteristic audios, the audio delay between the first user terminal and the second user terminal is determined.

2. The method according to claim 1, characterized in that, The first user terminal generates basic audio, which further includes: The first user terminal uses a random algorithm to generate the basic audio.

3. The method according to claim 2, characterized in that, The first user terminal generates basic audio using a random algorithm, which further includes: The first user terminal obtains the sampling value range of the sampling points of the basic audio, and generates the sampling values ​​of the sampling points of the basic audio using a random algorithm based on the sampling value range.

4. The method according to claim 3, characterized in that, The step of generating sample values ​​for the basic audio sample points using a random algorithm based on the aforementioned sample value range further includes: A random number is generated using a random algorithm, and the boundary values ​​of the range of sampled values ​​are determined; the modulus of the random number and the boundary values ​​is calculated, and the sampled values ​​of the sampled points of the basic audio are generated based on the modulus; And / or, using the range of sampled values ​​as the random range of a random algorithm, and generating random numbers based on the random range using a random algorithm, and generating sampled values ​​of the sampled points of the basic audio based on the random numbers.

5. The method according to claim 3, characterized in that, The range of sample values ​​for obtaining the sample points of the basic audio further includes: Obtain the historical audio stored in the first user terminal, and determine the range of sample values ​​for the sample points in the historical audio; The sampling value range of the sampling points of the basic audio is determined based on the sampling value range of the sampling points in the historical audio.

6. The method according to any one of claims 1-5, characterized in that, The characteristic audio is a silent audio; Alternatively, the characteristic audio may be an audio with a preset waveform.

7. The method according to claim 6, characterized in that, After obtaining the mixed audio, the method further includes: recording the insertion position of the feature audio in the mixed audio; The first user terminal sends the mixed audio, and recording the transmission time of the feature audio in the mixed audio further includes: the first user terminal divides the mixed audio into multiple audio segments and records the start and end positions of each audio segment in the mixed audio; the first user terminal sends an audio segment every preset time interval, and determines whether the insertion position of the feature audio is within the start and end positions of the audio segment when sending the audio segment; if so, the transmission time of the audio segment is determined as the transmission time of the feature audio in the audio segment.

8. The method according to claim 6, characterized in that, The first user terminal sends the mixed audio, and recording the transmission time of the characteristic audio in the mixed audio further includes: The first user terminal divides the mixed audio into multiple audio segments and assigns preset tags to the audio segments containing characteristic audio. The first user terminal sends an audio segment every preset time interval, and determines whether the audio segment has the preset tag when sending the audio segment; if so, the sending time of the audio segment is determined as the sending time of the characteristic audio in the audio segment.

9. The method according to claim 7 or 8, characterized in that, The second user terminal receives the mixed audio, identifies characteristic audio in the mixed audio, and records the reception time of the characteristic audio, which further includes: The second user terminal receives the audio segment and identifies whether the audio segment contains characteristic audio. If so, the reception time of the audio segment is determined as the reception time of the characteristic audio.

10. The method according to claim 6, characterized in that, If the characteristic audio is a silent audio; The step of identifying the feature audio in the mixed audio further includes: windowing the mixed audio; wherein the window length matches the audio length of the feature audio. For any window, calculate the short-time energy corresponding to that window. If the short-time energy is less than a preset energy threshold, then the audio segment corresponding to that window is determined to be the characteristic audio. And / or, for any window, calculate the zero-crossing rate corresponding to that window. If the zero-crossing rate is less than a preset zero-crossing rate threshold, then the audio segment corresponding to that window is determined to be the characteristic audio.

11. The method according to claim 6, characterized in that, If the characteristic audio is an audio with a preset waveform; The step of identifying the feature audio in the mixed audio further includes: windowing the mixed audio; wherein the window length matches the audio length of the feature audio. For each window, determine whether the waveform of the audio corresponding to that window is a preset waveform; if so, then determine the audio corresponding to that window as the feature audio.

12. The method according to any one of claims 1-5, characterized in that, The step of recording the transmission time of the characteristic audio in the mixed audio further includes: determining the system to which the first user terminal and the second user terminal belong, and recording the transmission time of the characteristic audio based on the system time of the system; The recording of the reception time of the characteristic audio further includes: recording the reception time of the characteristic audio based on the system time of the system.

13. The method according to any one of claims 1-5, characterized in that, The first user terminal generates a basic audio and multiple feature audios of different lengths. After inserting the multiple feature audios of different lengths into different positions of the basic audio, a mixed audio is obtained. This includes: the first client in the first user terminal generates a basic audio and multiple feature audios of different lengths. After inserting the multiple feature audios of different lengths into different positions of the basic audio, a mixed audio is obtained. The first user terminal sends the mixed audio, and the recording of the transmission time of the characteristic audio in the mixed audio further includes: the first client sends the mixed audio to the first audio processor in the first user terminal, and the first client records the transmission time of the characteristic audio in the mixed audio sent to the first audio processor; The second user terminal receiving the mixed audio, identifying the characteristic audio in the mixed audio, and recording the reception time of the characteristic audio further includes: the second client in the second user terminal receiving the mixed audio transmitted from the second audio processor in the second user terminal, the second client identifying the characteristic audio in the mixed audio, and recording the reception time of receiving the characteristic audio from the second audio processor; The step of determining the audio delay between the first user terminal and the second user terminal based on the statistical values ​​of audio delays corresponding to multiple characteristic audios further includes: determining the first audio delay between the first user terminal and the second user terminal based on the statistical values ​​of audio delays corresponding to multiple characteristic audios.

14. The method according to claim 13, characterized in that, The method further includes: After the first audio processor performs audio preprocessing, audio encoding, and / or sends to the transmission buffer, it transmits the mixed audio to the Ethernet. After the second audio processor performs audio retrieval from the buffer, audio decoding, and / or audio post-processing on the mixed audio, it sends the mixed audio to the second client.

15. The method according to claim 13, characterized in that, The method further includes: Obtain the second audio delay of the audio collector in the first user terminal, and obtain the third audio delay of the audio player in the second user terminal; The total audio delay between the first user terminal and the second user terminal is determined based on the first audio delay, the second audio delay, and the third audio delay.

16. The method according to any one of claims 1-5, characterized in that, The method further includes: The first user terminal monitors preset events; After detecting a preset event, the first user terminal executes the steps of generating basic audio and feature audio. The preset event includes at least one of the following events: Version update events, user audio latency detection trigger events, and network switching events.

17. An audio delay detection method, characterized in that, include: Generate a base audio and multiple feature audios of different lengths. Insert the multiple feature audios of different lengths into different positions of the base audio to obtain a mixed audio. This base audio is used to simulate a real voice interaction environment; The mixed audio is transmitted using a streaming method; The transmission time and audio length of the characteristic audio in the mixed audio are recorded so that the second user terminal can receive the mixed audio, identify the characteristic audio in the mixed audio, record the reception time and audio length of the characteristic audio, and determine the transmission time and reception time of characteristic audio with the same audio length. The audio delay corresponding to each characteristic audio is determined according to the transmission time and reception time of each characteristic audio with the same audio length. The audio delay between the second user terminal and the second user terminal is determined according to the statistical value of the audio delay corresponding to multiple characteristic audios.

18. The method according to claim 17, characterized in that, The generation of basic audio further includes: generating basic audio using a random algorithm.

19. The method according to claim 18, characterized in that, The step of generating basic audio using a random algorithm further includes: obtaining the sampling value range of the sampling points of the basic audio, and generating the sampling values ​​of the sampling points of the basic audio using a random algorithm based on the sampling value range.

20. The method according to claim 19, characterized in that, The step of generating sample values ​​for the basic audio sample points using a random algorithm based on the aforementioned sample value range further includes: A random number is generated using a random algorithm, and the boundary value of the range of the sampled values ​​is determined. The modulus of the random number and the boundary value is calculated, and the sampled values ​​of the sampled points of the basic audio are generated based on the modulus. And / or, using the range of sampled values ​​as the random range of a random algorithm, and generating random numbers based on the random range using a random algorithm, and generating sampled values ​​of the sampled points of the basic audio based on the random numbers.

21. The method according to claim 19, characterized in that, The range of sample values ​​for obtaining the sample points of the basic audio further includes: Obtain the stored historical audio and determine the range of sample values ​​for the sample points in the historical audio; The sampling value range of the sampling points of the basic audio is determined based on the sampling value range of the sampling points in the historical audio.

22. The method according to any one of claims 17-21, characterized in that, The characteristic audio is a silent audio; or, the characteristic audio is an audio with a preset waveform.

23. The method according to claim 22, characterized in that, After obtaining the mixed audio, the method further includes: recording the insertion position of the feature audio in the mixed audio; The method of transmitting the mixed audio using streaming transmission and recording the transmission time of the characteristic audio in the mixed audio further includes: dividing the mixed audio into multiple audio segments and recording the start and end positions of each audio segment in the mixed audio; transmitting an audio segment every preset duration, and determining whether the insertion position of the characteristic audio is within the start and end positions of the audio segment when transmitting the audio segment; if so, the transmission time of the audio segment is determined as the transmission time of the characteristic audio in the audio segment.

24. The method according to claim 22, characterized in that, The method of transmitting the mixed audio using streaming transmission and recording the transmission time of the characteristic audio in the mixed audio further includes: dividing the mixed audio into multiple audio segments and assigning a preset tag to the audio segment containing the characteristic audio; transmitting an audio segment every preset duration and determining whether the audio segment has the preset tag when transmitting the audio segment; if so, determining the transmission time of the audio segment as the transmission time of the characteristic audio in the audio segment.

25. The method according to any one of claims 17-21, characterized in that, The step of recording the transmission time of the characteristic audio in the mixed audio further includes: determining the system to which the second user terminal belongs, and recording the transmission time of the characteristic audio based on the system time of the system.

26. The method according to any one of claims 17-21, characterized in that, The process of generating a base audio and multiple feature audios of different lengths, and inserting the multiple feature audios of different lengths into different positions of the base audio to obtain a mixed audio, further includes: a first client generating a base audio and multiple feature audios of different lengths, and inserting the multiple feature audios of different lengths into different positions of the base audio to obtain a mixed audio. The step of sending the mixed audio using a streaming method and recording the sending time of the characteristic audio in the mixed audio further includes: sending the mixed audio to a first audio processor, and the first client recording the sending time of the characteristic audio in the mixed audio to the first audio processor; The step of determining the audio delay with the second user terminal based on the statistical values ​​of audio delays corresponding to multiple characteristic audios further includes: determining the first audio delay with the second user terminal based on the statistical values ​​of audio delays corresponding to multiple characteristic audios.

27. The method according to claim 26, characterized in that, The method further includes: After the first audio processor performs audio preprocessing, audio encoding, and / or sends to a buffer, it transmits the mixed audio to an Ethernet network.

28. The method according to claim 26, characterized in that, The method further includes: obtaining a second audio delay of the audio collector, so as to determine the total audio delay between the second user terminal and the first audio delay, the second audio delay, and the third audio delay of the audio player in the second user terminal.

29. The method according to any one of claims 17-21, characterized in that, The method further includes: Monitor preset events; After detecting a preset event, the steps of generating basic audio and feature audio are executed; The preset event includes at least one of the following events: Version update events, user audio latency detection trigger events, and network switching events.

30. An audio delay detection method, characterized in that, include: The system receives mixed audio; the mixed audio consists of a base audio generated by a first user terminal and multiple feature audios of different lengths inserted at different positions in the base audio; the base audio is used to simulate a real voice interaction environment; and the first user terminal sends the mixed audio using a streaming transmission method and records the sending time and audio length of the feature audios in the mixed audio. Identify the characteristic audio in the mixed audio; The reception time and audio length of the characteristic audio are recorded to determine the transmission time and reception time of characteristic audio with the same audio length. The audio delay corresponding to each characteristic audio is determined based on the transmission time and reception time of each characteristic audio with the same audio length. The audio delay between the characteristic audio and the first user terminal is determined based on the statistical value of the audio delay corresponding to multiple characteristic audios.

31. The method according to claim 30, characterized in that, The recording of the reception time of the characteristic audio further includes: determining the system to which the first user terminal belongs, and recording the reception time of the characteristic audio based on the system time of the system.

32. The method according to claim 30, characterized in that, If the characteristic audio is a silent audio; The step of identifying the feature audio in the mixed audio further includes: windowing the mixed audio; wherein the window length matches the audio length of the feature audio. For any window, calculate the short-time energy corresponding to that window. If the short-time energy is less than a preset energy threshold, then the audio segment corresponding to that window is determined to be the characteristic audio. And / or, for any window, calculate the zero-crossing rate corresponding to that window. If the zero-crossing rate is less than a preset zero-crossing rate threshold, then the audio segment corresponding to that window is determined to be the characteristic audio.

33. The method according to claim 30, characterized in that, If the characteristic audio is an audio with a preset waveform; The step of identifying the feature audio in the mixed audio further includes: performing windowing processing on the mixed audio; wherein the window length matches the audio length of the feature audio; for each window, determining whether the waveform of the audio corresponding to the window is a preset waveform; if so, then determining the audio corresponding to the window as the feature audio.

34. The method according to any one of claims 30-33, characterized in that, The receiving of the mixed audio further includes: the second client receiving the mixed audio transmitted from the second audio processor; The step of identifying the characteristic audio in the mixed audio further includes: the second client identifying the characteristic audio in the mixed audio; The recording of the reception time of the characteristic audio further includes: the second client recording the reception time of receiving the characteristic audio from the second audio processor.

35. The method according to any one of claims 30-33, characterized in that, The method further includes: The third audio delay of the audio player is obtained, which is used to determine the first audio delay between the first user terminal, the second audio delay of the audio collector in the first user terminal, and the total audio delay between the first user terminal based on the statistical values ​​of the audio delays corresponding to multiple feature audios, the third audio delay.

36. An audio delay detection method, characterized in that, include: The system receives the transmission time and audio length of the feature audio sent by the first user terminal; wherein, the first user terminal generates a base audio and multiple feature audios of different lengths, and inserts the multiple feature audios of different lengths into different positions of the base audio to obtain a mixed audio; the base audio is used to simulate a real voice interaction environment; the first user terminal sends the mixed audio using a streaming transmission method; The system receives the reception time and audio length of the characteristic audio sent by the second user terminal; wherein the second user terminal receives the mixed audio, identifies the characteristic audio in the mixed audio, and records the reception time and audio length of the characteristic audio. The transmission and reception times of characteristic audios with consistent audio lengths are determined. Based on the transmission and reception times of each characteristic audio with consistent audio length, the audio delay corresponding to each characteristic audio is determined. Based on the statistical values ​​of the audio delays corresponding to multiple characteristic audios, the audio delay between the first user terminal and the second user terminal is determined.

37. The method according to claim 36, characterized in that, Determining the audio delay between the first user terminal and the second user terminal based on statistical values ​​of audio delays corresponding to multiple characteristic audios further includes: The first audio delay between the first user terminal and the second user terminal is determined based on the statistical values ​​of the audio delay corresponding to multiple feature audios. Obtain the second audio delay of the audio collector in the first user terminal, and obtain the third audio delay of the audio player in the second user terminal; The total audio delay between the first user terminal and the second user terminal is determined based on the first audio delay, the second audio delay, and the third audio delay.

38. An audio delay detection device, characterized in that, include: The generation module is used to generate the base audio and multiple feature audios of different lengths. This base audio is used to simulate a real voice interaction environment; The insertion module is used to insert multiple feature audios of different lengths into different positions of the base audio to obtain a mixed audio; The sending module is used to send the mixed audio using a streaming transmission method; The recording module is used to record the transmission time and audio length of the characteristic audio in the mixed audio; so that the second user terminal can receive the mixed audio, identify the characteristic audio in the mixed audio, record the reception time and audio length of the characteristic audio, and determine the transmission time and reception time of characteristic audio with the same audio length. Based on the transmission time and reception time of each characteristic audio with the same audio length, the audio delay corresponding to each characteristic audio is determined, and the audio delay between the second user terminal is determined based on the statistical value of the audio delay corresponding to multiple characteristic audios.

39. An audio delay detection device, characterized in that, include: A receiving module is used to receive mixed audio; the mixed audio consists of a base audio generated by a first user terminal and multiple feature audios of different lengths inserted at different positions in the base audio; the base audio is audio used to simulate a real voice interaction environment; the first user terminal sends the mixed audio using a streaming transmission method and records the sending time and audio length of the feature audios in the mixed audio; The recognition module is used to identify the characteristic audio in the mixed audio; The recording module is used to record the reception time and audio length of the feature audio, so as to determine the transmission time and reception time of feature audio with the same audio length, determine the audio delay corresponding to each feature audio based on the transmission time and reception time of each feature audio with the same audio length, and determine the audio delay between the first user terminals based on the statistical value of the audio delay corresponding to multiple feature audios.

40. An audio delay detection device, characterized in that, include: The receiving module is used to receive the transmission time and audio length of the feature audio sent by the first user terminal; wherein, the first user terminal generates a base audio and multiple feature audios of different lengths, and inserts the multiple feature audios of different lengths into different positions of the base audio to obtain a mixed audio; and receives the reception time and audio length of the feature audio sent by the second user terminal; wherein, the second user terminal receives the mixed audio, identifies the feature audios in the mixed audio, and records the reception time and audio length of the feature audios; the base audio is audio used to simulate a real voice interaction environment; the first user terminal sends the mixed audio using a streaming transmission method; The determination module is used to determine the transmission and reception times of characteristic audios with consistent audio lengths, determine the audio delay corresponding to each characteristic audio based on the transmission and reception times of each characteristic audio with consistent audio lengths, and determine the audio delay between the first user terminal and the second user terminal based on the statistical values ​​of the audio delays corresponding to multiple characteristic audios.

41. A computing device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the audio delay detection method as described in any one of claims 17-37.

42. A computer storage medium, characterized in that, The storage medium stores at least one executable instruction that causes the processor to perform the operation corresponding to the audio delay detection method as described in any one of claims 17-37.

43. A computer program product, characterized in that, The computer program product is used to perform the operation corresponding to the audio delay detection method as described in any one of claims 17-37.

Images